Research · We checked 13 public APIs against their own OpenAPI specs. 9 had hard mismatches.· Markdown

Open Library: spec vs live

Summary

Spec
https://openlibrary.org/openapi.json (OpenAPI 3)
Base URL
https://openlibrary.org
Run
, read-only GET requests, no credentials
Coverage
58 operations in the spec, 20 requests sent, 9 successful,20 operations observed
Hard violations
1 distinct schema locations
Reviewer note
Confirmed: /health returns text/plain, documented as application/json; the spec's own example for search.json spellcheck_count violates its schema. Account endpoints 303-redirect when logged out.

Hard violations 1

A typed client generated from the spec would break on these.

Hide 1 finding
hardundocumented-response-media-typewarning
GET /health > response 200 > text/plain

media type "text/plain" is not documented for the 200 response

GET /health → 200 text/plain
expected ["application/json"] actual "text/plain"
  • If the spec is the source of truth: Return one of application/json from GET /health.
  • If the implementation is the source of truth: Add "text/plain" under paths["/health"].get.responses["200"].content.

Undocumented error responses 9

3xx and 4xx statuses the spec does not mention, usually auth or not-found cases.

Show 9 findings
errorsundocumented-statusbreaking
GET /account/loan-history.json > response 303

status 303 is not documented for "GET /account/loan-history.json"

GET /account/loan-history.json?page=1 → 303 text/html
expected ["200","422"] actual 303
  • If the spec is the source of truth: Return one of the documented statuses (200, 422) from GET /account/loan-history.json.
  • If the implementation is the source of truth: Add a "303" response under paths["/account/loan-history.json"].get.responses.
errorsundocumented-statusbreaking
GET /account/loans.json > response 303

status 303 is not documented for "GET /account/loans.json"

GET /account/loans.json → 303 text/html
expected ["200"] actual 303
  • If the spec is the source of truth: Return one of the documented statuses (200) from GET /account/loans.json.
  • If the implementation is the source of truth: Add a "303" response under paths["/account/loans.json"].get.responses.
errorsundocumented-statusbreaking
GET /import/preview.json > response 401

status 401 is not documented for "GET /import/preview.json"

GET /import/preview.json → 401 application/json
expected ["200","422"] actual 401
  • If the spec is the source of truth: Return one of the documented statuses (200, 422) from GET /import/preview.json.
  • If the implementation is the source of truth: Add a "401" response under paths["/import/preview.json"].get.responses.
errorsundocumented-statusbreaking
GET /lists/{olid}/editions.json > response 404

status 404 is not documented for "GET /lists/{olid}/editions.json"

GET /lists/OL1L/editions.json?limit=50&offset=0 → 404 application/json
expected ["200","422"] actual 404
  • If the spec is the source of truth: Return one of the documented statuses (200, 422) from GET /lists/{olid}/editions.json.
  • If the implementation is the source of truth: Add a "404" response under paths["/lists/{olid}/editions.json"].get.responses.
errorsundocumented-statusbreaking
GET /lists/{olid}/seeds.json > response 404

status 404 is not documented for "GET /lists/{olid}/seeds.json"

GET /lists/OL1L/seeds.json → 404 application/json
expected ["200","422"] actual 404
  • If the spec is the source of truth: Return one of the documented statuses (200, 422) from GET /lists/{olid}/seeds.json.
  • If the implementation is the source of truth: Add a "404" response under paths["/lists/{olid}/seeds.json"].get.responses.
errorsundocumented-statusbreaking
GET /lists/{olid}/subjects.json > response 404

status 404 is not documented for "GET /lists/{olid}/subjects.json"

GET /lists/OL1L/subjects.json?limit=20 → 404 application/json
expected ["200","422"] actual 404
  • If the spec is the source of truth: Return one of the documented statuses (200, 422) from GET /lists/{olid}/subjects.json.
  • If the implementation is the source of truth: Add a "404" response under paths["/lists/{olid}/subjects.json"].get.responses.
errorsundocumented-statusbreaking
GET /reading-goal.json > response 401

status 401 is not documented for "GET /reading-goal.json"

GET /reading-goal.json → 401 application/json
expected ["200","422"] actual 401
  • If the spec is the source of truth: Return one of the documented statuses (200, 422) from GET /reading-goal.json.
  • If the implementation is the source of truth: Add a "401" response under paths["/reading-goal.json"].get.responses.
errorsundocumented-statusbreaking
GET /search/editions.json > response 404

status 404 is not documented for "GET /search/editions.json"

GET /search/editions.json?fields=cover_i,ebook_access,isbn,key,language,publish_date,publisher,subtitle,title,work_key&limit=100&q=the lord of the rings&sort=&work_key=/works/OL82536W → 404
expected ["200","422"] actual 404
  • If the spec is the source of truth: Return one of the documented statuses (200, 422) from GET /search/editions.json.
  • If the implementation is the source of truth: Add a "404" response under paths["/search/editions.json"].get.responses.
errorsundocumented-statusbreaking
GET /series/{olid}/editions.json > response 404

status 404 is not documented for "GET /series/{olid}/editions.json"

GET /series/OL1L/editions.json?limit=50&offset=0 → 404 application/json
expected ["200","422"] actual 404
  • If the spec is the source of truth: Return one of the documented statuses (200, 422) from GET /series/{olid}/editions.json.
  • If the implementation is the source of truth: Add a "404" response under paths["/series/{olid}/editions.json"].get.responses.

Our input 4

Our own request values were wrong. Not the API's fault; listed for honesty.

Show 4 findings
inputrequest-accepted-but-invalidwarning
GET /search.json

request to "GET /search.json" violated the spec but was accepted with 200

GET /search.json?author=tolkien&author_facet=&author_key=&fields=author_key,author_name,cover_edition_key,cover_height,cover_i,cover_width,ebook_access,edition_count,first_publish_year,has_fulltext,ia,ia_collection,id_cita_press,id_librivox,id_openstax,id_project_gutenberg,id_project_runeberg,id_standard_ebooks,id_wikisource,key,language,lending_edition_s,lending_identifier_s,public_scan_b,series_key,series_name,series_position,subtitle,title&first_publish_year=&language=&limit=100&person_facet=&place_facet=&public_scan_b=&publisher_facet=&q=the lord of the rings&query={"q": "mark"}&spellcheck_count=3&subject_facet=&time_facet=&title=the hobbit → 200 application/json
actual 200
  • If the spec is the source of truth: Reject requests to GET /search.json that violate the spec; see the request findings for this observation.
  • If the implementation is the source of truth: Loosen the spec for GET /search.json to match what the implementation accepts, if that behavior is intended.
inputrequest-schema-violationinfo
GET /search.json > parameter query:spellcheck_count

query parameter spellcheck_count: value matches none of the anyOf alternatives

GET /search.json?author=tolkien&author_facet=&author_key=&fields=author_key,author_name,cover_edition_key,cover_height,cover_i,cover_width,ebook_access,edition_count,first_publish_year,has_fulltext,ia,ia_collection,id_cita_press,id_librivox,id_openstax,id_project_gutenberg,id_project_runeberg,id_standard_ebooks,id_wikisource,key,language,lending_edition_s,lending_identifier_s,public_scan_b,series_key,series_name,series_position,subtitle,title&first_publish_year=&language=&limit=100&person_facet=&place_facet=&public_scan_b=&publisher_facet=&q=the lord of the rings&query={"q": "mark"}&spellcheck_count=3&subject_facet=&time_facet=&title=the hobbit → 200 application/json
  • If the spec is the source of truth: Enforce the documented schema in the query parameter spellcheck_count.
  • If the implementation is the source of truth: Adjust the composition at paths["/search.json"].get.parameters[spellcheck_count].schema to describe what is sent.
inputrequest-constraint-violationinfo
GET /series/{olid}/seeds.json > parameter path:olid

path parameter olid: value does not match pattern "OL\\d+L"

GET /series/OL45804W/seeds.json → 422 application/json
expected "OL\\d+L" actual "OL45804W"
  • If the spec is the source of truth: Enforce pattern in the path parameter olid.
  • If the implementation is the source of truth: Relax pattern at paths["/series/{olid}/seeds.json"].get.parameters[olid].schema to cover what is sent.
inputrequest-constraint-violationinfo
GET /series/{olid}/subjects.json > parameter path:olid

path parameter olid: value does not match pattern "OL\\d+L"

GET /series/OL45804W/subjects.json?limit=20 → 422 application/json
expected "OL\\d+L" actual "OL45804W"
  • If the spec is the source of truth: Enforce pattern in the path parameter olid.
  • If the implementation is the source of truth: Relax pattern at paths["/series/{olid}/subjects.json"].get.parameters[olid].schema to cover what is sent.

Reproduce

The first hard violation came from this request. Compare the response to the schema at the location named in the finding.

curl -s -H 'Accept: application/json' 'https://openlibrary.org/health'

Data

If a finding is wrong, open an issue; if it is right, a pull request on the spec is welcome.