Research · We checked 13 public APIs against their own OpenAPI specs. 9 had hard mismatches.· Markdown

GitLab REST API (gitlab.com, unauthenticated): spec vs live

Summary

Spec
https://gitlab.com/gitlab-org/gitlab/-/raw/master/doc/api/openapi/openapi_v3.yaml (OpenAPI 3)
Base URL
https://gitlab.com
Run
, read-only GET requests, no credentials
Coverage
1864 operations in the spec, 15 requests sent, 8 successful,15 operations observed
Hard violations
3 distinct schema locations
Reviewer note
Confirmed: GET /projects/{id}/issues and /repository/contributors return arrays where the generated spec says object; namespace.parent_id is null but not nullable. gitlab.com rate-limits anonymous clients after a dozen calls, so coverage is thin.

Hard violations 3

A typed client generated from the spec would break on these.

Hide 3 findings
hardresponse-null-not-allowedbreaking
GET /api/v4/projects/{id} > response 200 > application/json > /namespace/parent_id

response body at /namespace/parent_id: value is null but the schema does not allow null

GET /api/v4/projects/278964?license=false&statistics=false&with_custom_attributes=false → 200 application/json
expected ["integer"] actual null
  • If the spec is the source of truth: Return a non-null value in response body at /namespace/parent_id.
  • If the implementation is the source of truth: Allow null at paths["/api/v4/projects/{id}"].get.responses["200"].content["application/json"].schema.properties["namespace"].properties["parent_id"] (add "null" to type, or nullable: true in OpenAPI 3.0).
hardresponse-type-mismatchbreaking
GET /api/v4/projects/{id}/issues > response 200 > application/json

the response body: expected "object" but got array

GET /api/v4/projects/278964/issues?order_by=created_at&page=1&per_page=20&sort=desc&state=all&with_labels_details=false → 200 application/json
expected ["object"] actual "array"
  • If the spec is the source of truth: Return "object" in the response body.
  • If the implementation is the source of truth: Change the type at paths["/api/v4/projects/{id}/issues"].get.responses["200"].content["application/json"].schema to "array".
hardresponse-type-mismatchbreaking
GET /api/v4/projects/{id}/repository/contributors > response 200 > application/json

the response body: expected "object" but got array

GET /api/v4/projects/278964/repository/contributors?order_by=commits&page=1&per_page=20&ref=main&sort=asc → 200 application/json
expected ["object"] actual "array"
  • If the spec is the source of truth: Return "object" in the response body.
  • If the implementation is the source of truth: Change the type at paths["/api/v4/projects/{id}/repository/contributors"].get.responses["200"].content["application/json"].schema to "array".

Undocumented error responses 6

3xx and 4xx statuses the spec does not mention, usually auth or not-found cases.

Show 6 findings
errorsundocumented-statusbreaking
GET /api/v4/groups/{id} > response 429

status 429 is not documented for "GET /api/v4/groups/{id}"

GET /api/v4/groups/278964?with_custom_attributes=false&with_projects=true → 429 text/plain
expected ["200","400","404"] actual 429
  • If the spec is the source of truth: Return one of the documented statuses (200, 400, 404) from GET /api/v4/groups/{id}.
  • If the implementation is the source of truth: Add a "429" response under paths["/api/v4/groups/{id}"].get.responses.
errorsundocumented-statusbreaking
GET /api/v4/projects/{id}/labels > response 401

status 401 is not documented for "GET /api/v4/projects/{id}/labels"

GET /api/v4/projects/278964/labels?include_ancestor_groups=true&page=1&per_page=20&with_counts=false → 401 application/json
expected ["200","400","404"] actual 401
  • If the spec is the source of truth: Return one of the documented statuses (200, 400, 404) from GET /api/v4/projects/{id}/labels.
  • If the implementation is the source of truth: Add a "401" response under paths["/api/v4/projects/{id}/labels"].get.responses.
errorsundocumented-statusbreaking
GET /api/v4/projects/{id}/members > response 401

status 401 is not documented for "GET /api/v4/projects/{id}/members"

GET /api/v4/projects/278964/members?page=1&per_page=20 → 401 application/json
expected ["200","400","404"] actual 401
  • If the spec is the source of truth: Return one of the documented statuses (200, 400, 404) from GET /api/v4/projects/{id}/members.
  • If the implementation is the source of truth: Add a "401" response under paths["/api/v4/projects/{id}/members"].get.responses.
errorsundocumented-statusbreaking
GET /api/v4/projects/{id}/milestones > response 401

status 401 is not documented for "GET /api/v4/projects/{id}/milestones"

GET /api/v4/projects/278964/milestones?page=1&per_page=20&sort=due_date_asc&state=all → 401 application/json
expected ["200","400","404"] actual 401
  • If the spec is the source of truth: Return one of the documented statuses (200, 400, 404) from GET /api/v4/projects/{id}/milestones.
  • If the implementation is the source of truth: Add a "401" response under paths["/api/v4/projects/{id}/milestones"].get.responses.
errorsundocumented-statusbreaking
GET /api/v4/users/{id} > response 429

status 429 is not documented for "GET /api/v4/users/{id}"

GET /api/v4/users/278964?with_custom_attributes=false → 429 text/plain
expected ["200","400","404"] actual 429
  • If the spec is the source of truth: Return one of the documented statuses (200, 400, 404) from GET /api/v4/users/{id}.
  • If the implementation is the source of truth: Add a "429" response under paths["/api/v4/users/{id}"].get.responses.
errorsundocumented-statusbreaking
GET /api/v4/version > response 429

status 429 is not documented for "GET /api/v4/version"

GET /api/v4/version → 429 text/plain
expected ["200","401"] actual 429
  • If the spec is the source of truth: Return one of the documented statuses (200, 401) from GET /api/v4/version.
  • If the implementation is the source of truth: Add a "429" response under paths["/api/v4/version"].get.responses.

Reproduce

The first hard violation came from this request. Compare the response to the schema at the location named in the finding.

curl -s -H 'Accept: application/json' 'https://gitlab.com/api/v4/projects/278964?license=false&statistics=false&with_custom_attributes=false'

Data

If a finding is wrong, open an issue; if it is right, a pull request on the spec is welcome.