CI API testing with Markdown specs

CI API testing fails when the test is too far away from the API contract. A shell script with five curl commands may be fast to write, but it rarely tells reviewers what behavior changed or what downstream workflow is now protected.

Markdown API specs give CI something better: readable contracts that can be validated before they run, executed against staging, and published as test output.

Validate before you send requests

The first CI step should not hit the network. It should validate the contracts.

- name: Validate API specs
  run: rqb validate api-docs/

This catches the mistakes that waste CI minutes:

• invalid frontmatter,
• missing request blocks,
• malformed expected responses,
• unresolved variables,
• secrets committed to Markdown.

That last point matters. A repo-native API testing workflow needs to be strict about secrets. If a token appears in a spec file, the build should fail before any request is sent.

Run contracts against staging

Once the files validate, run the endpoint specs that protect the change.

- name: Run API contracts
  env:
    RQB_BASE_URL: https://staging.example.com
    RQB_AUTH_TOKEN: ${{ secrets.STAGING_AUTH_TOKEN }}
  run: |
    rqb exec api-docs/apis/health/get-health.md --env=staging
    rqb exec api-docs/apis/users/get-user-by-id.md --env=staging

Use RQB_* variables for secrets and environment-specific values. Reqbook maps RQB_AUTH_TOKEN to {{authToken}} and RQB_BASE_URL to {{baseUrl}}.

That keeps the spec readable:

GET {{baseUrl}}/users/:id
Authorization: Bearer {{authToken}}
Accept: application/json

The CI job owns the secret. The Markdown file owns the contract.

Publish results people can read

CI output should help humans debug without expanding a thousand log lines. For test reporters, write JUnit:

rqb exec api-docs/apis/users/get-user-by-id.md \
  --env=staging \
  --output=junit > results.xml

Then upload the result as a CI artifact or test report.

The useful thing about JUnit is not the XML itself. The useful thing is that API contract failures show up in the same review surface as other tests. A reviewer can see which endpoint failed, whether the response mismatched, and which spec owns the behavior.

Keep network failures separate from contract failures

An API test can fail for different reasons. Treating every failure as “the API is broken” creates noisy builds.

Reqbook uses stable exit codes so CI can react differently:

rqb exec api-docs/apis/users/get-user-by-id.md --env=staging
CODE=$?

case $CODE in
  0) echo "contract passed" ;;
  1) echo "contract mismatch"; exit 1 ;;
  4) echo "network error"; exit 1 ;;
  5) echo "secret found in spec"; exit 1 ;;
  *) echo "unexpected Reqbook error"; exit 1 ;;
esac

A contract mismatch means the response did not match the expected behavior. A network error might mean staging is unavailable. A secret error means a review blocker.

That separation gives API testing in CI a cleaner signal.

A simple PR workflow

For most teams, start with this shape:

name: API spec tests

on:
  pull_request:
    paths:
      - 'api-docs/**'
      - 'src/**'

jobs:
  api:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Install Reqbook
        run: npm install -g reqbook
      - name: Validate specs
        run: rqb validate api-docs/
      - name: Run changed API contracts
        env:
          RQB_BASE_URL: https://staging.example.com
          RQB_AUTH_TOKEN: ${{ secrets.STAGING_AUTH_TOKEN }}
        run: rqb exec api-docs/ --env=staging

Then add targeted rqb flow runs for high-risk journeys like signup, checkout, onboarding, billing, or device pairing.

Run the right specs, not always every spec

Small teams can often run the whole api-docs/ directory on every pull request. Larger systems usually need a sharper strategy.

Use three tiers:

CI tier	When it runs	What it protects
Validate	Every pull request	Spec structure, secrets, missing variables
Targeted endpoint runs	Pull requests touching related code or specs	Changed API behavior
Flow runs	Main branch, release branches, or risky PRs	User journeys and cross-endpoint handoffs

This keeps API testing fast enough that developers do not bypass it. Validation should be cheap. Endpoint runs should be targeted. Flow runs should protect the business path that would hurt if it broke.

For example, a pull request that changes src/billing/ should run billing contracts and checkout flows. A pull request that only changes docs might validate specs but skip live staging calls.

Keep local and CI commands similar

The fastest way to make CI failures painful is to use commands that developers cannot run locally. Reqbook works best when the command in CI is almost the same command a developer runs before pushing.

Local:

rqb validate api-docs/
rqb exec api-docs/apis/users/get-user-by-id.md --env=dev

CI:

rqb validate api-docs/
rqb exec api-docs/apis/users/get-user-by-id.md --env=staging

The environment changes. The contract does not. That makes failures easier to reproduce.

If a CI result says the contract expected 201 but staging returned 200, the developer can run the same spec locally, inspect the Markdown, and decide whether the API behavior or expected response is wrong.

Add artifacts for review, not just logs

CI logs are useful while debugging. They are not a durable review artifact. For API testing, attach a machine-readable result whenever possible:

rqb exec api-docs/ --env=staging --output=json > rqb-results.json
rqb exec api-docs/ --env=staging --output=junit > rqb-results.xml

The JSON result is useful for agent workflows and custom dashboards. The JUnit result is useful for CI test reporters.

For pull requests, the best artifact answers three questions:

• Which contract ran?
• Did it pass?
• If not, was the problem a mismatch, missing variable, auth failure, network failure, or parse error?

That is enough for a human reviewer or coding agent to take the next step without reading every request body.

Do not hide API tests behind deploy-only jobs

If API contract tests only run after deployment, they catch drift late. A better pattern is to validate specs on every pull request and run a subset against a preview or staging environment before merge.

When staging is not always available, still run:

rqb validate api-docs/
rqb exec api-docs/apis/users/get-user-by-id.md --dry-run

Dry runs catch unresolved variables and malformed rendered requests without hitting the network. That is not a replacement for live API testing, but it keeps the contract layer healthy even when infrastructure is unavailable.

For multi-step tests, read API flow testing for onboarding and checkout. For variable handling, read Variables and secrets for local API testing.