ADR-005-a — Contract Testing Pipeline

Status

Accepted

Context

[[adr-005-api-contract-ssot|ADR-005]] establishes API.md as the SSOT for HTTP contracts and requires every endpoint to be documented before implementation. Without automated enforcement, the contract drifts from the codebase silently: endpoints get added without documentation, or documented endpoints never receive tests. A machine-parseable format in API.md and a coverage guard that reads it solve both problems.

Decision

API.md format contract. API.md uses a strict Markdown structure that is both human-readable and machine-parseable. Every endpoint is declared as a level-3 heading:

### METHOD /api/v1/path/

FSM transition endpoints may alternatively appear as table rows under a known section header:

### Employee FSM Transitions
| `POST /{id}/activate/` | ... |

These two patterns are the only entry points for the parser. Any endpoint not matching either pattern is invisible to the coverage guard.

Parser (test_contract_coverage.py). The file backend/tests/test_contract_coverage.py implements the extraction and verification logic:

Pass 1 — Primary extraction: Regex ^###\s+(GET|POST|PATCH|PUT|DELETE)\s+(/api/\S+) matches explicit full-path headers.

Pass 2 — FSM table rows: Detects section context (e.g., ### Employee FSM Transitions, ### Assignment FSM Transitions), then parses table rows matching | `POST /{id}/action/` | ... | and prefixes them with the section base path.

URL normalisation applied to both extracted and tested paths:

Strip query strings (?foo=...)
Replace {id}, {employee_id}, UUIDs, and integer IDs with {id}
Collapse {id}/{id} chains
Ensure trailing slash

Coverage guard (4 tests). The guard scans all test files in backend/tests/ (monolithic test_api_contract.py and modular contract/test_*_contract.py) for lines containing both an HTTP method call (.get(, .post(, etc.) and a URL literal starting with /api/. Each (METHOD, normalised_path) pair found is recorded as covered.

Test	Purpose
`test_all_api_md_endpoints_have_contract_tests`	Fails listing any documented endpoint without a matching contract test
`test_known_untested_has_no_stale_entries`	Fails if `KNOWN_UNTESTED` allowlist contains endpoints no longer in API.md
`test_api_md_has_no_duplicate_endpoint_definitions`	Fails if any `(METHOD, path)` appears more than once in API.md
`test_extraction_sanity_check`	Hardcoded smoke test of ~40 expected endpoints; fails fast if the regex breaks

Test execution. Run the coverage guard and contract tests with:

pytest tests/test_contract_coverage.py -v   # Coverage guard only
pytest tests/contract/ -v                   # Modular contract tests
pytest -m api                               # All API tests (guard + contracts)

The api marker is registered in pyproject.toml and applied via pytestmark in each contract file.

Docker volume requirement. API.md lives at the repository root, outside the backend/ directory. The Docker Compose service mounts it as a read-only volume (./API.md:/API.md:ro) so the parser can resolve the path inside the container.

Write-the-test-first workflow. Adding a ### METHOD /api/... line to API.md immediately causes test_all_api_md_endpoints_have_contract_tests to fail. The developer must then either write the contract test or add the endpoint to KNOWN_UNTESTED with a justification comment. This enforces the contract-first discipline from [[adr-005-api-contract-ssot|ADR-005]].

Consequences

API.md format is not arbitrary — the ### heading pattern is a machine contract. Changing the heading style breaks the parser.
Every new endpoint gets a failing test automatically, preventing undocumented surface area from growing.
The KNOWN_UNTESTED set provides a visible, auditable backlog of deferred tests rather than silent gaps.
The stale-entries test prevents KNOWN_UNTESTED from accumulating dead entries after endpoints are removed from API.md.
The sanity check catches regex regressions early — if the parser extracts fewer than ~40 endpoints, something is broken.