testing/surface_invariants.ts

Routes declaring 404 error schemas should use specific z.literal() or z.enum() error codes, not generic z.string().

A generic 404 schema (ApiError with z.string()) means the error code is unconstrained — the handler could return any string, making client error handling fragile. Routes with params (:id) are the primary 404 producers; their error schemas should use specific constants like ERROR_ACCOUNT_NOT_FOUND.

Only flags routes that have params_schema (param-driven resource lookup) — routes declaring 404 for other reasons (e.g., bootstrap not configured) may legitimately use generic schemas.

Every route has a non-empty description.

The same z.literal() error code should not appear at different HTTP status codes across routes.

Extracts const values from error schema error properties (which correspond to z.literal() in the Zod source). Flags when the same literal appears at different status codes — e.g., ERROR_INVALID_CREDENTIALS at both 401 and 403 would be a bug.

Only checks schemas with const values (literal schemas). Generic z.string() schemas (which produce {type: 'string'} in JSON Schema) are ignored.

Assert that all error schemas meet a minimum specificity threshold.

Calls audit_error_schema_tightness and fails on any entry below the configured threshold. Use allowlist and ignore_statuses to exclude known exceptions during progressive tightening.

Every route's declared error schemas must have an error field at the top level (conforming to the ApiError base shape {error: string}).

Catches typos in error schema definitions and ensures consumers can always read .error from error responses.

Every route with non-null input_schema has 400 in error_schemas.

Keeper-protected routes must be under expected path prefixes.

Catches keeper routes accidentally placed outside the API namespace (e.g., a keeper route at /health or /admin/ instead of /api/...).

Every applicable middleware that declares errors must have those status codes present in the route's error_schemas.

Routes with non-null input schemas should use POST (or other mutation methods), not GET.

GET routes with request bodies are technically allowed by HTTP but semantically suspicious — they bypass browser security assumptions about GET being idempotent. Query-string-driven filtering (audit log, list endpoints) should use params schemas or query string parsing, not input schemas.

No duplicate method+path pairs.

Public mutation routes (auth: none + is_mutation) must be in the allowlist.

Catches accidentally unprotected POST/PUT/DELETE routes. Routes like login and bootstrap are public mutations by design — they go in the allowlist.

Every route with non-null params_schema has 400 in error_schemas.

Every protected route has 401 in error_schemas.

Every route with non-null query_schema has 400 in error_schemas.

Every role/keeper route has 403 in error_schemas.

Sensitive routes must declare rate limiting (rate_limit_key is non-null) or have 429 in their error schemas.

Matches routes against sensitive patterns and flags any that lack rate limit declarations. Catches forgotten rate limiting on credential-handling routes.

Run all structural invariants. Options-free — applies universally.

Run security policy invariants. Configurable with sensible defaults.

Checks: - Sensitive routes are rate-limited - No unexpected public mutation routes - Input schemas use mutation methods (not GET) - Keeper routes under expected prefixes

Audit error schema tightness across all routes in a surface.

Reports which route x status code combinations use generic ApiError (z.string()) vs specific z.literal() or z.enum() error codes. Use the output to prioritize progressive tightening of error schemas.

Recommended baseline error schema tightness for consumer projects.

Uses min_specificity: 'enum' (the assertion default) with ignore_statuses for middleware-derived status codes that are commonly generic (auth middleware produces multiple error codes at 401/403, and 429 comes from rate limiters). Consumers can extend with project-specific allowlist entries.

A single entry in the error schema tightness audit report.

Specificity level of an error schema's error field.

Options for assert_error_schema_tightness.

Configuration for security policy invariants.

All fields have sensible defaults. Pass overrides for project-specific needs.