testing/error_coverage.ts

Error reachability coverage tracking.

Tracks which declared error statuses (and specific error codes) are actually exercised in tests. ErrorCoverageCollector records status codes (optionally with body error codes) observed during test runs, then assert_error_coverage compares against declared error schemas to find uncovered error paths — reporting per-code when the declared schema is a literal or enum, per-status otherwise.

Declarations
#

7 declarations

view source

assert_error_coverage
#

testing/error_coverage.ts view source

(collector: ErrorCoverageCollector, route_specs: RouteSpec[], options?: ErrorCoverageOptions | undefined): void

Assert error coverage meets a minimum threshold.

Computes the ratio of exercised error paths to total declared error paths. For routes whose status error schema names specific codes (z.literal or z.enum), each declared code counts as one coverage path; for schemas without declared codes (ApiError/z.string()), the status counts as one path. A status-only observation covers all declared codes for that status (the "any-code" rule).

When min_coverage is 0 (default), logs coverage info without failing. When > 0, fails if coverage is below the threshold.

collector

type ErrorCoverageCollector

route_specs

type RouteSpec[]

options?

type ErrorCoverageOptions | undefined
optional

returns

void

throws

  • AssertionError - if `min_coverage > 0` and the covered/total ratio

CoverageFilterOptions
#

testing/error_coverage.ts view source

CoverageFilterOptions

Options controlling which routes/statuses are considered for coverage.

ignore_routes

Routes to skip, in 'METHOD /path' format.

type Array<string>

ignore_statuses

HTTP status codes to skip.

type Array<number>

DEFAULT_INTEGRATION_ERROR_COVERAGE
#

testing/error_coverage.ts view source

0.2

Default minimum error coverage threshold for the standard integration and admin test suites. Conservative — not all error paths are exercisable in the composable suites. Consumers should increase as their test suites mature.

ErrorCoverageCollector
#

testing/error_coverage.ts view source

Tracks which route × status (and route × status × code) combinations have been exercised in tests.

Use record() to log an observed status (optionally with the body's error code), or assert_and_record() to combine response validation with tracking (auto-extracts body.error from the response when present). After all tests, call uncovered() to find declared error paths never exercised.

An observation recorded without a code still satisfies "any-code" coverage requirements for the same status — i.e., if a caller records just the status, all declared codes for that status are considered covered. Per-code tracking is additive: callers who know the body's error value should pass it to get precise per-code gap reporting on routes with literal/enum error schemas.

observed

Observed keys: "METHOD /spec-path:STATUS" or "METHOD /spec-path:STATUS:CODE".

Both shapes coexist — the code-less key marks the status as covered at any code; a code-bearing key adds per-code precision.

type Set<string>

readonly

record

Record an observed error status (optionally with the body's error code) for a route.

Resolves the concrete request path back to the spec template path (e.g., /api/accounts/abc → /api/accounts/:id). When code is provided, it is stored alongside the status for per-code coverage tracking.

type (route_specs: RouteSpec[], method: string, path: string, status: number, code?: string | undefined): void

route_specs
type RouteSpec[]
method
type string
path

request path (may be concrete)

type string
status
type number
code?

observed body error code (pass when the route's error schema declares specific codes via z.literal or z.enum)

type string | undefined
optional
returns void

assert_and_record

Validate a response against its route spec and record the status.

Wraps assert_response_matches_spec and records the status code. For error responses, auto-extracts body.error from the JSON body (via a cloned response, so the original stream stays usable) and records it for per-code coverage. Pass an explicit code to override the auto-extracted value or when the body was already consumed.

type (route_specs: RouteSpec[], method: string, path: string, response: Response, code?: string | undefined): Promise<void>

route_specs
type RouteSpec[]
method
type string
path
type string
response
type Response
code?

observed body error code (override; if omitted and the response body is a JSON object with a string error field, that value is auto-extracted)

type string | undefined
optional
returns Promise<void>
throws
  • Error - if the response body fails the route spec's declared

uncovered

Find declared error paths that were never observed.

Computes the declared set from merge_error_schemas for each route spec. For statuses whose error schema names specific codes (via z.literal or z.enum), reports per-code rows; otherwise reports one row per status. A status-only observation (no code) satisfies all declared codes for that status — the "any-code" rule.

type (route_specs: RouteSpec[], options?: CoverageFilterOptions | undefined): UncoveredEntry[]

route_specs
type RouteSpec[]
options?
type CoverageFilterOptions | undefined
optional
returns UncoveredEntry[]

ErrorCoverageOptions
#

extract_declared_error_codes
#

testing/error_coverage.ts view source

(schema: ZodType<unknown, unknown, $ZodTypeInternals<unknown, unknown>>): string[] | null

Extract declared error code values from an error response schema.

Recognizes schemas shaped like z.object({error: z.literal(...)}) or z.object({error: z.enum([...])}) (incl. looseObject/strictObject). Returns the set of declared code values, or null if the schema doesn't expose a literal/enum error field (e.g., bare ApiError with z.string()).

Used by coverage reporting to split a single declared status into per-code rows when the route's error schema names specific codes.

schema

type ZodType<unknown, unknown, $ZodTypeInternals<unknown, unknown>>

returns

string[] | null

UncoveredEntry
#

testing/error_coverage.ts view source

UncoveredEntry

Uncovered entry — either a status-level row (no code) or a specific-code row.

method

type string

path

type string

status

type number

code

Declared code value missing, when the status's error schema names specific codes.

type string

Depends on
#

Imported by
#