db/fact_store.ts

PG-backed FactStore implementation.

Wraps the raw queries in db/fact_queries.ts with the lifecycle the FactStore interface promises:

  • sync hash on put, stream hash on put_ref (counting bytes against the caller-supplied size)
  • idempotent insert (ON CONFLICT DO NOTHING in the queries layer)
  • JSON ref auto-extraction when content_type signals JSON and the caller didn't pass an explicit refs array
  • verify-on-read for external content; embedded reads skip verify because PG storage IS the hash table
  • mismatched external bytes return null + log warning (treat as unavailable; GC / repair is a separate concern)

Embedded vs disk split: writes route by size. Bytes <= embedded_threshold land in the PG bytes column; larger bytes go to the disk CAS at <facts_dir>/<shard>/<rest> (db/fact_disk_storage.ts) and the row records a file:<shard>/<rest> external_url. put takes fully-buffered bytes; put_stream is the bounded-memory streaming twin (hash BLAKE3 + SHA-256 in one pass, spill past the threshold, enforce max_bytes / ENOSPC). Both need disk_root + fs (the runtime/*Deps) configured for the over-threshold path; without them, an oversize put throws and the caller must put_ref against an externally-managed URL (federation / stub-fetcher tests).

view source

Declarations
#

5 declarations

create_default_fetcher
#

db/fact_store.ts view source

(): FactExternalFetcher import {create_default_fetcher} from '@fuzdev/fuz_app/db/fact_store.js';

Default fetcher backed by globalThis.fetch.

returns

FactExternalFetcher

FACT_EMBEDDED_THRESHOLD_DEFAULT
#

db/fact_store.ts view source

number import {FACT_EMBEDDED_THRESHOLD_DEFAULT} from '@fuzdev/fuz_app/db/fact_store.js';

Default embedded-vs-referenced cutoff (1 MiB).

FactExternalFetcher
#

db/fact_store.ts view source

FactExternalFetcher import type {FactExternalFetcher} from '@fuzdev/fuz_app/db/fact_store.js';

Fetcher abstraction so tests can stub external URL retrieval.

fetch_stream

type (url: string) => Promise<ReadableStream<Uint8Array>>

fetch_bytes

type (url: string) => Promise<Uint8Array>

PgFactStore
#

db/fact_store.ts view source

import {PgFactStore} from '@fuzdev/fuz_app/db/fact_store.js';

PG-backed FactStore. Delegates to db/fact_queries.ts for I/O and adds the lifecycle layer described in the module doc.

inheritance

implements: FactStore

constructor

type new (options: PgFactStoreDeps): PgFactStore

options

type PgFactStoreDeps

put

Store fully-buffered bytes, routing by size: <= embedded_threshold into the PG bytes column; larger into the disk CAS (when disk_root + fs are configured) at <facts_dir>/<shard>/<rest> with a file: URL. Oversize without a disk root throws so the caller routes it through put_ref explicitly. Idempotent — ON CONFLICT DO NOTHING + content-addressed disk filenames make a re-write a no-op.

type (bytes: Uint8Array<ArrayBufferLike>, options?: FactPutOptions | undefined): Promise<string & $brand<"FactHash">>

bytes

type Uint8Array<ArrayBufferLike>

options?

type FactPutOptions | undefined
optional
returns Promise<string & $brand<"FactHash">>

put_stream

Stream bytes into the store with bounded memory, returning the finalized digests + size. Delegates the byte path to stream_fact_to_disk (hash BLAKE3 + SHA-256 in one pass, buffer to the embedded threshold, spill to the disk CAS), then inserts the fact row by placement — embedded bytes go to the PG bytes column, disk-spilled bytes record the file: external_url. The cap is enforced mid-stream (PayloadTooLargeError); a disk-full mid- stream throws StorageFullError.

Refs: explicit options.refs are recorded; JSON auto-extraction is NOT attempted (it would need a buffered re-read, defeating the bounded-memory contract) — streamed uploads are opaque blobs.

Requires fs (and, for the over-threshold spill, disk_root) to be configured. The streaming twin of put; mirrors the Rust FactStore::put_stream.

type (stream: ReadableStream<Uint8Array<ArrayBufferLike>>, max_bytes: number, options?: FactPutOptions | undefined): Promise<...>

stream

type ReadableStream<Uint8Array<ArrayBufferLike>>

max_bytes

type number

options?

type FactPutOptions | undefined
optional
returns Promise<PutStreamOutcome>

put_ref

Stream-hash external content and record (hash, external_url, size). Throws when the streamed byte count disagrees with the caller's declared size — a size mismatch usually means the upload was truncated or the URL points at the wrong content.

type (url: string, size: number, options?: FactPutOptions | undefined): Promise<string & $brand<"FactHash">>

url

type string

size

type number

options?

type FactPutOptions | undefined
optional
returns Promise<string & $brand<"FactHash">>

get

Retrieve bytes. Embedded reads return PG bytes directly; external reads fetch + verify and return null (with a warning log) when the bytes don't match the stored hash.

type (hash: string & $brand<"FactHash">): Promise<Uint8Array<ArrayBufferLike> | null>

hash

type string & $brand<"FactHash">
returns Promise<Uint8Array<ArrayBufferLike> | null>

has

type (hash: string & $brand<"FactHash">): Promise<boolean>

hash

type string & $brand<"FactHash">
returns Promise<boolean>

get_meta

type (hash: string & $brand<"FactHash">): Promise<FactMeta | null>

hash

type string & $brand<"FactHash">
returns Promise<FactMeta | null>

get_refs

type (hash: string & $brand<"FactHash">): Promise<(string & $brand<"FactHash">)[]>

hash

type string & $brand<"FactHash">
returns Promise<(string & $brand<"FactHash">)[]>

delete

Drop a fact row. fact_ref rows referencing this hash as a source cascade via the FK; fact_ref targeting this hash do not — they remain as dangling pointers, consistent with the federation model where target_hash is intentionally not a FK.

Idempotent: deleting an absent fact returns null. The store does NOT verify the fact is unreferenced — that policy lives one layer up (the orphan-fact admin surface in the consumer; a future GC walker).

External-URL unlink is the caller's responsibility — the store doesn't know how to resolve file: / s3: / etc. URLs to a deletable handle. Caller iterates the returned external_url (when non-null) and dispatches to the appropriate cleanup routine. Mirrors the read-side FactExternalFetcher split.

type (hash: string & $brand<"FactHash">): Promise<{ size: number; external_url: string | null; } | null>

hash

type string & $brand<"FactHash">
returns Promise<{ size: number; external_url: string | null; } | null>

{size, external_url} for the deleted row, or null if no row matched the hash.

PgFactStoreDeps
#

db/fact_store.ts view source

PgFactStoreDeps import type {PgFactStoreDeps} from '@fuzdev/fuz_app/db/fact_store.js';

Construction-time deps for PgFactStore.

embedded_threshold (bytes) is the inline-vs-external cutoff: payloads at or under it store embedded in the fact row, larger ones route to the disk CAS. Defaults to FACT_EMBEDDED_THRESHOLD_DEFAULT (1 MiB). Consumers tune it per workload — e.g. a much lower bound (~16 KiB) keeps only small JSON inline and routes image originals + thumbnails to disk.

disk_root is the facts directory backing the <shard>/<rest> disk CAS; fs supplies the filesystem capabilities (a RuntimeDeps satisfies it). When both are set, oversize put + put_stream write to disk and the default fetcher reads from it. When unset, oversize put/put_stream spill throws and reads fall back to the globalThis.fetch-backed default fetcher (or an injected stub). log is optional — the only call site is the verify-mismatch warning path.

deps

type QueryDeps

embedded_threshold?

type number

disk_root?

type string

fs?

type FactDiskStorageDeps

fetcher?

type FactExternalFetcher

log?

type Logger

Depends on
#

Imported by
#