pgvector & Retrieval-Augmented Generation (RAG) — ADL v0.3 Reference

Status: Informative · Platform version: 0.3.1 · Display label: ADL v0.3

This chapter describes how to build a Retrieval-Augmented Generation (RAG) application on top of Kalos / ADL using pgvector, and documents the reference implementation — the 2cfr200 project (a research-administration regulatory assistant over 2 CFR 200, the Uniform Guidance).

Scope note. Unlike the geometry type (Chapter 41), ADL v0.3 has no native vector field type — its type system is closed (Chapter 7). This chapter is therefore Informative: §44.1–44.7 specify the integration pattern used today (ADL owns the structured records; a companion service owns the pgvector index), and §44.8 sketches a proposed native vector type for a future platform version. Nothing here changes the runtime; domains without RAG are unaffected.

44.1 Overview

A RAG system answers natural-language questions by retrieving the most relevant passages of a corpus and grounding a language model’s answer in them, with citations back to the source. It has two persistent concerns:

Structured records — documents, sections, chunks, queries, answers, citations. These are ordinary entities with relationships, permissions, validation, history — exactly what an ADL domain models well.
Embeddings + similarity search — each chunk is embedded as a high-dimensional vector; retrieval is approximate-nearest-neighbour (ANN) over those vectors. This is not CRUD and has no place in ADL’s closed type system.

The pattern splits along that seam:

Kalos / ADL domain = the system of record and REST API for all structured data.
pgvector = the vector index, in a companion table managed by a RAG service.
The two are joined by a shared chunk id: ADL owns Chunk metadata; pgvector owns chunk_vectors.embedding, keyed by that same id.

This keeps ADL’s wire contract and CRUD path untouched while letting Postgres do what it is good at. It mirrors the philosophy of Chapter 41 (GeoJSON is the source of truth; the PostGIS column is a sidecar) — here the ADL Chunk row is the source of truth and the pgvector row is the sidecar.

44.2 Why vectors live outside the ADL type system

ADL’s type system is closed — 37 fixed types, no user-defined types (Chapter 7). There is no vector type, and adding embeddings as array<float> would be a mistake:

Similarity search (ORDER BY embedding <=> $q) is not expressible in the filter[field][op]= grammar (Chapter 16).
Embeddings need an ANN index (HNSW / IVFFlat) that ADL’s index model (Chapter 19) does not emit.
An embedding is derived data tied to a specific model + dimension; it is not part of the business contract and should never appear in the resource’s OpenAPI schema.

So the embedding is stored in a separate, service-owned table (chunk_vectors) created and queried with direct SQL by the RAG service — not through ADL. The ADL Chunk resource carries only the metadata (section, ordinal, offsets, token count, embedding status).

44.3 Storage model & the pgvector image

Concern	Owner	Stored as
Document / section / chunk metadata, Q&A, citations	ADL domain (Kalos)	normal ADL tables (`<domain>_*`)
Chunk embedding vector	RAG service	`chunk_vectors.embedding VECTOR(N)` (pgvector), keyed by chunk id

Both live in the same PostgreSQL database, so they share one connection, one backup, and one transactional/provenance boundary.

Companion table

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE IF NOT EXISTS chunk_vectors (
    chunk_id   UUID PRIMARY KEY,      -- == the ADL Chunk.id
    corpus_id  UUID NOT NULL,
    section_id UUID NOT NULL,
    model      TEXT NOT NULL,         -- embedding model that produced the vector
    embedding  VECTOR(768) NOT NULL   -- dimension MUST match the embedding model
);
CREATE INDEX IF NOT EXISTS chunk_vectors_ann
    ON chunk_vectors USING hnsw (embedding vector_cosine_ops);
CREATE INDEX IF NOT EXISTS chunk_vectors_corpus ON chunk_vectors (corpus_id);

VECTOR(768) matches nomic-embed-text; the dimension is configuration, not a constant — changing the embedding model changes N and forces an index rebuild + re-embed.

The Postgres image (PostGIS + pgvector)

The standard postgres:17 image has neither PostGIS nor pgvector; postgis/postgis has PostGIS but not pgvector. To serve a Kalos store that may use both spatial (Chapter 41) and vector features, build one image with both:

# docker/postgres/Dockerfile
FROM postgis/postgis:17-3.5
RUN apt-get update \
 && apt-get install -y --no-install-recommends postgresql-17-pgvector \
 && rm -rf /var/lib/apt/lists/*
COPY init-extensions.sql /docker-entrypoint-initdb.d/20-extensions.sql

-- init-extensions.sql (runs only on a fresh data volume, against POSTGRES_DB)
CREATE EXTENSION IF NOT EXISTS postgis;
CREATE EXTENSION IF NOT EXISTS vector;

On an existing volume the init script does not run; enable the extension once per database:

CREATE EXTENSION IF NOT EXISTS vector;

Collation note. Switching a running container from postgres:17 (Debian bookworm) to a bullseye-based postgis image changes the glibc collation version. Postgres warns about the mismatch; clear it with ALTER DATABASE <db> REFRESH COLLATION VERSION; per database (reindex text indexes if they hold important data).

No-pgvector fallback

For a small corpus, the same companion table works without the extension: store the embedding as DOUBLE PRECISION[] and compute cosine similarity in the service (e.g. NumPy). This trades the ANN index for a brute-force scan — fine for thousands of chunks, not for millions. Keep retrieval behind a single module so pgvector ⇄ brute-force ⇄ a dedicated vector DB is a one-file swap.

44.4 RAG architecture on Kalos

The reference deployment reuses an existing Kalos stack rather than standing up its own database:

        HOST                                   DOCKER (shared Kalos network)
 ┌─────────────────────┐                 ┌────────────────────────────────────────────┐
 │ kalosd (ADL domain) │  Postgres:5432  │  postgres (PostGIS + pgvector)   db: <rag>   │
 │  system of record   │────────────────▶│   ├─ ADL tables (<domain>_*)                 │
 │  REST :<port>        │                 │   └─ chunk_vectors (pgvector, HNSW cosine)  │
 └─────────▲───────────┘                 │                                              │
           │                             │   ollama (embeddings + generation)           │
   rag service ── ADL REST ──────────────│   rag service (ingest / ask / eval)          │
   (writes records as a service user)    │   cli                                        │
                                         └────────────────────────────────────────────┘

Kalos serves the ADL domain (the REST API) and owns all structured rows.
RAG service (a small FastAPI app) does the compute ADL cannot: PDF parse, structure-aware chunking, embedding, ANN retrieval, prompt assembly, generation. It writes every structured result back through the Kalos REST API as a seeded service user (role admin), and owns chunk_vectors directly.
Ollama provides a local embedding model and a local generation model.
The generation call sits behind a Generator interface, so a frontier/cloud backend can be swapped in by config with no change to retrieval.

Ingestion pipeline

Parse the source document to text.
Structure-aware segmentation — split on the document’s own hierarchy (for a regulation: parts / sections / appendices), never a naive fixed window, so every chunk maps to exactly one citable unit.
Persist each section to the ADL Section resource (the citation anchor).
Chunk within a section (never crossing its boundary); persist Chunk metadata.
Embed each chunk via Ollama; upsert the vector into chunk_vectors.
Stamp the corpus row indexed with its provenance (source + version + date).

Ingestion is idempotent per corpus version (match on version label; re-ingest upserts).

Query path (`ask`)

Embed the question.
ANN search chunk_vectors for the corpus (ORDER BY embedding <=> $q LIMIT k).
Hydrate the matched chunks + their sections from the ADL API (?include=).
Assemble a grounded prompt with the passages and their citation labels.
Generate with the local model; capture model, tokens, latency.
Persist Query → Answer → one Citation per cited section, and return the answer plus its citations.

44.5 The ADL domain for RAG

A RAG domain models the corpus and the Q&A trail as ordinary resources. The key design rules:

Citations are first-class. A Citation resource links an Answer to the governing Section (plus the matched chunk + score). In a compliance setting an uncited answer is a failure, so citations are modeled, not stringified.
Corpus is versioned. Provenance (which edition/date the index was built from) is recorded on a Corpus resource — a compliance requirement and the basis for later version-diff features.
Section is a self-referential hierarchy — the citation anchor today and a natural graph node for a future GraphRAG extension.

domain:
  name: ug200                 # ADL identifiers cannot start with a digit
  version: "0.1.0"
  resources:
    Corpus:
      object:
        id:           { type: uuid, primaryKey: true }
        citation:     { type: string, required: true }     # e.g. "2 CFR 200"
        versionLabel: { type: string, required: true, unique: true }  # provenance
        versionDate:  { type: date }
        status:       { type: string, enum: [registered, ingesting, indexed, failed], default: registered }
    Section:
      object:
        id:            { type: uuid, primaryKey: true }
        corpusId:      { type: uuid, required: true, immutable: true }
        parentId:      { type: uuid }                       # self-referential hierarchy
        sectionNumber: { type: string }                    # e.g. "200.414"
        citationLabel: { type: string, required: true }    # e.g. "2 CFR § 200.414"
        heading:       { type: string }
        text:          { type: markdown }
      relationships:
        corpus:   { type: belongsTo, target: Corpus,  foreignKey: corpusId, onDelete: cascade }
        parent:   { type: belongsTo, target: Section,  foreignKey: parentId }
        children: { type: hasMany,   target: Section,  foreignKey: parentId }
        chunks:   { type: hasMany,   target: Chunk,    foreignKey: sectionId }
    Chunk:                                                  # metadata only; vector lives in pgvector
      object:
        id:              { type: uuid, primaryKey: true }
        corpusId:        { type: uuid, required: true, immutable: true }
        sectionId:       { type: uuid, required: true, immutable: true }
        ordinal:         { type: integer, required: true }
        text:            { type: text, required: true }
        embeddingModel:  { type: string }
        embeddingStatus: { type: string, enum: [pending, embedded, failed], default: pending }
      relationships:
        section: { type: belongsTo, target: Section, foreignKey: sectionId, onDelete: cascade }
    Query:    { object: { id: {type: uuid, primaryKey: true}, corpusId: {type: uuid, required: true}, text: {type: text, required: true} } }
    Answer:   { object: { id: {type: uuid, primaryKey: true}, queryId: {type: uuid, required: true}, text: {type: markdown, required: true}, model: {type: string, required: true}, latencyMs: {type: integer} } }
    Citation:
      object:
        id:        { type: uuid, primaryKey: true }
        answerId:  { type: uuid, required: true, immutable: true }
        sectionId: { type: uuid, required: true, immutable: true }
        score:     { type: float }
        snippet:   { type: text }
      relationships:
        answer:  { type: belongsTo, target: Answer,  foreignKey: answerId,  onDelete: cascade }
        section: { type: belongsTo, target: Section, foreignKey: sectionId, onDelete: restrict }

Naming conventions (verified against the runtime). REST paths are kebab-case plural and pluralization is mechanical: Corpus → /corpuses (not corpora), EvalQuestion → /eval-questions. Bus action ids are snake_case singular: adl.ug200.eval_question.create, adl.ug200.corpus.create. The seeder dispatches bus actions, so seed data: entries must use the action-id form.

44.6 Retrieval & generation contract

Grounding. The model is instructed to answer only from the retrieved passages and to cite the source unit for every claim.

Citation floor. If no retrieved chunk clears a relevance threshold, the service returns an explicit “not found in the indexed text of ()” rather than an uncited guess. An uncited substantive answer is treated as a bug.

Prompt skeleton.

System: Answer ONLY from the provided excerpts of {corpus} ({version}). Cite the
section number for every claim, e.g. (§200.414). If the excerpts do not contain the
answer, say so. You provide information, not legal/compliance determinations.

Context:
[2 CFR § 200.414 — Indirect (F&A) costs] <chunk text> ...
Question: {user question}
Answer (cite sections inline, then list the sections relied on):

A standing disclaimer accompanies every answer (the tool informs and points to sources; it does not render legal/compliance determinations).

44.7 Worked example — the `2cfr200` project

The reference implementation (2cfr200) answers research-administration questions over 2 CFR 200 with citations to the governing §200.x section.

Layout

2cfr200/
├── backend/      # Kalos ADL domain `ug200` (config + domain YAML + seeds)
├── pipeline/     # FastAPI RAG service: ingest / ask / eval; owns chunk_vectors
├── cli/          # regcli: ingest / ask / eval / sections / show
└── docker-compose.yml   # ollama + rag + cli (reuse existing Kalos Postgres + host Kalos)

Run

# 1. Postgres with PostGIS + pgvector (one DB per corpus), extension enabled:
docker exec <pg> psql -U postgres -c "CREATE DATABASE ug200"
docker exec <pg> psql -U postgres -d ug200 -c "CREATE EXTENSION IF NOT EXISTS vector"

# 2. Kalos serving the ug200 domain (host build or container), on its own port.

# 3. RAG service + local models:
docker compose up -d ollama rag
docker compose exec ollama ollama pull llama3.1:8b
docker compose exec ollama ollama pull nomic-embed-text

# 4. Ingest (provenance pinned) and ask:
regcli ingest --pdf /corpus/01_2CFR200_Uniform_Guidance.pdf --version "eCFR 2024-04-01"
regcli ask "What is the de minimis indirect cost rate?"
# → grounded answer citing 2 CFR § 200.414, with similarity scores + disclaimer.

Evaluation. A fixed EvalQuestion set (gold expectedSections per question) drives a citation-hit metric — did a cited section intersect the gold set — recorded as EvalResult rows. It is the basis for comparing generation backends (local vs frontier) and, later, vector-only vs graph-augmented retrieval.

44.8 Proposed: a native `vector` field type (non-normative)

A future platform version could fold embeddings into ADL the way Chapter 41 folded spatial data in — as a sidecar, with the ADL row staying the source of truth:

# PROPOSED — not implemented in v0.3.x
embedding: { type: vector, dim: 768, metric: cosine, index: hnsw, model: "nomic-embed-text" }

Sketch, mirroring the geometry design:

Portable layer: stored as JSON array (TEXT/JSONB); REST accepts/emits a number array; works with no extension.
Vector layer (PostgreSQL + pgvector): a managed <field>__vec VECTOR(dim) sidecar column + ANN index, probed for the vector extension at submit (fall back to JSON-only with a warning, exactly as PostGIS is probed today).
Query operator: a nearest / knn operator in the Chapter 16 grammar, e.g. ?nearest[embedding]=<vector|text>&limit=k, compiling to ORDER BY <field>__vec <=> $q.
Capability probe + fallback and OpenAPI publication identical in spirit to §41.3 / §41.6.

Until that exists, use the companion-table integration pattern in §44.3–44.5.

44.9 Limitations & notes

No native type yet. Embeddings are service-owned (companion table), not an ADL field; the ADL domain never sees the vector. This is intentional for v0.3.x.
Consistency. Because chunk_vectors is written outside ADL, deleting a Chunk via the API does not cascade to its vector. Either delete vectors in the same service flow, or periodically reconcile chunk_vectors.chunk_id against the Chunk table.
Dimension coupling. VECTOR(N) is fixed at table creation; changing the embedding model requires a new dimension, an index rebuild, and a full re-embed.
Image. Use one Postgres image carrying both PostGIS and pgvector if a deployment uses spatial and vector features (§44.3).
Scale. pgvector + HNSW scales to millions of vectors. Beyond that, or for advanced hybrid search, a dedicated vector DB behind the same retrieval seam is the next step; prefer a self-hostable one to preserve a local-first posture.

Chapter 5 — Store Adapters (PostgreSQL; the shared database)
Chapter 7 — Type System (closed type set; why vector is not a field type today)
Chapter 16 — Query Parameters (the filter[field][op]= grammar a nearest op would extend)
Chapter 19 — Indexes (ADL index model vs the ANN index pgvector needs)
Chapter 32 — Seeder System (seeding the service account + evaluation set)
Chapter 41 — GIS & Spatial Data (the geometry sidecar pattern this chapter mirrors)