- Question
- Dense embed Bedrock Cohere Keyword search Postgres
- Combine & rerank top 10 passages
-
Build
[S#]-labeled context - Generate answer Claude via Bedrock
-
Swap
[S#]→ real citations
Retrieval-augmented generation
Climbing Regulations RAG
A question-answering system over federal climbing regulations from the Forest Service, BLM, National Park Service, and Fish & Wildlife Service. Ask it "how many fixed anchors can I place without a permit?" and it answers with real citations to the actual rule text.
The interesting engineering problem here is not "can a model answer a question" — it is making it structurally impossible for the model to cite a rule that does not exist. Retrieved passages are labeled with placeholder IDs before the model ever sees them, the model is instructed to cite only those placeholders, and a separate post-processing step swaps each placeholder for the real citation. An unmapped placeholder is a hard error, not a guess.
- Chunks split on real regulatory structure (title / part / section / paragraph) so every chunk maps to one citable rule, never two
- Hybrid retrieval — dense embeddings plus Postgres keyword search, combined and reranked through Bedrock
- Placeholder-based citation pipeline: 100% valid citations across 86 generated answers, and it declined all 5 deliberately unanswerable questions
- 43-question eval harness (18 hand-written, 25 generated) comparing four retrieval strategies; each run tagged with the corpus version it saw
- Deployed as a FastAPI Lambda container with RDS Postgres + pgvector via AWS CDK — no NAT gateway, ~$0.13 for the live demo window
