Canadian Policy Navigator

Canadian Policy Navigator brings policy questions and document control into one local workbench. I built it around a simple operational boundary: a submitted document cannot become an answer source merely because somebody uploaded it.

Product preview

The hosted demo displays source passages with generated answers disabled. Its current library contains thirty-four documents. The screenshots below use fictional demonstration policies.

Under the hood

Flow: eligible document editions → section chunks → keyword and optional semantic search → rank fusion → optional reranking → five source passages → structured answer and citation checks.

The models and numeric settings below are source defaults, not confirmed hosted environment values. The current hosted demo exposes source passages with generated answers disabled.

Embeddings and chunking

Markdown is split at headings; HTML extraction removes navigation and scripts; PDF extraction retains page labels. Each section is split into 1,800-character chunks with 160-character overlap. These are character sizes, not token sizes. Passage identities bind the document edition, content checksum, section and chunk.

Semantic indexing embeds heading plus passage using OpenAI text-embedding-3-small in a persistent Chroma index. The code does not request a particular embedding dimension, and this review did not verify index width. SQLite remains authoritative for passages, versions, review records and budget reservations; the vector index is derived data.

Employee-submitted documents follow a separate path: independent review controls admission to the answer library, and their text is excluded from external embedding and generation calls. Private-document questions return local keyword passages.

Retrieval, fusion and reranking

Organization, effective date, department and selected-document filters establish eligible evidence before ranking. BM25Plus searches heading plus text and requires a shared query term. The semantic path prefilters Chroma by organization and allowed documents, then checks returned IDs against eligible passages.

Each path can contribute fifty candidates. Reciprocal-rank fusion uses a rank constant of sixty, rather than adding incompatible keyword and semantic scores. Five passages reach the answer context. Semantic failure preserves keyword results with a warning.

Cohere rerank-v3.5 is an optional top-five reranking adapter requiring enabled live calls and a key. It was not verified as active. Failure preserves fused ranking. Limited context bounds input size but can omit necessary cross-references; overlap can retain boundary context while repeating some text.

Answer flow and evidence checks

The application directs retrieval and generation through ordinary functions, with no LangGraph agent or autonomous document-write tool. The source generation model is gpt-4.1-mini, using structured OpenAI Responses output. Context includes source authority, date basis and fictional status. Short follow-ups reuse the prior user question; a prior model answer is not factual evidence.

Claims and steps must cite the provided short evidence labels, which map back to immutable passage IDs. Unknown citations or provider failures produce source passages and an unavailable answer. Citation membership does not establish semantic support, legal accuracy or complete policy coverage.

Evaluation, safeguards and recorded decisions

Offline authored fixtures check organization isolation, future and superseded dates, department filters, private no-provider retrieval, invented citations and budgets. Comprehensive retrieval benchmarking and independent expert review remain unfinished; no precision/recall claim follows from these tests.

Authenticated roles, CSRF checks, independent publication, immutable editions and stale-update rejection protect document workflows. Transactional spend reservations are capped at $5; provider, batch and context limits apply. Official-source changes enter review rather than silently publishing themselves.

Documented decisions include eligible-document allowlists before nearest-neighbor search, replacing old vectors only after new indexing succeeds, and short citation labels to reduce opaque-ID copying errors. RAPTOR, Flowise export and a LangSmith dashboard remain unfinished course deliverables, not technologies applied in this answer path.

The review boundary

Employees submit documents with ownership, dates and a change summary. A different reviewer can publish, reject or request changes with a recorded reason. Versions are immutable, acknowledgements identify the exact edition, and conflicting updates are rejected. Organization uploads stay local and are excluded from external embedding and answer providers.

Retrieval and operations

The fictional/public library combines lexical retrieval and optional Chroma embeddings, filters by organization and effective date, and merges candidates before answer generation. Citations must identify retrieved evidence. This checks citation membership, not whether every claim is semantically correct.

A registered-source monitor checks official pages, records snapshots and errors, and puts differences into review. It does not discover every new publication. The application also provides local role-based accounts, document readers, review records and editable forms.

Evidence and limits

Behavior tests cover independent approval, access across organizations, future effective dates, acknowledgement history and keeping internal text away from providers. The bank and hospital processes are independently authored fictional examples. They are not employer policies or a claim of legal compliance.

The core application is a working prototype with a hosted demonstration at the link above. Enterprise SSO, managed retention and institution-approved operations require separate production work. Additional course deliverables, including RAPTOR comparisons, the Flowise export and a LangSmith dashboard, are separate from that implemented application and remain unfinished.