The demo everyone loves, and the meeting after it
A knowledge assistant demo is easy to love. Someone asks a question in plain language, the answer arrives with three citations, and the room relaxes. We have sat in that room many times. The mood changes about forty minutes later, usually when someone from HR or legal asks a quieter question: what happens if a contractor asks about a colleague’s disciplinary record?
On one engagement the honest answer was that the assistant would have answered. The corpus had been assembled from every SharePoint site the crawler could reach, and the crawler ran with an account that could reach everything. Nothing had leaked, because only four engineers had access to the pilot. But the design had no way to say no, and that is what got the project paused for six weeks.
A vector cannot hold an access decision
The uncomfortable property of an embedding is that it discards exactly the information you need. Similarity is computed over meaning, and permission has nothing to do with meaning. Two paragraphs describing the same restructuring plan can sit millimetres apart in vector space while one is board-only and the other is a published memo.
So the permission decision has to live somewhere else: in metadata carried alongside the chunk, evaluated against the identity of the person asking, at the moment they ask. Every design that tries to approximate this — a separate index per department, a system prompt telling the model to be careful, a reranker trained to prefer public documents — is an approximation, and approximations are unpleasant things to defend in a security review.
Similarity is a property of the text. Authorisation is a property of the person asking. Nothing in a vector store knows the difference unless you tell it.
Filter inside the search, not after it
The most common implementation shortcut is to retrieve twenty chunks and then drop the ones the user cannot see. It works in testing and fails in two specific ways in production. If the user has thin permissions, most of the top twenty get dropped and the answer is built from whatever thin material survived — which looks like a quality problem and gets escalated to the wrong team.
Worse, the number of dropped results is itself a signal. A user who learns that a particular query returns nothing useful, while a similar query returns plenty, has learned something about what exists. Pre-filtering costs more to build because your index has to carry the access control list on every chunk, and your retriever has to accept a filter expression. It is still the only version that holds.
Reindexing is a permissions event, not a data event
Teams schedule reindexing around content freshness — nightly, weekly, whatever the crawler can manage. Then someone leaves a project, their group membership changes, and the index keeps the old answer for another nineteen hours. In a regulated estate that gap is the finding, not the theory.
Two things fix it cheaply. Subscribe to permission-change events from the source system and treat them with the same urgency as a deletion. And evaluate group membership at query time rather than baking a resolved user list into the index, so a revoked group takes effect on the next question rather than the next crawl.
The answer needs a receipt
Once retrieval is authorised, you get something valuable almost for free: every answer can carry the document identifiers, versions and the permission decision that let them through. That record is what turns a difficult conversation into a two-minute lookup. Someone asks why the assistant said that; you show what it read, when, and under whose entitlements.
We now treat the absence of that record as a blocking defect rather than a nice-to-have. It is also the cheapest debugging tool anybody on the project will ever get, because most “the model hallucinated” reports turn out to be retrieval returning something plausible and irrelevant.
Most reports of a hallucinating model are, on inspection, a retrieval layer that found something plausible and irrelevant and did its job too eagerly.
What we do in the first week
Before writing any retrieval code we ask for three things: the list of source systems, the identity groups that govern each one, and one worked example of a question the assistant must refuse. That last one is the useful artefact. It forces the organisation to name a boundary, and it becomes the first test in the evaluation suite.
From there the sequence is dull and reliable. Model the entitlements, index with them attached, push the filter down, log the decision, then start caring about chunk sizes and rerankers. Search quality is a tuning problem you can improve every sprint. Permissions are an architecture decision you get roughly one chance to make.
The short version
Get the permission model right and mediocre retrieval still ships. Get it wrong and world-class retrieval sits in a pilot forever, because no security review will sign a system that cannot explain why a document appeared in an answer.
Talk to an architect