Your AI should know its boundaries

A private AI assistant reads a firm’s documents so its people can find answers faster. That reach is also the risk. An assistant that can read everything can repeat everything, to whoever asks the right question. The design task is to give it the same boundaries the firm already expects of its staff.

Retrieval before generation

Most document assistants use retrieval-augmented generation, or RAG. The system splits documents into passages, indexes them, finds the passages that match a question, and hands those passages to the model to draft an answer. The model does not need training on client files. It reads them at the moment of the question.

This distinction matters for confidentiality. ANSSI’s generative AI recommendations draw the line with care. Access rights cannot be managed on data used to train a model. On additional data used in production, rights can be managed, within the limits of the tools that store it. Client files belong in the second category.

Permissions travel with every passage

The principle: the assistant establishes what the user may see before it searches, not after it writes. Each indexed passage carries the permissions of its source document. When a partner asks a question, the search runs only across passages that partner may open. The model never receives the rest.

OWASP’s entry on vector and embedding weaknesses describes the failure this prevents. Where several groups share one vector database, one group’s content can surface in another group’s answers. OWASP recommends permission-aware stores with strict logical partitioning. For a multi-family office, one family’s trust deeds should never share an unfiltered search space with another’s.

Filtering after generation is a weaker control. By then the model has read the restricted text, and a well-phrased question may draw it out.

Documents and permissions change

Mandates end, colleagues move on, files are reclassified, drafts are superseded. The index has to follow, and so does everything built on it:

• Sync permissions from the source system on a schedule, and on each change where the source allows it.
• Remove passages when a document is deleted or its retention period ends.
• Mark superseded versions so an older schedule is not quoted as current.
• Record which document versions contributed to each answer.

A permission change also has to reach what already exists: open sessions, cached search results and saved conversations. If a user loses access to a matter at noon, an answer cached at eleven should not remain available to them. Test revocation as carefully as the grant.

The NSA-led Deploying AI Systems Securely guidance recommends role-based or attribute-based access control for AI systems.

Logs that protect rather than expose

The same guidance recommends logs covering inputs, outputs and errors, and ANSSI asks for logging of processing on the AI system. Yet a prompt can contain a client name, an account balance or a medical detail. A full transcript store becomes a second copy of the most sensitive material in the firm.

Log with intent. Record who asked, when, which documents were retrieved and whether access was granted, as a first layer. Keep full prompt and answer text only where a defined purpose requires it, with restricted access, encryption and a retention limit. OWASP’s Logging Cheat Sheet advises masking or excluding sensitive personal data and protecting logs against tampering and unauthorised access.

Citations serve the reviewer

Each answer should link to the passages it drew on. A reviewer can open the source and check the wording in seconds.

A citation supports review; it is not proof. NIST’s Generative AI Profile defines confabulation as confidently stated but false content, and notes that outputs may include confabulated logic or citations. An interface that shows the retrieved passage beside the claim lets the reviewer compare text with text.

Prompt injection and human review

Prompt injection is an instruction hidden in content the model reads. OWASP’s LLM01 entry states that RAG does not fully mitigate it and that no fool-proof prevention is known. One scenario matches this setting: a document in a repository is altered, and the assistant relays the planted instruction to whoever retrieves it. A PDF received from a counterparty is untrusted input.

The controls limit the effect:

• Give the assistant read access only. It drafts; it does not send, sign, transfer or delete.
• Separate and label untrusted content apart from system instructions.
• Require human approval for consequential actions, as OWASP recommends.
• Test with adversarial documents before launch and after changes.

ANSSI also recommends that an AI system never execute critical actions automatically, and that the business keep a way to work when the AI is unavailable.

Before launch, an evaluation set drawn from the firm’s own work helps: real questions with known answers and known sources, including questions a given user must not be able to answer. The size and cadence depend on the use case; a few dozen questions rerun after each model, index or permission change is a reasonable starting point. The professional who signs a letter or memorandum remains accountable for its content.

The architecture described here is illustrative. To review these boundaries against your own document estate, write to contact@desgarde.com.