The failure that ends a B2B contract
There is a specific bug that is worse than downtime for a multi-tenant product: one customer's data appearing in another customer's answer. It is unrecoverable in a way outages are not, because it is not a reliability problem the customer can sympathise with — it is a breach.
RAG makes this bug easier to write than it should be, because the retrieval step is often built as a search feature rather than as an access-control boundary, and search features are typically written to be permissive.
Where the filter belongs
The tenant constraint must be applied in the query path as a non-optional condition, derived from a verified claim — a signed token carrying the tenant identity — rather than from a parameter the calling code passes.
The distinction matters more than it first appears. If scoping depends on each caller remembering to include a tenant filter, then isolation is a convention. Conventions are broken by the fourth engineer to touch the code, by a debugging endpoint someone forgot to remove, or by a new code path written in a hurry. The boundary should be impossible to omit, not merely customary to include.
Silo, pool, bridge
Three patterns are in common use. Silo gives each tenant separate infrastructure — the strongest isolation and the highest operational cost, appropriate where regulation or a specific contract demands it. Pool shares infrastructure with tenant identity enforced logically on every row and every query, which is where most products land. Bridge sits between: shared infrastructure with dedicated storage for tenants whose requirements justify it.
The choice is a regulatory and commercial question before it is a technical one. Deciding it by engineering preference alone tends to produce either unnecessary cost or an unhappy conversation during a security review.
In Postgres
For teams already on Postgres with pgvector, a disciplined tenant identifier on every table holding tenant data, plus row-level security as a backstop, gives real separation without operating a database per customer.
Row-level security is worth the setup specifically because it is a second line. Application code will eventually contain a query that forgot its filter. RLS turns that from a breach into an empty result set, which is the difference between a bug and an incident.
In a vector store
Where a dedicated vector database is in play, the equivalent is a namespace per tenant, or tenant identity as required metadata with the filter applied at query time as a hard constraint. Higher-sensitivity tenants can be moved to a dedicated index without changing the calling code.
The property to preserve is the same: the constraint is applied by the layer that serves the query, and it is not optional.
Permissions are not only tenant-level
Within a tenant, documents usually have their own access rules — this folder is restricted, that record belongs to one team. The correct approach is to mirror the source system's access control into retrieval metadata at ingestion time, and enforce it as a hard filter at query time alongside the tenant constraint.
The tempting alternative is to retrieve broadly and instruct the model not to reveal restricted material. This does not work and should be treated as an anti-pattern. A model asked to be discreet is not an access-control mechanism. If a document reaches the context window, treat it as disclosed.
Test it like a boundary
Isolation deserves its own evals, running in CI, asserting that a query authenticated as tenant A never returns content belonging to tenant B. This is one of the rare properties in an LLM system that is cheap to check mechanically and catastrophic to get wrong — which makes it the best possible first eval to write.
Include the awkward cases: a tenant with no data, a user whose permissions changed after ingestion, a document that moved between folders. Those are where mirrored permissions drift out of sync with the source of truth.