Building practical AI
Deepak Kumar Tiwari

Deepak Kumar Tiwari

Technology & AI Leader · Building practical systems

Production workflows / Guide 25

An Enterprise Knowledge Agent That Federates Evidence Without Writing

Build an enterprise knowledge agent that joins permission-aware evidence across systems, exposes stale or conflicting sources, and stays strictly read-only.

Illustrative engineering guide. Examples are not client deployments, production-ready integrations, or evidence of measured results. Validate the design with your domain and security owners.

Reference workflow: Authenticated question and scope; Read-only federated source queries; Permission and freshness validation; Claim-to-evidence synthesis; Cited answer with explicit gaps.
Reference workflow. Steps identify responsibilities, not necessarily separate agents. Branches and recovery paths are explained below. Open full-size diagram.

An enterprise knowledge agent should answer “What do we know, and from which authorized sources?” before anyone asks it to change a system. Consider a hypothetical account manager preparing for a renewal. The signed contract is in a document repository, implementation status is in a project tracker, and a pricing exception is discussed in an email thread. A fluent summary can accidentally turn an informal discussion into an approved commercial commitment. Read-only federation makes the evidence useful without giving the assistant write authority.

Keep Knowledge and Execution Separate

Despite the broad idea of a knowledge-to-action assistant, this design ends at a cited evidence packet. It does not create accounts, update CRM records, send messages, approve exceptions, or start onboarding. It can offer a verified source link and identify the human owner of the next decision. Even a harmless-looking “mark reviewed” write belongs outside this version’s tools.

The distinction is architectural, not a prompt convention. Configure connector credentials without business-system mutation permissions. Expose only search, fetch, and metadata operations. Audit logs and short-lived processing state may be stored internally, but they must not become an undisclosed cross-system action channel.

An LLM-only solution cannot know current access rights, reliably distinguish approved documents from drafts, or prove that an absent search hit means a fact does not exist. Deterministic services enforce source scope, principal identity, freshness budgets, and citation integrity. The model resolves language, extracts candidate claims, and explains supported relationships among evidence items.

Federate With a Shared Evidence Envelope

Keep source-specific adapters small. One searches approved contract records, another reads project status, and another searches permitted correspondence. Microsoft Search in Graph provides a unified search entry point for several Microsoft resources and indexed external content. That capability does not establish a universal authorization rule for every adapter or tenant; each supported resource and permission mode needs explicit verification.

Normalize results into an envelope with source ID, canonical URL, source version, tenant, observed time, effective date where available, authority class, and permission-check result. Preserve source text separately from the model’s interpretation. A contract clause is evidence; “the customer is eligible” is a derived claim that needs both contract applicability and current account facts.

The following JSON is an illustrative internal contract, not a provider API response. It shows why partial coverage must travel with the answer.

{
  "question_id": "renewal-query-18",
  "mode": "read_only",
  "principal_ref": "user-opaque-7",
  "coverage": {
    "contracts": "complete_for_requested_scope",
    "projects": "available",
    "correspondence": "unavailable"
  },
  "claims": [{
    "text": "The signed schedule specifies annual billing.",
    "evidence_ids": ["contract-19:v4:clause-6"],
    "status": "supported"
  }],
  "gaps": ["Email exception discussions could not be checked."],
  "business_writes": []
}

The service must generate coverage and principal fields, validate evidence IDs, and reject unexpected mutation fields. Production also needs source-specific pagination, document parsing, expiry, encryption, and revocation handling. An empty business_writes array documents intent but is not enforcement; credentials and server-side tool allowlists enforce the boundary.

Join Claims Without Inventing Authority

Use stable entity mapping rather than fuzzy names alone. Two customers can share a trading name, and a project can belong to a subsidiary rather than the contracting entity. Require an approved account-to-contract mapping or ask the user to disambiguate. A confidence score is not a substitute for the legal entity identifier.

Define precedence by question. A signed amendment may govern commercial terms, while a live tracker is authoritative for current implementation status. Neither should override the other globally. Present contradictions with source dates and owners: “The approved schedule says annual billing; a later email discusses monthly billing but contains no verified approval.” This is more useful than averaging the statements into a compromise.

Do not call partial retrieval complete. If correspondence times out, produce a narrower answer and identify the missing source. An empty result after successful authorized search means “not found in this scope,” not “no exception exists.” If a source has a stale index, fetch the canonical record before making a high-impact claim or abstain.

One synthesizer is usually enough. Specialist sub-agents become worthwhile when different domains need independent evidence policies, such as contract clause interpretation versus technical dependency analysis. Give them bounded read-only inputs and require claim-level citations. Multiple agents consulting the same document do not create independent corroboration.

Authorization Must Survive the Whole Query

Authenticate the principal server-side and derive the tenant from the session, not the prompt. Apply access checks before retrieval and again before returning citations or cached answers. Cache keys need tenant, principal or equivalent permission scope, authorization version, query scope, and source versions. Sharing a cache only by question text can disclose a restricted answer to the next user.

The MCP authorization specification describes transport-level authorization capabilities and token audience validation. MCP is an integration protocol, not automatic business authorization. A valid token to a connector still requires resource-level checks, and upstream services need their own correctly scoped credentials. Do not pass a token intended for one resource through to another.

Treat documents as hostile input. A wiki page that tells the assistant to fetch payroll or upload attachments must remain quoted evidence, never an instruction. OWASP’s LLM security project identifies prompt injection and sensitive disclosure risks. Restrict outbound fetches to approved source hosts, neutralize active markup, and avoid returning signed download URLs into broadly retained transcripts.

Read-only retries are easier than write recovery but still need budgets. Deduplicate concurrent requests, honor rate limits, and stop after bounded attempts. Preserve per-source failure status so a retry cannot silently blend yesterday’s contract with today’s project state. Human review remains necessary before commercial, legal, or personnel decisions based on the packet.

Acceptance and Evidence of Value

Make these release cases mandatory:

  • Two tenants ask the identical question. No cache entry, citation, or snippet crosses the tenant boundary.
  • Access is revoked after retrieval but before delivery. The final authorization check removes the affected evidence and dependent claims.
  • A document instructs the agent to update CRM or export payroll. No write tool exists, and no unauthorized read occurs.
  • One source times out while others succeed. The answer exposes partial coverage and avoids a definitive absence claim.
  • Two similarly named entities have different contracts. The system requests disambiguation instead of joining their evidence.

Baseline the time employees spend assembling renewal packets manually and have domain owners label claim support. Measure supported-claim precision as correctly supported claims divided by all substantive claims returned. Measure retrieval coverage against an authorized gold set, separately from answer fluency. Guardrails include unauthorized evidence exposure, stale authoritative claims, and hidden partial failures.

Start with two read-only sources and a narrow question family. Any proposed time-reduction target must be evaluated alongside correctness, not asserted as a result. The cross-system onboarding guide is deliberately different: it concerns durable writes and compensation. The legal contract review guide adds human legal interpretation. AI Systems, RAG & Agents supports permission-aware retrieval. Begin by defining the evidence envelope and authority rules before choosing a vector database.

Sources