Enterprise Agent Developer · RAG & Tool Use · L2 LAB
Enterprise RAG and Tool-Use Engineering
Design an enterprise retrieval and tool-use architecture that grounds model responses in authoritative knowledge, validates every context and tool boundary, preserves provenance, enforces least privilege, and fails safely when evidence or authorization is insufficient.
Overview
Enterprise RAG is not merely vector search attached to a prompt. It is a controlled evidence pipeline that begins with source authority and continues through ingestion, classification, chunking, indexing, retrieval, filtering, reranking, context assembly, citation, evaluation, and lifecycle ownership.
Tool use is a separate authority boundary. A model may propose a tool request, but identity, authorization, schema validation, policy, approval, rate limits, idempotency, execution, and system-of-record mutation must remain outside model judgment.
The core mental model is authoritative retrieval, bounded context, controlled action.
Concept Deep Dives
Expand each concept to understand how enterprise knowledge and action capabilities are engineered safely.
1. What makes a source authoritative?
An authoritative source has a defined owner, purpose, data classification, publication process, freshness expectation, access policy, retention rule, and system-of-record status. RAG should not treat every available document as equally trusted. Source authority must be explicit before ingestion.
2. How should ingestion and chunking be designed?
Ingestion should preserve document identity, version, ownership, timestamps, classification, tenant, permissions, and lineage. Chunking should follow semantic and structural boundaries while retaining parent-document references. Chunk size and overlap should be tested against retrieval quality rather than selected by habit.
3. What metadata and access filters are required?
Retrieval must apply identity-aware and tenant-aware filters before context reaches the model. Common controls include document classification, region, tenant, business unit, role, effective date, product, language, legal hold, and source status. Similarity score alone is never an authorization decision.
4. How should retrieval quality be evaluated?
Evaluate whether the system retrieves the correct evidence, excludes unauthorized or stale evidence, ranks useful chunks highly, and supports a grounded answer. Measures may include recall at k, precision at k, mean reciprocal rank, normalized discounted cumulative gain, answer faithfulness, citation correctness, refusal accuracy, latency, and cost.
5. How should context be assembled?
Context assembly should deduplicate overlapping chunks, preserve source labels, prioritize authoritative and current material, enforce token budgets, separate instructions from evidence, and include only the minimum context required. Retrieved content must be treated as untrusted data, not as executable instruction.
6. How are citations and provenance preserved?
Every material claim should be traceable to source identifiers, document versions, chunk identifiers, retrieval timestamps, and the retrieval query or intent. Provenance should survive reranking and context compression so reviewers can reproduce why evidence was selected.
7. How should tools be exposed to an agent?
Each tool requires a narrow purpose, explicit JSON Schema, dedicated identity, least-privilege permissions, data classification, timeout, rate limit, idempotency behavior, deterministic validation, approval requirement, error contract, audit evidence, and accountable owner. Tool descriptions must not grant authority.
8. What prevents retrieval and tool abuse?
Use source allowlists, pre-retrieval authorization, prompt-injection isolation, content scanning, output validation, tool allowlists, schema validation, policy checks, human approval for sensitive actions, no-write defaults, duplicate-action detection, circuit breakers, and complete trace correlation.
Visual Enterprise RAG and Tool-Use Model
The evidence path and action path remain distinct until deterministic controls authorize execution.
Learning rule: retrieval provides evidence, not authority; the model may propose an action, but controlled systems decide whether it can execute.
Example Scenario
A global enterprise wants an employee service agent that answers policy questions from approved documents and proposes service-ticket actions using a mock tool contract.
Synthetic policy documents with owners, effective dates, classifications, tenant metadata, and version history.
Identity-aware filtering, hybrid retrieval, reranking, citation capture, and refusal when no authoritative evidence is found.
A static ticket-proposal JSON Schema that stops before ticket creation and requires deterministic approval.
Source register, ingestion specification, retrieval evaluation set, context policy, tool contract, approval matrix, and trace design.
LAB Exercise
Produce a complete enterprise RAG and tool-use architecture package for the synthetic employee service agent. The exercise is design-only and does not execute a model, retrieval engine, vector database, API, or tool.
Required learner deliverable:
1. Authoritative source inventory and ownership matrix
2. Data classification, tenant, region, and access model
3. Ingestion, parsing, chunking, and versioning specification
4. Metadata schema and pre-retrieval authorization filters
5. Vector, keyword, or hybrid retrieval decision
6. Reranking and context-selection policy
7. Retrieval evaluation dataset and quality metrics
8. Prompt-injection and untrusted-content control design
9. Context token budget and deduplication rules
10. Citation, provenance, and source-version contract
11. Tool inventory and least-privilege permission matrix
12. JSON Schema for each proposed tool request
13. Approval, idempotency, timeout, retry, and rate-limit rules
14. No-evidence, no-authorization, and no-write fallback paths
15. Trace, audit, residual-risk, and governance-boundary statement
Acceptance criteria:
- every indexed source has an owner, authority status, classification, and freshness rule
- retrieval filters enforce identity, tenant, region, and data-policy boundaries
- retrieved text is treated as untrusted data rather than instruction
- retrieval quality is evaluated with reproducible test cases
- material claims preserve citation and provenance
- tools use narrow schemas and dedicated least-privilege identities
- model output cannot authorize or directly execute a write
- insufficient evidence or authority results in clarification, refusal, or handoff
- the learner can explain RAG and tool-use tradeoffs in client-ready languageHigh-Risk Anti-Pattern
A weak implementation uploads every available document into a vector store, retrieves by similarity alone, places raw chunks into the prompt, and gives the model broad tools because the demonstration appears accurate.
Unsafe pattern:
- no source owner or system-of-record designation
- no classification, tenant, or region filters
- stale and superseded documents remain searchable
- chunking strategy chosen without evaluation
- retrieved content can override system instructions
- similarity score treated as access control
- no citation or source-version evidence
- broad tool identity and permissive parameters
- free-form model text becomes an API request
- retries can duplicate irreversible actions
- no approval, idempotency, or no-write fallback
- no trace joining retrieval evidence to tool decisions
Risk:
cross-tenant or unauthorized disclosure
stale or contradictory answers
prompt injection through retrieved content
fabricated or unsupported claims
privilege escalation through tools
duplicate or irreversible system changes
unbounded latency and cost
inability to reproduce decisions
loss of stakeholder and regulator trust
Safe alternative:
register authoritative sources and owners
enforce access before retrieval
preserve metadata, lineage, and versions
evaluate retrieval and citation quality
separate instructions from retrieved evidence
use narrow schemas and least privilege
require deterministic policy and approval
apply idempotency, rate limits, and no-write defaults
trace every evidence and action decision
state residual risk and limitations clearlyGovernance Boundary
This LAB is static and educational. It teaches enterprise RAG and tool-use architecture only. It does not ingest documents, create embeddings, query a vector database, retrieve enterprise data, call a model, invoke a tool, execute an API, create a ticket, persist memory, export telemetry, deploy infrastructure, handle credentials, or mutate any system.
Runtime = read-only learning
Backend exposure = false
Public backend exposed = false
Live LLM inference = false
Agent runtime execution = false
Document ingestion execution = false
Embedding generation = false
Vector database execution = false
RAG retrieval execution = false
Enterprise source access = false
Cross-tenant retrieval = false
Live tool invocation = false
Live function calling = false
Live API call execution = false
Ticket creation execution = false
Approval execution = false
Memory persistence = false
MCP server execution = false
MCP client execution = false
Gateway integration = false
Persistent registry mutation = false
OPA policy execution = false
Responsible AI evaluation execution = false
OpenTelemetry export = false
Cloud deployment = false
Customer data access = false
Credential handling = false
Secret handling = false
Runtime mutation = false
Production enforcement claim = false