💡 An LLM answers. An agent acts. The difference is not a longer prompt — it is a control loop with tools, memory, a permission envelope, and a stop condition. Mix those wrong and you get a token furnace that can also DROP TABLE. Mix them right and you get boring, reversible work.

🗺️ The map

Read this as a curriculum, not a glossary. Five layers, each one only justified when the layer below is not enough:

1. 🎯 Definition — LLM vs workflow vs agent. Answers: who chooses the next step?
2. 🧩 Anatomy — model, planner, tools, memory, guardrails, orchestrator. Answers: what is the system besides the LLM?
3. 🔁 Patterns — ReAct, plan-and-execute, reflection, Anthropic’s five workflows. Answers: how does control flow?
4. 🔧 Integration — function calling, MCP, multi-agent, skills. Answers: how does it touch the world?
5. 🏭 Production — memory, LLMOps, HITL (or HIL), prompt injection, evals. Answers: how do you ship it without a pager?

flowchart TB
  subgraph LADDER["🪜 Complexity ladder"]
    direction TB
    A["1️⃣ 💬 Single LLM call"]
    B["2️⃣ 📚 RAG / in-context"]
    C["3️⃣ 🧱 Workflow
code chooses the path"]
    D["4️⃣ 🤖 Agent
model chooses the path"]
    E["5️⃣ 👥 Multi-agent
roles + handoffs"]
  end
  A --> B --> C --> D --> E
            

🎯 What is Agentic AI?

Classic AI already had the word. Russell & Norvig treat an agent as something that perceives and acts toward a performance measure [7]. PEAS — Performance, Environment, Actuators, Sensors — is still the right checklist. LLM-era “agentic AI” is that idea with a language model as the policy: the model interprets a goal, selects actions (usually tool calls), observes results, and continues until a stop condition.

The useful distinction is not “chatbot vs agent.” It is who owns control flow [1]:

• 💬 Traditional LLM generation — prompt → response. One shot, maybe a few turns of chat. No tools, no state machine, no side effects.
• 🧱 Workflow — LLMs and tools orchestrated through predefined code paths. You wrote the graph. The model fills in nodes. Predictable, cheap to debug.
• 🤖 Agent — the LLM dynamically directs its own process and tool use. Runtime observations change what happens next. Flexible, expensive, compounding-error-prone.

flowchart LR
  subgraph SHOT["💬 Single-shot LLM"]
    P1["📝 Prompt"] --> M1["🧠 Model"] --> R1["📤 Response"]
  end
  subgraph LOOP["🤖 Agentic loop"]
    G["🎯 Goal"] --> PL["🗺️ Plan"]
    PL --> ACT["🛠️ Action / tool"]
    ACT --> OBS["👁️ Observation"]
    OBS --> ST["📦 Updated state"]
    ST --> DEC{"✅ Done?"}
    DEC -->|no| PL
    DEC -->|yes| OUT["🏁 Result"]
  end
            

A system becomes meaningfully agentic when all of these are true:

• 🎯 A persistent goal, not only a response request
• 📦 State across multiple steps
• 🛠️ Dynamic selection of tools or actions
• 👁️ Feedback from an environment (API, codebase, browser, user)
• 🔄 Replanning after failures or new information
• 🛑 Explicit completion and stopping criteria
• 🛡️ Bounded autonomy — a permission envelope, not “do whatever”

A fixed chain of three predetermined LLM calls is a workflow, not an agent. The key test: do runtime observations change what the system decides to do next?

🧩 Anatomy of an agent

The LLM is not the agent. The agent is a system that uses an LLM as a policy. Strip the other parts and you have an eloquent intern with root access.

❌ When an agent is the wrong tool

An agent is usually the wrong choice when the process is predictable, rules-based, or high-risk without room for runtime judgment. Prefer a conventional workflow (or just software) when:

• 📋 Every step can be defined in advance
• 🎯 Deterministic output is required
• 🔌 A simple API, SQL query, or rules engine solves it
• ⚖️ Errors have severe legal, financial, or safety consequences
• ⏱️ Latency or cost budgets are extremely tight
• 🧪 There is no reliable way to verify success
• 🔓 The agent would have broad permissions but limited supervision

Rule: deterministic software for what can be specified; agentic decision-making only for the uncertain portion. Anthropic’s production note is the same: many apps only need a well-prompted single call with retrieval [1].

🛡️ Autonomy as a permission envelope

Autonomy is not a vibe in the system prompt. It is a permission envelope enforced outside the model. For every tool and action, specify:

• 📂 Which resources the agent can access
• 🔒 Read-only vs mutating operations
• 🏢 Data and tenant boundaries
• 💸 Transaction and spending limits
• 🙋 Actions that require human approval
• 🛑 Max steps, retries, runtime, and token budget
• 🚫 Prohibited actions
• 🚨 Escalation and shutdown conditions

flowchart TB
  USER["👤 User / goal"] --> ORCH["🎛️ Orchestrator"]
  ORCH --> LLM["🧠 LLM
untrusted proposer"]
  LLM -->|"function call request"| POL["🛡️ Policy engine
schema + auth + limits"]
  POL -->|allow| TOOL["🛠️ Tool / sandbox"]
  POL -->|deny / HITL| HUM["🙋 Human"]
  TOOL --> OBS["👁️ Observation"]
  HUM --> OBS
  OBS --> ORCH
            

Enforce this with authorization services, sandboxing, typed tool schemas, policy engines, database roles, and approval workflows. Prompt instructions are not a security boundary. If the model generates DROP TABLE, its database identity should lack permission to execute it.

🔁 Control-loop patterns

Three loops cover most “real” agents. Workflows (next section) cover the cases where you should not let the model own the path at all.

🔄 ReAct — reason, act, observe

ReAct (Yao et al., ICLR 2023) [2] interleaves verbal reasoning traces with actions so the model can reason to act and act to reason. The loop:

1. 🧠 Interpret the current goal and evidence (Thought)
2. 🛠️ Select an action or tool (Action)
3. ⚙️ Execute it
4. 👁️ Observe the result (Observation)
5. 🔁 Update the approach and repeat

flowchart LR
  T["💭 Thought"] --> A["🛠️ Action"]
  A --> O["👁️ Observation"]
  O --> T
            

WHERE. The correct next step depends on external information — support tickets, docs search, codebase inspection, browser tasks. Example: inspect an order → observe it already shipped → consult refund policy → propose the right resolution.

TRADE-OFFS. High latency, token use, error accumulation, runaway loops. In production, keep reasoning internal; log actions, observations, and decisions as structured traces. Cap turns. Prefer structured tool arguments over free-form “thought” leaking into the UI.

🗺️ Plan-and-Execute

Separate strategy from tactics. A planner creates a list or graph of subtasks. Executors complete them. The planner may revise based on results.

flowchart TB
  GOAL["🎯 Goal"] --> PLANNER["🗺️ Planner"]
  PLANNER --> GRAPH["📋 Task graph"]
  GRAPH --> E1["⚙️ Executor A"]
  GRAPH --> E2["⚙️ Executor B"]
  E1 --> RES["📦 Results"]
  E2 --> RES
  RES --> CHECK{"🧪 Still valid?"}
  CHECK -->|yes| DONE["🏁 Combine"]
  CHECK -->|stale| PLANNER
            

• ✅ Better organization for long tasks
• ✅ Easier parallelization and progress tracking
• ✅ Specialized models for plan vs execute
• ✅ Per-step retry and approval
• ⚠️ Original plan goes stale
• ⚠️ Planning adds latency and tokens
• ⚠️ A bad plan poisons many downstream tasks
• ⚠️ Overkill for simple tasks

Strong implementations support incremental replanning — patch the graph, do not regenerate it from scratch after every observation.

🪞 Reflection / self-correction

After a candidate answer or action, an evaluator scores it. The agent revises. Reflexion [4] stores those verbal critiques in an episodic buffer so later trials improve without weight updates — linguistic RL.

flowchart TB
  GEN["✍️ Generate candidate"] --> EV["🧪 Evaluate vs rubric"]
  EV --> OK{"✅ Pass?"}
  OK -->|yes| STOP["🏁 Accept"]
  OK -->|no, under N| FIX["🪞 Critique + revise"]
  FIX --> GEN
  OK -->|budget hit| FAIL["🚨 Escalate / best-so-far"]
            

Reflection works when grounded in objective signals: tests, schema validation, policy rules, compilers, citations. Asking the same model to “check itself” with no external signal often reinforces the original mistake and burns tokens. Cap attempts. Prefer a cheaper/different evaluator than the generator when you can.

🪓 Goal decomposition

Turn a wish into a task spec before you turn it into steps:

• 🏁 Desired final state
• 🚧 Constraints and permissions
• 📥 Required inputs
• 🛠️ Available tools
• 🔗 Dependencies
• ✅ Success criteria
• 🚨 Failure and escalation conditions

Subtasks should map to a tool or a verifiable operation — “retrieve account status,” “validate schema” — not “investigate the problem.” Represent complex work as a dependency graph, not only a linear checklist. After each step, update state and drop steps that are no longer necessary.

🌳 Chain vs tree vs graph

How the model (or orchestrator) explores the plan:

flowchart LR
  subgraph COT["⛓️ Chain"]
    C1["A"] --> C2["B"] --> C3["C"]
  end
  subgraph TOT["🌳 Tree"]
    T0["Start"] --> T1["A"]
    T0 --> T2["B"]
    T1 --> T3["A1"]
    T1 --> T4["A2"]
  end
  subgraph GP["🕸️ Graph"]
    G1["Parse"] --> G2["Plan"]
    G1 --> G3["Retrieve"]
    G2 --> G4["Edit"]
    G3 --> G4
    G4 --> G2
  end
            

Tree and graph search improve quality and explode cost. Use the simplest representation that meets the reliability bar. Store structured plans; do not stream raw private reasoning to users.

🧱 Workflow patterns — stay on the ladder

Anthropic’s production catalog [1] is five composable workflows built on an augmented LLM (retrieval + tools + memory). These are not agents. Code still owns the path. Use them until you genuinely cannot hardcode the next step.

⛓️ Prompt chaining

Decompose into a fixed sequence. Output of call n is input to n+1. Put a programmatic gate (schema, regex, cheap classifier) between steps so a bad intermediate cannot poison the rest.

flowchart LR
  IN["📥 Input"] --> L1["🧠 Step 1"]
  L1 --> GATE{"🚪 Gate"}
  GATE -->|pass| L2["🧠 Step 2"]
  GATE -->|fail| STOP["🛑 Halt"]
  L2 --> OUT["📤 Output"]
            

WHERE. Outline → (check outline) → draft. Marketing copy → translate. Each call is an easier task; you trade latency for accuracy.

🚏 Routing

Classify the input, dispatch to a specialized prompt / tool set / model. Optimizing one category no longer hurts the others.

flowchart TB
  IN["📥 Request"] --> R["🧭 Router"]
  R --> B["💬 Billing worker"]
  R --> T["🔧 Tech worker"]
  R --> G["🤝 General worker"]
            

WHERE. Support intents. Easy questions → small model; hard ones → large model. A router is a classifier; a manager (later) also plans and aggregates.

⚡ Parallelization

Two flavors:

• ✂️ Sectioning — independent subtasks at the same time (speed + focused attention)
• 🗳️ Voting — same task, many attempts, aggregate (confidence)

flowchart TB
  IN["📥 Task"] --> W1["🧠 Worker 1"]
  IN --> W2["🧠 Worker 2"]
  IN --> W3["🧠 Worker 3"]
  W1 --> AGG["🧩 Aggregate / vote"]
  W2 --> AGG
  W3 --> AGG
  AGG --> OUT["📤 Result"]
            

WHERE. Guardrail model in parallel with the answer model. Vulnerability review with several prompts. Evals that score different dimensions separately.

👷 Orchestrator–workers

A central LLM dynamically breaks the task, delegates, synthesizes. Topologically similar to parallelization; the difference is flexibility — subtasks are not hardcoded [1].

flowchart TB
  GOAL["🎯 Goal"] --> ORCH["👷 Orchestrator"]
  ORCH --> W1["⚙️ Worker A"]
  ORCH --> W2["⚙️ Worker B"]
  ORCH --> W3["⚙️ Worker C"]
  W1 --> SYN["🧩 Synthesize"]
  W2 --> SYN
  W3 --> SYN
            

WHERE. Multi-file code edits. Research across unknown sources. This is the closest workflow to a true agent — still wrap it in budgets and structured returns.

🪞 Evaluator–optimizer

Generator writes; evaluator critiques; loop until the rubric passes or the budget dies. Same shape as Reflection, but the path is a workflow you designed, not an open-ended ReAct wander.

Two signs of fit [1]: (1) human feedback would demonstrably improve the output; (2) an LLM can provide that feedback. Literary translation, iterative search, code that has tests.

🔧 Tool use, MCP, and trust

Treat the model’s function call as an untrusted request, not an authorized command.

Also: allowlists, idempotency keys, timeouts, rate limits, output-size caps, and protection against SSRF, SQL injection, path traversal, and command injection. Expose high-level parameterized tools (“refund order”) rather than arbitrary shells or SQL.

🔌 Model Context Protocol

MCP [5] is an open JSON-RPC protocol for connecting LLM apps to external context. Three roles:

• 🏠 Host — the app the user sees (IDE, desktop, your agent)
• 📎 Client — connector inside the host, 1:1 with a server
• 🖥️ Server — exposes capabilities

Servers offer:

• 🛠️ Tools — model-controlled functions with JSON Schema (side effects allowed)
• 📄 Resources — application-controlled, mostly read-only context (files, schemas, docs)
• 📝 Prompts — user-controlled templates for a domain workflow

flowchart LR
  U["👤 User"] --> HOST["🏠 Host / agent"]
  HOST --> C1["📎 Client"]
  HOST --> C2["📎 Client"]
  C1 --> S1["🖥️ MCP server
tools + resources"]
  C2 --> S2["🖥️ MCP server
prompts + data"]
            

MCP standardizes discovery and message exchange. It does not make tools safe. Auth, consent, sandboxing, and trust policy still live in the host and server. A random MCP server is a supply-chain decision.

🕸️ Frameworks and state

Frameworks (LangChain, LangGraph, CrewAI, AutoGen/AG2, OpenAI Agents SDK, Claude Agent SDK) are accelerators. They are also abstraction traps [1]. Start with the API. If you use a framework, you should be able to draw the underlying calls.

• 🔗 LangChain — models, prompts, tools, retrievers, message history. Fine for pipelines. Do not hide business state, permissions, or checkpoints inside a memory abstraction you cannot inspect.
• 🕸️ LangGraph [9] — stateful graph (Pregel-style), not a DAG. Nodes read and update a typed state. Edges can be conditional. Cycles are the point: ReAct, retries, HITL pauses. Checkpointers snapshot state each super-step → resume, time-travel, crash recovery. Requires recursion limits, reducers, and terminal states.
• 👥 CrewAI / AutoGen — role-playing multi-agent: sequential handoffs, manager delegation, group chat. Benefit is specialization. Risks: duplicated work, context loss, cost, undebuggable transcripts.
• 📦 OpenAI Agents SDK — hosted loop with handoffs, guardrails, tracing. Convenient on-rails; still version the whole config, not just the model.

flowchart LR
  subgraph DAG["➡️ DAG / chain"]
    D1["A"] --> D2["B"] --> D3["C"]
  end
  subgraph LG["🔄 Stateful graph"]
    L1["Reason"] --> L2["Tool"]
    L2 --> L3["Update state"]
    L3 --> L1
    L3 --> L4["HITL"]
    L4 --> L1
  end
            

A DAG assumes an acyclic, predetermined sequence. An agent graph revisits nodes based on updated state. That flexibility is why you need explicit stop conditions — an agent that can always return “needs more investigation” will.

👥 Multi-agent systems

Add a second agent only when roles are clearly separable or work is independently parallelizable. Otherwise you bought a meeting.

👔 Manager–worker vs 🚏 router–worker

• 🚏 Router — classify, send to one specialist. Light.
• 👔 Manager — plan, delegate, dependency management, retries, aggregation, conflict resolution. Heavy.

Workers should receive minimal, task-specific context and return structured payloads: result, evidence, uncertainty, errors, recommended next action. The manager validates workers; it does not automatically trust them.

flowchart TB
  U["👤 Goal"] --> M["👔 Manager"]
  M --> W1["🔬 Research"]
  M --> W2["💻 Coder"]
  M --> W3["🧪 Reviewer"]
  W1 -->|"structured result"| M
  W2 -->|"structured result"| M
  W3 -->|"structured result"| M
  M --> F["🏁 Final"]
            

📨 Safe handoffs

Prefer structured messages over dumping the full transcript. A handoff should contain:

• 🆔 Task ID and parent goal
• 🎯 Exact subtask
• 📌 Necessary facts only
• 🛠️ Allowed tools and resources
• 🏷️ Data sensitivity labels
• 📐 Expected output schema
• ⏱️ Deadline / resource budget
• ✅ Completion criteria

Redact secrets; pass access-controlled IDs. Authorize each receiving agent independently. Version shared state with provenance — which agent produced this fact, from which source.

♾️ Loops and circular handoffs

Enforce at the orchestrator, not in the prompt:

• 🔢 Max turns and handoff depth
• 🔁 Per-agent retry limits
• ⏱️ Global time and token budgets
• 👣 Visited-state / handoff history
• 🧬 Duplicate-task detection (hash normalized task + state)
• 📈 Progress metrics that must move
• 🏁 Explicit terminal states
• 🙋 Escalate after repeated failure

🎒 Skills

A skill is a packaged capability: instructions, tool defs, examples, validators, domain knowledge. Do not stuff every tool into the system prompt. Load on demand:

1. 🧭 Classify intent
2. 🔎 Search a skill registry (metadata / embeddings)
3. 🔐 Filter by user and tenant permissions
4. 📥 Load instructions + tools
5. 📦 Execute in a restricted environment
6. ✅ Validate output
7. 📜 Record skill version in the trace

Skills should be trusted/signed, versioned, tested, and permission-scoped. A skill file is prompt injection with a README if you load it from the open internet.

🧪 “Deterministic” multi-agent tests

The LLM is probabilistic. Make the contract around it deterministic:

• 📐 Fixed state schemas and tool interfaces
• 🌡️ Low temperature where routing matters
• 📌 Pinned model, prompt, tool, and skill versions
• 🎭 Mocked tools and replayable fixtures
• 🌱 Seeded execution where the API allows
• 🛑 Explicit routing and stopping policies
• 🥇 Golden scenarios + property / invariant tests
• 📜 Trace comparison and repeated-run stability

Measure separately: task correctness, route consistency, tool selection, argument accuracy, steps, cost, latency, policy compliance. Goal is controlled variability, not pretending the whole system is a pure function.

🧠 Memory, state, and context

Two memories, one trap:

• 🧾 Working memory — this run: goal, recent messages, plan, tool results, pending calls. Lives in the prompt and/or a workflow state store. Precise and temporary.
• 📚 Long-term semantic memory — cross-session facts, preferences, prior decisions. External store, retrieved when relevant. Selectively written, permission-aware, provenance-tracked, revisable.

flowchart TB
  RUN["▶️ Agent run"] --> WM["🧾 Working memory
prompt + state store"]
  RUN --> RET["🔎 Retrieve"]
  LTM["📚 Long-term store
profile / vectors / facts"] --> RET
  RET --> WM
  RUN --> WRITE["✍️ Validated write"]
  WRITE --> LTM
            

🧮 Context budget

Assign tokens to buckets before the model sees anything:

• 📜 System and policy (pinned — compaction cannot drop these)
• 🎯 Current user request
• 💬 Recent interactions
• 🗺️ Active plan and state
• 📚 Retrieved evidence
• 🛠️ Tool results
• 📤 Reserved output space

When you approach the limit: drop redundant tool output, replace older turns with structured summaries, persist large artifacts externally and pass references. Limit retrieval count and chunk size before data hits the model.

🗜️ Compaction and chunking

Chunk by semantic unit (turns, topics, tasks, documents, time), not arbitrary character counts. Overlap a little so references survive boundaries.

Compaction replaces old raw content with a structured summary: confirmed facts, preferences, decisions + rationale, open questions, completed actions, identifiers, source refs. Hierarchical: recent turns stay detailed; older sessions become task summaries; ancient history becomes durable facts. Keep links to originals when auditability matters. Compaction is lossy on purpose — be honest about it.

🐘 The agent that never forgets

Permanent memory is not a feature; it is a liability portfolio:

• 🔐 Sensitive or stale data retention
• 🚪 Cross-user / cross-tenant leakage
• ⚖️ Privacy and deletion compliance (right to be forgotten)
• ☠️ Memory poisoning via malicious inputs
• 📉 Irrelevant retrieval, slower vector search
• 💸 Larger prompts, higher cost
• 🧊 Personalization on outdated facts

Need: retention policies, user controls, tenant isolation, encryption, access logs, expiration, deletion, relevance thresholds. More memory ≠ better agent.

⚔️ Retrieval conflicts

Rank by recency, source authority, confidence, and scope. Current explicit user instructions normally override inferred historical preferences. Verified records may override an unsupported claim when the fact matters. Keep value and provenance — do not silently overwrite. For high-impact clashes, ask:

“Your current request says X; the saved config says Y. Which should I use?”

Memory is evidence, not truth.

👤 Safe personalization across stateless sessions

Keep the model call stateless. Store the profile externally, user-scoped. Inject only relevant authorized fields. Controls: strong identity, consent, field-level ACL, encryption, minimization, expiration, user review + deletion, provenance timestamps, separation of preferences vs sensitive records. The model must not decide which user’s memory to load.

🦠 Memorization of undesirable behavior

If a bad action, unsafe workaround, or injected “policy” gets written to memory, retrieval will teach the next run to repeat it. Example: a user convinces the agent that approval is unnecessary; that sentence is stored as procedure; later sessions skip HITL.

Mitigations: never learn security rules from conversation; separate user facts from immutable system policy; validate memory writes; trust + provenance scores; expire low-confidence memories; review high-impact procedural memories.

💾 State store and vectors

Durable transactional store as source of truth. Each run is a state machine: run ID, version, current node, completed / pending actions, tool results, approvals, retries, timestamps. Want: atomic transitions, optimistic concurrency, idempotency keys, checkpoint after material steps, replication, queues with DLQ, worker leases, recover from latest checkpoint, audit log. Side effects via outbox so state and the real world cannot diverge.

For vectors: stable ID + version + timestamp + source on every record. Write the primary row first; embed asynchronously. Readers filter by active version. Dedup with content hashes and novelty thresholds — long-running agents otherwise embed the same observation forever, inflating index size, latency, and prompt tokens.

🏭 LLMOps for agents

Classic MLOps versions datasets, training jobs, and model artifacts [10]. LLMOps versions the behavior-producing system:

• 🧠 Foundation model + provider version
• 📜 System prompt
• 🛠️ Tools and schemas
• 📚 Retrieval / memory config
• 🕸️ Agent graph
• 🛡️ Guardrails and evaluators
• 📖 Knowledge sources

The weights can sit still while a tool description change wrecks production. Treat the whole bundle as one release manifest.

🙋 Human-in-the-loop

Pause before irreversible, expensive, sensitive, or low-confidence actions: external messages, payments, deletes, infra changes, regulated decisions.

flowchart LR
  A["🛠️ Proposed action"] --> CP["💾 Checkpoint"]
  CP --> UI["🙋 Approval UI"]
  UI -->|approve, unexpired| X["▶️ Execute"]
  UI -->|deny| ALT["↩️ Replan"]
  UI -->|state drifted| RE["🔄 Re-request"]
            

The screen shows: proposed action, exact target, arguments, evidence, expected impact, risk, alternatives. Bind the approval to the exact action; expire it if state changed so an old yes cannot authorize a mutated call.

☠️ Prompt injection

OWASP still ranks prompt injection as LLM01 [6]. Retrieved pages, emails, PDFs, images, and tool outputs may contain hostile instructions. For agents this is not a parlor trick — it is confused-deputy with tools.

• 🧱 Separate trusted instructions from untrusted content; label origin and trust
• 🚫 Never treat retrieved text as system policy
• 🛠️ Least-privilege tools for the current task
• 🛡️ External policy checks on every action
• 🙋 HITL for high-risk ops
• 🔐 Do not put secrets in the prompt
• 🌐 Restrict network and filesystem
• 🧹 Sanitize tool output; test with adversarial content

Content can inform an answer. Content cannot grant itself permissions.

💣 Destructive actions

Defense in depth, not a polite system prompt:

• 👀 Read-only DB credentials by default
• 🧩 Parameterized tools, not arbitrary SQL
• ✅ Allowlisted statements and tables; AST-parse SQL; reject DDL
• 🙋 HITL for authorized destruction
• 🧪 Transactions, dry-run previews, row and time limits
• 💾 Backups and audit logs

💸 Cost of the loop

Optimize at the workflow, measure cost per successful task (a cheap model that retries forever is expensive):

• 🧭 Route simple work to small models; large models for hard planning / validation
• 🗜️ Compact context; retrieve fewer, better chunks
• 💾 Cache stable prompts, embeddings, tool results
• ⚡ Parallelize independent steps
• 📐 Structured tool outputs (less prose to re-ingest)
• 🛑 Stop when marginal gain is flat; cap reflection
• 🧱 Replace LLM decisions with rules where you can

📡 Tracing

One trace ID per user request. Every model call, retrieval, tool call, handoff, state transition, approval, and error is a child span (LangSmith, OpenTelemetry, your tracer). Record: model/prompt versions, sanitized I/O, tool name + validated args, latency, tokens, cost, retrieved IDs, routing decisions, retries, policy decisions, outcome, user feedback. Redact before log. Dashboards: success, latency, cost, loop rate, tool failures, escalations, policy violations.

🛡️ Layered guardrails

flowchart TB
  IN["📥 Input checks"] --> PLAN["🗺️ Plan checks"]
  PLAN --> TOOL["🛠️ Tool checks"]
  TOOL --> OUT["📤 Output checks"]
  OUT --> POST["🔍 Post-action checks"]
  POST --> RUN["⏱️ Runtime
budgets / kill switch"]
            

Prefer deterministic validators when the rule is code-shaped. Model-based judges are for semantic quality — calibrate them, version them, and never let a judge be the only lock on a high-risk action.

📺 Streaming UI

Separate user-visible events from internal tokens. Publish structured events: accepted, planning, searching, tool running, waiting for approval, partial answer, retrying, done/failed. Stream safe response text; show concise tool status. Do not stream private reasoning or unvalidated tool dumps. Support cancel, reconnectable streams, ordered event IDs, backpressure, resumable state.

📦 Versioning, rollback, A/B

Ship a manifest, canary it, keep the previous bundle deployable. Pin in-flight runs to the version they started on. For A/B: assign users or stable sessions so a conversation does not switch mid-thread; change one major factor at a time; pre-register metrics (completion, accuracy, corrections, tool success, steps, latency, cost per success, escalations, safety, satisfaction). Stratify by task difficulty — a win on easy tickets can hide a regression on hard ones. Sequence: replay → shadow → limited live → rollback thresholds. Same traffic-strategy instincts as model serving [10].

🪙 My 2 cents

Stop asking “should we use agents?” Ask the questions that force an architecture:

• 📋 Can I write the steps as a DAG today?
• 💥 What is the blast radius of a wrong tool call?
• 🧪 How do I know the task is done — test, schema, human?
• 💸 What is the token budget per successful outcome?
• 🙋 Who approves irreversible actions?

Answer those and the pattern almost picks itself:

• 💬 Single call + RAG when the path is a paragraph
• 🧱 Workflow when the path is a recipe
• 🤖 Agent when the path is unknown and the environment gives ground truth
• 👥 Multi-agent only when roles are separable
• 🛡️ Permission envelope outside the model — always

Contracts that are non-optional:

• 🎛️ Orchestrator-enforced budgets, not prompt manners
• 🎒 Skills over god-prompts
• 📜 Memory is evidence, never policy
• 📦 Version the whole agent config, not the weights alone
• 📡 Traces you can replay; evals you can fail a release on

🌱 Treat agentic AI as something you design — a control loop with a kill switch — not a prompt you hope stays on the rails.

📚 References

1. Erik S. and Barry Zhang, “Building effective agents,” Anthropic Engineering, anthropic.com/engineering/building-effective-agents
2. Shunyu Yao et al., “ReAct: Synergizing Reasoning and Acting in Language Models,” ICLR 2023, arxiv.org/abs/2210.03629
3. Shunyu Yao et al., “Tree of Thoughts: Deliberate Problem Solving with Large Language Models,” NeurIPS 2023, arxiv.org/abs/2305.10601
4. Noah Shinn et al., “Reflexion: Language Agents with Verbal Reinforcement Learning,” NeurIPS 2023, arxiv.org/abs/2303.11366
5. Model Context Protocol, specification, modelcontextprotocol.io
6. OWASP GenAI Security Project, “LLM01: Prompt Injection,” genai.owasp.org/llmrisk/llm01-prompt-injection
7. Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach, aima.cs.berkeley.edu
8. Jason Wei et al., “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models,” NeurIPS 2022, arxiv.org/abs/2201.11903
9. LangChain, “LangGraph checkpointers,” docs.langchain.com/…/langgraph/checkpointers
10. Amirhessam Tahmassebi, “MLOps Deployment Strategies,” 2-Cents, two-cents/mlops-deployment-strategies.html
11. LangChain, LangSmith tracing, docs.smith.langchain.com
12. OpenAI, Agents SDK, openai.github.io/openai-agents-python
13. CrewAI, documentation, docs.crewai.com
14. AG2 (AutoGen), ag2.ai
15. OWASP, “Top 10 for Large Language Model Applications,” owasp.org/www-project-top-10-for-large-language-model-applications