From Prompts to Autonomous Teams: The Four Architecture Patterns of Agentic AI

Agentic AI didn’t arrive as a single breakthrough. It emerged as a sequence of architectural patterns, each one built to overcome the limitations of the one before it. Understanding that progression — from a stateless language model answering one prompt at a time, to orchestrated teams of specialized agents with persistent memory — is the clearest way to understand where the technology stands today and how to build with it.

This post walks through four architecture patterns, then assembles them into a complete anatomy of a production AI agent, and closes with a deep look at the component that turns an agent from a clever tool into a compounding asset: persistent memory.

Pattern 1: Stateless Inference

The starting point is the pattern everyone knows: a large language model receives a prompt and produces a completion, with nothing persisting between interactions. The model reasons probabilistically over patterns learned during pre-training. Each interaction is fully independent — no memory of prior sessions, no connectivity to external systems, no control flow beyond a single input-output cycle.

Two mechanisms make this simple architecture surprisingly capable.

In-context learning is the model’s ability to infer the intended task from examples in the prompt itself, without any parameter updates or fine-tuning. The same model — same weights — handles translation, sentiment classification, summarization, or code generation based entirely on how the prompt is structured. Give it no examples and just an instruction (“Translate to French: Hello, how are you?”) and it performs zero-shot. Give it one worked example (“I love this product → positive”) and it performs one-shot. Give it several and it picks up more complex patterns few-shot. This is purely inference-time adaptation, and it eliminated the need to train a separate model for every task.

Chain-of-Thought reasoning activates multi-step logical decomposition by instructing the model to generate intermediate reasoning steps before committing to an answer. Ask a model directly “A store has 45 apples, sells 18, receives 30 more — how many now?” and it may skip the arithmetic and produce a wrong answer. Ask it to think step by step, and it writes out 45 − 18 = 27, then 27 + 30 = 57 — an explicit, verifiable reasoning trace. The Wei et al. finding in early 2022 was that this simple prompting change dramatically improved accuracy on arithmetic, commonsense, and symbolic reasoning tasks.

Beyond these basics, a broader toolkit of prompting styles emerged for shaping model behavior: self-consistency (generate multiple reasoning paths, take the majority answer), tree-of-thought (explore branching reasoning paths, evaluate and prune before committing), reflection/self-critique (the model reviews its own output for errors and revises), role/persona assignment, structured output (forcing JSON, XML, or tabular formats for machine-parsable responses), retrieval-augmented prompting, directional stimulus (a steering hint without giving away the answer), generated knowledge (produce background facts first, then answer using them), decomposition (break a hard problem into sub-problems and solve easiest-first), and contrastive prompting (show both correct and incorrect examples to teach the model what to avoid).

What can you actually do with Pattern 1? Quite a lot: summarize 200-page supplier quality submissions into executive briefs, generate SQL from plain-English questions like “which paint booth had the highest defect rate last month?”, extract structured bill-of-materials changes from engineering change notice PDFs, draft code across languages, translate and reformat unstructured text.

But the limitations are structural, not incidental. No memory — every interaction starts from zero. No live data access — the model is confined to its training data and whatever is in the prompt. Hallucination — it generates plausible but fabricated answers when it has no grounding. No agency — it responds when prompted but cannot initiate action or plan a workflow. Each of these gaps directly motivates the next pattern.

Pattern 2: Tool-Augmented Generation

The second pattern gives the model reach. Tool-Augmented Generation extends the inference architecture so the model can invoke external tools — APIs, databases, code interpreters, third-party services — during generation. The model remains the central reasoning component, but it can now step outside its training data to retrieve live information, execute computations, and trigger actions in external systems. Critically, control flow is still human-initiated: a person poses a query, the model determines which tool to call, a result comes back, and the session concludes.

Two mechanisms define this era.

Function calling lets the model invoke developer-defined tools by generating structured JSON that matches a predefined function schema. The flow has six steps: a user asks a natural-language question (“What’s the weather in Singapore?”); the LLM decides a function call is appropriate and selects one (get_weather); it generates JSON arguments ({“location”: “Singapore”}); the system executes the function against the real API; the result ({“temp”: 31, “condition”: “humid”}) returns to the model; and the model folds it into a natural-language answer. The key subtlety: the model generates the arguments but never executes anything itself — the surrounding system handles execution.

Retrieval-Augmented Generation (RAG) grounds model responses in enterprise data. It operates in two phases. In the indexing phase, done in advance, documents are split into chunks, converted into vector embeddings, and stored in a vector database. In the retrieval phase, at query time, the user’s question is itself embedded, semantically similar chunks are retrieved, and those chunks are injected into the context window alongside the prompt so the model generates an answer grounded in actual organizational data. The payoff is proprietary-data grounding without any model retraining.

In practice, Pattern 2 enables things like natural-language-to-SQL against a manufacturing execution system (“show me SPC trends for the torque spec on Drive Unit Line 3 this week”), RAG over quality manuals so a technician gets a grounded answer about weld inspection criteria, or a function call that pulls real-time OEE from a plant historian API and computes availability by shift. Early autonomous experiments like AutoGPT and BabyAGI belong to this era too — you could set a goal and watch the AI attempt to execute it, which proved the concept while exposing how unreliable it still was.

The limitations again point forward. The N×M integration problem: every model-tool connection is bespoke and brittle — 10 applications times 100 tools means 1,000 custom integrations. The system is human-dependent — the AI has reach but no initiative; it executes only when asked. Tool chaining stays shallow — multi-step plans across tools tend to fall apart. And there is still no persistent learning — each session is stateless, with no recall of what worked before.

Pattern 3: Self-Directed Execution

The third pattern is the pivotal shift: control flow transfers from the human to the model. Instead of responding to a single query, the agent receives a goal and drives its own execution cycle — decomposing the goal into sub-tasks, selecting and invoking tools, observing results, and adjusting its plan iteratively. The mental model changes from “AI as a tool you use” to “AI as a worker you delegate to.”

Three converging capabilities made this practical.

The ReAct loop is the DNA of every modern agent. First described by Yao et al. in October 2022, ReAct (Reason + Act) is a persistent cycle: the model reasons about what to do next (“I need to check supplier batch data”), acts by invoking a tool (query the procurement database), observes the result (“Vendor X, batch #4471, 15-day lead time”), reflects on progress (“that matches the cost-spike timeline — check quality data next”), and repeats until the goal is achieved. The model, not a pre-defined workflow, determines the next step at every iteration.

Around ReAct, a catalog of agentic design patterns has been documented in the research literature, each an architectural blueprint for how agents reason, plan, act, and self-correct: Reflection (critique your own output against quality criteria before delivering), Reflexion (store verbal “lessons learned” in memory across episodes), ReWOO (plan all steps upfront with variable placeholders and execute in one pass — faster and cheaper on predictable tasks), CodeAct (write and execute code as the reasoning mechanism instead of natural language), LATS (tree search over action paths with real-time tool feedback and backtracking), Tree-of-Thought, Plan-and-Solve (separate the planner from the executor and re-plan on failure), Self-Refine (generate, self-critique, iterate until a quality threshold is met), Self-Discovery (assemble a custom reasoning structure per task from a module library), STORM (multiple simulated perspectives research independently, then synthesize), and general Tool Use. Production systems typically combine several of these, and frameworks like LangGraph, CrewAI, and AutoGen package them into reusable form.

The Model Context Protocol (MCP), released in November 2024, solved the N×M integration problem. MCP defines a universal client-server interface between agents and external systems, built on three primitives: Tools (executable functions), Resources (structured data), and Prompts (interaction templates). The before-and-after is stark: three apps connecting to four tools previously required twelve bespoke integrations; with MCP as the universal interface, it takes three client integrations plus four server integrations — seven total. N×M collapses to N+M. This is the architectural insight that made autonomous agents practical at scale.

Context engineering supplies the domain intelligence. It is the systematic curation of everything that enters the model’s context window at each step: system instructions (role, behavioral rules, output format), tool definitions (available MCP tools and their schemas), domain ontology (KPI definitions, business rules, process taxonomies, thresholds), retrieved data (RAG results, query outputs, API responses from the current task), and memory plus conversation history. The same model produces radically different results depending on how these layers are assembled — which makes context design, including deciding what to leave out, the primary determinant of agent quality.

What does Pattern 3 look like in operation? Root-cause analysis: ask “why did the LV Battery module cost spike 18%?” and the agent queries the cost database, traces to a supplier batch, and pulls yield data on its own. Anomaly investigation: an agent detects a thickness drift in a CVD process, pulls SPC data, and correlates it with gas-flow logs. Warranty analysis: “what’s driving the increase in HVAC compressor claims?” triggers cross-referencing of field data against production lots. Supplier scorecards get assembled autonomously from delivery, PPM, and cost-variance data across five systems.

And the remaining gaps? Siloed intelligence — a self-directed agent can’t share findings with other specialized agents. No persistent learning — the agent resets after each task, losing the procedural knowledge it just gained. Reliability gaps — complex plans fail mid-execution and recovery is often crude. And a single-agent ceiling — context window limits and coherence loss in long reasoning chains cap how much one agent can hold. Enter Pattern 4.

Pattern 4: Orchestrated Multi-Agent Systems

The fourth pattern coordinates multiple specialized agents under a central orchestrator. Each agent operates within a defined domain; the orchestrator delegates tasks, manages handoffs, maintains shared context, and synthesizes results — mirroring how complex organizations distribute expertise across specialized teams. Picture a hub-and-spoke: an Orchestrator at the center delegating to a Quality Agent, a Procurement Agent, an Engineering Agent, and a Reporting Agent, while managing a shared context store, a task queue, result synthesis, and handoff management.

Three pillars support this architecture:

Protocol convergence. MCP handles agent-to-tool connectivity (with well over 97 million downloads, it’s the dominant standard), while agent-to-agent (A2A) protocols enable capability discovery via Agent Cards and cross-boundary delegation — agents finding out what other agents can do and handing work to them. Governance of these standards is consolidating under the Linux Foundation, though A2A, ACP, and AGNTCY remain fragmented.

Persistent memory across sessions, in three tiers — episodic, semantic, procedural — covered in depth below.

Context engineering as a formal discipline, applied now across a team of agents rather than one.

The capability jump is qualitative. Multi-agent systems produce cross-domain orchestrated intelligence — unified insights no single agent could reach. A battery cost-intelligence scenario: a Cost Agent, a Procurement Agent, and an Engineering Agent jointly correlate a cell-chemistry change with a cost spike across three suppliers. A paint-shop optimization: a Defect Agent monitors booth data, an Environment Agent tracks humidity and temperature, and a Process Agent adjusts parameters — yielding a 30% scrap reduction. Launch readiness: Design, Manufacturing, and Supplier agents perform a cross-domain risk assessment before start of production. These systems can also run proactively and always-on, surfacing root-cause hypotheses before humans notice anomalies, and even self-heal data pipelines by detecting schema changes and adjusting automatically.

An honest assessment demands naming the risks, because they are substantial. Governance is the number-one gap — it tops cybersecurity trend lists for 2026, and only about 6% of organizations report an advanced AI security strategy. Agent-scale security is a new attack surface: prompt injection combined with broad tool access means attacks can propagate at machine speed. Analysts project a 40% cancellation risk for agentic AI projects, driven by escalating costs, unclear ROI, and pervasive “agent washing” — relabeling ordinary automation as agents. And the standards are still converging. Meanwhile, demand signals are extraordinary — Gartner reported a 1,445% surge in multi-agent system inquiries. This is the frontier, not a finished state.

The framework landscape

For teams building Pattern 4 systems, the orchestration framework landscape is fragmented but converging. LangGraph (graph-based state machines with nodes, edges, branching, and cycles) has the most mature ecosystem at 80K+ GitHub stars. CrewAI (role-based crews coordinated by a crew manager) dominates enterprise adoption, reportedly used in over 60% of the Fortune 500. AG2/AutoGen (conversational multi-agent with async message passing) suits research and prototyping; Microsoft merged AutoGen and Semantic Kernel into the Microsoft Agent Framework in October 2025 for Azure/.NET environments. OpenAI’s Agents SDK (which replaced Swarm in March 2025) is the fastest on-ramp for OpenAI-stack teams. Google ADK offers hierarchical agent composition with native A2A support. Amazon Bedrock Agents is fully managed with no orchestration code. LlamaIndex Agents is the choice for RAG-heavy workloads, Pydantic AI adds type-safe structured outputs, MetaGPT simulates software engineering teams, and Smolagents from HuggingFace is the minimal, code-driven option for rapid prototyping. The practical insight: production systems increasingly combine frameworks — for example, LangGraph for orchestration, LlamaIndex for memory and retrieval, CrewAI for role-based collaboration.

Anatomy of a Production Agent

Put the pieces together and a complete agent architecture has six layers wrapped around a persistent memory system.

Layer 1 — Context window composition. When a goal arrives, the context window is assembled fresh from five components: system instructions, tool definitions, domain ontology, retrieved data, and memory recall. This is context engineering in action, and memory is read here — relevant episodes, facts, and procedures are pulled into context before reasoning begins.

Layer 2 — The ReAct reasoning loop. Reason → Act → Observe → Reflect, repeating until the goal is achieved.

Layer 3 — MCP tool connectivity. SQL, APIs, code execution, BI systems — all reached through the standardized protocol rather than bespoke integrations.

Layer 4 — The reflection gate. Before anything is delivered, the agent validates its own output: Is every claim grounded in data? Does it align with domain constraints? Are there internal contradictions? Is the format correct? A failure here sends the agent back into the reasoning loop.

Layer 5 — Human-in-the-loop. The agent escalates rather than acts when confidence is low, the action is high-stakes, ambiguity can’t be resolved, or policy requires sign-off. Autonomy is bounded by design, not by accident.

Layer 6 — Output and feedback. The agent delivers a natural-language summary, structured data, or a triggered action — and captures corrections, which are written back to memory.

Traced end-to-end, an execution looks like this. The goal arrives: “Why did the LV Battery module cost spike 18%?” — a goal, not a query. The context window is assembled, including a memory read. The agent reasons about what it knows and what it still needs, acts by invoking the right MCP tool, and observes: “Cathode material costs spiked 22%. Supplier X changed lot on Feb 3.” It reflects — goal met? If not, loop back. Once the answer holds, the reflection gate self-critiques every claim against data and domain rules. A human checkpoint intervenes if the decision is critical. Finally, the output is delivered and the feedback — corrections, new facts, successful tool sequences — is written to memory.

The single most important architectural insight: memory is read at the start and written at the end. That read/write cycle is the learning loop that makes the agent smarter over time.

Why Agents Need Persistent Memory

Without memory, every interaction starts from zero. The agent repeats mistakes, re-discovers facts, and re-invents workflows no matter how many times it has solved the same problem. The fix draws directly on Endel Tulving’s taxonomy of human memory, adapted into three complementary tiers.

The best way to grasp the tiers is the new-employee analogy. Imagine someone joining a support team. Over time they build three distinct kinds of knowledge:

Episodic memory — what happened. “Last Tuesday, Sarah called about a billing error. Escalation took 3 days. A direct refund took 5 minutes and resolved it.” Specific events, outcomes, and timestamps. This tier stops mistakes from repeating. It’s built by extracting noteworthy episodes after each interaction — significant events only: failures, successes, edge cases — stored with timestamp, context, and outcome. It’s used by retrieving episodes similar to the current situation and injecting them as few-shot examples, so the agent learns from its own past.

Semantic memory — what we know. “Refund policy: up to $500 without manager approval. Sarah is on the Enterprise plan. Maintenance window: Sundays 2–4am.” Structured facts, rules, and entity attributes — true regardless of context. This tier grounds responses in facts. It’s built by pre-loading knowledge bases, policies, and catalogs, then accumulating new facts extracted from conversations, stored in a vector database or knowledge graph. It’s used by retrieving facts relevant to the current query and injecting them into context, personalizing responses with real entity data.

Procedural memory — how we do things. “For billing complaints: verify the charge → check known errors → direct refund if under $500 → escalate only above threshold.” Learned workflows, decision heuristics, and optimized sequences, encoded as evolving prompts. This tier develops operational expertise. It’s built through feedback-driven prompt evolution — successful episode sequences consolidate into templates, and system prompts are updated based on what works. Its usage differs fundamentally from the other two: it is not retrieved by similarity. It lives in the system prompt, always loaded, shaping behavior at the deepest level as the agent’s default mode of operation.

The tiers also feed each other: events consolidate into facts, facts inform skills, skills improve execution.

The three tiers working together

A concrete scenario shows the whole system in motion. A customer messages: “I was charged twice for my subscription this month.”

Procedural memory activates first — it’s always in the system prompt — so the agent immediately follows its learned sequence: verify the charge, check known errors, process a refund if applicable. Semantic memory is retrieved next: the customer is on the Pro plan at $49/month, refund policy allows up to $500, and there’s a known duplicate-charge issue from March 1. Episodic memory is retrieved by similarity: last week the same issue was resolved in five minutes via direct refund, while an escalation on a similar case took three days and produced a negative satisfaction score.

The result is a fast, confident, grounded response: “I can see you were affected by a known billing issue on March 1. I’ve processed a refund of $49 to your card — you should see it within 2–3 business days.”

Then the feedback loop closes. The customer confirms the refund worked, so the interaction is stored as a successful episode. The “March 1 duplicate charge” pattern is reinforced as a semantic fact. The “skip escalation for known billing glitches” sequence is validated and refined in the procedural template. Every tier gets smarter from a single interaction — this is the compounding learning effect.

The memory platform landscape

Persistent memory is an active research frontier (there’s a dedicated ICLR 2026 workshop on memory in agents), and the tooling reflects both maturity and gaps. Mem0 is the fastest path to production — managed memory-as-a-service with vector plus graph memory, an MCP server, SOC 2 and HIPAA compliance, and a 50K+ developer community. Letta (formerly MemGPT) takes an OS-inspired approach in which agents self-edit their own memory blocks across core, archival, and recall tiers. Zep/Graphiti is the most architecturally sophisticated: a temporal knowledge graph that tracks how facts change over time, scoring 94.8% on the DMR benchmark with sub-200ms retrieval. LangMem is LangChain’s native LangGraph SDK covering all three memory types. Around these sit vector and storage infrastructure — Qdrant, Redis Agent Memory (with semantic caching and eviction policies for memory decay), MongoDB with vector search and TTL indexes, and Postgres with pgvector for teams that want SQL-native governance — plus emerging players like Graphlit, Cognee, MemoClaw, and the open-source, local-first Memoripy.

The honest state of the landscape: there is a real tension between simple and sophisticated approaches (a plain filesystem-based Letta setup scores 74% on one long-context memory benchmark, while Zep’s temporal graph reaches 94.8% on another), and no single platform delivers a turnkey three-tier solution. Semantic memory is largely solved; episodic memory is partially supported; procedural memory remains the most custom piece. Production systems assemble their memory architecture from components.

Closing Thoughts

The through-line across all four patterns is that each generation’s limitations were the next generation’s design requirements. Stateless inference lacked reach, so tool augmentation added it. Tool augmentation lacked initiative and standard connectivity, so self-directed execution added the ReAct loop and MCP. Self-directed agents lacked collaboration and memory, so orchestrated multi-agent systems added A2A protocols and three-tier persistence.

The destination is an architecture with a clear shape: context engineered per step, a ReAct core, standardized tool connectivity, a self-critique gate, bounded human oversight, and a memory read/write cycle that compounds capability over time. The frameworks and memory platforms to build it exist today — imperfect, fragmented, converging fast. The governance, security, and ROI discipline to run it responsibly is the part most organizations still have to build. That, more than any model capability, is likely to separate the teams that ship durable agentic systems from the 40% whose projects get cancelled.