Agent Loop & Architectures
|
This content was generated with the assistance of AI and should be verified against the official documentation of each tool and framework before being relied on in production. This section’s bibliography lists the reference material consulted while preparing these pages. |
This page’s examples target Python 3.12 with LangChain 1.4.2 / langchain-core 1.6.5 / LangGraph 1.2.12 and
langchain-ollama 1.1.0, Java 21 with LangChain4j 1.20.2 (its langchain4j-agentic module at 1.20.2-beta30), and
Spring Boot 4.1 with Spring AI 2.0.1, all calling DocsAssistant's Ollama llama3.1:8b chat model. Versions were
checked against PyPI and Maven Central on 2026-09-28; the AI Agents landing page’s
baseline wins if a page and that baseline ever disagree.
This page is about how one agent step reasons: the loop a single model runs, and the handful of architectures that decide what goes into each turn of that loop. Composing several model calls into a fixed, mostly deterministic pipeline (chaining, routing, parallelisation, orchestrator-workers, evaluator-optimizer) is a different design question, covered in Workflow Patterns. Splitting work across several agents is covered in Multi-Agent Systems.
The agent loop
Every agent, whatever the framework, runs the same four-step loop until it decides it is done or a limit stops it:
-
Perceive — read the input: the user’s message plus everything the loop has accumulated so far (earlier tool results, notes, memory).
-
Reason — one model call decides what to do next: answer, or request one or more tool calls.
-
Act — the runtime (not the model) executes the requested tool calls.
-
Observe — the tool results are appended to the context, and the loop goes back to perceive.
The loop is what separates an agent from a single LLM call: the model does not know the answer up front, so it is allowed to go and find it, one observation at a time. The two things that make that safe are also part of the loop, not optional extras: a stop condition the model controls (a reply with no tool calls) and a hard limit the runtime controls (a maximum number of steps, tokens or seconds), so a confused model cannot loop forever. The mechanics of a single tool call — the JSON schema the model sees, parallel calls, how a result is returned — are covered in Structured Output & Tool Calling; this page is about what gets built on top of that call.
Reflex agents
A reflex agent maps a condition straight to an action with no reasoning step at all: "if the question mentions the cheat sheet, return the cheat-sheet pointer". There is no model call, no memory and no plan. Reflex agents look out of place in an LLM guide, but they are the right tool for the (often large) share of requests that have one fixed correct answer, and they make an excellent first node in front of a more expensive agent: handle what a rule can handle, and only escalate the rest.
from typing import TypedDict
from langgraph.graph import END, START, StateGraph
class ReflexState(TypedDict):
question: str
answer: str
RULES = { # condition -> canned action: no model call, no memory, no planning
"cheat sheet": "Every sub-section has a one-page PDF; see its Cheat Sheet page.",
"licence": "This site's content is published under the repository's LICENSE file.",
}
def match_rule(state: ReflexState) -> str:
q = state["question"].lower()
return next((key for key in RULES if key in q), "no_rule")
def respond(state: ReflexState) -> dict:
return {"answer": RULES[match_rule(state)]}
def hand_off(state: ReflexState) -> dict:
return {"answer": "ESCALATE: route to the full DocsAssistant agent"}
builder = StateGraph(ReflexState)
builder.add_node("respond", respond)
builder.add_node("hand_off", hand_off)
builder.add_conditional_edges(
START, match_rule, {**{k: "respond" for k in RULES}, "no_rule": "hand_off"}
)
builder.add_edge("respond", END)
builder.add_edge("hand_off", END)
reflex_agent = builder.compile()
reflex_agent.invoke({"question": "Where is the vector-rag cheat sheet?"})
add_conditional_edges routes on the return value of a plain Python function; nothing here calls a model. See
LangGraph v1 release notes for the stable
StateGraph API this page uses throughout.
ReAct: reason and act
ReAct (Yao et al., 2022) interleaves reasoning and tool use in one loop: think about what is missing, call a tool to get it, read the result, think again. It is the default architecture of almost every modern agent framework, because it is exactly the agent loop above with the model deciding at each turn whether to act or answer. Its strength is adaptivity — the next query depends on the last result — and its weakness is myopia: it plans one step at a time, so long tasks can wander.
The graph below builds the ReAct loop by hand, so each step of the loop is a visible node. In production you
would usually call create_agent (see the note after the example), which builds the same loop for you.
from langchain_core.tools import tool
from langchain_ollama import ChatOllama
from langgraph.graph import END, START, MessagesState, StateGraph
from langgraph.prebuilt import ToolNode, tools_condition
from docsassistant.search import search_docs as _search_docs
@tool
def search_docs(query: str, component: str | None = None) -> list[dict]:
"""Search this site's documentation for chunks relevant to `query`.
component: optional filter, one of "vector-rag" or "spring-ai".
"""
return _search_docs(query, component=component, k=5)
tools = [search_docs]
llm = ChatOllama(model="llama3.1:8b", temperature=0).bind_tools(tools)
SYSTEM = (
"You are DocsAssistant. Think about what you still need to know, call search_docs "
"when the answer depends on what the documentation says, and answer only from the "
"chunks you retrieved, citing their source_path."
)
def reason(state: MessagesState) -> dict:
# Reason: the model reads everything observed so far and either answers or asks for a tool.
return {"messages": [llm.invoke([("system", SYSTEM), *state["messages"]])]}
builder = StateGraph(MessagesState)
builder.add_node("reason", reason)
builder.add_node("act", ToolNode(tools)) # Act: run the requested tool calls
builder.add_edge(START, "reason")
builder.add_conditional_edges("reason", tools_condition, {"tools": "act", END: END})
builder.add_edge("act", "reason") # Observe: tool results flow back into the loop
react_agent = builder.compile()
result = react_agent.invoke(
{"messages": [("user", "Which pgvector index type does DocsAssistant use, and why?")]},
{"recursion_limit": 12},
)
tools_condition returns "tools" when the last AI message contains tool calls and END otherwise — the
loop’s stop condition. recursion_limit is the runtime’s hard limit on the number of graph steps. The equivalent
one-liner is LangChain 1.x’s create_agent(model, tools=[search_docs], system_prompt=SYSTEM), which replaced the
0.x create_react_agent and adds a middleware hook at every step of the same loop. See
LangChain — Agents and
LangGraph v1 release notes.
In LangChain4j, an AiServices interface with @Tool-annotated methods runs the same loop: the service sends the
tool specifications with the request, executes whatever the model asks for, and re-invokes the model until it
returns text.
import dev.langchain4j.agent.tool.P;
import dev.langchain4j.agent.tool.Tool;
import dev.langchain4j.model.ollama.OllamaChatModel;
import dev.langchain4j.service.AiServices;
import dev.langchain4j.service.SystemMessage;
class DocsTools {
@Tool("Search this site's documentation for chunks relevant to a query")
List<DocHit> searchDocs(@P("natural-language search phrase") String query) {
return DocsSearch.search(query, 5); // DocsAssistant's retriever
}
}
interface DocsAssistant {
@SystemMessage("You are DocsAssistant. Use searchDocs and cite source paths.")
String answer(String question);
}
DocsAssistant assistant = AiServices.builder(DocsAssistant.class)
.chatModel(OllamaChatModel.builder()
.baseUrl("http://localhost:11434")
.modelName("llama3.1:8b")
.build())
.tools(new DocsTools())
.maxSequentialToolsInvocations(10) // the runtime's hard limit
.build();
See LangChain4j — Tools. Spring AI’s ChatClient does the same when
it is given @Tool-annotated objects via .tools(…) or .defaultTools(…): the tool-calling loop runs inside
the call, and .call().content() returns only once the model stops asking for tools — the exact code is on
Agentic RAG. See
Spring AI — Tool calling.
Plan-and-execute
Plan-and-execute (the "planner-executor" architecture, closely related to Plan-and-Solve prompting) splits the loop in two: one model call writes the whole plan up front, then an executor works through the steps, and a final call synthesises the answer. It trades ReAct’s adaptivity for focus: the plan keeps a long task on track, each executor step can use a smaller or cheaper model, and the plan itself is an inspectable artifact. Its failure mode is a bad plan executed faithfully, which is why production versions add a re-planning step (see Planning).
import operator
from typing import Annotated, TypedDict
from langchain_ollama import ChatOllama
from langgraph.graph import END, START, StateGraph
from pydantic import BaseModel, Field
from docsassistant.search import search_docs
llm = ChatOllama(model="llama3.1:8b", temperature=0)
class Plan(BaseModel):
steps: list[str] = Field(description="Ordered documentation lookups needed to answer")
class PlanState(TypedDict):
question: str
plan: list[str]
findings: Annotated[list[str], operator.add]
answer: str
def planner(state: PlanState) -> dict:
plan = llm.with_structured_output(Plan).invoke(
f"Break this question into at most 4 documentation lookups: {state['question']}"
)
return {"plan": plan.steps}
def executor(state: PlanState) -> dict:
step, *rest = state["plan"] # execute one step, keep the rest
chunks = search_docs(step, k=3)
note = f"{step}: " + " | ".join(c["content"] for c in chunks)
return {"plan": rest, "findings": [note]}
def more_steps(state: PlanState) -> str:
return "executor" if state["plan"] else "synthesize"
def synthesize(state: PlanState) -> dict:
notes = "\n".join(state["findings"])
reply = llm.invoke(f"Answer '{state['question']}' using only these notes:\n{notes}")
return {"answer": reply.content}
builder = StateGraph(PlanState)
builder.add_node("planner", planner)
builder.add_node("executor", executor)
builder.add_node("synthesize", synthesize)
builder.add_edge(START, "planner")
builder.add_conditional_edges("planner", more_steps, ["executor", "synthesize"]) # empty plan: skip ahead
builder.add_conditional_edges("executor", more_steps, ["executor", "synthesize"])
builder.add_edge("synthesize", END)
plan_and_execute = builder.compile()
plan_and_execute.invoke({"question": "Compare HNSW and IVFFlat for DocsAssistant's pgvector setup"})
findings is declared with an operator.add reducer, so each executor step appends to it rather than
overwriting it; plan has no reducer, so each step replaces it with the remaining steps. The planner’s edge uses the
same more_steps check as the executor’s, so a plan with no steps goes straight to synthesize instead of reaching
an executor that has nothing to unpack. See
LangChain — Structured output for
with_structured_output.
LangChain4j ships this pattern in its agentic module as a supervisor — an LLM planner that decides which
sub-agent to invoke next until it declares the task done (shown on
Multi-Agent Systems) — plus a lower-level AgenticServices.plannerBuilder()
for plugging in a custom planner. Spring AI has no planner abstraction; its
effective-agents examples build the
equivalent from plain ChatClient calls, as the orchestrator-workers pattern on
Workflow Patterns does.
Query decomposition
Query decomposition breaks one complex question into independent sub-questions, answers each (often in parallel), and combines the partial answers. It is plan-and-execute without the ordering: the sub-questions do not depend on each other, so they can fan out. For retrieval-heavy agents it is also a recall booster — one embedding of "how do chunking and re-ranking each affect recall?" retrieves worse than one embedding per half.
Send, fan-in with a reducerimport operator
from typing import Annotated, TypedDict
from langchain_ollama import ChatOllama
from langgraph.graph import END, START, StateGraph
from langgraph.types import Send
from pydantic import BaseModel
from docsassistant.search import search_docs
llm = ChatOllama(model="llama3.1:8b", temperature=0)
class SubQuestions(BaseModel):
questions: list[str]
class DecomposeState(TypedDict):
question: str
sub_questions: list[str]
partials: Annotated[list[str], operator.add] # parallel branches append here
answer: str
class SubState(TypedDict):
sub_question: str
def decompose(state: DecomposeState) -> dict:
out = llm.with_structured_output(SubQuestions).invoke(
f"Split into independent sub-questions: {state['question']}"
)
return {"sub_questions": out.questions}
def fan_out(state: DecomposeState) -> list[Send]:
return [Send("answer_one", {"sub_question": q}) for q in state["sub_questions"]]
def answer_one(state: SubState) -> dict:
chunks = search_docs(state["sub_question"], k=3)
reply = llm.invoke(f"{state['sub_question']}\nContext: {chunks}")
return {"partials": [f"{state['sub_question']} -> {reply.content}"]}
def combine(state: DecomposeState) -> dict:
reply = llm.invoke(f"Combine into one answer to '{state['question']}':\n{state['partials']}")
return {"answer": reply.content}
builder = StateGraph(DecomposeState)
builder.add_node("decompose", decompose)
builder.add_node("answer_one", answer_one)
builder.add_node("combine", combine)
builder.add_edge(START, "decompose")
builder.add_conditional_edges("decompose", fan_out, ["answer_one"])
builder.add_edge("answer_one", "combine")
builder.add_edge("combine", END)
decomposer = builder.compile()
A conditional edge that returns a list of Send objects starts one answer_one branch per sub-question, each with
its own small input; the branches run in the same superstep, and combine runs once they have all written to
partials. See LangGraph v1 release notes. The
same fan-out is LangChain4j’s parallelMapperBuilder() and Spring AI’s parallelisation example, both shown on
Workflow Patterns. The retrieval-specific variant
(multi-query expansion) is on Agentic RAG.
Reflection and Reflexion
Reflection adds a critique step after a draft: the same model (or a second one) grades the output against a rubric, and the draft is revised until it passes or a retry budget runs out. It is the single most reliable quality lift for tasks with a checkable criterion — "is every claim supported by a retrieved chunk?"
Reflexion (Shinn et al., 2023) goes one step further: the critique is turned into a short verbal lesson ("search the pgvector page, not the HNSW one, for index parameters") and stored, so the next attempt — on this task or, with long-term memory, on the next similar task — starts from the lessons rather than from scratch. It is learning without changing any weights; the persistent-memory side is on Learning & Improvement.
import operator
from typing import Annotated, TypedDict
from langchain_ollama import ChatOllama
from langgraph.graph import END, START, StateGraph
from pydantic import BaseModel
from docsassistant.search import search_docs
llm = ChatOllama(model="llama3.1:8b", temperature=0)
MAX_ATTEMPTS = 3
class Critique(BaseModel):
grounded: bool # every claim is supported by a retrieved chunk
lesson: str # what to do differently next time, in one sentence
class ReflexionState(TypedDict):
question: str
draft: str
attempts: int
lessons: Annotated[list[str], operator.add] # verbal "memory" carried between attempts
def draft(state: ReflexionState) -> dict:
chunks = search_docs(state["question"], k=5)
guidance = "\n".join(f"- {l}" for l in state.get("lessons", []))
reply = llm.invoke(
f"Question: {state['question']}\nContext: {chunks}\n"
f"Lessons from earlier attempts:\n{guidance or '- none yet'}"
)
return {"draft": reply.content, "attempts": state.get("attempts", 0) + 1}
def reflect(state: ReflexionState) -> dict:
critique = llm.with_structured_output(Critique).invoke(
f"Is this answer fully grounded in DocsAssistant's docs? If not, give one lesson.\n"
f"{state['draft']}"
)
return {"lessons": [] if critique.grounded else [critique.lesson]}
def done(state: ReflexionState) -> str:
stuck = state["attempts"] >= MAX_ATTEMPTS
passed = len(state["lessons"]) < state["attempts"] # last reflection added no lesson
return END if passed or stuck else "draft"
builder = StateGraph(ReflexionState)
builder.add_node("draft", draft)
builder.add_node("reflect", reflect)
builder.add_edge(START, "draft")
builder.add_edge("draft", "reflect")
builder.add_conditional_edges("reflect", done, ["draft", END])
reflexion = builder.compile()
reflexion.invoke({"question": "What does ef_search control?", "attempts": 0})
Plain reflection is the same graph without lessons — the critique goes straight into the revision prompt and
is then discarded. Because the stop condition is checked in code (MAX_ATTEMPTS), a critic that is never
satisfied cannot keep the loop alive. See
LangGraph v1 release notes.
LangChain4j’s agentic module expresses the draft-critique loop directly as a loop workflow: a scorer agent writes
a score to the shared AgenticScope, and the loop exits when it passes.
import dev.langchain4j.agentic.Agent;
import dev.langchain4j.agentic.AgenticServices;
import dev.langchain4j.agentic.UntypedAgent;
import dev.langchain4j.service.UserMessage;
import dev.langchain4j.service.V;
public interface GroundednessScorer {
@UserMessage("Score 0.0-1.0 how fully this answer is supported by the docs: {{answer}}")
@Agent("Scores how well an answer is grounded in the documentation")
double score(@V("answer") String answer);
}
public interface AnswerReviser {
@UserMessage("Revise this answer so every claim is supported by the docs: {{answer}}")
@Agent("Revises an answer to remove unsupported claims")
String revise(@V("answer") String answer);
}
GroundednessScorer scorer = AgenticServices.agentBuilder(GroundednessScorer.class)
.chatModel(chatModel).outputKey("score").build();
AnswerReviser reviser = AgenticServices.agentBuilder(AnswerReviser.class)
.chatModel(chatModel).outputKey("answer").build();
UntypedAgent reflectionLoop = AgenticServices.loopBuilder()
.subAgents(scorer, reviser)
.maxIterations(3)
.exitCondition(scope -> scope.readState("score", 0.0) >= 0.8)
.build();
See LangChain4j — Agents and Agentic AI (loop workflow). Spring AI’s version is the evaluator-optimizer example, shown on Workflow Patterns.
Deep research
Deep research is the long-running composition of everything above: plan a set of research questions, fan out to investigate them (each branch a small ReAct loop over search tools), reflect on what the notes still do not answer, and loop until there are no gaps or a round budget runs out — then write a cited report. It is the architecture behind the "research" modes of the major chat products, and the one most in need of the context discipline on Context Engineering for Agents: each branch returns a condensed note, never its raw search results.
import operator
from typing import Annotated, TypedDict
from langchain_ollama import ChatOllama
from langgraph.graph import END, START, StateGraph
from langgraph.types import Send
from pydantic import BaseModel
from docsassistant.search import search_docs
llm = ChatOllama(model="llama3.1:8b", temperature=0)
MAX_ROUNDS = 3
class ResearchPlan(BaseModel):
questions: list[str] # empty list means "no gaps left"
class ResearchState(TypedDict):
topic: str
open_questions: list[str]
notes: Annotated[list[str], operator.add]
rounds: int
report: str
def plan_round(state: ResearchState) -> dict:
plan = llm.with_structured_output(ResearchPlan).invoke(
f"Topic: {state['topic']}\nNotes so far: {state.get('notes', [])}\n"
"List up to 3 questions the notes do not answer yet, or none."
)
return {"open_questions": plan.questions, "rounds": state.get("rounds", 0) + 1}
def dispatch(state: ResearchState):
if not state["open_questions"] or state["rounds"] > MAX_ROUNDS:
return "write_report"
return [Send("investigate", {"question": q}) for q in state["open_questions"]]
def investigate(state: dict) -> dict:
chunks = search_docs(state["question"], k=4)
summary = llm.invoke(f"Summarise what these chunks say about '{state['question']}': {chunks}")
return {"notes": [f"Q: {state['question']}\nA: {summary.content}"]}
def write_report(state: ResearchState) -> dict:
report = llm.invoke(f"Write a cited report on {state['topic']} from:\n" + "\n\n".join(state["notes"]))
return {"report": report.content}
builder = StateGraph(ResearchState)
builder.add_node("plan_round", plan_round)
builder.add_node("investigate", investigate)
builder.add_node("write_report", write_report)
builder.add_edge(START, "plan_round")
builder.add_conditional_edges("plan_round", dispatch, ["investigate", "write_report"])
builder.add_edge("investigate", "plan_round")
builder.add_edge("write_report", END)
deep_research = builder.compile()
The loop-back edge from investigate to plan_round is what makes this research rather than decomposition: the
next round’s questions depend on what the last round found. See
LangGraph v1 release notes. LangChain4j and Spring AI
have no dedicated deep-research component; the LangChain4j version is a loopBuilder() whose sub-agents are a
planner and a parallelMapperBuilder() investigator, and the Spring AI version is the orchestrator-workers
example run in a loop.
Reasoning aids: tree of thoughts and self-consistency
Two well-known techniques are often listed as agent architectures but are better understood as aids that plug into any step of the patterns above, wherever a single sample is not trustworthy enough:
-
Self-consistency (Wang et al., 2022) samples the same step several times at a non-zero temperature and takes the majority answer. It is cheap to add to a planner or a router whose output is a small discrete choice, and a lack of majority is itself a useful uncertainty signal to escalate on (see Human-in-the-Loop).
-
Tree of thoughts (Yao et al., 2023) keeps several partial candidates (plans, drafts) alive at once, scores each with an evaluator call, and expands only the best — a beam search over reasoning steps. It suits planning steps where an early wrong choice is expensive, at a cost of evaluator calls for beam width and depth .
If a self-consistency vote over independent samples each has probability of being right, the majority is right with probability
which grows with — five samples at give roughly . The formula assumes independent errors; samples from one model share its blind spots, so the real gain is smaller.
from collections import Counter
from langchain_ollama import ChatOllama
from pydantic import BaseModel
sampler = ChatOllama(model="llama3.1:8b", temperature=0.8) # diversity is the point
class Choice(BaseModel):
index_type: str # e.g. "hnsw" or "ivfflat"
def self_consistent_choice(question: str, context: str, n: int = 5) -> str:
votes = Counter(
sampler.with_structured_output(Choice).invoke(f"{question}\n{context}").index_type
for _ in range(n)
)
winner, count = votes.most_common(1)[0]
return winner if count > n // 2 else "no-majority: escalate"
The single-call versions of both techniques are prompting patterns, covered on Prompting Techniques; what changes in an agent is that the samples or branches are separate graph steps with their own budget. See LangChain — Structured output.
Choosing an architecture
| Architecture | Use when | Watch out for |
|---|---|---|
Reflex |
One fixed correct answer per condition |
Rules silently going stale; always keep an escalation path |
ReAct |
The next step depends on the last result; short to medium tasks |
Wandering on long tasks; enforce a step limit |
Plan-and-execute |
Long tasks with a knowable shape; cheaper executor models |
A bad plan executed faithfully; add re-planning |
Query decomposition |
Independent sub-questions; retrieval recall matters |
Sub-questions that are not actually independent |
Reflection / Reflexion |
There is a checkable quality criterion |
A critic that is never satisfied; cap the retries |
Deep research |
Open-ended investigation with a report as output |
Context bloat from raw results; condense per branch |
In other frameworks
| Framework | How the loop and these architectures appear |
|---|---|
OpenAI Agents SDK |
|
Claude Agent SDK |
|
Microsoft Agent Framework |
An |
CrewAI |
An |
See OpenAI Agents SDK, Claude Agent SDK, Microsoft Agent Framework and CrewAI, and The 2026 Agent Frameworks Landscape for the full comparison.
References
-
Yao et al. — ReAct: Synergizing Reasoning and Acting in Language Models
-
Shinn et al. — Reflexion: Language Agents with Verbal Reinforcement Learning
-
Yao et al. — Tree of Thoughts: Deliberate Problem Solving with Large Language Models
-
Wang et al. — Self-Consistency Improves Chain of Thought Reasoning in Language Models