Tool Design

This content was generated with the assistance of AI and should be verified against the official documentation of each tool and framework before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

This page’s examples target Python 3.12 with LangChain 1.4.2 / langchain-core 1.6.5 / LangGraph 1.2.12 and langchain-ollama 1.1.0, Java 21 with LangChain4j 1.20.2, and Spring Boot 4.1 with Spring AI 2.0.1, using DocsAssistant's Ollama llama3.1:8b chat model and nomic-embed-text embedding model, as checked on 2026-09-28 (see the AI Agents baseline).

Tools are the agent’s hands: everything it can find out or change beyond its own weights goes through one. They are also the part of an agent the developer controls completely, which makes tool design the highest-leverage work in most agent projects. A model can only be as good as the interface it is given, and Anthropic’s Writing effective tools for agents makes the point bluntly: tools are a contract between deterministic code and a non-deterministic caller, and should be designed for that caller, not wrapped around an existing API as-is.

Function-calling mechanics

Each tool reaches the model as a name, a description and a JSON Schema for its arguments. The model never executes anything: it returns one or more structured tool calls (name, arguments, an ID), the runtime executes them, and each result goes back to the model as a tool result message carrying the same ID. Most providers let the model request several independent calls in one turn (parallel tool calls); the runtime can execute them concurrently and must return every result before the next model call. The wire formats, tool_choice, strict schemas and structured output are covered in Structured Output & Tool Calling; this page is about what to put inside that envelope.

Everything below builds on DocsAssistant’s search tool, rewritten for agents. Here is what LangChain’s @tool decorator turns it into — the JSON the model actually reads:

What the model sees for docs_search

{
  "type": "function",
  "function": {
    "name": "docs_search",
    "description": "Search this site's documentation and return matching sections.\n\n    Start with response_format=\"concise\" (page path, section, one-line snippet).\n    Ask for \"detailed\" only for the one or two results you intend to quote.\n    If has_more is true, call again with page + 1 rather than a longer query.\n    Each page is a repository path (modules/ROOT/pages/<dir>/<page>.adoc); pass it unchanged to other tools.",
    "parameters": {
      "properties": {
        "query": {"type": "string"},
        "component": {"anyOf": [{"enum": ["vector-rag", "spring-ai"], "type": "string"}, {"type": "null"}],
                      "default": null},
        "page": {"default": 1, "type": "integer"},
        "response_format": {"default": "concise", "enum": ["concise", "detailed"], "type": "string"}
      },
      "required": ["query"],
      "type": "object"
    }
  }
}

The docstring became the description, the type hints became the schema, and the Literal types became enum constraints. That is the whole interface: if something matters to how the tool should be used, it has to be in one of those three places.

Designing tools for agents

The Python version of the tool, applying the rules listed after it:

DocsAssistant’s search tool, designed for an agent caller

from typing import Literal

from langchain.tools import tool

from docsassistant.search import search_docs

PAGE_SIZE = 5


@tool
def docs_search(
    query: str,
    component: Literal["vector-rag", "spring-ai"] | None = None,
    page: int = 1,
    response_format: Literal["concise", "detailed"] = "concise",
) -> dict:
    """Search this site's documentation and return matching sections.

    Start with response_format="concise" (page path, section, one-line snippet).
    Ask for "detailed" only for the one or two results you intend to quote.
    If has_more is true, call again with page + 1 rather than a longer query.
    Each page is a repository path (modules/ROOT/pages/<dir>/<page>.adoc); pass it unchanged to other tools.
    """
    if not query.strip():
        return {"error": "query is empty; pass a key phrase from the question, e.g. 'HNSW ef_search'"}
    hits = search_docs(query, component=component, k=50)
    window = hits[(page - 1) * PAGE_SIZE: page * PAGE_SIZE]
    if not window:
        return {"error": f"no results on page {page}; try page=1 or drop the component filter"}
    results = []
    for h in window:
        item = {"page": h["source_path"], "section": h["section"]}   # human-readable IDs: the repo path
        if response_format == "concise":
            item["snippet"] = h["content"][:160]
        else:
            item |= {"content": h["content"], "score": round(h["score"], 3)}
        results.append(item)
    return {"results": results, "page": page, "has_more": len(hits) > page * PAGE_SIZE}

See LangChain — Tools for @tool and its schema inference.

Narrow and typed. One tool does one thing an agent would naturally want to do, with a small, typed argument list. component is an enum, not a free string the model has to guess the spelling of. Prefer a few purpose-built tools (docs_search, docs_get_page) over one docs_api(method, path, body) passthrough.
Concise vs. detailed responses. A response_format argument lets the agent skim first and pay for detail only where it needs it. Concise results are what keep a long agent loop inside its context budget (see Context Engineering for Agents).
Pagination and truncation. Never return an unbounded list. Page, cap, and say so (has_more), and tell the agent in the description what to do next. A truncated result that says it is truncated is far more useful than a silently truncated one.
Actionable errors. Return errors as data the model can act on — what was wrong and what to try instead — rather than a stack trace or an opaque code. "no results on page 3; try page=1" gets a correct retry; "IndexError" gets a hallucinated answer.
Namespacing. Prefix tools by service or resource (docs_search, docs_get_page, github_open_issue) so an agent with many tools — especially tools from several MCP servers — can tell overlapping ones apart.
Human-readable IDs. Return modules/ROOT/pages/database/vector-rag/ann-indexes.adoc and a section title, not a chunk UUID. Models copy meaningful identifiers into the next call and into citations reliably; they mangle opaque ones. Use the same ID format in every tool: docs_search returns exactly the page_path that github_open_issue accepts, so the model never has to translate one into the other.
Descriptions are prompts. Write the description for a new colleague: when to use the tool, when not to, what each argument means, and what comes back. Evaluate it like a prompt, too (see tool-call accuracy).

LangChain4j and Spring AI express the same contract with an annotation on a Java method; the description and parameter descriptions are what the model reads.

LangChain4j: @Tool with described, typed parameters

import dev.langchain4j.agent.tool.P;
import dev.langchain4j.agent.tool.Tool;

public class DocsTools {

    public enum Component { VECTOR_RAG, SPRING_AI }
    public enum Detail { CONCISE, DETAILED }

    @Tool(name = "docs_search", value = """
            Search this site's documentation and return matching sections.
            Start with CONCISE; ask for DETAILED only for results you will quote.
            If hasMore is true, call again with page + 1.
            Each page is a repository path (modules/ROOT/pages/<dir>/<page>.adoc); pass it unchanged to other tools.""")
    public SearchPage search(@P("key phrase from the question") String query,
                             @P(value = "optional component filter", required = false) Component component,
                             @P(value = "1-based page number", required = false) Integer page,
                             @P(value = "CONCISE or DETAILED", required = false) Detail detail) {
        return DocsSearch.page(query, component, page == null ? 1 : page,
                detail == null ? Detail.CONCISE : detail);
    }
}

See LangChain4j — Tools.

Spring AI: @Tool and @ToolParam

import org.springframework.ai.tool.annotation.Tool;
import org.springframework.ai.tool.annotation.ToolParam;

public class DocsTools {

    public enum Component { VECTOR_RAG, SPRING_AI }
    public enum Detail { CONCISE, DETAILED }

    @Tool(name = "docs_search", description = """
            Search this site's documentation and return matching sections.
            Start with CONCISE; ask for DETAILED only for results you will quote.
            If hasMore is true, call again with page + 1.
            Each page is a repository path (modules/ROOT/pages/<dir>/<page>.adoc); pass it unchanged to other tools.""")
    public SearchPage search(@ToolParam(description = "key phrase from the question") String query,
                             @ToolParam(description = "optional component filter", required = false)
                             Component component,
                             @ToolParam(description = "1-based page number", required = false) Integer page,
                             @ToolParam(description = "CONCISE or DETAILED", required = false) Detail detail) {
        return DocsSearch.page(query, component, page == null ? 1 : page,
                detail == null ? Detail.CONCISE : detail);
    }
}

See Spring AI — Tool calling. In both frameworks a RuntimeException thrown from a tool is, by default, turned into an error message sent back to the model, so an exception message should be as actionable as the Python error values above. Spring AI’s spring.ai.tools.throw-exception-on-error=true switches that to failing the call instead.

Stateful tools with least privilege

A stateful tool reads or changes something that outlives the conversation — a database, a ticket system, a file. The rule for those is least privilege, applied at three layers:

Narrow, parameterised operations. Give the agent docs_count_chunks(component), not run_sql(query). A raw SQL or shell tool turns every prompt injection into arbitrary code execution; a parameterised operation limits the blast radius to what that one operation can do.
Validate every argument in code. The schema is a hint to the model, not a guarantee — validate types, ranges, allow-lists and formats again inside the tool, and reject with an actionable error.
Scope the credential, not just the code. The tool runs under an identity that can only do what the tool does: a read-only database role for read tools, a token scoped to one repository and one permission for the write tool. Log every call (arguments, caller, outcome) so actions are auditable.

A parameterised, validated, logged, read-only tool

import logging
from typing import Literal

import psycopg
from langchain.tools import tool

log = logging.getLogger("docsassistant.tools")
READONLY_DSN = "postgresql://docs_readonly@localhost/docsassistant"   # SELECT-only role, no password here


@tool
def docs_count_chunks(component: Literal["vector-rag", "spring-ai"]) -> dict:
    """Count how many indexed chunks DocsAssistant holds for one documentation component."""
    with psycopg.connect(READONLY_DSN) as conn:           # credentials come from the environment/.pgpass
        (count,) = conn.execute(
            "SELECT count(*) FROM chunks WHERE component = %s", (component,)
        ).fetchone()
    log.info("docs_count_chunks component=%s count=%s", component, count)
    return {"component": component, "chunks": count}

The query text is fixed; only the validated component value is bound as a parameter. docs_readonly is the same read-only role Agentic RAG uses for its retriever-as-tool, and the password is supplied by libpq’s environment or .pgpass handling rather than the source. See LangChain — Tools.

Generating tools from OpenAPI

When a service already has an OpenAPI description, tools can be generated from it rather than written by hand: each operation’s operationId, summary and parameter schemas map directly onto a tool’s name, description and argument schema. Generation is a starting point, not an end state — an API designed for programs is rarely a good tool set for an agent, so generate only an allow-listed subset of read-only operations, then rename, describe and wrap them by hand where the generated shape is awkward.

Typed tools from allow-listed, read-only OpenAPI operations

from typing import Any, Callable

from langchain_core.tools import StructuredTool
from pydantic import Field, create_model

READ_ONLY = {"get"}                                        # never generate write tools blindly
JSON_TYPES = {"string": str, "integer": int, "number": float, "boolean": bool}


def tools_from_openapi(spec: dict, call: Callable[[str, str, dict], Any],
                       allow: set[str]) -> list[StructuredTool]:
    """Turn allow-listed, read-only OpenAPI operations into typed tools."""
    tools = []
    for path, operations in spec["paths"].items():
        for method, op in operations.items():
            if method not in READ_ONLY or op.get("operationId") not in allow:
                continue
            fields = {}
            for p in op.get("parameters", []):
                py_type = JSON_TYPES[p["schema"]["type"]]
                default = ... if p.get("required") else None
                fields[p["name"]] = (py_type if p.get("required") else py_type | None,
                                     Field(default, description=p.get("description", "")))
            args = create_model(f"{op['operationId']}_args", **fields)
            tools.append(StructuredTool.from_function(
                func=lambda _m=method, _p=path, **kw: call(_m, _p, kw),
                name=f"docsapi_{op['operationId']}",                 # namespaced
                description=op.get("summary", op["operationId"]),
                args_schema=args,
            ))
    return tools

call is the one place that performs HTTP, so authentication, rate limiting and logging live there rather than in every generated tool. Even an allow-listed DELETE is skipped by the READ_ONLY check. See LangChain — Tools for StructuredTool. In the Java stacks the same idea usually arrives through MCP instead: an OpenAPI-backed MCP server exposes the operations, and both LangChain4j and Spring AI consume MCP tools natively — the Model Context Protocol (MCP) sub-section of this AI section covers building such servers.

Semantic and hierarchical tool selection

Every tool definition costs context on every model call, and selection accuracy drops as the list grows: past a few dozen tools, models pick the wrong one more often. Two techniques keep the visible list short:

Semantic selection embeds each tool’s name and description once, embeds the user’s request per turn, and shows the model only the top- tools by similarity — retrieval, applied to tools instead of documents.
Hierarchical selection groups tools into toolsets (search, GitHub, analytics) and lets the model, or a cheap router, pick a toolset first and a tool second — the same idea as the routing pattern.

LangChain 1.x middleware can rewrite the tool list before every model call. The middleware below does semantic selection with DocsAssistant’s own embedding model:

Semantic tool selection as create_agent middleware

import math

from langchain.agents.middleware import ModelRequest, ModelResponse, wrap_model_call
from langchain_ollama import OllamaEmbeddings

embeddings = OllamaEmbeddings(model="nomic-embed-text")    # DocsAssistant's embedding model


def cosine(a: list[float], b: list[float]) -> float:
    dot = sum(x * y for x, y in zip(a, b))
    return dot / (math.sqrt(sum(x * x for x in a)) * math.sqrt(sum(y * y for y in b)))


def semantic_tool_selector(all_tools, top_k: int = 4, always: tuple[str, ...] = ("docs_search",)):
    vectors = embeddings.embed_documents([f"{t.name}: {t.description}" for t in all_tools])

    @wrap_model_call
    def select_tools(request: ModelRequest, handler) -> ModelResponse:
        question = next(m.content for m in reversed(request.messages) if m.type == "human")
        q = embeddings.embed_query(question)
        ranked = sorted(zip(all_tools, vectors), key=lambda tv: cosine(q, tv[1]), reverse=True)
        chosen = [t for t, _ in ranked[:top_k]]
        chosen += [t for t in all_tools if t.name in always and t not in chosen]
        return handler(request.override(tools=chosen))

    return select_tools


# agent = create_agent(model, tools=ALL_TOOLS, middleware=[semantic_tool_selector(ALL_TOOLS)])

Every tool is still registered with create_agent, so any of them can execute; the middleware only narrows what each model call sees. LangChain also ships LLMToolSelectorMiddleware, which asks a (cheaper) model to pick the relevant tools instead of using embeddings, with max_tools and always_include options. See LangChain — Agents for middleware.

LangChain4j inverts the approach: with a tool search strategy, the model initially sees only a search tool, calls it with a natural-language description of what it needs, and the matching tools are added to the following requests.

LangChain4j: vector tool search

DocsAssistant assistant = AiServices.builder(DocsAssistant.class)
        .chatModel(chatModel)
        .tools(new DocsTools(), new GitHubTools(), new AnalyticsTools())
        .toolSearchStrategy(VectorToolSearchStrategy.builder()
                .embeddingModel(embeddingModel)       // nomic-embed-text via OllamaEmbeddingModel
                .maxResults(4)
                .build())
        .build();

See LangChain4j — Tools. Spring AI 2.0 implements the same progressive disclosure as ToolSearchToolCallingAdvisor (module spring-ai-autoconfigure-tool-search-advisor), a drop-in replacement for the default tool-calling advisor that indexes the tool set and exposes a built-in tool search tool; see Spring AI — Tool calling ("Scaling to Hundreds of Tools").

Write-action safety

A read tool that misbehaves returns a wrong answer; a write tool that misbehaves changes the world. DocsAssistant has exactly one write action — opening a GitHub issue when a page is wrong — and it is the running example for how to build one:

One repository, one permission. The token is a fine-grained GitHub token scoped to albertoirurueta/docs with only Issues: read and write. It is read from the environment (or a secret manager) inside the tool, never placed in the prompt, the tool arguments or the logs, so the model cannot leak or misuse it.
The model supplies content, not targets. The repository is a constant; the model provides a page path, a one-sentence problem and the contradicting evidence, each validated in code.
Allow-lists over free text for anything structural (labels).
A human approves every call before it runs. The tool description says so, and the runtime enforces it — the model cannot opt out. The approval-gate mechanics (interrupt, approve/edit/reject, resume) are on Human-in-the-Loop.
Errors stop the loop. A failed write says "do not retry, tell the user" so the agent does not hammer the API.

The least-privilege, approval-gated write tool

import logging
import os
import re

import httpx
from langchain.tools import tool

log = logging.getLogger("docsassistant.tools")

REPO = "albertoirurueta/docs"                      # the only repository this tool can touch
ALLOWED_LABELS = {"documentation", "bug"}
PAGE_PATH = re.compile(r"^modules/ROOT/pages/[a-z0-9/_-]+\.adoc$")


@tool
def github_open_issue(page_path: str, problem: str, evidence: str,
                      labels: list[str] | None = None) -> dict:
    """Open a GitHub issue reporting that a documentation page is wrong.

    page_path: repository path of the page, e.g. modules/ROOT/pages/ai/agents/memory.adoc
    problem: one sentence stating what is wrong.
    evidence: the official source that contradicts the page (a URL plus a short quote).
    This is a write action: it always requires human approval before it runs.
    """
    if not PAGE_PATH.match(page_path):
        return {"error": "page_path must look like modules/ROOT/pages/<dir>/<page>.adoc"}
    if not (10 <= len(problem) <= 200):
        return {"error": "problem must be one sentence of 10-200 characters"}
    labels = sorted(set(labels or ["documentation"]) & ALLOWED_LABELS) or ["documentation"]

    token = os.environ["DOCSASSISTANT_GITHUB_TOKEN"]   # fine-grained token: this repo, Issues write only
    response = httpx.post(
        f"https://api.github.com/repos/{REPO}/issues",
        headers={"Authorization": f"Bearer {token}", "Accept": "application/vnd.github+json"},
        json={"title": f"Docs: {problem[:80]}", "labels": labels,
              "body": f"Page: `{page_path}`\n\n{problem}\n\nEvidence: {evidence}"},
        timeout=10,
    )
    log.info("github_open_issue page=%s status=%s", page_path, response.status_code)
    if response.status_code != 201:
        return {"error": f"GitHub returned {response.status_code}; do not retry, tell the user"}
    return {"issue": response.json()["html_url"]}

The approval gate is attached where the agent is assembled, not inside the tool, so it cannot be bypassed by a prompt:

Wiring the gate: interrupt before the write tool runs

from langchain.agents import create_agent
from langchain.agents.middleware import HumanInTheLoopMiddleware
from langchain_ollama import ChatOllama
from langgraph.checkpoint.memory import InMemorySaver

agent = create_agent(
    ChatOllama(model="llama3.1:8b", temperature=0),
    tools=[docs_search, github_open_issue],
    middleware=[HumanInTheLoopMiddleware(interrupt_on={
        "github_open_issue": {"allowed_decisions": ["approve", "edit", "reject"]},
        "docs_search": False,                      # reads never need approval
    })],
    checkpointer=InMemorySaver(),                  # interrupts need persisted state to resume from
)

See LangChain — Human-in-the-loop and GitHub REST API — Create an issue.

The Java versions keep the same shape. Spring AI’s ToolContext carries caller data — here, the authenticated user who will be recorded on the issue — to the tool without ever sending it to the model:

Spring AI: the write tool with ToolContext

import java.util.List;

import org.springframework.ai.chat.model.ToolContext;
import org.springframework.ai.tool.annotation.Tool;
import org.springframework.ai.tool.annotation.ToolParam;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.web.client.RestClient;
import org.springframework.web.client.RestClientResponseException;

public class GitHubTools {

    private static final String REPO = "albertoirurueta/docs";
    private final RestClient github;

    public GitHubTools(RestClient.Builder builder,
                       @Value("${docsassistant.github.token}") String token) {  // bound from the environment
        this.github = builder.baseUrl("https://api.github.com")
                .defaultHeader("Authorization", "Bearer " + token)
                .defaultHeader("Accept", "application/vnd.github+json")
                .build();
    }

    @Tool(name = "github_open_issue", description = """
            Open a GitHub issue reporting that a documentation page is wrong.
            This is a write action: it always requires human approval before it runs.""")
    public String openIssue(@ToolParam(description = "repository path of the page, as docs_search returns it, "
                                    + "e.g. modules/ROOT/pages/ai/agents/memory.adoc") String pagePath,
                            @ToolParam(description = "one sentence stating what is wrong") String problem,
                            @ToolParam(description = "official source URL plus a short quote") String evidence,
                            ToolContext toolContext) {
        if (!pagePath.matches("^modules/ROOT/pages/[a-z0-9/_-]+\\.adoc$")) {
            return "error: pagePath must look like modules/ROOT/pages/<dir>/<page>.adoc";
        }
        if (problem.length() < 10 || problem.length() > 200) {
            return "error: problem must be one sentence of 10-200 characters";
        }
        String reporter = (String) toolContext.getContext().get("reporter");   // never seen by the model
        record IssueRequest(String title, String body, List<String> labels) {}
        record Issue(String html_url) {}
        try {
            Issue issue = github.post()
                    .uri("/repos/{repo}/issues", REPO)
                    .body(new IssueRequest("Docs: " + problem.substring(0, Math.min(80, problem.length())),
                            "Page: `" + pagePath + "`\n\n" + problem + "\n\nEvidence: " + evidence
                                    + "\n\nReported via DocsAssistant by " + reporter,
                            List.of("documentation")))
                    .retrieve()                               // throws on a 4xx or 5xx response
                    .body(Issue.class);
            return issue.html_url();
        } catch (RestClientResponseException e) {             // an error the model can act on, not a stack trace
            return "error: GitHub returned " + e.getStatusCode().value() + "; do not retry, tell the user";
        }
    }
}

retrieve() throws a RestClientResponseException for any 4xx or 5xx status, so the tool catches it and returns the same "do not retry" error as the Python version rather than letting the failure bubble up, and problem is held to the same 10-200 characters. docsassistant.github.token is resolved from the DOCSASSISTANT_GITHUB_TOKEN environment variable by Spring Boot’s relaxed binding, so no credential appears in source or configuration files. The caller passes the reporter with .toolContext(Map.of("reporter", user.getName())) on the ChatClient request. See Spring AI — Tool calling (Tool Context). Spring AI and LangChain4j have no built-in approval middleware; the manual approval pattern for both is on Human-in-the-Loop.

Tool calling elsewhere on this site

Agentic RAG — the retriever-as-tool pattern, routing between retrieval tools, and exposing retrieval through MCP. The retrieval side of every tool on this page lives there.
Apple Intelligence & Machine Learning — tool calling with Apple’s on-device Foundation Models framework, the same contract (name, description, typed arguments) running entirely on the device.
The Model Context Protocol (MCP) sub-section of this AI section covers exposing tools to any agent through an MCP server (see also Designing Good MCP Servers); the Building CLIs for AI Agents sub-section covers designing CLIs that agents drive well.

In other frameworks

Framework Tool definition and safety hooks

Framework	Tool definition and safety hooks
OpenAI Agents SDK	`@function_tool` infers the schema from the signature and docstring; `needs_approval=True` pauses the run for approval, and tool input/output guardrails validate calls.
Claude Agent SDK	`@tool` plus `create_sdk_mcp_server` defines in-process tools; `allowed_tools`, `disallowed_tools`, `permission_mode` and a `can_use_tool` callback gate what runs.
Microsoft Agent Framework	The `@tool` decorator (or plain functions) defines tools; `approval_mode="always_require"` makes a tool wait for approval.
CrewAI	Tools are `BaseTool` subclasses or `@tool`-decorated functions assigned per agent, so each agent sees only its own tool set.

OpenAI Agents SDK

@function_tool infers the schema from the signature and docstring; needs_approval=True pauses the run for approval, and tool input/output guardrails validate calls.

Claude Agent SDK

@tool plus create_sdk_mcp_server defines in-process tools; allowed_tools, disallowed_tools, permission_mode and a can_use_tool callback gate what runs.

Microsoft Agent Framework

The @tool decorator (or plain functions) defines tools; approval_mode="always_require" makes a tool wait for approval.

CrewAI

Tools are BaseTool subclasses or @tool-decorated functions assigned per agent, so each agent sees only its own tool set.

See OpenAI Agents SDK, Claude Agent SDK, Microsoft Agent Framework and CrewAI.