Observability

This content was generated with the assistance of AI and should be verified against the official documentation of each tool and framework before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

Written against Spring AI 2.0.1, Spring Boot 4.1.x and Java 21 (verified 2026-09-29). A model call is slow, metered and non-deterministic, which are three good reasons to measure it. Spring AI instruments its own components with Micrometer, so the metrics and traces arrive through the same Actuator, Prometheus and OpenTelemetry machinery as the rest of the application. This page adds that visibility to DocsAssistant: which observations exist, how they show up in Prometheus, how one request looks as a trace, how to turn token usage into an estimated cost, and how to log prompts and completions without leaking what users typed. The classes and tests were compiled against the 2.0.1 jars. The metric and log output below was captured from a Spring Boot 4.1.1 context that called a real OllamaChatModel pointed at a WireMock server (no model server was available), so the numbers are those of scripted replies, not of a live model. Spring Boot’s own metrics and tracing setup is covered on Metrics and Observability; this page only covers what Spring AI adds.

What Spring AI observes

Spring AI creates a Micrometer Observation around each operation. One observation feeds several backends: a timer (and counters) for metrics, a span for tracing, and, when switched on, a log line. Every observation carries low-cardinality keys, whose values come from a small fixed set and become metric tags and span tags, and high-cardinality keys, such as a conversation ID or a temperature, which go to the span only because they would create a new time series for every value.

Table 1. The observations in Spring AI 2.0.1
Component	Observation name	Recorded when
`ChatClient`	`spring.ai.chat.client`	A `call()` or `stream()` runs. It measures the whole request, including advisors
Advisors	`spring.ai.advisor`	An advisor runs. The time includes the advisors inside it
`ChatModel`	`gen_ai.client.operation` and the counter `gen_ai.client.token.usage`	One request goes to the model. A tool loop makes several
Tools	`spring.ai.tool`, span name `execute_tool <tool name>`	The framework executes a tool for the model
`EmbeddingModel`, `ImageModel`	`gen_ai.client.operation`	An embedding or image call. Only some providers are instrumented
`VectorStore`	`db.vector.client.operation`	`add`, `delete` or `query` on a vector store

Source: Spring AI: Observability. Two names are easy to mix up. gen_ai.client.operation is the model call, one per round trip to the provider, while spring.ai.chat.client is the whole ChatClient request, which may contain several model calls when tools are involved. The tag gen_ai.system says which is which: ollama (or the provider name) on the model call, spring_ai on the framework observations.

Table 2. Keys worth knowing (a selection; the reference lists all of them)
Key	Kind	Meaning
`gen_ai.request.model`, `gen_ai.response.model`, `gen_ai.system`, `gen_ai.operation.name`	low	The model asked for, the model that answered, the provider and the operation (`chat`)
`gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`, `gen_ai.usage.total_tokens`	high	Token counts on the span. Provider-managed cache reads and writes have their own keys
`gen_ai.request.temperature`, `gen_ai.request.max_tokens`, `gen_ai.response.finish_reasons`	high	What was asked and why the model stopped
`spring.ai.chat.client.conversation.id`	high	The memory conversation ID when memory is used. It is hashed in DocsAssistant, see Chat Memory
`spring.ai.advisor.name`	low	Which advisor. The framework’s own terminal advisor is named `call`
`spring.ai.tool.definition.name`, `spring.ai.tool.type`	low	Which tool and its type (`function` by default). The type is new in 2.0
`spring.ai.tool.call.arguments`, `spring.ai.tool.call.result`	high	Only when tool content is switched on, see the privacy section

Turning it on

Metrics and tracing need Actuator and a registry or exporter, which are ordinary Spring Boot dependencies:

pom.xml (the observability starters)

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
  <groupId>io.micrometer</groupId>
  <artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-opentelemetry</artifactId>
</dependency>

Source: Spring Boot: Observability.

application.properties (observability)

# expose only what the monitoring system needs
management.endpoints.web.exposure.include=health,metrics,prometheus
# a model call is slow and costly to trace, but a small sample of them is cheap: keep 10 percent
management.tracing.sampling.probability=0.1
# OTLP over HTTP to an OpenTelemetry collector or Jaeger
management.opentelemetry.tracing.export.otlp.endpoint=http://localhost:4318/v1/traces
# latency histograms for the model call, so Grafana can draw percentiles
management.metrics.distribution.percentiles-histogram.gen_ai.client.operation=true

Source: Spring Boot: Tracing and Spring Boot: Metrics.

All four keys exist in the Spring Boot 4.1.1 configuration metadata. The Actuator’s Prometheus endpoint is an authenticated endpoint like any other: a scrape without credentials returned 401 in the tests below. In DocsAssistant, SecurityConfig (see Security & Guardrails) restricts /actuator/** to a monitoring role, and Prometheus sends a bearer token for that role, read from a file that a secret store mounts. Spring AI’s auto-configuration registers the meter handlers for you when a MeterRegistry is present. A unit test without Spring has to register them by hand, as the first test does.

Seeing the observations in a unit test

The fastest feedback is a test that calls a real chat model class against a stub server and reads the MeterRegistry. The stub replies with token counts the way Ollama reports them (prompt_eval_count and eval_count), so the token metrics have something to record.

OllamaStubs.java: canned Ollama replies for WireMock

package com.example.docsassistant;

import static com.github.tomakehurst.wiremock.client.WireMock.aResponse;
import static com.github.tomakehurst.wiremock.client.WireMock.equalTo;
import static com.github.tomakehurst.wiremock.client.WireMock.matchingJsonPath;
import static com.github.tomakehurst.wiremock.client.WireMock.post;
import static com.github.tomakehurst.wiremock.client.WireMock.urlEqualTo;

import com.github.tomakehurst.wiremock.WireMockServer;

/** Canned Ollama /api/chat replies, served by WireMock. */
final class OllamaStubs {

    private OllamaStubs() {
    }

    /** A complete, non-streaming reply with token counts, the way Ollama reports usage. */
    static void chatReply(WireMockServer server, String text, int promptTokens, int completionTokens) {
        chatReply(server, "llama3.1:8b", text, promptTokens, completionTokens);
    }

    /** The same, but only for requests addressed to one model, so an answer model and a judge can share a server. */
    static void chatReply(WireMockServer server, String model, String text, int promptTokens, int completionTokens) {
        String body = """
                {"model":%s,"created_at":"2026-09-29T10:00:00Z",
                 "message":{"role":"assistant","content":%s},
                 "done":true,"done_reason":"stop",
                 "prompt_eval_count":%d,"eval_count":%d}
                """.formatted(quote(model), quote(text), promptTokens, completionTokens);
        server.stubFor(post(urlEqualTo("/api/chat"))
                .withRequestBody(matchingJsonPath("$.model", equalTo(model)))
                .willReturn(aResponse().withHeader("Content-Type", "application/json").withBody(body)));
    }

    private static String quote(String text) {
        return "\"" + text.replace("\\", "\\\\").replace("\"", "\\\"") + "\"";
    }
}

Source: Spring AI: Observability, whose token metrics this helper feeds.

ChatObservationTest.java: one call, four kinds of meter

package com.example.docsassistant;

import static org.assertj.core.api.Assertions.assertThat;

import com.github.tomakehurst.wiremock.WireMockServer;
import com.github.tomakehurst.wiremock.core.WireMockConfiguration;
import io.micrometer.core.instrument.MeterRegistry;
import io.micrometer.core.instrument.simple.SimpleMeterRegistry;
import io.micrometer.observation.ObservationRegistry;
import io.micrometer.core.instrument.observation.DefaultMeterObservationHandler;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.client.advisor.observation.DefaultAdvisorObservationConvention;
import org.springframework.ai.chat.client.observation.DefaultChatClientObservationConvention;
import org.springframework.ai.chat.observation.ChatModelMeterObservationHandler;
import org.springframework.ai.ollama.OllamaChatModel;
import org.springframework.ai.ollama.api.OllamaApi;
import org.springframework.ai.ollama.api.OllamaChatOptions;

class ChatObservationTest {

    private WireMockServer ollama;
    private final MeterRegistry meters = new SimpleMeterRegistry();
    private final ObservationRegistry observations = ObservationRegistry.create();

    @BeforeEach
    void start() {
        this.ollama = new WireMockServer(WireMockConfiguration.options().dynamicPort());
        this.ollama.start();
        // Spring Boot registers both handlers for you; a plain unit test has to do it by hand
        this.observations.observationConfig().observationHandler(new DefaultMeterObservationHandler(this.meters));
        this.observations.observationConfig().observationHandler(new ChatModelMeterObservationHandler(this.meters));
    }

    @AfterEach
    void stop() {
        this.ollama.stop();
    }

    @Test
    void oneCallProducesClientModelAndTokenMetrics() {
        OllamaStubs.chatReply(this.ollama, "Use MatrixInverse.", 26, 12);
        OllamaChatModel model = OllamaChatModel.builder()
                .ollamaApi(OllamaApi.builder().baseUrl(this.ollama.baseUrl()).build())
                .options(OllamaChatOptions.builder().model("llama3.1:8b").temperature(0.0).build())
                .observationRegistry(this.observations)
                .build();
        ChatClient client = ChatClient.builder(model, this.observations,
                new DefaultChatClientObservationConvention(), new DefaultAdvisorObservationConvention()).build();

        String answer = client.prompt().user("How do I invert a matrix?").call().content();

        assertThat(answer).isEqualTo("Use MatrixInverse.");
        assertThat(this.meters.get("spring.ai.chat.client").timer().count()).isEqualTo(1);
        assertThat(this.meters.get("gen_ai.client.operation").tag("gen_ai.request.model", "llama3.1:8b").timer().count())
                .isEqualTo(1);
        assertThat(this.meters.get("gen_ai.client.token.usage").tag("gen_ai.token.type", "input").counter().count())
                .isEqualTo(26);
        assertThat(this.meters.get("gen_ai.client.token.usage").tag("gen_ai.token.type", "output").counter().count())
                .isEqualTo(12);
    }
}

Source: Spring AI: Observability.

The test passed. Beyond the assertions, the registry contained a spring.ai.chat.client timer, a spring.ai.advisor timer for each advisor that ran (Tool Calling Advisor and call), the gen_ai.client.operation timer with the tags gen_ai.system=ollama, gen_ai.operation.name=chat and the request and response model, and one gen_ai.client.token.usage counter per gen_ai.token.type (input, output, total). Note the tag name: it is gen_ai.token.type. Every ChatClient call passes through Tool Calling Advisor because the 2.0 ChatClient registers it automatically, which is why it appears even in an application that has no tools, as Tool Calling Advisor and Tool Search explains.

What Prometheus sees

With a Prometheus registry, Micrometer renames the dots to underscores and adds suffixes. A Spring Boot 4.1.1 context, called once through a stubbed Ollama server, exposed these series on /actuator/prometheus:

Table 3. Series exported for one chat call (labels abbreviated)
Series	Type	Notes
`gen_ai_client_operation_seconds_count`, `_sum`, `_max`, `_bucket`	timer	One per model round trip. `_bucket` exists because of the histogram property above. Labels include `gen_ai_request_model`, `gen_ai_system` and `error`
`gen_ai_client_token_usage_total`	counter	Labels `gen_ai_token_type` (`input`, `output`, `total`) and the model. In the run: 26 input, 12 output, 38 total tokens
`spring_ai_chat_client_seconds_count`, `_sum`, `_max`	timer	One per `ChatClient` request, label `spring_ai_chat_client_stream`
`spring_ai_advisor_seconds_count`, `_sum`, `_max`	timer	One series per advisor, label `spring_ai_advisor_name`
`gen_ai_client_operation_active_seconds_count`, `spring_ai_chat_client_active_seconds_count`, `spring_ai_advisor_active_seconds_count`	gauge	In-flight operations. The reference documents these as `…_active_count`; the names above are what the 4.1.1 run exposed

The total token series is the sum of input and output. If you add all three label values in a query you count every token twice, so filter gen_ai_token_type!="total" as the rules file below does. Tool calls have an observation but did not appear as a dedicated series in the metrics run, because no tool was called there. The tool observation is exercised in the tracing section.

Prometheus, Grafana and Jaeger locally

A compose file is enough for a local dashboard and trace viewer. The Grafana admin password comes from the environment, never from the file.

compose.yml

services:
  prometheus:
    image: prom/prometheus:v3.13.4
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
    ports:
      - "9090:9090"
  grafana:
    image: grafana/grafana:13.0
    ports:
      - "3000:3000"
    environment:
      GF_SECURITY_ADMIN_PASSWORD: ${GRAFANA_ADMIN_PASSWORD:?set GRAFANA_ADMIN_PASSWORD}
  jaeger:
    image: jaegertracing/jaeger:2.21.0
    ports:
      - "16686:16686"
      - "4318:4318"

Source: Spring Boot: Tracing, for the OTLP endpoint that Jaeger receives; the compose file itself is this page’s own local setup.

prometheus.yml

scrape_configs:
  - job_name: docsassistant
    metrics_path: /actuator/prometheus
    scrape_interval: 15s
    authorization:
      type: Bearer
      credentials_file: /etc/prometheus/scrape-token
    static_configs:
      - targets: ["host.docker.internal:8080"]

Source: Prometheus: Configuration.

The compose file passed docker compose config, and promtool check config accepted the Prometheus file once the token file existed. The images were not started as a stack. Once Prometheus is added to Grafana as a data source, four panels answer most questions about a model-backed service. The recording and alerting rules below were checked with promtool check rules; they were not evaluated against live traffic.

rules.yml: the queries behind the panels and two alerts

groups:
  - name: docsassistant-llm
    rules:
      - record: docsassistant:llm_tokens:rate5m
        expr: sum by (gen_ai_request_model, gen_ai_token_type) (rate(gen_ai_client_token_usage_total{gen_ai_token_type!="total"}[5m]))
      - record: docsassistant:llm_latency_seconds:p95
        expr: histogram_quantile(0.95, sum by (le, gen_ai_request_model) (rate(gen_ai_client_operation_seconds_bucket[5m])))
      - alert: LlmErrorRateHigh
        expr: |
          sum(rate(gen_ai_client_operation_seconds_count{error!="none"}[5m]))
            / sum(rate(gen_ai_client_operation_seconds_count[5m])) > 0.05
        for: 10m
        labels:
          severity: page
        annotations:
          summary: More than 5 percent of model calls are failing
      - alert: LlmCostBurn
        expr: sum(increase(docsassistant_llm_cost_total[1h])) > 5
        for: 5m
        labels:
          severity: ticket
        annotations:
          summary: Estimated model spend in the last hour is above the budget

Source: Prometheus: Recording rules.

Table 4. Dashboard panels for DocsAssistant
Panel	Question it answers
Tokens per minute by model and type	How fast are we spending context? A jump in `input` without a jump in traffic means prompts or retrieved context grew
p50 and p95 latency of `gen_ai.client.operation`	Is the model slow, or is it something around the model? Compare with `spring_ai_chat_client_seconds` to see the advisors' share
Error ratio (`error!="none"`)	Provider outages, rate limits (`429`) and timeouts, before users report them
Estimated cost per hour	The custom counter below, so a runaway loop shows up before the invoice does

Adding your own tags and metrics

Two extension points cover most needs: an ObservationFilter changes every observation, and an ObservationHandler reacts to a finished one. Both are picked up by Spring Boot as beans.

FeatureTagFilter.java: a low-cardinality tag on every Spring AI observation

package com.example.docsassistant;

import io.micrometer.common.KeyValue;
import io.micrometer.observation.Observation;
import io.micrometer.observation.ObservationFilter;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/** Adds a low-cardinality key to every Spring AI observation so dashboards can split traffic by feature. */
@Configuration(proxyBeanMethods = false)
class FeatureTagFilter {

    @Bean
    ObservationFilter docsAssistantFeatureFilter() {
        return (Observation.Context context) -> {
            String name = context.getName();
            if (name != null && (name.startsWith("gen_ai.") || name.startsWith("spring.ai."))) {
                context.addLowCardinalityKeyValue(KeyValue.of("docsassistant.feature", "chat"));
            }
            return context;
        };
    }
}

Source: Micrometer: Observation.

Keep custom tags low-cardinality. A user ID, a prompt or a conversation ID as a tag would create one time series per value and overwhelm Prometheus; such values belong on spans (high-cardinality keys) or nowhere.

The second extension turns token usage into money. With and the token counts of a call and and the prices per million tokens, the estimated cost of a call is

For the 26 input and 12 output tokens of the test run at illustrative prices of 0.20 and 0.60 per million tokens, . The prices are configuration, not values taken from any provider; set them from your contract, and leave them at zero for a local model.

LlmCostMetrics.java: an estimated-cost counter from the usage in each response

package com.example.docsassistant;

import io.micrometer.core.instrument.MeterRegistry;
import io.micrometer.observation.ObservationHandler;
import org.springframework.ai.chat.metadata.Usage;
import org.springframework.ai.chat.observation.ChatModelObservationContext;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/** Turns the token usage of every chat model call into an estimated cost, in a counter Prometheus can graph. */
@Configuration(proxyBeanMethods = false)
class LlmCostMetrics {

    @Bean
    ObservationHandler<ChatModelObservationContext> costHandler(MeterRegistry registry,
            @Value("${docsassistant.pricing.input-per-million:0.0}") double inputPerMillion,
            @Value("${docsassistant.pricing.output-per-million:0.0}") double outputPerMillion) {
        return new ObservationHandler<>() {

            @Override
            public boolean supportsContext(io.micrometer.observation.Observation.Context context) {
                return context instanceof ChatModelObservationContext;
            }

            @Override
            public void onStop(ChatModelObservationContext context) {
                if (context.getResponse() == null || context.getResponse().getMetadata() == null) {
                    return;
                }
                Usage usage = context.getResponse().getMetadata().getUsage();
                if (usage == null || usage.getPromptTokens() == null || usage.getCompletionTokens() == null) {
                    return;
                }
                double cost = (usage.getPromptTokens() * inputPerMillion
                        + usage.getCompletionTokens() * outputPerMillion) / 1_000_000.0;
                // only low-cardinality tags: the model name, never a user, prompt or conversation id
                registry.counter("docsassistant.llm.cost", "model", modelName(context)).increment(cost);
            }
        };
    }

    private static String modelName(ChatModelObservationContext context) {
        var options = context.getRequest().getOptions();
        return options != null && options.getModel() != null ? options.getModel() : "unknown";
    }
}

Source: Micrometer: Observation.

MetricsExportTest.java: the Prometheus scrape of a Spring Boot context

package com.example.docsassistant;

import static org.assertj.core.api.Assertions.assertThat;
import static org.springframework.security.test.web.servlet.request.SecurityMockMvcRequestPostProcessors.httpBasic;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;

import com.github.tomakehurst.wiremock.WireMockServer;
import com.github.tomakehurst.wiremock.core.WireMockConfiguration;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.micrometer.metrics.test.autoconfigure.AutoConfigureMetrics;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc;
import org.springframework.context.annotation.Import;
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import org.springframework.test.web.servlet.MockMvc;

@SpringBootTest(classes = MetricsExportTest.App.class, properties = {
        "spring.ai.model.chat=ollama",
        "spring.ai.model.embedding=none",
        "spring.ai.model.moderation=none",
        "spring.ai.model.image=none",
        "spring.ai.model.audio.speech=none",
        "spring.ai.model.audio.transcription=none",
        "spring.ai.ollama.init.pull-model-strategy=never",
        "spring.ai.ollama.chat.model=llama3.1:8b",
        "management.endpoints.web.exposure.include=health,metrics,prometheus",
        "spring.security.user.name=ops",
        "spring.security.user.password=pw",
        "management.metrics.distribution.percentiles-histogram.gen_ai.client.operation=true",
        "docsassistant.pricing.input-per-million=0.20",
        "docsassistant.pricing.output-per-million=0.60"
})
@AutoConfigureMockMvc
@AutoConfigureMetrics
class MetricsExportTest {

    @EnableAutoConfiguration(excludeName = {
            "org.springframework.ai.vectorstore.pgvector.autoconfigure.PgVectorStoreAutoConfiguration",
            "org.springframework.boot.jdbc.autoconfigure.DataSourceAutoConfiguration",
            "org.springframework.ai.model.chat.memory.repository.jdbc.autoconfigure.JdbcChatMemoryRepositoryAutoConfiguration",
            "org.springframework.boot.jdbc.autoconfigure.JdbcTemplateAutoConfiguration",
            "org.springframework.boot.hibernate.autoconfigure.HibernateJpaAutoConfiguration"
    })
    @Import({ LlmCostMetrics.class, FeatureTagFilter.class })
    static class App {
    }

    static WireMockServer ollama;

    @BeforeAll
    static void startOllama() {
        ollama = new WireMockServer(WireMockConfiguration.options().dynamicPort());
        ollama.start();
        OllamaStubs.chatReply(ollama, "Use MatrixInverse.", 26, 12);
    }

    @AfterAll
    static void stopOllama() {
        ollama.stop();
    }

    @DynamicPropertySource
    static void ollamaUrl(DynamicPropertyRegistry registry) {
        registry.add("spring.ai.ollama.base-url", () -> ollama.baseUrl());
    }

    @Autowired
    ChatClient.Builder builder;

    @Autowired
    MockMvc mvc;

    @Test
    void prometheusEndpointExposesTheGenAiMetrics() throws Exception {
        this.builder.build().prompt().user("How do I invert a matrix?").call().content();

        String scrape = this.mvc.perform(get("/actuator/prometheus").with(httpBasic("ops", "pw")))
                .andExpect(status().isOk()).andReturn().getResponse().getContentAsString();

        assertThat(scrape).contains("gen_ai_client_operation_seconds_bucket",
                "gen_ai_client_token_usage_total", "gen_ai_client_operation_seconds_count",
                "docsassistant_llm_cost_total", "docsassistant_feature=\"chat\"",
                "gen_ai_token_type=\"input\"");
    }
}

Source: Spring Boot: Metrics.

The test passed and the scrape contained docsassistant_llm_cost_total{model="llama3.1:8b"} 1.24E-5, the value computed by hand above, and docsassistant_feature="chat" on the Spring AI series. The counter records estimates from the usage the provider reported. Providers that do not report usage (some local servers) produce no cost, and the handler skips those responses.

Traces

Each observation is also a span. The tool-call flow is the interesting one: a ChatClient request that needs a tool contains two model calls and one tool execution. The test below runs that flow against a stub Ollama that first asks for a tool and then answers, and records every observation that finished.

ToolObservationTest.java: a tool call as observations

package com.example.docsassistant;

import static com.github.tomakehurst.wiremock.client.WireMock.aResponse;
import static com.github.tomakehurst.wiremock.client.WireMock.post;
import static com.github.tomakehurst.wiremock.client.WireMock.urlEqualTo;
import static org.assertj.core.api.Assertions.assertThat;

import com.github.tomakehurst.wiremock.WireMockServer;
import com.github.tomakehurst.wiremock.core.WireMockConfiguration;
import io.micrometer.observation.Observation;
import io.micrometer.observation.ObservationHandler;
import io.micrometer.observation.ObservationRegistry;
import java.util.ArrayList;
import java.util.List;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.client.advisor.ToolCallingAdvisor;
import org.springframework.ai.chat.client.advisor.observation.DefaultAdvisorObservationConvention;
import org.springframework.ai.chat.client.observation.DefaultChatClientObservationConvention;
import org.springframework.ai.model.tool.DefaultToolCallingManager;
import org.springframework.ai.ollama.OllamaChatModel;
import org.springframework.ai.ollama.api.OllamaApi;
import org.springframework.ai.ollama.api.OllamaChatOptions;
import org.springframework.ai.tool.annotation.Tool;

class ToolObservationTest {

    static class ClockTool {
        @Tool(description = "Return the current documentation build id")
        String buildId() {
            return "build-2026-09-29";
        }
    }

    private WireMockServer ollama;
    private final List<String> seen = new ArrayList<>();
    private final ObservationRegistry observations = ObservationRegistry.create();

    @BeforeEach
    void start() {
        this.observations.observationConfig().observationHandler(new ObservationHandler<Observation.Context>() {
            public boolean supportsContext(Observation.Context c) { return true; }
            public void onStop(Observation.Context c) { seen.add(c.getName() + " | " + c.getContextualName()); }
        });
        this.ollama = new WireMockServer(WireMockConfiguration.options().dynamicPort());
        this.ollama.start();
    }

    @AfterEach
    void stop() {
        this.ollama.stop();
    }

    @Test
    void toolCallsGetTheirOwnObservation() {
        // first request: the model asks for the tool; second request (it carries the tool result): the final text
        this.ollama.stubFor(post(urlEqualTo("/api/chat"))
                .inScenario("tool").whenScenarioStateIs("Started").willSetStateTo("answered")
                .willReturn(aResponse().withHeader("Content-Type", "application/json").withBody("""
                        {"model":"llama3.1:8b","created_at":"2026-09-29T10:00:00Z",
                         "message":{"role":"assistant","content":"",
                           "tool_calls":[{"function":{"name":"buildId","arguments":{}}}]},
                         "done":true,"done_reason":"stop","prompt_eval_count":30,"eval_count":8}""")));
        this.ollama.stubFor(post(urlEqualTo("/api/chat"))
                .inScenario("tool").whenScenarioStateIs("answered")
                .willReturn(aResponse().withHeader("Content-Type", "application/json").withBody("""
                        {"model":"llama3.1:8b","created_at":"2026-09-29T10:00:01Z",
                         "message":{"role":"assistant","content":"The build is build-2026-09-29."},
                         "done":true,"done_reason":"stop","prompt_eval_count":45,"eval_count":10}""")));

        OllamaChatModel model = OllamaChatModel.builder()
                .ollamaApi(OllamaApi.builder().baseUrl(this.ollama.baseUrl()).build())
                .options(OllamaChatOptions.builder().model("llama3.1:8b").build())
                .observationRegistry(this.observations)
                .build();

        String answer = ChatClient.builder(model, this.observations, new DefaultChatClientObservationConvention(),
                new DefaultAdvisorObservationConvention(),
                ToolCallingAdvisor.builder().toolCallingManager(
                        DefaultToolCallingManager.builder().observationRegistry(this.observations).build()))
                .build().prompt().user("Which build is this?")
                .tools(new ClockTool()).call().content();

        assertThat(answer).contains("build-2026-09-29");
        assertThat(this.seen).contains("spring.ai.tool | execute_tool buildId", "spring.ai.chat.client | spring_ai chat_client");
        assertThat(this.seen.stream().filter(o -> o.startsWith("gen_ai.client.operation")).count()).isEqualTo(2);
    }
}

Source: Spring AI: Observability.

The observations finished in this order: gen_ai.client.operation | chat llama3.1:8b, spring.ai.advisor | call, spring.ai.tool | execute_tool buildId, a second model call, the second call advisor, spring.ai.advisor | tool _calling, and last spring.ai.chat.client | spring_ai chat_client. The text after the bar is the span name. As a trace this is:

flowchart TD A["spring_ai chat_client"] --> B["advisor: Tool Calling Advisor"] B --> C1["advisor: call"] C1 --> D1["chat llama3.1:8b"] B --> E["execute_tool buildId"] B --> C2["advisor: call"] C2 --> D2["chat llama3.1:8b"]

Read it as the answer to two questions. If the request is slow, the widest child says whether the time went to a model call or to a tool. If the answer is wrong after a tool ran, execute_tool <name> is the span that belongs to that tool, and with content switched on its arguments and result are attributes of the span. The span name execute_tool <name> is new in 2.0; the 1.x name was tool_call <name>.

Three limits from the reference apply. Streaming requests are observed, but a tool executed while streaming can appear as a span detached from the ChatClient span; the OpenAI and Anthropic HTTP spans for a stream are recorded but not parented under the chat-model span; and the servlet thread and the reactive threads do not share the security or tracing context unless context propagation is on (see Streaming & Reactive, which verified this for the security context). Check the parent of your tool spans in a streaming request before relying on them. For agents, where a single user request contains many such loops, see Observability for Agents.

The exporter needs nothing Spring AI specific. Start the compose file above, run DocsAssistant with the OTLP endpoint, ask a question that needs searchDocs, and search Jaeger at http://localhost:16686 for the service. This step was not run here; the span names are those of the observations captured above.

Prompt and completion logging, and privacy

Prompts and completions are big and often personal, so none of them is exported by default. Spring AI has switches that log them, and one that puts tool data on spans. Each is off by default:

Table 5. Content switches (all default to false)
Property	What it exposes
`spring.ai.chat.observations.log-prompt`	The messages sent to the model, logged by `ChatModelPromptContentObservationHandler`
`spring.ai.chat.observations.log-completion`	The model’s reply, logged by `ChatModelCompletionObservationHandler`
`spring.ai.chat.observations.include-error-logging`	Errors from the model call
`spring.ai.chat.client.observations.log-prompt`, `spring.ai.chat.client.observations.log-completion`	The same, at the `ChatClient` level
`spring.ai.tools.observations.include-content`	Tool arguments and results as span attributes, which reach the tracing backend
`spring.ai.vectorstore.observations.log-query-response`	Retrieved passages
`spring.ai.image.observations.log-prompt`	Image prompts

With the first two on, a test that asked "My secret question" produced two log lines from the handlers, each with the trace and span identifiers in the log prefix so a log line can be tied to its trace:

Log output with the chat switches on (abbreviated)

INFO ... [<trace id>-<span id>] ChatModelPromptContentObservationHandler : Chat Model Prompt Content:
["My secret question"]

Source: Spring AI: Observability.

That is the point of the warning in the reference: anything a user typed is now in your log files, and, with include-content, tool results such as retrieved documentation or issue drafts are in your trace store. Treat these switches as you would a database dump:

leave them off in production and turn them on for one environment or one incident, with an owner and an end date;
send them to a log sink with restricted access and a short retention, not to the general application log;
remember that prompts include what retrieval and tools added, not only what the user typed, so a permission-filtered retrieval (see Security & Guardrails) does not make the logs safe;
redact or hash identifiers before logging, and note the legal basis and retention if the prompts can contain personal data.

The switches were verified with a test that captured the console output of a Spring Boot context (PromptLoggingTest): with them on, the prompt and the completion appeared in the log; the metrics run above had them off and contained neither.

Operating notes

Cardinality. Do not add the model name from user input, a conversation ID or a user name as a metric tag.
Sampling. Trace sampling applies to the whole request. A rate of 0.1 keeps one request in ten, so tune it per environment and keep errors by tail sampling in the collector if you need every failed request.
Cost of observing. The observations add microseconds next to a model call that takes seconds. The expensive part is the content switches, which serialize whole prompts.
Provider coverage. Check that the model class for your provider is instrumented before you build a dashboard on it. Embedding instrumentation covers only some providers (Mistral AI, Ollama and OpenAI in the reference), and image instrumentation only OpenAI.
Beyond metrics. Quality, drift and cost governance over weeks are the subject of LLMOps: Drift & Regression Monitoring and LLMOps: Cost Management. Automatic quality checks are on Evaluation, and deployment settings on Production & Deployment.

Where older material differs

Table 6. Older observability material and the 2.0 equivalent
Older text	Spring AI 2.0
Tool span named `tool_call <name>`	`execute_tool <name>`, with the tag `gen_ai.operation.name=execute_tool` and the new key `spring.ai.tool.type`
Internal tool execution inside each `ChatModel`	The tool loop runs in `ToolCallingAdvisor`, so a tool call is a child of an advisor span, as in the trace above
`_active_count` gauges in the metric reference	`…_active_seconds_count` in the Prometheus exposition of Spring Boot 4.1.1

Observability

What Spring AI observes

Turning it on

Seeing the observations in a unit test

What Prometheus sees

Prometheus, Grafana and Jaeger locally

Adding your own tags and metrics

Traces

Prompt and completion logging, and privacy

Operating notes

Where older material differs

References