Observability: Health, Logging, Metrics, Tracing

@Liveness answers "should this instance be restarted" — a failing liveness check should mean the process is in a state it cannot recover from on its own (deadlocked, unrecoverable internal error), not merely "temporarily overloaded"; a flapping liveness probe causes an orchestrator to restart-loop an instance that would otherwise have recovered. @Readiness answers "should traffic be routed to this instance right now" — it can legitimately flip between up and down repeatedly (a downstream dependency blips) without the process being restarted. @Startup gates readiness until an initial startup sequence (cache warm-up, initial data load) completes, and is checked once before the instance is considered ready for the first time.

Each responds 200 with a JSON body listing every check’s individual status when all checks in the group pass, and 503 the moment any one of them reports DOWN. The quarkus-kubernetes extension reads these same paths to generate the Deployment manifest’s own liveness/readiness probes automatically:

which is also the reason the liveness/readiness distinction above matters operationally, not just semantically — a check registered under the wrong annotation changes what Kubernetes actually does when it fails (restart the pod vs. pull it out of the Service’s endpoint list).

quarkus-info exposes /q/info, a build/runtime metadata endpoint distinct from health — it never reports a failure, it just describes the running instance: build timestamp, Git commit (when quarkus.info.git.enabled=true and the module was built inside a Git working copy), OS and Java runtime details, and any extension-contributed section (a datasource’s JDBC URL host, without credentials). It is useful for confirming exactly which build and commit is running in a given environment without cross-referencing a separate deployment log:

Quarkus logging is configured per category (usually a package or class name), with quarkus.log.level setting the root:

quarkus.log.min-level caps the lowest level any category can ever be raised to (at build time, categories below it are compiled out entirely for performance in native builds), so a category-level override above it has no effect unless the floor is lowered too. Per-category levels set via quarkus.log.category.* are runtime-reloadable in dev mode and, in a packaged application, mutable without a restart via the same configuration mechanism (an updated environment variable or a mounted config file picked up on the next config refresh), which is the practical way to raise a specific category’s verbosity in production for a short window without redeploying.

quarkus-logging-json switches any of these handlers to structured JSON output (one JSON object per log line, including MDC and, when quarkus-opentelemetry is present, the current trace/span ID) instead of the quarkus.log.console.format pattern — structured output is what makes centralised log aggregation practical, since the collector can then index individual fields instead of regex-parsing free text.

Quarkus does not ship a Graylog/Logstash/Fluentd client of its own for the general case — the standard pattern is: emit JSON (quarkus.log.console.json=true), let the container runtime capture stdout, and let a sidecar or node-level agent (Filebeat, Fluent Bit, the Fluentd daemonset) ship it to the aggregator, which is what most Kubernetes logging stacks already do regardless of the application inside the pod. quarkus-logging-gelf is the one extension that talks a specific backend’s wire protocol directly — the Graylog Extended Log Format — useful when there is no log-shipping sidecar available:

With quarkus-opentelemetry present, every log record’s MDC automatically carries traceId/spanId for the request it was emitted during, so a JSON log line and a distributed trace can be correlated in whichever backend receives both (see OpenTelemetry). Quarkus can additionally export log records themselves as OTLP log records, alongside traces and metrics, rather than (or in addition to) writing them to a file/console handler:

@Counted/@Timed (io.micrometer.core.annotation) instrument a method declaratively without touching its body; injecting MeterRegistry directly is the escape hatch for a metric whose tags depend on runtime data (here, a channel tag) that an annotation’s static value cannot express.

Once quarkus-micrometer is present, several meter families are registered automatically with no code: HTTP server request metrics (http.server.requests, tagged by method/status/URI template), JVM metrics (heap and non-heap memory, garbage-collection pause time, live thread count), and datasource metrics for Agroal-managed pools (agroal.active.count, agroal.available.count, agroal.acquire.count). Each binder can be disabled individually if it is not wanted:

micrometer-registry-prometheus exposes every registered meter, in Prometheus’s own text exposition format, at /q/metrics — see the Prometheus reference for how a Prometheus server scrapes that endpoint, the resulting metric types, and building alerting/dashboards on top of it. Distinguishing this from the OpenTelemetry metrics pipeline below matters: the Prometheus registry is pull-based (Prometheus scrapes /q/metrics on its own schedule) while OTLP metrics export is push-based (Quarkus sends metrics to a collector on its own interval) — a project can run either or, via the bridge described next, both from the same set of meters.

quarkus-opentelemetry instruments the extensions most likely to matter for a distributed trace without any code change: Quarkus REST server and client calls, JDBC statements (through the Agroal datasource), Kafka producers/consumers, and gRPC calls each create their own span automatically, and the trace context (the current trace/span ID) propagates across each of those boundaries — an inbound HTTP request’s trace ID is the same one attached to the JDBC query and the outbound REST client call it triggers, and the same one Kafka carries in message headers to a downstream consumer. Propagation across reactive/async hops (a Uni continuation, an event-loop-to-worker-thread handoff, a @RunOnVirtualThread method) is handled by the same context-propagation machinery Quarkus already uses for CDI request-scoped beans, so a span started before an asynchronous boundary still shows up as the parent of spans created after it.

quarkus.otel.traces.sampler controls what fraction of traces are actually recorded and exported — parentbased_always_on (the default; export everything, but never override a sampling decision already made by an upstream caller), parentbased_traceidratio with quarkus.otel.traces.sampler.arg as the ratio (shown above), or always_off to disable trace export entirely while keeping metrics/logs. Consistent sampling across a distributed system relies on every service using a parentbased_* sampler, so a downstream service does not independently decide to drop a trace its caller chose to keep.

@WithSpan (io.opentelemetry.instrumentation.annotations) is the declarative shortcut for a method whose whole body should be one span, with @SpanAttribute binding parameters onto it; injecting the Tracer directly is needed for anything an annotation cannot express — a span that only covers part of a method, or one whose attributes depend on values not known until partway through execution.

quarkus-micrometer-opentelemetry re-exports every meter already recorded through MeterRegistry (including the built-in HTTP/JVM/datasource binders and any @Counted/@Timed metric) as OpenTelemetry metrics, so a project that has already instrumented its code with Micrometer does not need to duplicate that work with the OTel Metrics API just to get metrics into an OTLP-based backend:

With the bridge active, Micrometer remains the instrumentation API used in application code; OpenTelemetry becomes purely an additional export path for the same data, alongside (or instead of) the Prometheus registry.

In dev mode and for @QuarkusTest, adding quarkus-opentelemetry (or quarkus-micrometer-opentelemetry) without configuring quarkus.otel.exporter.otlp.endpoint triggers Dev Services to start a containerised Grafana OTel-LGTM stack automatically — an all-in-one image bundling an OTLP collector, Loki (logs), Grafana (the UI), Tempo (traces), and Mimir/Prometheus (metrics) — and points the application’s exporters at it, with the Grafana UI’s URL printed to the console/Dev UI. This gives full local trace/metric/log exploration with zero backend setup during development; it is a development-time-only convenience and plays no part in a production deployment, which should point quarkus.otel.exporter.otlp.endpoint at a real collector as shown earlier.

Independently of any Quarkus extension, the JDK’s own Java Flight Recorder can profile a running Quarkus application — JVM-level events (GC pauses, allocation, thread contention, and, for virtual threads, pinning via jdk.VirtualThreadPinned, see Virtual Threads) with very low overhead, viewable afterwards in JDK Mission Control:

JFR is complementary to, not a replacement for, the application-level signals above: it answers "what was the JVM itself doing" during an incident, where Micrometer/OpenTelemetry answer "what was the application doing" at the request/span level.