Messaging Reference

Getting Started with Messaging — the problem messaging solves, synchronous request/response vs. asynchronous messaging, brokered vs. brokerless, the shared vocabulary mapped across Kafka and RabbitMQ, running both brokers locally, and a first send/receive with each plain Java client.

Messaging Models and Patterns — point-to-point and publish/subscribe as the two base models, request/reply on top of them, and the Enterprise Integration Patterns the rest of this section refers to, each mapped onto its Kafka and RabbitMQ realisation.

Delivery Guarantees — at-most-once, at-least-once and exactly-once from both the producer and the consumer side, acknowledgements and redelivery, duplicates and idempotent consumers, ordering, and durability, with a configuration comparison table.

Messaging Protocols — AMQP 0-9-1, AMQP 1.0, the Kafka binary protocol, MQTT, STOMP, the RabbitMQ Stream protocol, WebSocket transports, and JMS/Jakarta Messaging, with a protocol-by-broker matrix.

Kafka Architecture — events, topics, partitions and offsets, segments and the on-disk log, brokers, KRaft controllers and the metadata log, replication, retention, log compaction, and tiered storage.

Kafka Producers — the Producer API, the record accumulator and batching, the partitioner, acks and min.insync.replicas, the idempotent producer, retries, compression, and a plain-Java producer example.

Kafka Consumers — consumer groups, the poll loop, partition assignment and rebalancing (including the broker-driven KIP-848 protocol), static membership, offsets and commits, and a plain-Java consumer example.

Kafka Share Groups — Queues for Kafka (KIP-932): share groups vs. consumer groups, record acquisition locks, per-record outcomes and lock renewal, delivery counts and poison-message protection, and a plain-Java ShareConsumer example.

Kafka Reliability and Exactly-Once — the reliability toolkit end to end — replication factor, min.insync.replicas, producer and consumer discipline — then Kafka transactions (transactional.id, read_committed) and what exactly-once does and does not cover.

Kafka Streams — topology, KStream/KTable/GlobalKTable, stream-table duality, windowing, joins, state stores and changelog topics, exactly-once, and testing with TopologyTestDriver.

Kafka Connect — source vs. sink connectors, standalone vs. distributed workers, tasks, converters, single message transforms, error handling and the sink dead-letter queue, and the transactional-outbox pattern via CDC.

Kafka Serialization and Schemas — serializers/deserializers as a contract, bytes vs. String vs. JSON vs. Avro/Protobuf/JSON Schema, and the case for a schema registry.

Kafka Operations and Monitoring — the CLI tools, partition-count decisions, dynamic configuration, key broker metrics and consumer lag, quotas, and geo-replication with MirrorMaker 2.

Kafka Security — listeners and security protocols, TLS encryption and mTLS authentication, SASL mechanisms, ACLs, and multi-tenancy.

RabbitMQ Architecture and the AMQP Model — connections, channels, virtual hosts, exchanges, bindings, queues and their properties, the Erlang/BEAM runtime, and clustering basics.

RabbitMQ Queue Types — classic, quorum and stream queues (and JMS queues), how to choose between them, and the migration path off classic mirrored queues.

RabbitMQ Publishing — publishing to an exchange, the mandatory flag and returns, alternate exchanges, publisher confirms, message properties, and publishing to streams.

RabbitMQ Consuming — push vs. pull consumption, acknowledgement modes, prefetch (QoS), consumer priorities and exclusivity, and consuming from streams.

RabbitMQ Routing Patterns — the six official tutorials as reusable patterns — work queues, publish/subscribe, routing, topics, RPC — plus headers-exchange routing and exchange-to-exchange bindings.

RabbitMQ Reliability, Dead-Lettering and TTL — connection recovery, confirms and manual acks, dead-letter exchanges, per-queue and per-message TTL, queue length limits, priorities, and retry topologies.

RabbitMQ and AMQP 1.0 — the v2 address format, sessions and links vs. channels, credit-based flow control, settlement outcomes, and the new amqp-client Java library.

RabbitMQ Clustering, Federation and Shovel — clustering and the Khepri metadata store, federation, shovel, and when to use each versus a single cluster.

RabbitMQ Operations, Monitoring and Security — the management UI and HTTP API, CLI tools, policies, Prometheus/Grafana monitoring, and security (virtual hosts, permissions, TLS, OAuth 2.0/OIDC).

Kafka vs. RabbitMQ — a dimension-by-dimension comparison — design origin, data model, routing, consumer model, delivery guarantees, ordering, retention, per-message features, protocols, security, operations and runtime — closing with a decision table.

Messaging Patterns in Practice — the cross-broker patterns every production system needs: transactional outbox, idempotent consumers, retry and dead-letter handling, ordering keys and hot partitions, backpressure, correlation IDs, schema evolution, and saga choreography vs. orchestration.

Spring Boot Messaging: Overview — the Spring messaging landscape and how to choose between Spring for Apache Kafka, Spring AMQP, Spring Cloud Stream and Spring Integration, the starters, and what this repository already documents elsewhere.

Spring for Apache Kafka — topic provisioning, request/reply, non-blocking retries with @RetryableTopic, share consumers, exactly-once, and Kafka Streams — only what Messaging with Kafka does not already cover.

Spring AMQP — spring.rabbitmq.* configuration, topology as beans, @RabbitListener, retry and dead-letter configuration, request/reply, streams, and the AMQP 1.0 path.

Spring Cloud Stream — the binder abstraction and the functional programming model, consumer groups and partitioning, StreamBridge, and the Kafka and RabbitMQ binder specifics.

Other Spring Messaging Options — application events, JMS/Jakarta Messaging, Spring Integration, WebSocket/STOMP, and Apache Pulsar.

Testing Messaging — Testcontainers with @ServiceConnection, @EmbeddedKafka, spring-rabbit-test, the Spring Cloud Stream test binder, and AsyncAPI contract testing.

Messaging with SmallRye Reactive Messaging — the equivalent mapping of this section’s concepts onto Quarkus, via channels, @Incoming/@Outgoing, and the Kafka, RabbitMQ, AMQP 1.0, Pulsar and JMS connectors.

Messaging Cheat Sheet — a one-page, printable summary of the whole section, with a downloadable PDF.