What is a Message Broker?

How applications communicate is becoming an increasingly large challenge. Using messaging middleware simplifies this challenge and allows for common communications infrastructure that grows and scales to meet the most demanding conditions. But communication can happen in many different forms and how messaging middleware provides communication is no different.

One method of communication used by messaging middleware is a server based model that uses a message broker. With a message broker, the source application (producer) sends a message to a server process that can provide data marshaling, routing, message translation, persistence and delivery to all the appropriate destinations (consumers). The defining characteristic of a message broker is that the broker itself is a discrete service. Producers and consumers communicate with the broker using standard or proprietary protocols. The message broker typically provides all the state management and tracking of clients so that individual applications do not need to take on this responsibility and the complexity of message delivery is built into the message broker itself.

There are two basic forms of communications with a message broker:

• Publish and Subscribe (Topics)
• Point-to-Point (Queues)

Publish and Subscribe Messaging

In a publish and subscribe message system, producers send messages on a topic. In this model, the producer is known as a publisher and the consumer is known as a subscriber. One or many publishers can publish on the same topic, and a message from one or many publishers can be received by many subscribers. Subscribers subscribe to topics, and all messages published to the topic are received by all subscribers on the topic. This model provides for simple interest driven delivery based on what topics applications subscribe to.

Point-to-point communication

Point-to-point communications in its simplest form has one producer and one consumer. This style of messaging typically uses a queue to store messages sent by the producer until they are received by the consumer. The message producer sends the message to the queue; the message consumer retrieves messages from the queue and sends an acknowledgement that the message was received. More than one producer can send messages to the same queue, and more than one consumer can retrieve messages from the same queue. When multiple consumers are used each consumer typically receives a portion of the message stream to allow for concurrent processing.

Server-based Message Brokers

Server-based message brokers have many advantages, including centralized processing, centralized message distribution, and centralized data persistence. Examples include many proprietary message brokers, Java Message Service (JMS) implementations, and open source products like Apache Kafka, Apache Pulsar, and Eclipse Mosquitto.

In some cases a decentralized communications model is needed. This "Brokerless" or peer-to-peer approach typically provides a mechanism to support direct communication between applications for use cases where very low latency is required.