Message Oriented Middleware — RabbitMQ — Part 1

This article is originally published at https://www.learncsdesign.com

Messaging that just works — RabbitMQ

Inan era of distributed computing, services must communicate and interact. There are several mechanisms by which these services can communicate.

1. File-Based Interaction — In this interaction, one service creates a file that must be processed by another service into a well-known network location that can be accessed by both services. In that case, the new file is automatically fetched by another service and processed as soon as it appears in that location.

File-Based Interaction

2. Shared Database Interaction — One of the services adds records to a shared database table in this interaction. Another service watches for new entries in that table, and if there are any unprocessed entries, they are fetched, processed, and updated in the table.

Shared Database Interaction

3. Direct Connection Interaction — Through this interaction, services connect and send messages directly to one another. Pipe connections or TCP/IP connections can be used for this purpose. Payloads can be binary, like protocol buffers, or text-based, like XML or JSON.

Direct Connection Interaction

4. Asynchronous Messaging — During this interaction, services send messages to each other using middleware, which is mostly known as Message Brokers or Message Buses. A producer publishes messages to these Message Brokers, and a consumer consumes them. This approach has many advantages, first of all, it decouples publishers from consumers. In addition, it provides a reliable means of asynchronous processing and communication without the need to poll. Furthermore, it provides a way to implement horizontally scalable architectures.

Asynchronous Messaging

Introducing RabbitMQ

1. The RabbitMQ message broker receives messages from producers and routes them to one or more consumers. It’s an open-source broker written in Erlang that implements Advanced Message Queuing Protocol (AMQP). It supports many messages protocols, both directly and through plugins, such as AMQP 0–9–1, STOMP, MQTT, and AMQP 1.0.

RabbitMQ Message Broker

Installation on Docker

Instructions are available https://hub.docker.com/_/rabbitmq to download and run the docker image of RabbitMQ.

Pull the image of RabbitMQ with below command
$ docker pull rabbitmq:3-managementStart a RabbitMQ container listening on default port of 5672
$ docker run -d --hostname my-rabbit --name some-rabbit -p 15672:15672 -p 5672:5672 rabbitmq:3-managementVerify docker started, running below command will display
$ docker logs some-rabbitStarting broker...
[info] <0.222.0>  node           : rabbit@my-rabbit
[info] <0.222.0>  home dir       : /var/lib/rabbitmq
[info] <0.222.0>  config file(s) : /etc/rabbitmq/conf.d/10-default-guest-user.conf
[info] <0.222.0>                 : /etc/rabbitmq/conf.d/management_agent.disable_metrics_collector.conf
[info] <0.222.0>  cookie hash    : 7ni8xRk4CF2Jgs7NTiXXgQ==
[info] <0.222.0>  log(s)         : /var/log/rabbitmq/rabbit@my-rabbit_upgrade.log
[info] <0.222.0>                 : <stdout>
[info] <0.222.0>  database dir   : /var/lib/rabbitmq/mnesia/rabbit@my-rabbit

Verify management plugin for RabbitMQ is running, open browser URL http://localhost:15672, enter the default username guest and default password as guest.

Overview of RabbitMQ management web interface

RabbitMQ Management panel has 6 tabs as :

1. Overview — Besides general status information for the RabbitMQ service, it also contains statistics about channel and queue operations, along with configured ports that include export and import definitions.

Overview Management Interface

2. Connections — It contains all the connections made between producers and consumers.

Connections Management Interface

3. Channels — There are some applications that require multiple logical connections to the broker. The problem with keeping many TCP connections open at the same time is that it consumes system resources and makes it more difficult to configure firewalls. Multiple channels are multiplexed into a single TCP connection, which can be thought of as lightweight connections.

Channels Management Interface

4. Exchanges — These are the message routers of RabbitMQ. Different exchanges are created by default, such as direct, fanout, headers, and topics.

Exchanges Management Interface

5. Queues — The message queue in which actual messages are stored.

Queues Management Interface

6. Admin — With this panel, one can manage users, virtual hosts, feature flags, policies, limits, and clusters.

Admin Management Interface

Messaging Concepts

By understanding the concepts of messaging, we can get a better understanding of the building blocks of the messaging system.

1. Message — Every message has routing information along with the actual payload, which is any information or data to be transmitted between services.
2. Producer/Publisher — Service that creates messages and sends them to a message broker.
3. Consumer/Receiver — A service that receives messages from a message broker and processes them.
4. Message Queue — Queues are lists of messages sent by producers.
5. Message Broker — This is the middleware that transmits messages from senders to receivers.
6. Exchanges/Routers — The business logic of the message broker determines which messages should be sent to which queue, based on the configuration.
7. Connection — The TCP connection between producer and message broker, as well as between consumer and message broker.
8. Channel — In a TCP connection, it is a virtual connection. Channels are where the message is actually transmitted; a single TCP connection can have multiple channels.
9. Binding — Describes the relationship between an exchange and a queue. As part of the binding definition, you can specify routing keys and headers that will be used to filter messages that will be sent to the bound queue.

Attributes of Message

1. Routing key — A single or multiple words that are used when distributing a message to the queues.
2. Headers — A collection of key-value pairs that are also used for routing messages and passing additional information.
3. Payload — Actual data that a message carries.
4. Publishing Timestamp — Optional timestamp provided by the publisher.
5. Expiration — TTL of the message in a queue.
6. Delivery Mode — It can be persistent or transient. Persistent messages are written to disk and if the RabbitMQ service restarts, they are available whereas transient messages are lost.
7. Priority — Priority of the message between 0 ~ 255.
8. Message Id — Optional unique message id set by the publisher to distinguish a message.
9. Correlation Id — Optional id for matching a request and a response in RPC scenarios.
10. Reply To — Optional queue or exchange name used in request-response scenarios.

Attributes of Queue

1. Name — Unique name of queue max up to 255 characters UTF-8.
2. Durable — Whether to preserve or delete the queue when RabbitMQ restarts.
3. Auto Delete — Whether to delete this queue if no one is subscribed to it.
4. Exclusive — Used only by one connection and deleted when the connection is closed.
5. Max Length — Maximum number of messages in a queue. Overflow behavior can be set as dropping the oldest or rejecting the newest messages.
6. Max Priority — Maximum number of priority values that this queue supports.
7. Message TTL — Lifetime of each message that is added to this queue. If both message and queue have a TTL value, the lowest one will be chosen.
8. Dead-letter Exchange — Name of the exchange that expired or dropped message with be automatically sent.
9. Binding Configuration — Associations between queues and exchanges. A queue must be bound to exchange, in order to receive messages from it.

Attributes of Exchange

1. Name — Unique name of the exchange.
2. Type — Types of the exchange, like fanout, direct, topic, and headers.
3. Durability — Same as queue durability, durable exchange survives after RabbitMQ service restarts.
4. Auto Delete — If true, the exchange will be deleted when there is no bound queue left.
5. Internal — Internal exchanges can only receive messages from other exchanges.
6. Alternate exchange — The name of the exchange where unroutable messages will be sent.

In this post, we discussed different ways services can communicate with one another. RabbitMQ was discussed along with message concepts and attributes of messages, queues, and exchanges.

I will cover more about the different types of exchanges in Part — 2 and show real-life examples of RabbitMQ at work.

If you like the post, don’t forget to clap. If you’d like to connect, you can find me on LinkedIn.