MQTT examples with Mosquitto
We propose some scenarios to exemplify MQTT by using the Open source broker Mosquitto and its client tools.
Software installation and configuration
We assume that you are running a GNU/Linux system, e.g. Debian, and that you have installed the corresponding packages, e.g. mosquitto and mosquitto-clients on Debian.
We do not want the broker to be installed and started as a service:
First example
The mosquitto command launches the broker. We add the option "-v" to see all the logs. By default, we do not change the port number of the broker, i.e. 1883.
The mosquitto_sub command starts a client that subscribes to a topic filter. As a first set of options, we use the following ones:
- "-i xxx": the name of the client, e.g. Consumer in our illustrative scripts;
- "-d": to see all the logs;
- "-h xxx": the name of the host running the broker, e.g. localhost in our illustrative scripts;
- "-t xxx": the topic filter to subscribe to.
The mosquitto_pub command starts a client that publishes a message to a topic filter. As a first set of options, we use the following ones:
- "-i xxx": the name of the client, e.g. Producer in our illustrative scripts;
- "-d": to see all the logs;
- "-h xxx": the name of the host running the broker, e.g. localhost in our illustrative scripts;
- "-t xxx": the topic name for routing at the broker;
- "-m xxx": the message to send. In our illustrative scripts, this is a string provided as a shell argument (a shell word).
For testing your installation and as a first example, download the following script, change the mode to be executable, and run it:
When executing the script with the --xterm option, two xterm windows are opened: the first one for running the broker, the second one for running the consumer, the producer being executed in the main terminal in which you execute the script.
Let us provide some explanations. The text displayed in the consumer's terminal is as follows:
The connection is acknowledged by the broker by a CONNACK message, 0 indicating that the connection is accepted.
When subscribing, Mid stands for "message identifier" and we ignore this field in our explanations. The topic being test/#, the consumer is going to receive messages with a topic that has string test as the first word. We are going to see that the # sign means "any number of levels, with whatever string at each level". In addition, by default, the quality of service is 0, meaning "at most one" or "best effort" (see more on QoS below). The SUBSCRIBE message is acknowledged by the broker with sucess (0).
When there is a matching message, the broker sends it to the consumer as a PUBLISH message. The flags "d0, q0, r0, m0" mean: d0 is the duplicate flag that here says that this message is sent for the first time (not resent), q0 is the QoS flag that here says that the QoS is "at most once", r0 is the retain flag that here says "do not retain at the broker" (see more on the retain flag below), and m0 is ? (don't know :-().
Similarly, the terminal of the producer shows the following log:
The connection is acknowledged (CONNACK) with success (0). After sending its PUBLISH message, the producer disconnects (DISCONNECT).
Topic names and topic filters
A producer uses topic names to tag its messages when publishing. A consumer subscribes to a topic filter and receive messages when the topic name of the message matches the topic filter of the subscription.
The topic level separator (forward slash "/" or U+002F) is used to introduce structure into the topic name. If present, it divides the topic name into multiple "topic levels".
Single-level wildcard (the plus sign '+' or U+002B)
A subscription’s topic filter can contain special wildcard characters, which allow a Client to subscribe to multiple topics at once. The plus sign "+" or U+002B is a wildcard character that matches only one topic level. The single-level wildcard can be used at any level in the topic filter, including first and last levels. Where it is used, it MUST occupy an entire level of the filter. It can be used at more than one level in the topic tilter. For example, "sport/tennis/+" matches "sport/tennis/player1" and "sport/tennis/player2", but not "sport/tennis/player1/ranking".
You can experiment these examples with the following script:
Other examples of topic filters with single-level wildcards:
- "+" is valid;
- "sport+" is not valid;
- "sport/+/player1" is valid;
- "/finance" matches "+/+" and "/+", but not "+".
Multi-level wildcard (the number sign '#' or U+0023)
The number sign "#" or U+0023 is a wildcard character that matches any number of levels within a topic. The multi-level wildcard character MUST be specified either on its own or following a topic level separator. In either case, it MUST be the last character specified in the topic filter.
For example, if a client subscribes to "sport/tennis/player1/#", it would receive messages published using these topic names:
- "sport/tennis/player1",
- "sport/tennis/player1/ranking",
- "sport/tennis/player1/score/wimbledon".
You can experiment these examples with the following script:
Other examples of topic filter using multi-level wildcards:
- "sport/#" also matches the singular "sport", since "#" includes the parent level;
- "#" is valid and will receive every application message;
- "sport/tennis/#" is valid;
- "sport/tennis#" is not valid;
- "sport/tennis/#/ranking" is not valid.
Topic alias
Topic alias is new in MQTT v5.
A topic alias is an integer value that is used to identify the topic instead of using the topic name. This reduces the size of the PUBLISH packet, and is useful when the topic names are long and the same topic names are used repetitively within a network connection. The sender (e.g. a publisher or a consumer) decides whether to use a topic alias and chooses the value. It sets a topic alias mapping by including a non-zero length topic name and a topic alias in the PUBLISH packet.
Topic alias mappings exist only within a network connection and last only for the lifetime of that network connection. A receiver (e.g. a broker) MUST NOT carry forward any topic alias mappings from one network connection to another.
How many topic aliases the client (i.e. a publisher or a consumer) can publish on is determined by the broker. Mosquitto is set to 10 topic aliases by default. How many topic aliases the broker can publish on is determined by the client. Most clients use 0 by default, which means that the broker isn’t allowed to use topic aliases when publishing messages to the client. If a client tries to publish using a topic alias and the number of topic aliases has been exceeded then the connection is dropped by the broker.
Here follows a scenario. Please note that option -l of command mosquitto_pub is for reading messages from stdin, splitting separate lines into separate messages. In addition, option -V 5 states that this is MQTT v5 that is going to be used: this option is necessary for the producer because topic alias is a new feature of MQTT v5.
Observe topic name null for the second and the third message.
Qualities of service
In the following examples, we display the messages sent when publishing a message with QoS 0, QoS 1, and QoS 2.
For more explanations, please refer to the slides.
QoS 0 (at most once)
The default quality of service is QoS 0: the message is sent once and that delivery isn’t guaranteed.
QoS 0 is experimented in the following scenario. The producer sends its message with QoS (q0):
In the scenario, three consumers are started with subscribing with QoS 0, 1, and 2, respectively. First of all, we present the exchange of messages between the broker and the consumer for a subscription with QoS 0:
Now, we present the exchange of messages between the broker and the consumer for a subscription with QoS 1. Note that, since there are no message losses, the receiving of the PUBLISH message is the same.
Finally, we present the exchange of messages between the broker and the consumer for a subscription with QoS 2. Note that, since there are no message losses, the receiving of the PUBLISH message is still the same.
QoS 1 (at least once)
Quality of service 1 guarantees that the message will be delivered to the receiver while not preventing the message being received multiple times. To achieve this the message is queued on the sender. In the following experiment, we do not have any message losses (the flag is always d0) and we do not show buffering.
QoS 1 is experimented in the following scenario. The producer sends its message with QoS 1 (q1). The novelty on the producer side is the acknowledgement of the PUBLISH message by the reception of a PUBACK message from the broker (i.e. not from a consumer).
In the scenario, three consumers are started with subscribing with QoS 0, 1, and 2, respectively. First of all, we present the exchange of messages between the broker and the consumer for a subscription with QoS 0:
Now, we present the exchange of messages between the broker and the consumer for a subscription with QoS 1. Let us observe that the reception of the PUBLISH is followed by the sending of the corresponding PUBACK message to the broker (i.e. not to the producer).
Finally, we present the exchange of messages between the broker and the consumer for a subscription with QoS 2. Note that the sequence of messages on the consumer side is the same: message PUBLISH received and message PUBACK sent.
QoS 2 (exactly once)
Quality of service 2 additionally guarantees delivery with no duplicates. In the following experiment, we do not have any message losses (the flag is always d0) and we do not show buffering.
QoS 2 is experimented in the following scenario. The producer sends its message with QoS 2 (q2). The sequence of messages shows the "consensus" protocol (on the producer side) between the producer and the broker: message PUBLISH sent to the broker, message PUBREC received from the broker to acknowledge the PUBLISH message, message PUBREL sent to the broker to ask for releasing, message PUBCOMP to complete the "consensus" by acknowledging the release message.
In the scenario, three consumers are started with subscribing with QoS 0, 1, and 2, respectively. First of all, we present the exchange of messages between the broker and the consumer for a subscription with QoS 0:
Now, we present the exchange of messages between the broker and the consumer for a subscription with QoS 1:
Finally, we present the exchange of messages between the broker and the consumer for a subscription with QoS 2. In the sequence of message, we can observe that this is the dual of the one of the producer, which publishes also with QoS 2.
Sessions and client's clean start flag
A session is a stateful interaction between a client and a broker. Some sessions last only as long as the network connection, i.e. they are non persistent, others can span multiple consecutive network connections, i.e. they are persistent.
Regardless the quality of service, messages that have been delivered to a consumer are deleted from the broker.
On the client side, the clean start flag specifies whether the connection starts a new session or is a continuation of an existing session. If a CONNECT packet is received with Clean Start set to true, the client and broker MUST discard any existing session and start a new session. If a CONNECT packet is received with Clean Start set to false and there is a session associated with the client identifier, the broker MUST resume communications with the client based on state from the existing session. If a CONNECT packet is received with Clean Start set to false and there is no session associated with the client identifier, the broker MUST create a new session.
In MQTTv3.1.1, the Clean Start flag was called the Clean Session flag.
Clean Start flag set to true and QoS 0, i.e. non persistent session
In the following example, a consumer is created, connects with clean start set to true, subscribes to a filter. Then, a producer publishes message message0 that is received by the client. Next, the consumer disconnects and the newly published message message1 is "lost" by the broker because there is no consumer. Finally, the consumer reconnects without re-subscribing, i.e. with subscribing to another topic filter, and does not receive the last published message message2, thus showing that the session was not persistent.
By default, command mosquitto_sub starts a consumer with Clean Start set to true. Here follows the screnario. Note that option "-C 1" states that the consumer disconnects after receiving one message.
If you execute the same scenario, but with the --xterm, you can observe that the first consumer xterm terminal disappear as soon as a message is received, and the the second one is not closed. As a consequence, the fact that the second terminal is not closed is a proof that only one message is received, then showing that the session was not persistent.
Clean Start flag set to false, i.e. persistent session
In the following example, a consumer is created, connects with clean start set to false, subscribes to a filter. Then, a producer publishes message message0 that is received by the client. Next, the consumer disconnects and the newly published message message1 is "lost" by the broker because there is no consumer. Finally, the consumer reconnects without re-subscribing, i.e. with subscribing to another topic filter, and does receive the last published message message2, thus showing that the session was persistent.
We use option "-c" to start a consumer with clean start set to false. Here follows the scenario:
If you execute the same scenario, but with the --xterm, you can observe that the consumer xterm terminals disappear as soon as a message is received. As a consequence, the fact that the two terminals are closed is a proof that two messages are received, then showing that the session was persistent.
Clean Start flag set to false and QoS 1, i.e. persistent session
Let us use the same scenario: consumer connects with the clean start flag set to false, subscribes, and disconnects; producer publishes message0 and message1 while the consumer was connected and after consumer's disconnection, respectively. With QoS 0 (subscription and publication), message1 was "lost" (forever). But, as follows, with QoS 1 (subscription and publication), message1 is received when the consumer reconnects (in the context of the persistent session):
Do not execute the same scenario with the --xterm option set because, when the second consumer is started, there already exists a message to receive, and therefore, you will not have enough time to see the terminal that is opened and immediately closed. Indeed, this is why we added the echo command that displays the PID of this volatile consumer.
Retain flag in publish packet
By default, when a producer publishes a message, if there are no consumers (the topic name of the message matches no topic filter), the broker simply discards the message. By setting the Retain flag, the producer can tell the broker to keep the last message on that topic name. It is important to note that only one message is retained per topic name: The next message published on that topic name replaces the last retained message for that topic name.
More precisely, when functionality "Retained message" is provided by the broker, if the Retain flag is set to 1 in a PUBLISH packet sent by a client to a broker, the broker MUST replace any existing retained message for this topic and store the application message. If the Retain flag is 0 in a PUBLISH packet sent by a client to a broker, the broker MUST NOT store the message as a retained message and MUST NOT remove or replace any existing retained message. If the broker receives a PUBLISH packet with the Retain flag set to 1 and QoS 0, it SHOULD store the new QoS 0 message as the new retained message for that topic name, but MAY choose to discard it at any time. If this happens there will be no retained message for that topic.
Producers can mix retained and non-retained messages.
Publishing a retained message with command mosquitto_pub is done with option "-r". The set of flags of a PUBLISH message contains the Retain flag set to either "0" ("r0" in the console) or "1" ("r1" in the console).
We sequence different situations in the same scenario, with all the topic names that potentially match the topic filters:
- Produce message0 with the retained flag set (observe the r1 flag);
- When subscribing, the consumer receives message0 and observe the r1 flag (This consumer is then removed [killed].);
- When starting another consumer, it also receives message0; (This consumer is then removed [killed].)
- Produce message1 with the retained flag set;
- A new consumer receives message1; (This consumer is then removed [killed].)
- Produce message2, but with the retained flag not set. There are no consumers and message message2 is not kept in memory by the broker;
- A new consumer receives message1, i.e. the last retained message; (This consumer is then removed [killed].)
- Produce an empty message (option -n) with the retained flag set. From now on, the broker has no more retained message;
- A new consumer receives no retained message when subscribing.
Here follows the scenario:
