MQTT Test Suite

XMQ is distributed with a set of command-line utilities used to load-test and functionally test MQTT brokers, XMQ included. They implement MQTT protocol versions 3.1, 3.1.1, and 5.0, so they can be pointed at any compliant broker (e.g. XMQ, eMQX, Mosquitto) for an apples-to-apples comparison. All of them share a common set of connection options, such as --host/--port, --username/--password, --cafile/--cert/--key for TLS, --client-id, --protocol-version, and --sessions for opening several client sessions from a single process.

xmq_con — connection tester

Opens a number of client connections to the broker, either as fast as possible or at a controlled rate, and reports how quickly the connections are established. It doesn't publish or subscribe to anything — it is a pure connect/disconnect load generator used to find out how many clients per second a broker can accept, and to compare that rate across brokers.

• --sessions/-n — number of client sessions to connect.
• --connect-rate/-R — connections per second; unpaced when omitted.
• --show-counters / --show-counters-csv — print progress counters as plain text or CSV while the test is running.

xmq_pub — publisher

Connects one or more publishing clients and sends messages to one or more topics. It is used both to generate publish load and to functionally verify that a broker accepts and routes messages correctly.

• --topic/-t — one or more topics (separated by ‘;’); a topic may contain %ClientIndex%, substituted with the client's index.
• Message source: --message/-m (inline text), --message-file/-f (file contents), or --stdin/-s (one message per line).
• --qos/-q, --repeat/-r — QoS level and how many times the message is sent.
• --message-rate/-g — publish rate in messages per second; unpaced when omitted.
• --max-inflight/-M — maximum in-flight messages for QoS 1/2.
• --send-timestamp/-l — injects a send timestamp at the start of each message, used by xmq_sub to measure end-to-end latency.
• --session-expiry-interval/-x, --disable-clean-session/-c — MQTT session persistence controls.

MQTT 5 CONNECT and PUBLISH properties (such as message-expiry-interval or user-property) can be set with repeated -D arguments; the full property list is printed by xmq_pub --help.

xmq_sub — subscriber

Connects one or more subscribing clients to one or more topics and receives messages, used to verify delivery and, together with xmq_pub's --send-timestamp, to measure publish-to-receive latency.

• --topic/-t — one or more topics to subscribe to, with the same ‘;’ and %ClientIndex% support as xmq_pub.
• --qos/-q — subscription QoS level.
• --receive-count/-C — disconnect and exit after receiving this many messages.
• --disconnect-after/-W — disconnect and exit after waiting this many seconds for messages.
• --print-messages/-v — print each received message to stdout.

xmq_scn — scenario runner

Runs a complete multi-client test — a mix of publishers, subscribers, and a target broker — described by a single JSON scenario file, instead of requiring separate xmq_con/xmq_pub/xmq_sub processes to be started and coordinated by hand. Any value from the scenario file (rate, duration, QoS, broker address, and so on) can be overridden from the command line for a one-off run.

• --scenario/-s — scenario JSON file. An absolute path is used as is; a relative path is looked up in the current directory first, then in the installed scenario directory (share/xmq next to the xmq_scn executable).
• --list-scenarios — list the scenario files installed under share/xmq, grouped by sub-directory, and exit.
• --publish-rate/-r, --publish-count/-C, --duration/-d, --connection-rate/-R, --payload-size/-m, --qos/-q — override the matching scenario parameters.
• --bind-to-interfaces/-I — spread clients across several local network interfaces (a comma-separated IP list, or a mask such as 192.0.2.1/24), needed to run more than 64K clients from one host.
• --id-prefix — prepended to the publishers' and subscribers' id_prefix from the scenario file, so client IDs don't collide when the same scenario is run from several hosts against one broker.

Scenario file format

A scenario file is a JSON document with the following top-level sections:

• name — scenario name, used in reports and by --list-scenarios.
• type — one of Connections, Point-To-Point, Fan-Out, Fan-In; selects the traffic pattern the scenario exercises.
• publishers — the publishing client group: id_prefix, protocol_version, client_count, qos, clean_session, and topics (which may use $clientid/$clientindex substitution).
• subscribers — the subscribing client group, same shape as publishers; omitted for scenarios that don't subscribe (e.g. a pure connection test). Its topics can use the shared-subscription syntax $share/<group>/<filter> so several subscribers split one topic's messages.
• server — the broker under test: hostname, port, username, password.
• parameters — test sizing and pass/fail thresholds: message_count or duration_sec (how long/how much to publish), payload_size, publish_rate, connection_rate, and the expected results expected_connect_latency/expected_message_latencythe run is compared against.

The four scenario types model different delivery topologies:

• Connections — clients only connect (and stay connected) at a given connection rate; there are no publishers or subscribers, so it is a pure connect-scaling test.
• Point-To-Point — matching numbers of publishers and subscribers, each pair using its own topic; models one-to-one message delivery.
• Fan-Out — one (or a few) publisher(s) send to a single topic that many subscribers listen to; models broadcast-style delivery.
• Fan-In — many publishers, each on its own topic, consumed by a small number of subscribers through a shared subscription; models fan-in aggregation load.

A Point-To-Point example, 1,000 publishers and 1,000 subscribers over 1,000 topics:

{
  "name": "Point-To-Point-1K-1K-1K-1K",
  "type": "Point-To-Point",
  "publishers": {
    "id_prefix": "test-publisher-",
    "protocol_version": 3,
    "client_count": 1000,
    "qos": 1,
    "clean_session": true,
    "topics": "test/topic$clientindex"
  },
  "subscribers": {
    "id_prefix": "test-subscriber-",
    "protocol_version": 3,
    "client_count": 1000,
    "qos": 1,
    "clean_session": true,
    "topics": "test/topic$clientindex"
  },
  "server": {
    "hostname": "localhost",
    "port": 1880,
    "username": "user",
    "password": "secret"
  },
  "parameters": {
    "duration_sec": 10,
    "payload_size": 16,
    "publish_rate": 1,
    "expected_message_latency": 2
  }
}

Run it with xmq_scn --scenario Basic/Point-To-Point-1K-1K-1K-1K.json, or override just the publish rate for one run with xmq_scn --scenario Basic/Point-To-Point-1K-1K-1K-1K.json --publish-rate 10.