Credits:The XMQ web site is provided by Linotex.The XMQ and SPTK Windows installers are created with Advanced Installer free Open-Source license.
There were 0 unique visitors to this page

Webhook and Data Integration

Webhook and data integration

The webhook extension posts what the broker sees to one HTTP(S) endpoint. It has two inputs - the MQTT messages the broker accepts, and the broker's own log records - and either or both can be sent. Every request carries an HMAC-SHA256 signature when a secret is configured.

It is an extension, not a core feature: a broker with no webhook loaded pays nothing for it. It is built for Linux, FreeBSD and Windows: the socket layer uses Winsock on Windows and the platform's POSIX sockets elsewhere, and TLS is OpenSSL on all of them. On Windows the installed fragment nameslib\xmq-extensions\xmq_webhook.dll; on the other platforms the library is libxmq_webhook.so.

One extension is one endpoint. To send the same stream to several receivers, put a small fan-out service in front of this one; it is a better place for per-receiver filters and credentials than a JSON document typed into a settings field.

Enabling it

The shipped fragment xmq_extensions.d/50-webhook.conf is disabled and has an empty URL, so nothing is posted until both are set. Either edit that file, or switch the extension on in the interface - which writesxmq_extensions.d/90-webhook.conf - and fill in the settings there. Loading, ordering and the Required flag are described on theExtensions page.

A URL that is missing or malformed is refused and logged when the extension starts; it does not start in silence and lose records. The same holds for a plainhttp:// URL while allow_http is false.

What is sent

sources chooses the input. Send messages posts the messages the broker accepted, Send log posts the broker's log records, and Send log and messages posts both, told apart by the source field in the body. There is no "neither": to send nothing, disable the extension.

SettingNarrows
topic_filterMessages. A list of MQTT filters; a message matching any item is sent.+ and # must each be a whole level, and# the last one.
max_payloadMessages. Larger bodies are dropped and counted, not truncated.
log_min_levelLog records. debug, info, warning orerror, the lowest level that is sent.
log_subjectsLog records. A comma-separated list of the broker's log subjects, for example publish,connect,server_connections; empty means all of them.

The log input is what makes an unreachable-account or refused-publication incident visible: refusals from the account database, the permission evaluator and the extension host arrive even when no client can connect, because they are records of why a message does not exist.

What the endpoint receives

A POST with Content-Type: application/json andUser-Agent: xmq-webhook/1.2. With no template set, a message arrives as:

{"topic":"sensors/water/1","message":"aGVsbG8=","level":"message"}

The payload is base64 because MQTT bodies are binary and JSON strings are not, and a message has no severity of its own, so level is the literalmessage - which is also what tells the two inputs apart at the receiver. A log record arrives in the same three fields: topic empty,message the log text, and level its severity.

A 2xx answer means the record was delivered. The body is not interpreted otherwise. The connection is kept open between records and re-established if the endpoint closes it while idle.

Body templates

Set template when the receiver wants its own envelope. Names written as{{name}} are replaced, and the value is JSON-escaped, so a quote or a newline in a topic cannot break the body:

"template": "{\"text\":\"[{{level}}] {{subject}}: {{message}}\",\"topic\":\"{{topic}}\"}"

The names available for a message are id, source,timestamp, topic, qos,retain, client_id, username,payload_base64, message and level. A log record has id, source, timestamp,level, subject and message, withtopic empty. For a message, message andpayload_base64 are the same base64 text.

A name the template does not know is left exactly as written rather than replaced with nothing, so a typo fails at the receiver instead of silently emptying a field. One template serves both inputs, which is why the default one names only fields that exist in both.

Proving the sender

With secret set, every request carriesX-XMQ-Signature: sha256=<hex>, the HMAC-SHA256 of the exact request body keyed with that secret. A receiver should compute the same digest over the bytes it received and compare in constant time; it should reject a request with no signature when one is expected.

secret is a secret setting: the interface masks it and the broker keeps it out of the log. Empty means unsigned requests, which is acceptable only on a link that is already trusted.

headers adds request headers of your own, one Name: valueitem per entry - an authorization token for the receiver, for example. A header that is malformed is refused when the settings are applied.

Configuration

These are the settings of the extension. Each is checked on theExtensions page and again by the broker; the format column is what the value must match as a whole.

SettingTypeDefaultMeaning
urlstringemptyWhere the bodies are posted, http:// orhttps://. Empty means the extension has nothing to do.
sourceschoiceSend messagesSend log, Send messages, or Send log and messages.
templatestringthe three-field JSON bodyBody to post, with {{...}} names substituted and JSON-escaped.
headerslist of stringsemptyExtra request headers, one Name: value per item.
secretsecretemptyHMAC-SHA256 key for X-XMQ-Signature. Masked in the interface and never logged.
topic_filterlist of strings#Which messages are sent; one filter per item. The interface shows one field per filter, with Add and Remove beside the list.
max_payloadinteger1048576Largest message body to send, in bytes; larger ones are dropped and counted. 0 means no limit.
timeout_msinteger5000Connect and read timeout of one request, 1 ms to 999999 ms.
retriesinteger3Attempts after the first one, 0 to 100.
concurrencyinteger4Requests in flight to the endpoint, 1 to 64. Each is a worker thread waiting for one synchronous reply.
allow_httpbooleanfalsePermit a plain http:// URL. Meant for a receiver on the same machine and for tests, not for a production endpoint.
log_min_levelchoiceerrorLowest log level to send: debug, info, warning or error.
log_subjectsstringemptyComma-separated log subjects; empty means all.

_comment keys in the shipped fragment are not settings and are not edited on the page; name and library identify the extension itself and live in the configuration file.

Delivery, and what is lost

The publish path never waits for the endpoint. An accepted message copies its body into the extension's queue and the broker continues; a slow or unreachable receiver costs deliveries, not broker throughput.

Answer or failureWhat happens
HTTP 2xxDelivered. Counted as sent.
HTTP 5xx, 429, timeout, connection failureRetried, up to retries times, with a backoff that starts at 100 ms and doubles to at most 2 s.
Any other 4xxRefused without a retry: the endpoint would say no again.
The queue is fullThe newest record is dropped and the drop counter rises. Delivery is at-most-once and a restart loses what has not been sent.

The extension holds 1024 bodies in its queue, and concurrency workers send them. A worker waits for one synchronous reply, so an endpoint 10 ms away carries about 100 × concurrency bodies a second at best; raiseconcurrency before raising timeouts.

What it is not, and what it costs

  • One endpoint, no transformations. The body arrives as it was published, or in the fixed shape the template gives it. There is no rule language, no JSONPath and no per-record routing. Kafka, database and object-store connectors are further extensions on the same message tap.
  • No replay and no durable queue. The queues are in memory, bounded by count rather than bytes. With the default max_payload of 1 MiB and a stalled endpoint, the queued bodies can be a gigabyte of memory: setmax_payload to the largest message the deployment really sends.
  • Base64 adds a third to the body. A 1 MiB payload travels as roughly 1.4 MB of JSON.
  • HTTPS is verified. Certificates are checked against the system store, so a private CA has to be installed where the broker can see it.
  • A URL set by an administrator can reach internal services. The payload is the message body, so do not point this at an endpoint that logs bodies which carry credentials.

The counters for each input are shown on the Extensions page as messages and log records delivered and dropped. A non-zero drop count means the endpoint or the network is slower than the traffic that matches the filters.