Kafka vs MQTT

Quick verdict

This page is for architects of building management, CAFM, CMMS and industrial IoT systems who see both names on a reference architecture and want to know which does what. A typical example: a BMS gateway reads AHU supply-air temperature, chiller kW and zone CO2 every 1 to 60 seconds from BACnet or Modbus, publishes them, and the readings must reach a time-series database, an alarm service that raises maintenance work orders, and a long-term analytics store. For the full data path from devices to dashboards, see the time-series and IoT data hub.

MQTT is an OASIS standard messaging protocol. The specification describes it as a client-server publish/subscribe transport that is light weight and simple, intended for constrained environments such as M2M and IoT. Versions 5.0 and 3.1.1 are OASIS standards (3.1.1 is also an ISO standard). MQTT is a protocol, not a product: you run a broker such as Eclipse Mosquitto, the MQTT plugin in RabbitMQ, or a commercial or cloud broker.

Apache Kafka is an open source event streaming platform from the Apache Software Foundation. Its documentation lists three capabilities: publishing and subscribing to streams of events, storing them durably for as long as you want, and processing them as they occur or retrospectively. The latest release when checked was 4.4.0 (22 October 2026). See the Apache Kafka guide for architecture detail.

Side by side

Key differences

Different jobs: device messaging versus an event log

MQTT solves the device problem. mqtt.org describes it as designed for constrained devices and low-bandwidth, high-latency or unreliable networks, with small headers to save bandwidth and persistent sessions that shorten reconnection. A field gateway on a 4G link can publish a reading in a few bytes of overhead, subscribe to commands such as a setpoint change, and register a last-will message so the broker announces when the gateway drops off. Topic names are hierarchical, for example:

site/hq/ahu/ahu-01/supply_air_temp
site/hq/chiller/ch-02/kw
site/hq/floor-3/zone-12/co2
site/hq/+/+/status        (filter: every asset status on site hq)

Kafka solves the back-end problem. Events are appended to partitioned, replicated topics and kept for a configured retention period; the documentation stresses that, unlike traditional messaging systems, events are not deleted after consumption. Several consumer groups can read the same topic independently and at their own pace, and a consumer can rewind to an earlier offset to reprocess data after a bug. That makes Kafka a durable buffer and distribution layer between producers and many downstream systems.

Our opinion (editorial judgement, not a statement from either project's documentation): Kafka clients and the Kafka protocol are aimed at servers and applications rather than battery-powered microcontrollers on unreliable links, and MQTT brokers are not designed to be the long-term, replayable record of every reading. That is why, in our view, the two are usually combined rather than compared.

The bridge pattern: MQTT at the edge, Kafka in the back end

A typical building or plant architecture has four layers, and it helps to keep their roles separate:

1. Devices and gateways read BACnet, Modbus or OPC UA points, record a collection time for each (plus the source's event time where the device supplies one) and normalise them, and publish over MQTT (often with TLS on port 8883).
2. MQTT broker (streaming at the edge) authenticates devices, routes messages to subscribers and handles retained values, last-will messages and offline sessions.
3. Bridge into Kafka: a Kafka Connect source connector for MQTT, a broker's own Kafka integration, or a small service that subscribes to MQTT topics and produces to Kafka. It maps topic segments to fields and sets the Kafka key, for example to the asset or point ID, so that all readings for one point land in one partition and stay in order.
4. Kafka consumers: a sink writing to the time-series database (storage and query), an alarm or rule service that raises CMMS work orders, and a job that lands history in a warehouse or lakehouse (analytics).

Kafka Connect is the documented framework for streaming data between Kafka and other systems. The Apache Kafka 4.4 Kafka Connect documentation we checked does not mention an MQTT connector, so an MQTT source connector would come from a third party or a broker vendor; we did not review any specific connector, so check the licence, support and delivery guarantees of the one you choose. The Kafka IoT data pipeline guide covers topic design, partitioning and sinks in depth, and MQTT to TimescaleDB covers the simpler path without Kafka.

Delivery guarantees and duplicates end to end

MQTT offers three quality-of-service levels per message: at most once, at least once and exactly once. The specification gives ambient sensor data as an example where QoS 0 is acceptable because the next reading follows soon after. QoS applies to each hop (device to broker, broker to subscriber), and brokers differ: the RabbitMQ MQTT plugin, for example, supports QoS 0 and 1 but not QoS 2.

Kafka guarantees at-least-once delivery by default. The idempotent producer avoids duplicates caused by producer retries, and transactions give exactly-once processing for reads and writes inside Kafka, but the documentation notes that exactly-once delivery to other destination systems generally requires cooperation from those systems. In practice an MQTT QoS 1 publish followed by an at-least-once bridge and sink can deliver the same reading twice, so make the database write idempotent with a natural key. Such a sink is at-least-once with idempotent writes; it should only be described as exactly once when offsets are stored in the same transaction as the data and a fencing check rejects writes from a consumer that has lost its partitions, as set out in Kafka consumer idempotency for sensor streams:

-- PostgreSQL: run on PostgreSQL 18.6 (PGlite)
CREATE TABLE sensor_reading (
  device_id text NOT NULL,
  point     text NOT NULL,
  ts        timestamptz NOT NULL,
  value     double precision NOT NULL,
  PRIMARY KEY (device_id, point, ts)
);

-- the same reading delivered twice by the bridge
INSERT INTO sensor_reading VALUES ('ahu-01', 'supply_air_temp', '2026-10-27 09:00:00+00', 13.4)
ON CONFLICT (device_id, point, ts) DO NOTHING;
INSERT INTO sensor_reading VALUES ('ahu-01', 'supply_air_temp', '2026-10-27 09:00:00+00', 13.4)
ON CONFLICT (device_id, point, ts) DO NOTHING;

SELECT count(*) AS stored_rows FROM sensor_reading;
-- stored_rows = 1

Without ON CONFLICT, the second insert fails with the 23505 duplicate key error. For consumer-side patterns such as offset handling and deduplication tables, see Kafka consumer idempotency for sensor streams and handling duplicate telemetry events.

Is there an MQTT database?

No. MQTT has no query language and no history. The nearest thing to storage in the protocol is the retained message: when a client publishes with the RETAIN flag, the broker replaces any existing retained message for that topic and stores the new one, so a new subscriber immediately receives the last known value. That is useful for a dashboard showing current zone CO2, but it is one value per topic, not a time series. Persistent sessions only queue messages for a disconnected subscriber until it returns.

To keep MQTT data you need a subscriber (or Kafka sink) that writes to a database: PostgreSQL or TimescaleDB, InfluxDB, QuestDB, ClickHouse or another store. Some broker products bundle such integrations, but the storage is still a database behind the broker. See how to store IoT sensor data for schema and sizing, and the time-series database selection guide to choose the store.

Retention, sizing and operations

Kafka retention is set per topic. retention.ms defaults to 7 days and can be set to -1 for no time limit; retention.bytes limits size, log compaction keeps the latest value per key, and tiered storage can move older segments to remote storage. Disk needed is roughly the write rate multiplied by retention and the replication factor (for example three copies; that figure is an illustrative assumption, and the right factor depends on your durability and availability needs). Kafka 4.x runs only in KRaft mode, in which some servers act as controllers for cluster metadata; clusters still on ZooKeeper must migrate through the 3.9 bridge release first.

An MQTT broker's sizing is driven by concurrent connections, subscriptions, message rate and the queues held for offline persistent sessions. Retained and queued messages are broker-specific; the RabbitMQ plugin, for example, keeps retained messages node-local. Plan for the slow consumer: if the database subscriber stops, an MQTT-only design has to queue or drop messages at the broker, while a Kafka design keeps them in the topic until the sink catches up. See batch vs stream ingestion and sizing storage for sensor telemetry.

Security

MQTT clients can authenticate with a user name and password or modern methods such as OAuth, and connections are encrypted with TLS (port 8883 is registered for MQTT over TLS). Topic-level authorisation is a broker feature, so define per-site and per-device permissions so a compromised gateway cannot publish to another building's command topics.

Kafka supports TLS encryption, SASL authentication and ACL-based authorisation on topics and consumer groups. Keep the two trust zones apart: devices should not connect to Kafka directly, and the bridge is the controlled crossing point between the device network and the back-end platform.

Pricing and licensing

Apache Kafka is distributed under the Apache License, Version 2.0, whose copyright grant reads in part: "each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license", subject to the licence's terms and conditions. The running cost is the cluster you run (brokers, controllers, disks, network) and the people to operate it. Kafka's documentation notes that fully managed services are offered by a variety of vendors; they price by throughput, partitions, storage or cluster size in different ways, so compare them on your own write rate and retention. We did not check individual vendors' price lists for this page.

MQTT is an OASIS standard; the protocol itself is not a product you license, but every broker and client library has its own licence. Eclipse Mosquitto is an open source project, and RabbitMQ (which ships the MQTT plugin) states that it is "Free under the Mozilla Public License 2.0"; read each project's licence file for the terms. Commercial and cloud IoT brokers set their own pricing models (for example by connected devices, connection time or message volume); we did not check individual price lists for this page. As with Kafka, estimate from device count, publish interval and message size.

Licence descriptions here are factual summaries with quotes from the official licence texts and vendor pages; they are not legal advice. Read the licence and take advice for your situation.

Pricing checked on the vendors' official pages on 27 October 2026. Prices change; confirm before buying.

Where each one leads

Limitations

When to choose each

When neither is right

• Data arrives as files or periodic exports from a BMS or meter portal: batch loading into the database is simpler; see batch vs stream ingestion.
• You need task queues for work orders, notifications or integration jobs with per-message acknowledgement and retries: compare Kafka vs RabbitMQ.
• The data is infrastructure metrics scraped from exporters: a metrics system fits better; see InfluxDB vs Prometheus.
• You are choosing where readings are stored and queried, not how they move: see TimescaleDB vs InfluxDB and ClickHouse vs TimescaleDB.

Final recommendation

Frequently asked questions

Sources

• Apache Kafka: introduction
• Apache Kafka 4.4: design (consumers, delivery semantics)
• Apache Kafka 4.4: topic configuration
• Apache Kafka 4.4: Kafka Connect overview
• Apache Kafka 4.4: KRaft operations
• Apache Kafka 4.4: upgrade notes (share groups)
• Apache Kafka downloads (releases)
• MQTT.org: the standard for IoT messaging
• MQTT.org: FAQ
• OASIS: MQTT Version 5.0 specification
• Eclipse Mosquitto repository
• RabbitMQ: MQTT plugin
• RabbitMQ home page (licence statement)
• Apache Software Foundation: Apache License, Version 2.0
• Apache Kafka 4.4 javadoc: ConsumerRebalanceListener (partitions reassigned before revocation)

Checked October 2026.

How we research comparisons: our editorial method.