Apache Druid for Real-Time Analytics on Sensor and Event Data

Short answer

Apache Druid is an Apache-licensed, real-time analytics database for aggregation queries on event-oriented data; the project lists IoT and server metrics among its use cases.
A cluster runs separate services (Coordinator, Overlord, Broker, Router, Historical, Middle Manager or Indexer) and depends on deep storage, a metadata database and ZooKeeper.
Data is always partitioned by time into columnar, immutable segments; rollup can combine rows with identical dimensions and truncated timestamps at ingestion time.
Kafka and Kinesis ingestion run as supervisors, which the documentation says read partitions and offsets to guarantee exactly-once ingestion (a guarantee about reading offsets, not about duplicates a producer wrote to the topic); streamed rows are queryable as soon as they are ingested; Druid SQL is translated into native queries.
There are no single-row updates by primary key and no UPDATE statement; Druid suits large append-only telemetry, not CMMS records or small datasets.

How we know: Research-based from the Apache Druid documentation (introduction, architecture, segments, deep storage, ingestion overview, rollup, partitioning, schema design, Kafka ingestion, supervisor reference, ingestion spec, query granularities, SQL-based ingestion concepts, Druid SQL, SQL functions, lookups, data updates, retention rules, query from deep storage, Indexer and security overview), the Druid downloads page and the apache/druid LICENSE file, checked 27 October 2026. The plain PostgreSQL rollup illustration on this page was run on PostgreSQL 18.6 (PGlite 0.5.11). The Kafka supervisor spec, retention rule and Druid SQL are adapted from the documentation and were not run here. No performance figures are quoted.

What Apache Druid is, and who this is for

Apache Druid is a real-time analytics database designed for fast slice-and-dice (OLAP) queries on large data sets. The project says it works best with event-oriented data and is commonly used behind analytical user interfaces and highly concurrent APIs that need fast aggregations. Its documentation lists network telemetry, server metrics and IoT analytics among the common application areas.

For building and equipment data, Druid is a candidate when many people or applications query recent and historical telemetry at the same time: portfolio dashboards that group meter and plant readings by site, region and equipment type, or an API that serves energy totals to tenants. This guide is for data engineers comparing Druid with ClickHouse and dedicated time-series databases. On 27 October 2026 the latest stable release was Druid 38.0.0, published on 1 October 2026. For the general concepts, start with the time-series database overview.

The Druid documentation is direct about fit. It suggests Druid when insert rates are high but updates rare, most queries are aggregations, target latencies are from about 100 milliseconds to a few seconds, data has a time component, and each query hits one large table plus optional small lookup tables. It suggests looking elsewhere when you need low-latency updates of existing records by primary key, when query latency does not matter, or when you need large joins between big fact tables.

Architecture: services and external dependencies

Druid is a distributed system made of several service types that can be scaled independently. The documentation recommends grouping them on three server types:

Master servers run the Coordinator, which assigns segments to Historical services and balances them, and the Overlord, which assigns ingestion tasks and coordinates segment publishing.
Query servers run the Broker, which receives queries, fans them out to data servers and merges the results, and the Router, an API gateway that also serves the web console.
Data servers run Historical services, which download segments from deep storage and answer queries on them but never accept writes, and Middle Managers with Peons (or the experimental Indexer) that run ingestion tasks.

Three external dependencies are required in a cluster: deep storage (typically S3-compatible object storage, Google Cloud Storage, Azure Blob Storage or HDFS) holding every ingested segment, a metadata store (typically PostgreSQL or MySQL) for segment and task metadata, and ZooKeeper for service discovery, coordination and leader election. Because all data is kept in deep storage, the documentation states that a cluster can bootstrap again even if every data server is lost.

The data flow for telemetry is therefore: devices and gateways publish to an MQTT broker, a bridge or connector writes to Kafka, Druid's ingestion tasks read Kafka and build segments, segments are published to deep storage and loaded by Historicals, and dashboards query the Broker. Kafka is the streaming layer, Druid is the analytical store, and a warehouse or lakehouse can still hold the long-range history joined with CAFM and CMMS data.

Segments, time partitioning and rollup

Time chunks and segments. Druid always partitions a datasource (table) by time, using the segmentGranularity in an ingestion spec or PARTITIONED BY in SQL. Each time chunk holds one or more immutable segment files, which are columnar: timestamps and metrics are compressed numeric arrays, while each dimension (string attribute) has a dictionary, the encoded values and a bitmap index per value for fast filtering. The documentation recommends segment files of roughly 300 to 700 MB and names hour and day as the most common time chunk sizes, with hour common for streaming. Within a time chunk, secondary partitioning (CLUSTERED BY or upstream Kafka partitioning) groups rows with the same dimension values for better locality.

Rollup. With rollup enabled (the default for ingestion specs), Druid combines into one row all input rows that have identical dimension values and the same timestamp after truncation to the queryGranularity. Metrics such as sum, min, max and count are pre-aggregated. Rollup can cut row counts by orders of magnitude, but individual events can no longer be queried. Streaming ingestion always performs best-effort rollup, so rows that could be combined may sit in different segments until compaction rewrites them.

Modelling sensor data. The schema design guide notes that Druid is not a time-series database but is a natural choice for storing time-series data. Its advice: add a dimension that names the series (for example point_id) and place it first in the dimension list, keep tags such as building and equipment type as further dimensions, and define metrics for the aggregations you need (sum, min, max, plus approximate sketches for percentiles). The one-minute rollup in the example below is an illustrative assumption: choose the query granularity from the finest time resolution your dashboards, energy reports and fault analysis actually need, because rollup discards anything finer. Keep raw readings elsewhere (or in a second datasource without rollup) if engineers need individual samples for fault diagnosis.

Kafka supervisor spec: BMS readings rolled up to one minute, hourly segments (Illustrative configuration, not run here: the field layout follows the example in the Druid Kafka ingestion documentation; names, granularities, taskCount and taskDuration are ours. That example writes granularities in upper case (HOUR, NONE) while the granularities page lists lower-case strings, so check which form your version accepts; check every setting against the documentation for your version before use)

{
  "type": "kafka",
  "spec": {
    "dataSchema": {
      "dataSource": "bms_readings",
      "timestampSpec": { "column": "ts", "format": "iso" },
      "dimensionsSpec": { "dimensions": ["point_id", "building", "equipment_type", "unit"] },
      "metricsSpec": [
        { "name": "num_rows",  "type": "count" },
        { "name": "value_sum", "type": "doubleSum", "fieldName": "value" },
        { "name": "value_min", "type": "doubleMin", "fieldName": "value" },
        { "name": "value_max", "type": "doubleMax", "fieldName": "value" }
      ],
      "granularitySpec": {
        "type": "uniform",
        "segmentGranularity": "hour",
        "queryGranularity": "minute",
        "rollup": true
      }
    },
    "ioConfig": {
      "topic": "bms.readings",
      "consumerProperties": { "bootstrap.servers": "kafka-1:9092" },
      "inputFormat": { "type": "json" },
      "taskCount": 2,
      "taskDuration": "PT1H"
    },
    "tuningConfig": { "type": "kafka" }
  }
}

What rollup stores, shown in plain PostgreSQL

Rollup is easiest to understand as a GROUP BY on the truncated timestamp and every dimension, with the metrics aggregated. The query below reproduces that effect for two chillers sampled every 10 seconds: 36 ingested events become 6 stored rows, a rollup ratio of 6. It ran on PostgreSQL 18.6 and illustrates the concept only; it is not Druid code. In Druid, the same ratio is measured by comparing SUM(num_rows) with COUNT(*), and averages are computed at query time as SUM(value_sum) / SUM(num_rows).

One-minute rollup of 10-second chiller kW readings (run on PostgreSQL 18.6)

SET TimeZone = 'UTC';

CREATE TABLE chiller_kw_raw (
    ts        timestamptz  NOT NULL,
    building  text         NOT NULL,
    chiller   text         NOT NULL,
    kw        numeric(6,1) NOT NULL
);

INSERT INTO chiller_kw_raw
SELECT '2026-10-27 09:00+00'::timestamptz + (n * interval '10 seconds'),
       'HQ', c, 180 + (n % 6) * 2.5 + CASE c WHEN 'CH-2' THEN 40 ELSE 0 END
FROM generate_series(0, 17) AS n, unnest(ARRAY['CH-1', 'CH-2']) AS c;

-- what rollup with a one-minute query granularity stores: one row per minute per dimension combination
SELECT date_trunc('minute', ts)::text AS minute, building, chiller,
       count(*) AS num_rows, sum(kw) AS kw_sum, min(kw) AS kw_min, max(kw) AS kw_max
FROM chiller_kw_raw
GROUP BY 1, 2, 3
ORDER BY 1, 3;

-- rollup ratio: ingested events per stored row
SELECT count(*) AS ingested_events,
       (SELECT count(*) FROM (SELECT 1 FROM chiller_kw_raw GROUP BY date_trunc('minute', ts), building, chiller) s) AS stored_rows
FROM chiller_kw_raw;

Result on PostgreSQL 18.6 (PGlite)

         minute         | building | chiller | num_rows | kw_sum | kw_min | kw_max
------------------------+----------+---------+----------+--------+--------+--------
 2026-10-27 09:00:00+00 | HQ       | CH-1    |        6 | 1117.5 |  180.0 |  192.5
 2026-10-27 09:00:00+00 | HQ       | CH-2    |        6 | 1357.5 |  220.0 |  232.5
 2026-10-27 09:01:00+00 | HQ       | CH-1    |        6 | 1117.5 |  180.0 |  192.5
 2026-10-27 09:01:00+00 | HQ       | CH-2    |        6 | 1357.5 |  220.0 |  232.5
 2026-10-27 09:02:00+00 | HQ       | CH-1    |        6 | 1117.5 |  180.0 |  192.5
 2026-10-27 09:02:00+00 | HQ       | CH-2    |        6 | 1357.5 |  220.0 |  232.5

 ingested_events | stored_rows
-----------------+-------------
              36 |           6

Streaming and batch ingestion

Streaming (Kafka or Kinesis). A continuously running supervisor on the Overlord manages indexing tasks that read Kafka partitions and offsets directly, which the documentation says guarantees exactly-once ingestion. As described, that guarantee is about the partition and offset mechanism; the documentation we checked does not describe removing duplicate messages that a gateway or producer wrote to the topic twice, so plan to deduplicate upstream or tolerate duplicates in rollup metrics such as counts and sums. Streamed rows can be queried as soon as they are ingested, because ingestion tasks answer queries for data not yet handed off to Historicals. Late messages are accepted unless the supervisor is configured to reject them (lateMessageRejectionPeriod or lateMessageRejectionStartDateTime). The druid-kafka-indexing-service extension must be loaded, consumerProperties must at least set bootstrap.servers, and a single supervisor can read several topics with topicPattern.
SQL-based batch (multi-stage query engine). INSERT appends and REPLACE ... OVERWRITE replaces data for a whole table or a time range, reading files through EXTERN or other datasources through FROM. There is no CREATE TABLE: the first INSERT creates the datasource. The documentation warns against using INSERT for micro-batches; that is what streaming ingestion is for. SQL batch ingestion always produces perfect rollup.
Native batch. JSON index_parallel specs submitted to the Tasks API, used for backfills from object storage or HDFS.

For building telemetry, one possible split (a design choice, not a Druid recommendation) is streaming from Kafka for live data, and a SQL REPLACE over a day or hour when a gateway backfill or a corrected meter export arrives, because overwriting a time range is how Druid applies corrections. The Kafka side of the pipeline (topics, keys, retention) is covered in Designing an IoT Data Pipeline with Kafka.

Querying with Druid SQL

Druid supports two query languages: Druid SQL and native JSON queries. SQL is planned on the Broker and translated into native queries, and is available over HTTP and through a JDBC driver. Every table has a primary timestamp column called __time; filtering on it lets Druid prune segments outside the time range. TIME_FLOOR rounds timestamps down to an ISO 8601 period for bucketing. Joins are supported but the design favours one large table per query plus small lookups; a lookup maps a dimension value (such as a point ID) to another value (such as a display name) and is preloaded in memory on all servers, and the LOOKUP function is usually faster than an explicit join to a lookup.

15-minute average and peak per point for one building over one day (Druid SQL. Illustrative query, not run here: uses TIME_FLOOR and LOOKUP as documented; the lookup name bms_point_names is a placeholder for a lookup you would define; check it against the documentation for your version before use)

SELECT TIME_FLOOR(__time, 'PT15M')               AS bucket,
       point_id,
       LOOKUP(point_id, 'bms_point_names')       AS point_name,
       SUM(value_sum) / SUM(num_rows)            AS avg_value,
       MAX(value_max)                            AS peak_value
FROM bms_readings
WHERE building = 'HQ'
  AND equipment_type = 'chiller'
  AND __time >= TIMESTAMP '2026-10-27 00:00:00'
  AND __time <  TIMESTAMP '2026-10-28 00:00:00'
GROUP BY 1, 2, 3
ORDER BY 1, 2;

Retention, deep storage and updates

Retention rules. The Coordinator applies load and drop rules per datasource (or a cluster default), evaluated in order, using a period relative to now, a fixed interval or forever. One telemetry pattern is "load recent data onto Historicals, drop the rest from the cluster"; the 90 days and two replicas in the example are illustrative assumptions, chosen from how far back dashboards query with low latency and how much Historical disk you can afford. Segments dropped by rules are marked unused but stay in deep storage until unused segments are cleaned up automatically or by a kill task; they are not queryable unless reloaded. To keep older data queryable without loading it onto Historicals, the rule documentation describes a load rule with no Historical replicas for those segments, so they are available to query from deep storage, which the documentation says is slower than querying segments loaded on Historicals.

Updates. Druid does not support single-record updates by primary key and has no UPDATE or ALTER TABLE statements. Changes are made by overwriting a time range (SQL REPLACE or a native batch job), by reindexing existing data into itself, or by appending to rolled-up datasources and letting compaction combine rows. Dimension values that change often, such as asset names or zone labels, belong in lookups rather than in the segments.

Retention rules: keep an illustrative 90 days on the default tier with two replicas, drop older data from the cluster (Illustrative configuration, not run here: adapted from the Druid rule documentation; check every setting against the documentation for your version before use)

[
  {
    "type": "loadByPeriod",
    "period": "P90D",
    "includeFuture": true,
    "tieredReplicants": { "_default_tier": 2 }
  },
  { "type": "dropForever" }
]

Sizing, security and operations

Two storage budgets. Deep storage must hold everything you ingest; Historical disks must hold the segments you load for low-latency queries. Retention rules set the split.
Segment sizing. Aim for the documented 300 to 700 MB per segment; too-fine time chunks create many small segments, which hurts performance. Schedule compaction after streaming ingestion to merge small segments and complete rollup.
Cardinality. Rollup only helps when dimension combinations repeat. A raw reading_id or high-resolution timestamp as a dimension defeats it; the documentation suggests fewer, lower-cardinality dimensions and sketches for unique counts.
Security. Security features are disabled by default. The documentation says to enable authentication for production and any environment reachable from untrusted networks, enable TLS for traffic between clients and the cluster and between services, and avoid running Druid as the operating system root user.
Operations. A production cluster means several service types plus ZooKeeper, a metadata database and object storage to run, monitor and upgrade. Running Druid yourself costs infrastructure and staff time. Commercial managed services exist, but their pricing is outside the scope of this page.
Licence. The LICENSE file in the apache/druid repository is the "Apache License Version 2.0, January 2004". This is a factual description, not legal advice: read the licence, and the terms of any managed service, and take advice for your situation.

When not to use Apache Druid

Transactional and frequently updated records. Work orders, asset registers and contracts need row-level updates and constraints; Druid has neither.
Small teams or small data. Our editorial view: if one PostgreSQL server or a single-node time-series database can hold the data, the operational overhead of a Druid cluster is hard to justify. See How to Store IoT Sensor Data for the simpler options.
Raw-sample diagnostics only. If every query needs individual readings, you lose the main storage benefit of rollup.
Large joins. Joining two big fact tables, for example every reading with every alarm event, fits a warehouse or lakehouse better.
Offline reporting. Our editorial view: if nightly reports are fine, a warehouse is likely to be simpler.

For the trade-offs between Druid, ClickHouse and specialised time-series engines, see Time-Series Database Performance and the guide to choosing a time-series database.

Frequently asked questions

Is Apache Druid a time-series database?

Not strictly. Its documentation says Druid is not a time-series database but is a natural choice for time-series data, because every table has a primary timestamp, data is partitioned by time and rollup can pre-aggregate readings.

Can Druid ingest directly from Kafka?

Yes. The Kafka indexing service runs supervisors that read topics directly and track partitions and offsets, which the documentation says guarantees exactly-once ingestion; that refers to the offset mechanism, and the documentation we checked does not describe removing duplicate messages a producer wrote to the topic. New rows are queryable as soon as they are ingested.

What is rollup in Druid?

Pre-aggregation at ingestion time: rows with the same dimension values and the same truncated timestamp are stored as one row with summed, minimum, maximum or counted metrics. It saves storage and speeds queries, but you can no longer query individual events.

Can I update or delete a row in Druid?

Not by primary key. You overwrite or reindex a time range, use lookups for values that change, or delete whole segments. Druid SQL has no UPDATE statement.

What does Druid need besides its own services?

Deep storage such as S3-compatible object storage or HDFS, a metadata database such as PostgreSQL or MySQL, and ZooKeeper. A single-server setup can use local disk and an embedded Derby database for testing.

Sources

Checked 27 October 2026.

How we research these articles: our editorial method. Research-based from official documentation; SQL examples are run on a local database where the page says so. CodeWithSQL earns nothing from the vendors mentioned.