Setting the file. One moment.
Chapter 03 · Clickhouse Architecture Advisor
Subchapter 3.3
examples/observability-high-throughput.mdMarkdown3 KBView on GitHub
This is a high-ingest, append-friendly, time-series workload. The main architectural risks are:
What
Use Kafka as the decoupling layer and load ClickHouse through Kafka engine tables and downstream MVs.
Why
The producer fleet is bursty and distributed. This pattern improves replayability and isolates producers from storage behavior.
How
Category
derived
Confidence
medium
Source
What
Use PARTITION BY toYYYYMM(event_time) for the main raw table.
Why
This workload is time-bounded and retention-based, but daily partitioning would likely create unnecessary operational overhead at scale.
Category
derived
Confidence
medium
Source
What
Create rollup tables for repeated service-health queries.
Why
Dashboards and alerts should not repeatedly scan the raw log corpus.
Category
official
Confidence
high
Source
CREATE TABLE logs_raw
(
event_time DateTime64(3),
service LowCardinality(String),
level LowCardinality(String),
host String,
message String,
attrs JSON
)
ENGINE = MergeTree
PARTITION BY toYYYYMM(event_time)
ORDER BY (service, event_time, host);CREATE TABLE logs_rollup_1m
(
bucket DateTime,
service LowCardinality(String),
level LowCardinality(String),
count_state AggregateFunction(count)
)
ENGINE = AggregatingMergeTree
PARTITION BY toYYYYMM(bucket)
ORDER BY (service, level, bucket);