ClickHouse Log Data Flow

ClickHouse Log Data Flow A sequence diagram generated by Archify. Generate logs Buffering Kafka Engine ingestion Stored in MergeTree table Move cold data Based on TTL policy Ingestion Tiering (TTL) Application · Generate logs · Sequence participant Application Generate logs FluentBit · collector · Sequence participant FluentBit collector Kafka · optional · buffer · Sequence participant Kafka optional · buffer ClickHouse · MergeTree · Sequence participant ClickHouse MergeTree S3 · cold volume · Sequence participant S3 cold volume Legend request async trace default message