feat: add clickhouse micro-batch storage

README.md

@@ -28,8 +28,12 @@ Redpanda / cdr.raw
 Processing Worker
     ↓
 Validation + Normalization
-    ├── valid → processed by worker
-    └── invalid → cdr.raw.dlq
+    ├── invalid → cdr.raw.dlq
+    └── valid
+          ↓
+      Micro-Batch
+          ↓
+      ClickHouse
 ```
 
 Infrastructure already available:
@@ -72,6 +76,11 @@ worker processing is not connected yet.
 * DLQ routing for malformed and unprocessable Kafka records
 * DLQ records with failure reason, failed field, timestamp, and worker instance
 * Kafka offsets committed only after the current processing outcome succeeds
+* ClickHouse bulk storage through the official Python client
+* Configurable CDR micro-batching
+* Flush on 1,000 records or 2-second timeout
+* Kafka offsets committed only after successful ClickHouse batch writes
+* Remaining worker buffer flushed during graceful shutdown
 
 ## Project Structure
 
@@ -106,6 +115,8 @@ telecom-cdr-pipeline/
 │       │
 │       └── worker/
 │           ├── __init__.py
+│           ├── batch.py
+│           ├── clickhouse.py
 │           ├── consumer.py
 │           ├── dlq.py
 │           ├── main.py
@@ -116,6 +127,7 @@ telecom-cdr-pipeline/
 │   │   └── test_ingestion_api.py
 │   │
 │   └── unit/
+│       ├── test_batch.py
 │       ├── test_dlq.py
 │       ├── test_normalization.py
 │       ├── test_project_setup.py