## Purpose Defines the ClickHouse database schema, table engines, deduplication semantics, and automated idempotent initialization for storing normalized CDRs. ## ADDED Requirements ### Requirement: Database and Table Schema Definition The storage layer SHALL provide a database named `telecom` and a table named `cdr_records` optimized for high-volume append and analytical queries. #### Scenario: Database and table initialized idempotently - **WHEN** the initialization script or container entrypoint executes - **THEN** ClickHouse creates database `telecom` (`CREATE DATABASE IF NOT EXISTS`) and table `telecom.cdr_records` (`CREATE TABLE IF NOT EXISTS`) using `ReplacingMergeTree(processed_at)`, partitioned by `toYYYYMM(session_start_time)`, ordered by `(service_type, radio_access_type, cell_id, session_start_time, msisdn)`, with `TTL session_start_time + INTERVAL 90 DAY` and `SETTINGS index_granularity = 8192` ### Requirement: Deduplication on At-Least-Once Replay The storage layer SHALL support idempotent re-insertion of records without creating duplicate entries in analytical views. #### Scenario: Duplicate record inserted after crash replay - **WHEN** a batch containing previously written records is re-inserted due to consumer restart before offset commit - **THEN** ClickHouse accepts the rows into `ReplacingMergeTree(processed_at)` and deduplicates rows sharing the same order key upon merge or when queried with `FINAL` ### Requirement: Source IP Column Type Compatibility The storage layer and worker client SHALL store `source_ip` using the ClickHouse `IPv4` data type. #### Scenario: IPv4 string inserted into table - **WHEN** a normalized record with an IPv4 address string (e.g. `"10.240.12.1"`) is inserted - **THEN** the value is inserted as a valid IPv4 representation compatible with ClickHouse's internal `IPv4` (`UInt32`) structure using `toIPv4()` casting or native client IP types