amiirkhl/telecom-cdr-pipeline-ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(pipeline): align delivery contracts and verification requirements

Browse Source
This commit is contained in:
Amirhossein Khalili
2026-09-19 21:31:05 +03:30
parent 977f838eb2
commit bf439ab03c
10 changed files with 209 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
openspec/changes/telecom-cdr-pipeline/specs/clickhouse-storage/spec.md
View File

@@ -9,7 +9,7 @@ The storage layer SHALL provide a database named `telecom` and a table named `cd
#### Scenario: Database and table initialized idempotently
- **WHEN** the initialization script or container entrypoint executes
- **THEN** ClickHouse creates database `telecom` (`CREATE DATABASE IF NOT EXISTS`) and table `telecom.cdr_records` (`CREATE TABLE IF NOT EXISTS`) using `ReplacingMergeTree(processed_at)`, partitioned by `toYYYYMM(session_start_time)`, ordered by `(service_type, radio_access_type, cell_id, session_start_time, msisdn)`, with `TTL session_start_time + INTERVAL 90 DAY` and `SETTINGS index_granularity = 8192`
- **THEN** ClickHouse creates database `telecom` (`CREATE DATABASE IF NOT EXISTS`) and table `telecom.cdr_records` (`CREATE TABLE IF NOT EXISTS`) using `ReplacingMergeTree(processed_at)`, partitioned by `toYYYYMM(session_start_time)`, ordered by `(service_type, radio_access_type, cell_id, session_start_time, msisdn, cdr_id)`, with `TTL session_start_time + INTERVAL 90 DAY` and `SETTINGS index_granularity = 8192`
### Requirement: Deduplication on At-Least-Once Replay
The storage layer SHALL support idempotent re-insertion of records without creating duplicate entries in analytical views.
@@ -24,3 +24,14 @@ The storage layer and worker client SHALL store `source_ip` using the ClickHouse
#### Scenario: IPv4 string inserted into table
- **WHEN** a normalized record with an IPv4 address string (e.g. `"10.240.12.1"`) is inserted
- **THEN** the value is inserted as a valid IPv4 representation compatible with ClickHouse's internal `IPv4` (`UInt32`) structure using `toIPv4()` casting or native client IP types
### Requirement: Identity preservation and schema compatibility
Initialization SHALL use one canonical schema, verify existing table compatibility and never reset data automatically.
#### Scenario: Distinct identities share analytical metadata
- **WHEN** records have different raw_id values but identical other sorting fields
- **THEN** both survive FINAL and background replacement, while exact replay yields one logical row
#### Scenario: Existing incompatible table
- **WHEN** an existing table lacks the corrected identity sorting key
- **THEN** initialization fails with explicit assessment reset instructions without deleting data