amiirkhl/telecom-cdr-pipeline-ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(pipeline): align worker and storage contracts

Browse Source
This commit is contained in:
Amirhossein Khalili
2026-09-20 10:40:58 +03:30
parent 81cef7f732
commit dc66e7fe97
4 changed files with 13 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
openspec/changes/telecom-cdr-pipeline/specs/clickhouse-storage/spec.md
View File

@@ -16,7 +16,7 @@ The storage layer SHALL support idempotent re-insertion of records without creat
#### Scenario: Duplicate record inserted after crash replay
- **WHEN** a batch containing previously written records is re-inserted due to consumer restart before offset commit
- **THEN** ClickHouse accepts the rows into `ReplacingMergeTree(processed_at)` and deduplicates rows sharing the same order key upon merge or when queried with `FINAL`
- **THEN** ClickHouse accepts the rows into `ReplacingMergeTree(processed_at)` and deduplicates exact replays sharing the deterministic `cdr_id`-bearing order key upon merge or when queried with `FINAL`, without collapsing distinct `raw_id` values whose session metadata happens to match
### Requirement: Source IP Column Type Compatibility
The storage layer and worker client SHALL store `source_ip` using the ClickHouse `IPv4` data type.
@@ -29,9 +29,9 @@ The storage layer and worker client SHALL store `source_ip` using the ClickHouse
Initialization SHALL use one canonical schema, verify existing table compatibility and never reset data automatically.
#### Scenario: Distinct identities share analytical metadata
- **WHEN** records have different raw_id values but identical other sorting fields
- **THEN** both survive FINAL and background replacement, while exact replay yields one logical row
- **WHEN** records have different `raw_id` values but identical other sorting fields
- **THEN** both survive `FINAL` and background replacement, while an exact replay yields one logical row
#### Scenario: Existing incompatible table
- **WHEN** an existing table lacks the corrected identity sorting key
- **WHEN** an existing table does not use the canonical identity-bearing sorting key
- **THEN** initialization fails with explicit assessment reset instructions without deleting data