## Purpose Defines the automated test traffic generator, scenario coverage, and end-to-end pipeline verification reporting requirements. ## ADDED Requirements ### Requirement: Automated Traffic Generator Scenario Coverage The test suite SHALL provide a script that generates synthetic CDR traffic to exercise all core pipeline behaviors. #### Scenario: Valid records ingestion - **WHEN** the generator sends 1,000 valid records with varied RAT and service types - **THEN** all 1,000 records are successfully stored in ClickHouse and `cdr_processed_total` increments accordingly #### Scenario: Missing required field validation - **WHEN** the generator sends 20 records missing `caller_num` - **THEN** all 20 records are rejected by the API with HTTP 400 or routed to DLQ with `missing_required_field` #### Scenario: Malformed JSON validation - **WHEN** the generator sends 10 invalid JSON payloads - **THEN** the API returns HTTP 400 with structured JSON error details #### Scenario: Unknown RAT routing - **WHEN** the generator sends 10 records with unsupported `rat` values (such as `rat: 99`) - **THEN** all 10 records land in `cdr.raw.dlq` with `invalid_rat_value` #### Scenario: Invalid epoch timestamps - **WHEN** the generator sends 10 records with `epoch_start > epoch_end` - **THEN** all 10 records land in `cdr.raw.dlq` with `invalid_epoch` #### Scenario: Duplicate record deduplication - **WHEN** the generator sends 20 duplicate records with identical `raw_id` - **THEN** ClickHouse deduplicates the rows so that only 1 record per unique `raw_id` is retained #### Scenario: High-volume burst resilience - **WHEN** the generator sends a burst of 500 records in less than 5 seconds - **THEN** the API accepts the requests without returning 429 and all 500 records are processed without loss ### Requirement: Verification Summary Reporting The test script SHALL inspect ClickHouse, the DLQ, and API metrics after test execution and output a verification summary. #### Scenario: Summary report printed - **WHEN** all traffic generation scenarios complete and assertions are verified - **THEN** the script prints a formatted summary comparing: - Records sent - Records in ClickHouse - Records in DLQ - API 400 errors - Duplicate `raw_id`s resolved