## Purpose Defines message broker topics, partitioning, retention, and dead-letter queue envelope specifications for decoupling and reliable asynchronous buffer management. ## ADDED Requirements ### Requirement: Raw Ingestion Topic Configuration The message broker SHALL provide a topic named `cdr.raw` configured for high-throughput buffering of raw vendor CDR payloads. #### Scenario: Raw topic configuration applied - **WHEN** the message broker initializes or topic metadata is queried - **THEN** topic `cdr.raw` exists with 3 partitions, replication factor of 1, retention time of 86,400,000 ms (24 hours), and LZ4 compression enabled ### Requirement: Dead Letter Queue Topic Configuration The message broker SHALL provide a topic named `cdr.raw.dlq` to retain unprocessable or malformed records for inspection and replay. #### Scenario: DLQ topic configuration applied - **WHEN** the message broker initializes or topic metadata is queried - **THEN** topic `cdr.raw.dlq` exists with 1 partition, replication factor of 1, and retention time of 604,800,000 ms (7 days) ### Requirement: Standardized DLQ Message Envelope All messages published to `cdr.raw.dlq` SHALL be wrapped in a standardized JSON envelope containing error diagnostic metadata. #### Scenario: DLQ envelope structure enforced - **WHEN** a message is routed to `cdr.raw.dlq` due to processing or validation failure - **THEN** the message payload contains a JSON object with top-level fields: `original_payload` (object or string), `failure_reason` (string), `failed_field` (nullable string), `failed_at` (ISO 8601 UTC timestamp), and `worker_instance` (string identifier) #### Scenario: Failure reason taxonomy enforced - **WHEN** any record is routed to `cdr.raw.dlq` - **THEN** the `failure_reason` value matches one of: `missing_required_field`, `invalid_rat_value`, `invalid_epoch`, `malformed_json`, or `normalization_error` - **THEN** the `failure_reason` value matches one of: `missing_required_field`, `invalid_rat_value`, `invalid_epoch`, `malformed_json`, `normalization_error`