docs(specs): initialize OpenSpec and define telecom CDR pipeline specifications

openspec/changes/telecom-cdr-pipeline/specs/observability/spec.md

@@ -0,0 +1,33 @@
+## Purpose
+
+Defines Prometheus metrics collection, operational stats reporting, and structured JSON logging across pipeline components.
+
+## ADDED Requirements
+
+### Requirement: Prometheus Metrics Exposition
+Each pipeline service SHALL expose standard Prometheus metrics at `GET /metrics`.
+
+#### Scenario: Ingestion API metrics exposed
+- **WHEN** Prometheus scrapes `GET /metrics` on the Ingestion API
+- **THEN** the response includes counter `cdr_api_received_total`
+
+#### Scenario: Processing Worker metrics exposed
+- **WHEN** Prometheus scrapes `GET /metrics` on the Processing Worker
+- **THEN** the response includes counters (`cdr_processed_total{service_type, radio_access_type}`, `cdr_dlq_total{reason}`), histograms (`cdr_kafka_to_clickhouse_latency_seconds`, `cdr_batch_flush_duration_seconds`, `cdr_batch_size_at_flush`), and gauges (`cdr_in_flight_buffer_size`, `cdr_kafka_consumer_lag{partition}`)
+
+### Requirement: Human-Readable Pipeline Stats JSON
+The Ingestion API SHALL serve a unified JSON summary at `GET /stats` representing end-to-end pipeline health.
+
+#### Scenario: Stats endpoint returns unified summary
+- **WHEN** a client issues `GET /stats` to the Ingestion API
+- **THEN** the API returns HTTP 200 with JSON matching the structure:
+  - `pipeline`: `total_received`, `total_processed`, `total_dlq`, `in_flight_buffer`, `last_flush_at`
+  - `breakdown`: `by_service_type`, `by_radio_access_type`, `dlq_by_reason`
+
+### Requirement: Structured JSON Logging
+All pipeline services SHALL emit log lines formatted as structured JSON to stdout.
+
+#### Scenario: Operational event logged
+- **WHEN** any service logs an event (info, warning, or error)
+- **THEN** the log output is a single-line JSON object containing at minimum `timestamp` (ISO 8601 UTC), `level` (INFO, WARN, ERROR), `service` (`ingestion-api` or `processing-worker`), `raw_id` (record identifier or null), and `event` (short descriptive string)
+