From 2753162a75acfc3a2c0ec94e1f36b19f521ef597 Mon Sep 17 00:00:00 2001 From: Amirhossein Khalili Date: Thu, 20 Aug 2026 13:15:11 +0330 Subject: [PATCH] docs: define source timestamp normalization --- AGENTS.md | 2 +- docs/DATA-FOUNDATION-SPEC.md | 11 ++++---- .../0004-source-timezone-normalization.md | 25 +++++++++++++++++++ 3 files changed, 32 insertions(+), 6 deletions(-) create mode 100644 docs/decisions/0004-source-timezone-normalization.md diff --git a/AGENTS.md b/AGENTS.md index b9ffa64..577a307 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -12,7 +12,7 @@ For every Go-related task, always invoke and follow the `golang-how-to` skill, t - Every merchant-scoped query derives merchant identity from the authenticated server-side session; never trust a client-supplied merchant identifier as authorization. - Cross-merchant results are minimum-cohort, anonymized aggregates only. - Use integer minor currency units or an explicitly documented fixed-precision representation; never floating point for money. -- Preserve source timestamps and document their timezone assumptions. Normalize operational timestamps consistently. +- Interpret timezone-naive challenge timestamps as `Asia/Tehran`, preserve their source provenance, and normalize canonical database/API timestamps to UTC `timestamptz`/ISO 8601. Any change to this assumption requires a new import version and ADR. - Raw imports are immutable inputs; corrections are new import versions, not silent mutation. - `adjusted_fee` is obfuscated and may be used only for relative analysis. diff --git a/docs/DATA-FOUNDATION-SPEC.md b/docs/DATA-FOUNDATION-SPEC.md index 636ae31..3012c8c 100644 --- a/docs/DATA-FOUNDATION-SPEC.md +++ b/docs/DATA-FOUNDATION-SPEC.md @@ -113,7 +113,7 @@ erDiagram } ``` -This ERD is a logical baseline, not an approved physical schema. In particular, source key types, timestamp timezone interpretation, card-key retention, analytics projections/materialized views, and whether session-level fields can conflict across retries must be resolved from a profiling report before migrations are approved. +This ERD is a logical baseline, not an approved physical schema. In particular, source key types, card-key retention, analytics projections/materialized views, and whether session-level fields can conflict across retries must be resolved from a profiling report before migrations are approved. ## Import contract @@ -152,9 +152,10 @@ Resolved: use manual constructor injection with explicit wiring at thin `cmd/` c Resolved: use Ent for application persistence/modeling and pgx/PostgreSQL `COPY` for streamed bulk-ingestion staging. Migrations are explicit and versioned; production startup never performs automatic schema changes. See ADR 0003. +Resolved: interpret timezone-naive source timestamps as `Asia/Tehran`, normalize canonical timestamps to UTC, expose ISO-8601 timestamps through APIs, and retain the immutable source/import provenance needed to reproduce the conversion. See ADR 0004. + Remaining approval gates: -1. Source timestamp timezone and daylight-saving assumptions. -2. Whether `payer_card_key` must be retained, irreversibly transformed, or omitted after derived analytics. -3. Minimum cohort size and suppression policy for peer benchmarks. -4. Physical partitioning/materialization strategy, which must follow measured query needs rather than the logical ERD alone. +1. Whether `payer_card_key` must be retained, irreversibly transformed, or omitted after derived analytics. +2. Minimum cohort size and suppression policy for peer benchmarks. +3. Physical partitioning/materialization strategy, which must follow measured query needs rather than the logical ERD alone. diff --git a/docs/decisions/0004-source-timezone-normalization.md b/docs/decisions/0004-source-timezone-normalization.md new file mode 100644 index 0000000..d8c492a --- /dev/null +++ b/docs/decisions/0004-source-timezone-normalization.md @@ -0,0 +1,25 @@ +# ADR 0004: Interpret source timestamps as Asia/Tehran and normalize to UTC + +- Status: Accepted +- Date: 2026-08-20 + +## Context + +The challenge archive contains timezone-naive timestamp strings such as `2026-01-02 11:52:56`. It supplies no offset or timezone column. Analytics depend on consistent daily/hourly windows, retry durations, settlement timing, and period comparisons, so leaving interpretation implicit would make results environment-dependent. + +## Decision + +Interpret every valid timezone-naive source timestamp as local civil time in the IANA `Asia/Tehran` zone. Convert canonical persisted timestamps to UTC-backed PostgreSQL `timestamptz` values. API contracts expose ISO-8601 timestamps with an explicit UTC offset (`Z`); merchant-facing presentation converts them to `Asia/Tehran` and uses the Jalali calendar. + +The importer records the assumed zone and timezone-data/runtime version in its manifest. The raw archive remains immutable operational input identified by checksum, preserving the original textual values and allowing a new import version if authoritative source metadata later contradicts the assumption. A changed interpretation must never silently update an existing completed import. + +Missing or invalid timestamp values remain missing/rejected according to the import validation policy; they are not replaced with import time. Timestamp-order anomalies are reported as data-quality evidence and are not automatically “corrected.” + +## Consequences + +- Day/hour analytics have one documented merchant-relevant timezone. +- Database comparisons and API interchange use explicit instants rather than server-local time. +- The assumption is traceable and reversible through source checksum, immutable input, manifest metadata, and importer revision. +- Tests must cover conversion around relevant timezone-rule boundaries and ensure host/container timezone does not change results. +- Calendar conversion is a presentation concern; backend period boundaries are calculated from explicit `Asia/Tehran` local ranges and converted to UTC for querying. +