docs: limit analytics reporting periods

This commit is contained in:
2026-08-20 16:03:52 +03:30
parent f472ad5b1d
commit 7c8afeaa4b
6 changed files with 34 additions and 59 deletions

View File

@@ -14,7 +14,7 @@ Perform a one-time, repeatable bootstrap of the supplied payment-attempt archive
- One-time administrative bootstrap/recovery execution; no ingestion scheduler or file watcher
- Import runs, checksums, schema version, row counts, rejection/error reporting
- Normalized merchant, category, terminal, payment-session, payment-attempt, and account/profile concepts
- Versioned Gregorian/Jalali calendar dates and curated Iranian event/holiday periods
- Gregorian/Jalali reporting dates with day, week, month, and Persian-season attributes
- Rebuildable daily base summaries and metric-specific period projections for approved analytics
- Synthetic merchant display profiles and one demo account per merchant
- Idempotent re-runs and explicit import lifecycle
@@ -40,8 +40,6 @@ erDiagram
MERCHANT ||--o{ PAYMENT_SESSION : receives
TERMINAL ||--o{ PAYMENT_SESSION : handles
PAYMENT_SESSION ||--|{ PAYMENT_ATTEMPT : contains
CALENDAR_DATE ||--o{ CALENDAR_DATE_EVENT : includes
CALENDAR_EVENT ||--o{ CALENDAR_DATE_EVENT : occurs_on
CALENDAR_DATE ||--o{ MERCHANT_DAY_SUMMARY : groups
MERCHANT ||--o{ MERCHANT_DAY_SUMMARY : summarizes
IMPORT_RUN ||--o{ PAYMENT_SESSION : introduced_by
@@ -126,17 +124,7 @@ erDiagram
int jalali_day
text persian_season
int iso_weekday
}
CALENDAR_EVENT {
bigint id PK
text code UK
text title_fa
text event_type
text source_version
}
CALENDAR_DATE_EVENT {
date date FK
bigint calendar_event_id FK
date reporting_week_start
}
MERCHANT_DAY_SUMMARY {
bigint merchant_id FK
@@ -192,11 +180,11 @@ This contract applies only to the one-time challenge-data bootstrap and explicit
- Missing values remain missing rather than being silently replaced with factual-looking defaults.
- The import report captures checksum, importer revision, timings, counts, validation failures, and quality metrics.
- Raw data and database state remain ignored by Git; code, schema, rules, docs, and safe templates are tracked.
- Daily, Jalali-month, Persian-season, and approved calendar-event analytics reconcile to deterministic source queries for additive metrics.
- Daily, weekly, Jalali-month, and Persian-season analytics reconcile to deterministic source queries for additive metrics.
- Rates are recomputed from their numerator and denominator over the requested period rather than averaged from daily percentages.
- Non-additive metrics such as unique payer-token counts are never produced by summing daily distinct counts.
- Incremental projections become queryable within 15 minutes of eligible live writes, and nightly reconciliation detects/corrects drift without double counting.
- Calendar rows covering the reporting period reconcile with reviewed official snapshots and record provenance/checksums.
- Reporting-date rows covering the dataset range reconcile deterministically between Gregorian and Jalali day, week, month, and season attributes.
## Approval gates / unresolved decisions
@@ -212,12 +200,12 @@ Resolved: retain `payer_card_key` unchanged as a restricted, merchant-scoped opa
Resolved: peer benchmarks exclude the active merchant and require at least 10 other merchants plus 1,000 combined peer sessions in the approved cohort/window. Otherwise the benchmark is suppressed as insufficient data; the merchant's own analytics remain available. See ADR 0006.
Resolved: use normalized source-of-truth entities plus versioned calendar/event dimensions, rebuildable daily base summaries, and metric-specific projections for non-additive or measured high-cost period analytics. Monthly, Persian-season, and holiday results use explicit calendar definitions. Keep fact tables unpartitioned initially and introduce partitioning only from measured need. See ADR 0007.
Resolved: use normalized source-of-truth entities plus a reporting-date dimension, rebuildable daily base summaries, and metric-specific projections for non-additive or measured high-cost period analytics. Daily, weekly, Jalali-month, and Persian-season results use explicit date boundaries. Keep fact tables unpartitioned initially and introduce partitioning only from measured need. See ADR 0007.
Resolved: ZarinPal writes future production rows directly to the normalized tables. Our backend owns schema/read/analytics contracts, refreshes projections within 15 minutes, and performs nightly reconciliation. CSV ingestion remains one-time only. See ADR 0008.
Resolved: commit reviewed yearly Iranian calendar snapshots sourced from the University of Tehran Institute of Geophysics Calendar Center; do not call a calendar API at runtime. Official holidays and analytical campaign/event windows remain distinguishable and versioned. See ADR 0009.
Resolved: holiday and named-event reporting are removed from scope. MVP reporting periods are limited to daily, weekly, Jalali-month, and Persian-season views. See ADR 0010.
Resolved: only source session status `Verified` is a definitely successful payment. Keep `Paid` separate until an authoritative ZarinPal definition changes the metric catalogue. See `METRIC-CATALOGUE.md`.
No broad architecture approval gates remain for this capability. Physical columns, indexes, constraints, calendar source, projection shapes, and migration details still require profiling evidence and acceptance in the implementation-ready revision of this spec.
No broad architecture approval gates remain for this capability. Physical columns, indexes, constraints, reporting-period boundaries, projection shapes, and migration details still require profiling evidence and acceptance in the implementation-ready revision of this spec.

View File

@@ -25,9 +25,7 @@
| `merchant_categories` | Five source business categories | unique source category key |
| `psps` | Opaque PSP reference codes | unique source PSP code |
| `issuer_banks` | Opaque bank reference codes | unique source bank code |
| `calendar_dates` | Gregorian/Jalali reporting attributes | Gregorian date PK; unique Jalali date tuple |
| `calendar_events` | Versioned holidays/events | stable event code + source version |
| `calendar_date_events` | Many-to-many event membership | unique date/event pair; FKs |
| `calendar_dates` | Gregorian/Jalali day, week, month, and Persian-season reporting attributes | Gregorian date PK; unique Jalali date tuple |
Switch response codes remain nullable text on attempts until an authoritative description catalogue exists. Normalizing an undocumented 97.4%-missing field into a lookup table adds little value.
@@ -59,7 +57,7 @@ Both fact tables distinguish `bootstrap` from `zarinpal_live` origin. Bootstrap
|---|---|---|
| `merchant_day_metrics` | merchant + Tehran local date | Additive counts/sums and metric numerators/denominators |
| `merchant_day_*_breakdowns` | merchant + date + approved dimension | Only metric-catalogue breakdowns proven useful, such as PSP/failure groups |
| `merchant_period_*` | merchant + approved period/event | Non-additive or measured expensive metrics only |
| `merchant_period_*` | merchant + approved day/week/month/season period | Non-additive or measured expensive metrics only |
| `cohort_period_*` | approved cohort + period | Suppressed peer benchmark inputs/outputs with provenance and freshness |
Projection names/columns are finalized by the analytics metric catalogue. Avoid a generic key/value metrics table because it weakens types, constraints, and discoverability.

View File

@@ -5,7 +5,7 @@
## Context
The source contains approximately 2.2 million payment-attempt rows, but attempts are only one domain entity. Merchant, terminal, session, attempt, import, identity, and calendar concepts require separate normalized relations connected by constraints. The product also needs daily, Jalali-month, Persian-season, and Iranian holiday/event analysis, including evidence that may later support bounded AI recommendations.
The source contains approximately 2.2 million payment-attempt rows, but attempts are only one domain entity. Merchant, terminal, session, attempt, import, identity, and reporting-date concepts require separate normalized relations connected by constraints. The product needs daily, weekly, Jalali-month, and Persian-season analysis.
Calculating every dashboard result from normalized attempt rows on every request creates avoidable latency. Conversely, splitting attempts into unrelated tables by month/category or adding PostgreSQL partitions before measuring query and maintenance behavior introduces complexity without evidence. Some metrics are additive across days, while distinct payer-token counts and percentiles are not.
@@ -13,9 +13,9 @@ Calculating every dashboard result from normalized attempt rows on every request
Keep normalized business/source-of-truth relations: categories, merchants, merchant users, terminals, payment sessions, payment attempts, and import provenance. The attempt relation retains one row per source attempt; normalization does not arbitrarily distribute a single entity's rows across category/month tables.
Add a versioned calendar model containing Gregorian and Jalali date attributes, Persian seasons, weekdays, and curated named events such as Nowruz. Calendar-event definitions have explicit provenance/version and are deterministic inputs to analytics, not LLM-generated labels.
Add a reporting-date model containing Gregorian and Jalali date attributes, explicit reporting-week boundaries, Persian months/seasons, and weekdays.
Build idempotent, rebuildable daily base projections containing additive components such as session count, attempt count, successful count, amount sum, and fee sum as approved by the metric catalogue. Derive month/season/event totals by grouping these components through the calendar model and recompute ratios from their numerators and denominators.
Build idempotent, rebuildable daily base projections containing additive components such as session count, attempt count, successful count, amount sum, and fee sum as approved by the metric catalogue. Derive week/month/season totals by grouping these components through the reporting-date model and recompute ratios from their numerators and denominators.
Create separate metric-specific period projections only for non-additive metrics or query paths proven expensive—for example, exact unique payer-token counts cannot be obtained by summing daily distinct counts. Projection rows record source import/version and freshness.
@@ -25,8 +25,6 @@ Keep normalized fact tables unpartitioned initially. Add targeted indexes from a
- The normalized ERD remains semantically correct while dashboards read smaller, purpose-built projections.
- Daily components support many calendar groupings without precomputing every possible period.
- Calendar definitions and metric provenance make future AI observations traceable.
- A single observed Nowruz period may be reported as an observation but not claimed as recurring seasonality; multiple comparable years are needed for that claim.
- Explicit reporting-period definitions and metric provenance make future AI observations traceable.
- Projection refresh, failure, stale-state visibility, and reconciliation tests become required operational behavior.
- Schema/index/projection details remain subject to data profiling and approved metric/API requirements.

View File

@@ -1,24 +0,0 @@
# ADR 0009: Version reviewed Iranian calendar snapshots in Git
- Status: Accepted
- Date: 2026-08-20
## Context
Monthly, Persian-season, Nowruz, and public-holiday analytics need deterministic Gregorian/Jalali mappings and named date ranges. Runtime dependence on a public calendar API would add availability, version drift, and reproducibility risks. Official holiday dates can change across years, especially events derived from other calendars.
## Decision
Commit a reviewed, machine-readable calendar snapshot for every supported Jalali year. Use the Calendar Center of the University of Tehran Institute of Geophysics as the authoritative annual source (`https://calendar.ut.ac.ir/`). The source URL/document identity, retrieval date, checksum, reviewer, and snapshot schema version accompany each file.
For the challenge period, cover both Jalali 1404 and 1405 because Gregorian January–June 2026 crosses the Jalali year boundary. Runtime code reads validated database calendar tables populated from these versioned snapshots and never calls an external calendar API.
Differentiate official public holidays from analytical event/campaign windows. A Nowruz analytical window may extend beyond officially closed days, but its exact dates, rationale, and version are explicit rather than inferred by an LLM.
## Consequences
- Historical analytics are reproducible even if websites/APIs change.
- Annual calendar updates are reviewed source changes with validation tests.
- Unsupported years fail clearly instead of guessing holiday membership.
- Calendar data is committed reference data; raw payment archives remain ignored.

View File

@@ -0,0 +1,20 @@
# ADR 0010: Limit MVP reporting periods
- Status: Accepted
- Date: 2026-08-20
## Context
Earlier drafts included named Iranian holidays and analytical event windows. That scope adds calendar sourcing, versioning, UX distinctions, and analytics definitions that are not required for the challenge MVP.
## Decision
Limit reporting periods to Tehran-local daily, weekly, Jalali-month, and Persian-season views. Remove named holidays, Nowruz-specific periods, public-holiday calendars, and generic event-window reporting from the current product and data model.
The exact weekly boundary remains an explicit reporting-period decision; it must not be inferred differently by the database, API, and frontend.
## Consequences
- No holiday/event entities, imported holiday snapshots, runtime calendar API, or holiday-specific UI are required.
- A reporting-date dimension remains useful for deterministic Gregorian/Jalali conversion and consistent week, month, and season grouping.
- Holiday/event analytics may return only through a future approved spec and ADR.