docs: limit analytics reporting periods
This commit is contained in:
@@ -32,13 +32,9 @@ One traceable attempt to ingest a specific source archive/schema using a specifi
|
||||
|
||||
One civil date enriched with Gregorian and Jalali calendar attributes, Persian month/season, weekday, and reporting attributes.
|
||||
|
||||
## Calendar Event
|
||||
|
||||
A curated, versioned named period relevant to analysis, such as Nowruz or another Iranian holiday. Event membership is evidence supplied by the calendar model, not inferred by an LLM.
|
||||
|
||||
## Reporting Period
|
||||
|
||||
An approved time window used for metrics: day, Jalali month, Persian season, calendar event, or another explicitly defined comparison window.
|
||||
An approved time window used for metrics: Tehran-local day, week, Jalali month, or Persian season.
|
||||
|
||||
## Analytical Metric
|
||||
|
||||
@@ -51,4 +47,3 @@ A rebuildable summary derived from normalized source-of-truth records to answer
|
||||
## Peer Cohort
|
||||
|
||||
An approved group of other merchants used for anonymized comparison. The active merchant is excluded, and a benchmark is suppressed unless at least 10 peers and 1,000 peer sessions contribute.
|
||||
|
||||
|
||||
@@ -14,7 +14,7 @@ Perform a one-time, repeatable bootstrap of the supplied payment-attempt archive
|
||||
- One-time administrative bootstrap/recovery execution; no ingestion scheduler or file watcher
|
||||
- Import runs, checksums, schema version, row counts, rejection/error reporting
|
||||
- Normalized merchant, category, terminal, payment-session, payment-attempt, and account/profile concepts
|
||||
- Versioned Gregorian/Jalali calendar dates and curated Iranian event/holiday periods
|
||||
- Gregorian/Jalali reporting dates with day, week, month, and Persian-season attributes
|
||||
- Rebuildable daily base summaries and metric-specific period projections for approved analytics
|
||||
- Synthetic merchant display profiles and one demo account per merchant
|
||||
- Idempotent re-runs and explicit import lifecycle
|
||||
@@ -40,8 +40,6 @@ erDiagram
|
||||
MERCHANT ||--o{ PAYMENT_SESSION : receives
|
||||
TERMINAL ||--o{ PAYMENT_SESSION : handles
|
||||
PAYMENT_SESSION ||--|{ PAYMENT_ATTEMPT : contains
|
||||
CALENDAR_DATE ||--o{ CALENDAR_DATE_EVENT : includes
|
||||
CALENDAR_EVENT ||--o{ CALENDAR_DATE_EVENT : occurs_on
|
||||
CALENDAR_DATE ||--o{ MERCHANT_DAY_SUMMARY : groups
|
||||
MERCHANT ||--o{ MERCHANT_DAY_SUMMARY : summarizes
|
||||
IMPORT_RUN ||--o{ PAYMENT_SESSION : introduced_by
|
||||
@@ -126,17 +124,7 @@ erDiagram
|
||||
int jalali_day
|
||||
text persian_season
|
||||
int iso_weekday
|
||||
}
|
||||
CALENDAR_EVENT {
|
||||
bigint id PK
|
||||
text code UK
|
||||
text title_fa
|
||||
text event_type
|
||||
text source_version
|
||||
}
|
||||
CALENDAR_DATE_EVENT {
|
||||
date date FK
|
||||
bigint calendar_event_id FK
|
||||
date reporting_week_start
|
||||
}
|
||||
MERCHANT_DAY_SUMMARY {
|
||||
bigint merchant_id FK
|
||||
@@ -192,11 +180,11 @@ This contract applies only to the one-time challenge-data bootstrap and explicit
|
||||
- Missing values remain missing rather than being silently replaced with factual-looking defaults.
|
||||
- The import report captures checksum, importer revision, timings, counts, validation failures, and quality metrics.
|
||||
- Raw data and database state remain ignored by Git; code, schema, rules, docs, and safe templates are tracked.
|
||||
- Daily, Jalali-month, Persian-season, and approved calendar-event analytics reconcile to deterministic source queries for additive metrics.
|
||||
- Daily, weekly, Jalali-month, and Persian-season analytics reconcile to deterministic source queries for additive metrics.
|
||||
- Rates are recomputed from their numerator and denominator over the requested period rather than averaged from daily percentages.
|
||||
- Non-additive metrics such as unique payer-token counts are never produced by summing daily distinct counts.
|
||||
- Incremental projections become queryable within 15 minutes of eligible live writes, and nightly reconciliation detects/corrects drift without double counting.
|
||||
- Calendar rows covering the reporting period reconcile with reviewed official snapshots and record provenance/checksums.
|
||||
- Reporting-date rows covering the dataset range reconcile deterministically between Gregorian and Jalali day, week, month, and season attributes.
|
||||
|
||||
## Approval gates / unresolved decisions
|
||||
|
||||
@@ -212,12 +200,12 @@ Resolved: retain `payer_card_key` unchanged as a restricted, merchant-scoped opa
|
||||
|
||||
Resolved: peer benchmarks exclude the active merchant and require at least 10 other merchants plus 1,000 combined peer sessions in the approved cohort/window. Otherwise the benchmark is suppressed as insufficient data; the merchant's own analytics remain available. See ADR 0006.
|
||||
|
||||
Resolved: use normalized source-of-truth entities plus versioned calendar/event dimensions, rebuildable daily base summaries, and metric-specific projections for non-additive or measured high-cost period analytics. Monthly, Persian-season, and holiday results use explicit calendar definitions. Keep fact tables unpartitioned initially and introduce partitioning only from measured need. See ADR 0007.
|
||||
Resolved: use normalized source-of-truth entities plus a reporting-date dimension, rebuildable daily base summaries, and metric-specific projections for non-additive or measured high-cost period analytics. Daily, weekly, Jalali-month, and Persian-season results use explicit date boundaries. Keep fact tables unpartitioned initially and introduce partitioning only from measured need. See ADR 0007.
|
||||
|
||||
Resolved: ZarinPal writes future production rows directly to the normalized tables. Our backend owns schema/read/analytics contracts, refreshes projections within 15 minutes, and performs nightly reconciliation. CSV ingestion remains one-time only. See ADR 0008.
|
||||
|
||||
Resolved: commit reviewed yearly Iranian calendar snapshots sourced from the University of Tehran Institute of Geophysics Calendar Center; do not call a calendar API at runtime. Official holidays and analytical campaign/event windows remain distinguishable and versioned. See ADR 0009.
|
||||
Resolved: holiday and named-event reporting are removed from scope. MVP reporting periods are limited to daily, weekly, Jalali-month, and Persian-season views. See ADR 0010.
|
||||
|
||||
Resolved: only source session status `Verified` is a definitely successful payment. Keep `Paid` separate until an authoritative ZarinPal definition changes the metric catalogue. See `METRIC-CATALOGUE.md`.
|
||||
|
||||
No broad architecture approval gates remain for this capability. Physical columns, indexes, constraints, calendar source, projection shapes, and migration details still require profiling evidence and acceptance in the implementation-ready revision of this spec.
|
||||
No broad architecture approval gates remain for this capability. Physical columns, indexes, constraints, reporting-period boundaries, projection shapes, and migration details still require profiling evidence and acceptance in the implementation-ready revision of this spec.
|
||||
|
||||
@@ -25,9 +25,7 @@
|
||||
| `merchant_categories` | Five source business categories | unique source category key |
|
||||
| `psps` | Opaque PSP reference codes | unique source PSP code |
|
||||
| `issuer_banks` | Opaque bank reference codes | unique source bank code |
|
||||
| `calendar_dates` | Gregorian/Jalali reporting attributes | Gregorian date PK; unique Jalali date tuple |
|
||||
| `calendar_events` | Versioned holidays/events | stable event code + source version |
|
||||
| `calendar_date_events` | Many-to-many event membership | unique date/event pair; FKs |
|
||||
| `calendar_dates` | Gregorian/Jalali day, week, month, and Persian-season reporting attributes | Gregorian date PK; unique Jalali date tuple |
|
||||
|
||||
Switch response codes remain nullable text on attempts until an authoritative description catalogue exists. Normalizing an undocumented 97.4%-missing field into a lookup table adds little value.
|
||||
|
||||
@@ -59,7 +57,7 @@ Both fact tables distinguish `bootstrap` from `zarinpal_live` origin. Bootstrap
|
||||
|---|---|---|
|
||||
| `merchant_day_metrics` | merchant + Tehran local date | Additive counts/sums and metric numerators/denominators |
|
||||
| `merchant_day_*_breakdowns` | merchant + date + approved dimension | Only metric-catalogue breakdowns proven useful, such as PSP/failure groups |
|
||||
| `merchant_period_*` | merchant + approved period/event | Non-additive or measured expensive metrics only |
|
||||
| `merchant_period_*` | merchant + approved day/week/month/season period | Non-additive or measured expensive metrics only |
|
||||
| `cohort_period_*` | approved cohort + period | Suppressed peer benchmark inputs/outputs with provenance and freshness |
|
||||
|
||||
Projection names/columns are finalized by the analytics metric catalogue. Avoid a generic key/value metrics table because it weakens types, constraints, and discoverability.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
## Context
|
||||
|
||||
The source contains approximately 2.2 million payment-attempt rows, but attempts are only one domain entity. Merchant, terminal, session, attempt, import, identity, and calendar concepts require separate normalized relations connected by constraints. The product also needs daily, Jalali-month, Persian-season, and Iranian holiday/event analysis, including evidence that may later support bounded AI recommendations.
|
||||
The source contains approximately 2.2 million payment-attempt rows, but attempts are only one domain entity. Merchant, terminal, session, attempt, import, identity, and reporting-date concepts require separate normalized relations connected by constraints. The product needs daily, weekly, Jalali-month, and Persian-season analysis.
|
||||
|
||||
Calculating every dashboard result from normalized attempt rows on every request creates avoidable latency. Conversely, splitting attempts into unrelated tables by month/category or adding PostgreSQL partitions before measuring query and maintenance behavior introduces complexity without evidence. Some metrics are additive across days, while distinct payer-token counts and percentiles are not.
|
||||
|
||||
@@ -13,9 +13,9 @@ Calculating every dashboard result from normalized attempt rows on every request
|
||||
|
||||
Keep normalized business/source-of-truth relations: categories, merchants, merchant users, terminals, payment sessions, payment attempts, and import provenance. The attempt relation retains one row per source attempt; normalization does not arbitrarily distribute a single entity's rows across category/month tables.
|
||||
|
||||
Add a versioned calendar model containing Gregorian and Jalali date attributes, Persian seasons, weekdays, and curated named events such as Nowruz. Calendar-event definitions have explicit provenance/version and are deterministic inputs to analytics, not LLM-generated labels.
|
||||
Add a reporting-date model containing Gregorian and Jalali date attributes, explicit reporting-week boundaries, Persian months/seasons, and weekdays.
|
||||
|
||||
Build idempotent, rebuildable daily base projections containing additive components such as session count, attempt count, successful count, amount sum, and fee sum as approved by the metric catalogue. Derive month/season/event totals by grouping these components through the calendar model and recompute ratios from their numerators and denominators.
|
||||
Build idempotent, rebuildable daily base projections containing additive components such as session count, attempt count, successful count, amount sum, and fee sum as approved by the metric catalogue. Derive week/month/season totals by grouping these components through the reporting-date model and recompute ratios from their numerators and denominators.
|
||||
|
||||
Create separate metric-specific period projections only for non-additive metrics or query paths proven expensive—for example, exact unique payer-token counts cannot be obtained by summing daily distinct counts. Projection rows record source import/version and freshness.
|
||||
|
||||
@@ -25,8 +25,6 @@ Keep normalized fact tables unpartitioned initially. Add targeted indexes from a
|
||||
|
||||
- The normalized ERD remains semantically correct while dashboards read smaller, purpose-built projections.
|
||||
- Daily components support many calendar groupings without precomputing every possible period.
|
||||
- Calendar definitions and metric provenance make future AI observations traceable.
|
||||
- A single observed Nowruz period may be reported as an observation but not claimed as recurring seasonality; multiple comparable years are needed for that claim.
|
||||
- Explicit reporting-period definitions and metric provenance make future AI observations traceable.
|
||||
- Projection refresh, failure, stale-state visibility, and reconciliation tests become required operational behavior.
|
||||
- Schema/index/projection details remain subject to data profiling and approved metric/API requirements.
|
||||
|
||||
|
||||
@@ -1,24 +0,0 @@
|
||||
# ADR 0009: Version reviewed Iranian calendar snapshots in Git
|
||||
|
||||
- Status: Accepted
|
||||
- Date: 2026-08-20
|
||||
|
||||
## Context
|
||||
|
||||
Monthly, Persian-season, Nowruz, and public-holiday analytics need deterministic Gregorian/Jalali mappings and named date ranges. Runtime dependence on a public calendar API would add availability, version drift, and reproducibility risks. Official holiday dates can change across years, especially events derived from other calendars.
|
||||
|
||||
## Decision
|
||||
|
||||
Commit a reviewed, machine-readable calendar snapshot for every supported Jalali year. Use the Calendar Center of the University of Tehran Institute of Geophysics as the authoritative annual source (`https://calendar.ut.ac.ir/`). The source URL/document identity, retrieval date, checksum, reviewer, and snapshot schema version accompany each file.
|
||||
|
||||
For the challenge period, cover both Jalali 1404 and 1405 because Gregorian January–June 2026 crosses the Jalali year boundary. Runtime code reads validated database calendar tables populated from these versioned snapshots and never calls an external calendar API.
|
||||
|
||||
Differentiate official public holidays from analytical event/campaign windows. A Nowruz analytical window may extend beyond officially closed days, but its exact dates, rationale, and version are explicit rather than inferred by an LLM.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Historical analytics are reproducible even if websites/APIs change.
|
||||
- Annual calendar updates are reviewed source changes with validation tests.
|
||||
- Unsupported years fail clearly instead of guessing holiday membership.
|
||||
- Calendar data is committed reference data; raw payment archives remain ignored.
|
||||
|
||||
20
docs/decisions/0010-limit-reporting-periods.md
Normal file
20
docs/decisions/0010-limit-reporting-periods.md
Normal file
@@ -0,0 +1,20 @@
|
||||
# ADR 0010: Limit MVP reporting periods
|
||||
|
||||
- Status: Accepted
|
||||
- Date: 2026-08-20
|
||||
|
||||
## Context
|
||||
|
||||
Earlier drafts included named Iranian holidays and analytical event windows. That scope adds calendar sourcing, versioning, UX distinctions, and analytics definitions that are not required for the challenge MVP.
|
||||
|
||||
## Decision
|
||||
|
||||
Limit reporting periods to Tehran-local daily, weekly, Jalali-month, and Persian-season views. Remove named holidays, Nowruz-specific periods, public-holiday calendars, and generic event-window reporting from the current product and data model.
|
||||
|
||||
The exact weekly boundary remains an explicit reporting-period decision; it must not be inferred differently by the database, API, and frontend.
|
||||
|
||||
## Consequences
|
||||
|
||||
- No holiday/event entities, imported holiday snapshots, runtime calendar API, or holiday-specific UI are required.
|
||||
- A reporting-date dimension remains useful for deterministic Gregorian/Jalali conversion and consistent week, month, and season grouping.
|
||||
- Holiday/event analytics may return only through a future approved spec and ADR.
|
||||
Reference in New Issue
Block a user