docs: adopt lazy analytics caching

CONTEXT.md

@@ -44,6 +44,10 @@ A deterministic numerical measure with a definition, source fields, filters, per
 
 A rebuildable summary derived from normalized source-of-truth records to answer an approved metric efficiently. A projection is not independent business truth.
 
+## Analytics Revision
+
+A monotonically increasing merchant or cohort version changed when a related payment fact is inserted. Cached projections record the revision they represent; a revision mismatch makes them stale and triggers lazy recomputation on the next relevant dashboard request.
+
 ## Peer Cohort
 
 An approved group of other merchants used for anonymized comparison. The active merchant is excluded, and a benchmark is suppressed unless at least 10 peers and 1,000 peer sessions contribute.

README.md

@@ -21,6 +21,6 @@ One repository may expose several small `cmd/` entry points while sharing domain
 - HTTP API
 - dataset importer
 - database migration command
-- analytics refresh/background job, only if the approved spec requires one
+- on-demand analytics calculation within the HTTP API, persisted as merchant-scoped PostgreSQL projections
 
 The exact package layout is deferred until the Go architecture and DI decisions are approved.

docs/DATA-FOUNDATION-SPEC.md

@@ -125,6 +125,7 @@ erDiagram
       text persian_season
       int iso_weekday
       date reporting_week_start
+      date reporting_week_end
     }
     MERCHANT_DAY_SUMMARY {
       bigint merchant_id FK
@@ -150,17 +151,20 @@ This contract applies only to the one-time challenge-data bootstrap and explicit
 5. Validate uniqueness of `(session_key, try_seq)`, referential consistency, parseability, null rates, ranges, timestamp ordering, and merchant/terminal/category relationships.
 6. Promote accepted data transactionally or through a resumable, explicitly versioned workflow; a failed run cannot appear complete.
 7. Emit a machine-readable and human-readable import report. Do not log raw card keys or credentials.
-8. Populate and validate the approved calendar range, then build affected analytics projections idempotently after source promotion. Projection failure must not make an incomplete projection appear fresh.
+8. Populate and validate the approved reporting-date range, then initialize merchant/cohort analytics revisions. Do not eagerly calculate dashboard projections; first relevant access owns lazy calculation.
-9. Never schedule, watch, or poll for another CSV. After bootstrap, production source-row creation is an external ZarinPal responsibility. Analytics freshness over those future rows has a separate read/refresh contract.
+9. Never schedule, watch, or poll for another CSV. After bootstrap, production source-row creation is an external ZarinPal responsibility. Insert-triggered revisions invalidate affected lazy analytics projections.
 
-## Production write and analytics freshness contract
+## Production write and lazy analytics-cache contract
 
 - ZarinPal is the authoritative writer of future production payment sessions/attempts in the normalized PostgreSQL schema.
 - This project owns the normalized schema contract, migrations, read models, and analytics; it does not recreate ZarinPal's payment-write workflow.
-- Live rows are distinguishable from bootstrap rows and include stable creation/update metadata for incremental analytics. Bootstrap-only import provenance is not forged for live rows.
+- Payment facts are append-only in the MVP. Existing session/attempt updates and hard deletion are out of scope; a future mutation requirement needs a new invalidation contract.
-- Incremental analytics projections refresh within 15 minutes of an eligible source change.
+- The first relevant dashboard request calculates typed merchant analytics projections and persists them in PostgreSQL. Later requests reuse projections for the same metric-definition and source revisions.
-- A nightly idempotent reconciliation checks/recomputes the approved authoritative window and detects drift.
+- Every inserted related payment fact atomically advances the affected merchant analytics revision. It also advances the relevant cohort/category revision when peer metrics can change. Database-managed revision changes cover direct ZarinPal inserts that bypass Go request handlers.
-- Failed refreshes never advance the API/UI freshness watermark.
+- A request with missing or stale projections obtains a merchant-scoped PostgreSQL lock, recalculates once, publishes atomically, and records the exact source/cohort and metric-definition revisions represented. Concurrent requests reuse the completed result rather than duplicate expensive queries.
+- Invalidation marks cached results stale through revision mismatch; it does not synchronously recalculate or delete the last successful snapshot.
+- Failed calculations never mark a revision current. The API may expose the prior snapshot only with an explicit stale warning and last-calculated metadata.
+- No scheduled analytics refresh worker, nightly reconciliation job, Redis, or time-based expiry is required for the static MVP dataset.
 
 ## Synthetic merchant/account rules
 
@@ -183,8 +187,11 @@ This contract applies only to the one-time challenge-data bootstrap and explicit
 - Daily, weekly, Jalali-month, and Persian-season analytics reconcile to deterministic source queries for additive metrics.
 - Rates are recomputed from their numerator and denominator over the requested period rather than averaged from daily percentages.
 - Non-additive metrics such as unique payer-token counts are never produced by summing daily distinct counts.
-- Incremental projections become queryable within 15 minutes of eligible live writes, and nightly reconciliation detects/corrects drift without double counting.
+- First access creates reusable merchant-scoped projections; a second equivalent request reads the current cache without repeating source-fact aggregation.
+- Inserting a related payment fact advances only the affected merchant revision and any affected cohort revision; the next relevant request recomputes once and publishes atomically.
+- Concurrent stale-cache requests do not duplicate the same merchant calculation, and failed calculation does not claim the new revision is cached.
 - Reporting-date rows covering the dataset range reconcile deterministically between Gregorian and Jalali day, week, month, and season attributes.
+- Every reporting week starts Saturday at `00:00:00` and ends Friday at `23:59:59.999…` in `Asia/Tehran`; API ranges use an equivalent half-open `[start, next_start)` interval.
 
 ## Approval gates / unresolved decisions
 
@@ -192,7 +199,7 @@ Resolved: use a capability-oriented modular monolith, with lightweight ports/ada
 
 Resolved: use manual constructor injection with explicit wiring at thin `cmd/` composition roots. See ADR 0002.
 
-Resolved: use Ent for application persistence/modeling and pgx/PostgreSQL `COPY` for streamed bulk-ingestion staging. Migrations are explicit and versioned; production startup never performs automatic schema changes. See ADR 0003.
+Resolved: use Ent for application persistence/modeling, pgx/PostgreSQL `COPY` for streamed bulk-ingestion staging, and Atlas for reviewed versioned SQL migrations. Production startup never performs automatic schema changes. See ADR 0003.
 
 Resolved: interpret timezone-naive source timestamps as `Asia/Tehran`, normalize canonical timestamps to UTC, expose ISO-8601 timestamps through APIs, and retain the immutable source/import provenance needed to reproduce the conversion. See ADR 0004.
 
@@ -202,10 +209,12 @@ Resolved: peer benchmarks exclude the active merchant and require at least 10 ot
 
 Resolved: use normalized source-of-truth entities plus a reporting-date dimension, rebuildable daily base summaries, and metric-specific projections for non-additive or measured high-cost period analytics. Daily, weekly, Jalali-month, and Persian-season results use explicit date boundaries. Keep fact tables unpartitioned initially and introduce partitioning only from measured need. See ADR 0007.
 
-Resolved: ZarinPal writes future production rows directly to the normalized tables. Our backend owns schema/read/analytics contracts, refreshes projections within 15 minutes, and performs nightly reconciliation. CSV ingestion remains one-time only. See ADR 0008.
+Resolved: ZarinPal writes future append-only production rows directly to normalized tables. Analytics are calculated lazily on first relevant dashboard access, stored as typed PostgreSQL projections, and invalidated through merchant/cohort source revisions advanced on insert. No scheduled refresh or Redis is required. See ADR 0011.
 
 Resolved: holiday and named-event reporting are removed from scope. MVP reporting periods are limited to daily, weekly, Jalali-month, and Persian-season views. See ADR 0010.
 
+Resolved: reporting weeks follow the Iranian week: Saturday through Friday in `Asia/Tehran`, represented as half-open API/database ranges. See ADR 0012.
+
 Resolved: only source session status `Verified` is a definitely successful payment. Keep `Paid` separate until an authoritative ZarinPal definition changes the metric catalogue. See `METRIC-CATALOGUE.md`.
 
 No broad architecture approval gates remain for this capability. Physical columns, indexes, constraints, reporting-period boundaries, projection shapes, and migration details still require profiling evidence and acceptance in the implementation-ready revision of this spec.

docs/PHYSICAL-SCHEMA-PROPOSAL.md

@@ -49,7 +49,7 @@ Synthetic profile fields carry an explicit `profile_is_synthetic` marker. Creden
 
 `adjusted_fee` uses fixed precision and is labeled as an obfuscated relative value. The physical precision is finalized from exact decimal profiling before migration approval.
 
-Both fact tables distinguish `bootstrap` from `zarinpal_live` origin. Bootstrap rows reference an import run; live rows do not invent one. Live creation/update metadata forms the incremental analytics watermark contract, including status changes to an existing session.
+Both fact tables distinguish `bootstrap` from `zarinpal_live` origin. Bootstrap rows reference an import run; live rows do not invent one. Payment facts are append-only for the MVP; inserts advance database-managed analytics revisions.
 
 ### Rebuildable analytics projections
 
@@ -59,10 +59,12 @@ Both fact tables distinguish `bootstrap` from `zarinpal_live` origin. Bootstrap
 | `merchant_day_*_breakdowns` | merchant + date + approved dimension | Only metric-catalogue breakdowns proven useful, such as PSP/failure groups |
 | `merchant_period_*` | merchant + approved day/week/month/season period | Non-additive or measured expensive metrics only |
 | `cohort_period_*` | approved cohort + period | Suppressed peer benchmark inputs/outputs with provenance and freshness |
+| `merchant_analytics_state` | one row per merchant | current source revision, cached revision, metric-definition version, calculation status/time |
+| `cohort_analytics_state` | one row per benchmark cohort/category | current source revision and cached revision for peer projections |
 
 Projection names/columns are finalized by the analytics metric catalogue. Avoid a generic key/value metrics table because it weakens types, constraints, and discoverability.
 
-Projection metadata records source watermark, calculation version, calculated time, and status. An incremental refresh runs at least every 15 minutes; a nightly reconciliation checks/rebuilds an authoritative window and advances freshness only after successful atomic publication.
+Projection rows record merchant/cohort source revision, metric-definition version, calculated time, and publication status. Inserts atomically increment the affected revision through schema-owned database logic so direct upstream writes cannot bypass invalidation. A dashboard cache miss or revision mismatch acquires a PostgreSQL advisory/row lock, builds typed projections, and advances the cached revision only after atomic publication. No generic JSON metric cache, Redis, scheduled refresh, or time-to-live expiry is required.
 
 ## Initial index policy
 
@@ -77,9 +79,10 @@ Projection metadata records source watermark, calculation version, calculated ti
 
 - Payment/import provenance uses restrictive foreign keys; no cascading deletion of analytical facts from merchant/profile operations.
 - Imported source facts are immutable after a completed import. Corrections create a new versioned recovery/import decision.
+- Production payment facts are append-only in the MVP. Update/delete semantics require a future ADR and corresponding cache-invalidation rules.
 - Synthetic display profiles and credentials can change independently from source facts.
-- Derived projections may be rebuilt and atomically replaced; freshness/provenance prevents stale data from appearing current.
+- Derived projections may be rebuilt and atomically replaced; source and metric-definition revisions prevent stale data from appearing current.
 
 ## Production population boundary
 
-After the one-time CSV bootstrap, ZarinPal—not this project—writes future production payment records directly to the normalized PostgreSQL session/attempt tables. This project owns the schema/migration and analytics contract but not ZarinPal's write workflow. Compatibility, required fields, source origin, and creation/update watermarks are versioned interfaces; recurring CSV ETL is prohibited.
+After the one-time CSV bootstrap, ZarinPal—not this project—writes future production payment records directly to the normalized PostgreSQL session/attempt tables. This project owns the schema/migration, insert-triggered analytics-revision, and read contract but not ZarinPal's write workflow. Compatibility, required fields, source origin, and append-only insertion semantics are versioned interfaces; recurring CSV ETL is prohibited.

docs/decisions/0003-ent-pgx-data-access.md

@@ -13,7 +13,7 @@ Use Ent for application persistence, relationship modeling, and type-safe access
 
 Use pgx directly for the importer’s PostgreSQL staging boundary. Feed `CopyFrom` with a streaming `CopyFromSource` (or equivalent bounded batches) so decompressed CSV rows are not accumulated in memory. Validate and promote staged data through explicit transactions/steps described by the import spec; bulk loading must not bypass reconciliation or data-quality rules.
 
-Use versioned SQL migrations generated/planned from the reviewed schema and committed to Git. Apply them through a dedicated migration command or deployment step. Ent automatic schema creation is allowed only in disposable development/test contexts if the later implementation spec permits it; it must never run as part of production application startup.
+Use Atlas, Ent's supported migration engine, to generate/plan versioned SQL migrations from the reviewed Ent schema. Review and commit the SQL to Git, then apply it through a dedicated deployment migration step. Atlas is build/deployment tooling, not a runtime service. Ent automatic schema creation is allowed only in disposable development/test contexts if the later implementation spec permits it; it must never run as part of production application startup.
 
 ## Consequences
 
@@ -21,10 +21,10 @@ Use versioned SQL migrations generated/planned from the reviewed schema and comm
 - Engineers must still inspect generated SQL, query plans, indexes, transaction boundaries, and possible N+1 behavior; the ORM is not a performance or correctness boundary.
 - The importer owns a deliberate lower-level pgx path and cannot reuse ordinary Ent writes for convenience.
 - Both Ent and pgx require compatible pool/transaction ownership and context propagation; adapters must not hide unbounded work.
-- Migration generation tooling and the migration runner remain to be selected in the implementation spec. Versioned migration files—not runtime diffs—are the production contract.
+- Atlas-generated versioned migration files—not runtime diffs—are the production contract. CI validates migration integrity and drift before deployment.
 
 ## References
 
 - Ent versioned migrations: <https://entgo.io/docs/versioned/intro/>
+- Atlas versioned migrations: <https://atlasgo.io/concepts/declarative-vs-versioned#versioned-migrations>
 - pgx v5 `CopyFrom`: <https://pkg.go.dev/github.com/jackc/pgx/v5#Conn.CopyFrom>
-

docs/decisions/0007-calendar-aware-analytics-projections.md

@@ -26,5 +26,5 @@ Keep normalized fact tables unpartitioned initially. Add targeted indexes from a
 - The normalized ERD remains semantically correct while dashboards read smaller, purpose-built projections.
 - Daily components support many calendar groupings without precomputing every possible period.
 - Explicit reporting-period definitions and metric provenance make future AI observations traceable.
-- Projection refresh, failure, stale-state visibility, and reconciliation tests become required operational behavior.
+- Lazy calculation, revision invalidation, concurrency deduplication, failure, and stale-state visibility tests become required behavior.
 - Schema/index/projection details remain subject to data profiling and approved metric/API requirements.

docs/decisions/0008-production-write-and-analytics-freshness.md

@@ -1,25 +0,0 @@
-# ADR 0008: ZarinPal owns live writes; analytics refresh within 15 minutes
-
-- Status: Accepted
-- Date: 2026-08-20
-
-## Context
-
-The challenge archive is a one-time bootstrap source. In production, ZarinPal already creates payment data and will write future records directly to the normalized PostgreSQL schema. Building a recurring CSV importer, polling source, or competing payment-write workflow would duplicate upstream responsibility. Derived dashboard projections still need bounded freshness and correction when live source rows arrive or change status.
-
-## Decision
-
-ZarinPal is the authoritative producer of live `payment_sessions` and `payment_attempts`. This backend owns the normalized schema contract, versioned migrations, analytical reads/projections, and tenant-safe APIs, but not the business workflow that produces live payment records.
-
-Fact rows identify `bootstrap` versus `zarinpal_live` origin. Bootstrap rows reference import provenance; live rows do not fabricate import runs. The live-write contract supplies stable creation/update metadata so an idempotent high-watermark process can include new rows and status updates.
-
-Refresh affected analytics projections within 15 minutes. Run a nightly idempotent reconciliation over an approved authoritative window to detect and correct missed/changed rows. Publish projections atomically and advance the visible freshness watermark only after success.
-
-## Consequences
-
-- No recurring CSV ingestion service, file watcher, or source polling is built.
-- Schema migrations require compatibility coordination with ZarinPal's writer.
-- Analytics refresh is separate from ingestion and requires scheduling, checkpoints, idempotency, metrics, and stale-state UX.
-- If upstream cannot provide reliable update metadata, revise the refresh algorithm/window before implementation rather than silently missing mutations.
-- The static hackathon dataset builds projections once after bootstrap; production scheduling remains testable with synthetic live writes.
-

docs/decisions/0010-limit-reporting-periods.md

@@ -11,7 +11,7 @@ Earlier drafts included named Iranian holidays and analytical event windows. Tha
 
 Limit reporting periods to Tehran-local daily, weekly, Jalali-month, and Persian-season views. Remove named holidays, Nowruz-specific periods, public-holiday calendars, and generic event-window reporting from the current product and data model.
 
-The exact weekly boundary remains an explicit reporting-period decision; it must not be inferred differently by the database, API, and frontend.
+The reporting week follows the separately approved Saturday-through-Friday boundary in ADR 0012 and must not be inferred differently by the database, API, and frontend.
 
 ## Consequences
 

docs/decisions/0011-lazy-merchant-analytics-cache.md

@@ -0,0 +1,26 @@
+# ADR 0011: Calculate and cache merchant analytics lazily
+
+- Status: Accepted
+- Date: 2026-08-20
+
+## Context
+
+The hackathon dataset is static after its one-time bootstrap. Continuously refreshing analytics would spend resources before any merchant requests the data. The production boundary may later receive new append-only rows written directly by ZarinPal, bypassing Go request handlers, so any invalidation contract must also work at the database boundary.
+
+## Decision
+
+Calculate a merchant's approved analytics projections on the first relevant dashboard request and persist typed projections in PostgreSQL. Reuse them while their merchant source revision, relevant cohort revision, and metric-definition version remain current.
+
+Schema-owned database logic atomically increments the affected merchant revision whenever a related payment fact is inserted. Inserts that affect peer benchmarks also increment the corresponding category/cohort revision. This covers direct ZarinPal writes without a refresh worker or Redis.
+
+On a cache miss or revision mismatch, the backend obtains a merchant-scoped PostgreSQL advisory/row lock, computes the projections once, publishes them atomically, and then advances the cached revision. Concurrent requests wait for or reuse that result. A failed calculation does not advance the cached revision; an older snapshot may be returned only when clearly marked stale.
+
+Payment facts are append-only for this MVP. Cache invalidation for updates or hard deletion is out of scope and requires a future contract change.
+
+## Consequences
+
+- The static dataset is aggregated at most once per merchant and metric-definition revision when requested.
+- No scheduled analytics worker, nightly reconciliation, Redis, or time-based cache expiry is required.
+- Cache correctness is version-based, not age-based.
+- Cohort revision tracking prevents a new merchant fact from silently leaving peer comparisons current for other merchants.
+- Trigger/revision behavior, concurrency deduplication, failure recovery, and traceability require integration tests.

docs/decisions/0012-iranian-reporting-week.md

@@ -0,0 +1,18 @@
+# ADR 0012: Use Saturday-through-Friday reporting weeks
+
+- Status: Accepted
+- Date: 2026-08-20
+
+## Context
+
+Weekly analytics require one boundary shared by PostgreSQL grouping, API filters, cache keys, comparisons, and Persian UI labels.
+
+## Decision
+
+Use the Iranian week in `Asia/Tehran`: Saturday is the first day and Friday is the last day. Store the Tehran-local Saturday date as the stable week key. API and query ranges use a half-open interval from Saturday `00:00` inclusive to the following Saturday `00:00` exclusive.
+
+## Consequences
+
+- Week boundaries remain correct and unambiguous across daylight/timezone handling.
+- A week crossing a Jalali month or year remains identified by its starting Saturday date.
+- Previous-period comparison uses the immediately preceding Saturday-through-Friday interval.