docs: move lazy analytics cache to redis

docs/decisions/0007-calendar-aware-analytics-projections.md

@@ -15,16 +15,16 @@ Keep normalized business/source-of-truth relations: categories, merchants, merch
 
 Add a reporting-date model containing Gregorian and Jalali date attributes, explicit reporting-week boundaries, Persian months/seasons, and weekdays.
 
-Build idempotent, rebuildable daily base projections containing additive components such as session count, attempt count, successful count, amount sum, and fee sum as approved by the metric catalogue. Derive week/month/season totals by grouping these components through the reporting-date model and recompute ratios from their numerators and denominators.
+Build idempotent, rebuildable analytics-section queries containing additive components such as session count, attempt count, successful count, amount sum, and fee sum as approved by the metric catalogue. Derive week/month/season totals through the reporting-date model and recompute ratios from their numerators and denominators.
 
-Create separate metric-specific period projections only for non-additive metrics or query paths proven expensive—for example, exact unique payer-token counts cannot be obtained by summing daily distinct counts. Projection rows record source import/version and freshness.
+Calculate non-additive metrics such as exact unique payer-token counts directly at their requested grain; never obtain them by summing daily distinct counts. Cache each typed section in Redis under source/cohort and metric-definition revisions.
 
 Keep normalized fact tables unpartitioned initially. Add targeted indexes from approved API access paths and verified query plans. Introduce PostgreSQL partitioning only through a future ADR supported by measured table growth, query latency, vacuum/maintenance behavior, or retention operations.
 
 ## Consequences
 
-- The normalized ERD remains semantically correct while dashboards read smaller, purpose-built projections.
-- Daily components support many calendar groupings without precomputing every possible period.
+- The normalized ERD remains semantically correct while dashboards reuse disposable Redis results.
+- Section calculations support approved period groupings without storing calculated facts as PostgreSQL business truth.
 - Explicit reporting-period definitions and metric provenance make future AI observations traceable.
-- Lazy calculation, revision invalidation, concurrency deduplication, failure, and stale-state visibility tests become required behavior.
-- Schema/index/projection details remain subject to data profiling and approved metric/API requirements.
+- Lazy parallel calculation, Redis revision keys, job deduplication, partial failure, and SSE visibility tests become required behavior.
+- Schema/index/cache details remain subject to data profiling and approved metric/API requirements.

docs/decisions/0011-lazy-merchant-analytics-cache.md

@@ -1,26 +0,0 @@
-# ADR 0011: Calculate and cache merchant analytics lazily
-
-- Status: Accepted
-- Date: 2026-08-20
-
-## Context
-
-The hackathon dataset is static after its one-time bootstrap. Continuously refreshing analytics would spend resources before any merchant requests the data. The production boundary may later receive new append-only rows written directly by ZarinPal, bypassing Go request handlers, so any invalidation contract must also work at the database boundary.
-
-## Decision
-
-Calculate a merchant's approved analytics projections on the first relevant dashboard request and persist typed projections in PostgreSQL. Reuse them while their merchant source revision, relevant cohort revision, and metric-definition version remain current.
-
-Schema-owned database logic atomically increments the affected merchant revision whenever a related payment fact is inserted. Inserts that affect peer benchmarks also increment the corresponding category/cohort revision. This covers direct ZarinPal writes without a refresh worker or Redis.
-
-On a cache miss or revision mismatch, the backend obtains a merchant-scoped PostgreSQL advisory/row lock, computes the projections once, publishes them atomically, and then advances the cached revision. Concurrent requests wait for or reuse that result. A failed calculation does not advance the cached revision; an older snapshot may be returned only when clearly marked stale.
-
-Payment facts are append-only for this MVP. Cache invalidation for updates or hard deletion is out of scope and requires a future contract change.
-
-## Consequences
-
-- The static dataset is aggregated at most once per merchant and metric-definition revision when requested.
-- No scheduled analytics worker, nightly reconciliation, Redis, or time-based cache expiry is required.
-- Cache correctness is version-based, not age-based.
-- Cohort revision tracking prevents a new merchant fact from silently leaving peer comparisons current for other merchants.
-- Trigger/revision behavior, concurrency deduplication, failure recovery, and traceability require integration tests.

docs/decisions/0011-lazy-redis-analytics-cache.md

@@ -0,0 +1,29 @@
+# ADR 0011: Calculate analytics sections lazily and cache them in Redis
+
+- Status: Accepted
+- Date: 2026-08-20
+
+## Context
+
+The hackathon dataset is static after its one-time bootstrap. Continuously refreshing analytics would spend resources before a merchant requests them. A first calculation may contain several independent dashboard sections, and the user should see completed sections without waiting for the slowest query. Future append-only rows may be written directly by ZarinPal, bypassing Go request handlers.
+
+## Decision
+
+On the first relevant dashboard access, enqueue every approved analytics section for the authenticated merchant. A bounded on-demand worker pool calculates independent sections in parallel. This worker is event-driven, not scheduled.
+
+Cache each completed typed section in Redis using a key derived from the merchant id, merchant source revision, relevant cohort revision, metric-definition version, and section id. Use a configurable high TTL. Redis-backed idempotency/single-flight coordination prevents duplicate equivalent jobs. Old-revision keys expire naturally and are never treated as current.
+
+Stream progress and section availability through an authenticated, merchant-scoped SSE connection. Emit stable event ids and typed events for section started, section ready, section failed, overall complete, and heartbeat. Support safe reconnect without exposing another merchant’s jobs or values. One section failure does not suppress sections that succeeded.
+
+Schema-owned PostgreSQL logic atomically increments the affected merchant revision whenever a related payment fact is inserted. Inserts affecting peer benchmarks also increment the corresponding category/cohort revision. This covers direct ZarinPal writes. Payment facts are append-only for the MVP; invalidation for update/delete remains out of scope.
+
+Redis is a private disposable cache and coordination system, not a source of truth. PostgreSQL remains authoritative. Cached values contain approved aggregates only and exclude raw rows, payer-card keys, credentials, session tokens, and cross-tenant details.
+
+## Consequences
+
+- All dashboard analytics begin together, while the UI reveals each section independently as it completes.
+- Redis loss or key expiry causes safe lazy recomputation; Redis is excluded from backup/restore requirements.
+- No scheduled refresh or time-based freshness claim is required.
+- The deployment includes private Redis and on-demand analytics-worker services.
+- Bounded global/per-merchant concurrency, job timeouts, retry limits, SSE connection limits, and cache TTL still require explicit values.
+- Trigger/revision behavior, tenant isolation, job deduplication, partial failures, SSE reconnect, and cache data minimization require integration/security tests.