docs: adopt lazy analytics caching
This commit is contained in:
@@ -13,7 +13,7 @@ Use Ent for application persistence, relationship modeling, and type-safe access
|
||||
|
||||
Use pgx directly for the importer’s PostgreSQL staging boundary. Feed `CopyFrom` with a streaming `CopyFromSource` (or equivalent bounded batches) so decompressed CSV rows are not accumulated in memory. Validate and promote staged data through explicit transactions/steps described by the import spec; bulk loading must not bypass reconciliation or data-quality rules.
|
||||
|
||||
Use versioned SQL migrations generated/planned from the reviewed schema and committed to Git. Apply them through a dedicated migration command or deployment step. Ent automatic schema creation is allowed only in disposable development/test contexts if the later implementation spec permits it; it must never run as part of production application startup.
|
||||
Use Atlas, Ent's supported migration engine, to generate/plan versioned SQL migrations from the reviewed Ent schema. Review and commit the SQL to Git, then apply it through a dedicated deployment migration step. Atlas is build/deployment tooling, not a runtime service. Ent automatic schema creation is allowed only in disposable development/test contexts if the later implementation spec permits it; it must never run as part of production application startup.
|
||||
|
||||
## Consequences
|
||||
|
||||
@@ -21,10 +21,10 @@ Use versioned SQL migrations generated/planned from the reviewed schema and comm
|
||||
- Engineers must still inspect generated SQL, query plans, indexes, transaction boundaries, and possible N+1 behavior; the ORM is not a performance or correctness boundary.
|
||||
- The importer owns a deliberate lower-level pgx path and cannot reuse ordinary Ent writes for convenience.
|
||||
- Both Ent and pgx require compatible pool/transaction ownership and context propagation; adapters must not hide unbounded work.
|
||||
- Migration generation tooling and the migration runner remain to be selected in the implementation spec. Versioned migration files—not runtime diffs—are the production contract.
|
||||
- Atlas-generated versioned migration files—not runtime diffs—are the production contract. CI validates migration integrity and drift before deployment.
|
||||
|
||||
## References
|
||||
|
||||
- Ent versioned migrations: <https://entgo.io/docs/versioned/intro/>
|
||||
- Atlas versioned migrations: <https://atlasgo.io/concepts/declarative-vs-versioned#versioned-migrations>
|
||||
- pgx v5 `CopyFrom`: <https://pkg.go.dev/github.com/jackc/pgx/v5#Conn.CopyFrom>
|
||||
|
||||
|
||||
@@ -26,5 +26,5 @@ Keep normalized fact tables unpartitioned initially. Add targeted indexes from a
|
||||
- The normalized ERD remains semantically correct while dashboards read smaller, purpose-built projections.
|
||||
- Daily components support many calendar groupings without precomputing every possible period.
|
||||
- Explicit reporting-period definitions and metric provenance make future AI observations traceable.
|
||||
- Projection refresh, failure, stale-state visibility, and reconciliation tests become required operational behavior.
|
||||
- Lazy calculation, revision invalidation, concurrency deduplication, failure, and stale-state visibility tests become required behavior.
|
||||
- Schema/index/projection details remain subject to data profiling and approved metric/API requirements.
|
||||
|
||||
@@ -1,25 +0,0 @@
|
||||
# ADR 0008: ZarinPal owns live writes; analytics refresh within 15 minutes
|
||||
|
||||
- Status: Accepted
|
||||
- Date: 2026-08-20
|
||||
|
||||
## Context
|
||||
|
||||
The challenge archive is a one-time bootstrap source. In production, ZarinPal already creates payment data and will write future records directly to the normalized PostgreSQL schema. Building a recurring CSV importer, polling source, or competing payment-write workflow would duplicate upstream responsibility. Derived dashboard projections still need bounded freshness and correction when live source rows arrive or change status.
|
||||
|
||||
## Decision
|
||||
|
||||
ZarinPal is the authoritative producer of live `payment_sessions` and `payment_attempts`. This backend owns the normalized schema contract, versioned migrations, analytical reads/projections, and tenant-safe APIs, but not the business workflow that produces live payment records.
|
||||
|
||||
Fact rows identify `bootstrap` versus `zarinpal_live` origin. Bootstrap rows reference import provenance; live rows do not fabricate import runs. The live-write contract supplies stable creation/update metadata so an idempotent high-watermark process can include new rows and status updates.
|
||||
|
||||
Refresh affected analytics projections within 15 minutes. Run a nightly idempotent reconciliation over an approved authoritative window to detect and correct missed/changed rows. Publish projections atomically and advance the visible freshness watermark only after success.
|
||||
|
||||
## Consequences
|
||||
|
||||
- No recurring CSV ingestion service, file watcher, or source polling is built.
|
||||
- Schema migrations require compatibility coordination with ZarinPal's writer.
|
||||
- Analytics refresh is separate from ingestion and requires scheduling, checkpoints, idempotency, metrics, and stale-state UX.
|
||||
- If upstream cannot provide reliable update metadata, revise the refresh algorithm/window before implementation rather than silently missing mutations.
|
||||
- The static hackathon dataset builds projections once after bootstrap; production scheduling remains testable with synthetic live writes.
|
||||
|
||||
@@ -11,7 +11,7 @@ Earlier drafts included named Iranian holidays and analytical event windows. Tha
|
||||
|
||||
Limit reporting periods to Tehran-local daily, weekly, Jalali-month, and Persian-season views. Remove named holidays, Nowruz-specific periods, public-holiday calendars, and generic event-window reporting from the current product and data model.
|
||||
|
||||
The exact weekly boundary remains an explicit reporting-period decision; it must not be inferred differently by the database, API, and frontend.
|
||||
The reporting week follows the separately approved Saturday-through-Friday boundary in ADR 0012 and must not be inferred differently by the database, API, and frontend.
|
||||
|
||||
## Consequences
|
||||
|
||||
|
||||
26
docs/decisions/0011-lazy-merchant-analytics-cache.md
Normal file
26
docs/decisions/0011-lazy-merchant-analytics-cache.md
Normal file
@@ -0,0 +1,26 @@
|
||||
# ADR 0011: Calculate and cache merchant analytics lazily
|
||||
|
||||
- Status: Accepted
|
||||
- Date: 2026-08-20
|
||||
|
||||
## Context
|
||||
|
||||
The hackathon dataset is static after its one-time bootstrap. Continuously refreshing analytics would spend resources before any merchant requests the data. The production boundary may later receive new append-only rows written directly by ZarinPal, bypassing Go request handlers, so any invalidation contract must also work at the database boundary.
|
||||
|
||||
## Decision
|
||||
|
||||
Calculate a merchant's approved analytics projections on the first relevant dashboard request and persist typed projections in PostgreSQL. Reuse them while their merchant source revision, relevant cohort revision, and metric-definition version remain current.
|
||||
|
||||
Schema-owned database logic atomically increments the affected merchant revision whenever a related payment fact is inserted. Inserts that affect peer benchmarks also increment the corresponding category/cohort revision. This covers direct ZarinPal writes without a refresh worker or Redis.
|
||||
|
||||
On a cache miss or revision mismatch, the backend obtains a merchant-scoped PostgreSQL advisory/row lock, computes the projections once, publishes them atomically, and then advances the cached revision. Concurrent requests wait for or reuse that result. A failed calculation does not advance the cached revision; an older snapshot may be returned only when clearly marked stale.
|
||||
|
||||
Payment facts are append-only for this MVP. Cache invalidation for updates or hard deletion is out of scope and requires a future contract change.
|
||||
|
||||
## Consequences
|
||||
|
||||
- The static dataset is aggregated at most once per merchant and metric-definition revision when requested.
|
||||
- No scheduled analytics worker, nightly reconciliation, Redis, or time-based cache expiry is required.
|
||||
- Cache correctness is version-based, not age-based.
|
||||
- Cohort revision tracking prevents a new merchant fact from silently leaving peer comparisons current for other merchants.
|
||||
- Trigger/revision behavior, concurrency deduplication, failure recovery, and traceability require integration tests.
|
||||
18
docs/decisions/0012-iranian-reporting-week.md
Normal file
18
docs/decisions/0012-iranian-reporting-week.md
Normal file
@@ -0,0 +1,18 @@
|
||||
# ADR 0012: Use Saturday-through-Friday reporting weeks
|
||||
|
||||
- Status: Accepted
|
||||
- Date: 2026-08-20
|
||||
|
||||
## Context
|
||||
|
||||
Weekly analytics require one boundary shared by PostgreSQL grouping, API filters, cache keys, comparisons, and Persian UI labels.
|
||||
|
||||
## Decision
|
||||
|
||||
Use the Iranian week in `Asia/Tehran`: Saturday is the first day and Friday is the last day. Store the Tehran-local Saturday date as the stable week key. API and query ranges use a half-open interval from Saturday `00:00` inclusive to the following Saturday `00:00` exclusive.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Week boundaries remain correct and unambiguous across daylight/timezone handling.
|
||||
- A week crossing a Jalali month or year remains identified by its starting Saturday date.
|
||||
- Previous-period comparison uses the immediately preceding Saturday-through-Friday interval.
|
||||
Reference in New Issue
Block a user