docs: pin analytics queue safety limits

This commit is contained in:
2026-08-20 16:41:07 +03:30
parent ae551c02c1
commit 96a27e1b98
2 changed files with 30 additions and 3 deletions

View File

@@ -149,12 +149,17 @@ This contract applies only to the one-time challenge-data bootstrap and explicit
- This project owns the normalized schema contract, migrations, read models, and analytics; it does not recreate ZarinPal's payment-write workflow.
- Payment facts are append-only in the MVP. Existing session/attempt updates and hard deletion are out of scope; a future mutation requirement needs a new invalidation contract.
- An idempotent authenticated `POST` starts or returns the current analytics run for the merchant/revision. It enqueues the four MVP sections—`overview`, `payment_performance`, `fees`, and `peer_comparison`—for bounded parallel calculation by a separate Go worker service.
- Use Asynq pinned to `v0.26.0` as the Redis-backed task queue. Asynq is pre-`v1.0.0`; never use a floating version, and review API/behavior compatibility before upgrading the pin.
- Each completed section is serialized through its approved typed contract and cached in Redis under a key containing merchant id, merchant source revision, relevant cohort revision, metric-definition version, and section id. Cache entries have a 30-day TTL.
- A separate authenticated `GET` SSE stream for that run emits section progress and makes each successful section available immediately; the dashboard does not wait for every job to finish.
- Redis-backed idempotency/single-flight coordination prevents equivalent concurrent requests from duplicating the same section calculation. Parallelism is bounded globally and per merchant.
- Worker concurrency defaults to 16 jobs globally and 4 jobs per merchant, configurable downward/upward through validated environment settings.
- Every inserted related payment fact atomically advances the affected PostgreSQL merchant analytics revision. It also advances the relevant cohort/category revision when peer metrics can change. Database-managed revision changes cover direct ZarinPal inserts that bypass Go request handlers.
- Revision changes invalidate logically by producing new cache keys; old values are not trusted and expire through TTL without requiring a Redis key scan.
- Each section attempt has a 60-second timeout. Transient failures receive at most two automatic retries (three total attempts) with exponential backoff and jitter; deterministic validation/authorization failures are not retried. Exhausted sections emit a typed failure event without marking the section ready. Other successful sections remain available.
- Rate limits use independent authenticated-user and merchant buckets: analytics starts allow 10 requests/minute with burst 3; manual failed-section retries allow 3 requests/10 minutes with burst 1; SSE allows 2 concurrent connections/user and 4/merchant. Rejections return `429` with `Retry-After` and do not enqueue work.
- SSE sends a heartbeat every 15 seconds and retains replayable run events for one hour. Reconnects resume from the last authorized event id where available; after retention, the client fetches the current run snapshot and reopens the stream.
- Manual retry is available only for an exhausted failed section, is idempotent for the current revisions, and never clears or restarts successful sibling sections.
- Redis is cache/coordination only. PostgreSQL normalized facts and revision rows remain authoritative; loss or flush of Redis causes safe lazy recomputation.
- Redis and the on-demand job worker are private services, never exposed through Nginx. Cache values exclude payer-card keys, credentials, raw payment rows, and cross-tenant details.
@@ -185,6 +190,8 @@ This contract applies only to the one-time challenge-data bootstrap and explicit
- Concurrent equivalent requests do not duplicate section jobs; configured global/per-merchant limits prevent unbounded query fan-out.
- One failed section does not hide ready sections and does not create a ready cache entry.
- A transient section failure performs no more than two retries with exponential backoff/jitter; each attempt ends within 60 seconds, while non-retryable failures fail immediately.
- Start, manual-retry, and SSE connection limits return a typed `429` with `Retry-After`; rejected requests enqueue no jobs and cannot affect another merchant.
- SSE emits heartbeats within 15 seconds and supports authorized replay for one hour; replay expiry falls back to the current run snapshot without duplicating work.
- Redis loss produces a cold-cache calculation rather than loss of source data.
- Reporting-date rows covering the dataset range reconcile deterministically between Gregorian and Jalali day, week, month, and season attributes.
- Every reporting week starts Saturday at `00:00:00` and ends Friday at `23:59:59.999…` in `Asia/Tehran`; API ranges use an equivalent half-open `[start, next_start)` interval.
@@ -205,7 +212,7 @@ Resolved: peer benchmarks exclude the active merchant and require at least 10 ot
Resolved: use normalized PostgreSQL source-of-truth entities plus a reporting-date dimension, then compute versioned analytics sections lazily into Redis. Daily, weekly, Jalali-month, and Persian-season results use explicit date boundaries. Keep fact tables unpartitioned initially and introduce partitioning only from measured need. See ADR 0007.
Resolved: ZarinPal writes future append-only production rows directly to normalized tables. An idempotent `POST` starts four parallel sections in a separate Go worker; Redis caches each typed result for 30 days; an authenticated `GET` SSE stream reveals sections as ready. Each section has a 60-second attempt timeout and at most two transient retries using exponential backoff/jitter. PostgreSQL merchant/cohort revisions provide logical invalidation. See ADR 0011.
Resolved: ZarinPal writes future append-only production rows directly to normalized tables. An idempotent `POST` starts four sections through Asynq `v0.26.0` (explicitly pre-1.0) in a separate Go worker, limited to 4 concurrent jobs/merchant and 16 globally. Redis caches results for 30 days; authenticated SSE uses 15-second heartbeats and one-hour replay. Sections have a 60-second attempt timeout and two transient exponential-backoff retries. Start, SSE, and manual-retry operations are rate-limited. PostgreSQL revisions provide invalidation. See ADR 0011.
Resolved: holiday and named-event reporting are removed from scope. MVP reporting periods are limited to daily, weekly, Jalali-month, and Persian-season views. See ADR 0010.
@@ -213,4 +220,4 @@ Resolved: reporting weeks follow the Iranian week: Saturday through Friday in `A
Resolved: only source session status `Verified` is a definitely successful payment. Keep `Paid` separate until an authoritative ZarinPal definition changes the metric catalogue. See `METRIC-CATALOGUE.md`.
No broad architecture approval gates remain for this capability. Physical columns, indexes, exact SSE schemas/reconnect retention, worker concurrency, retry classification, analytics-section response contracts, and migration details still require acceptance in the implementation-ready specs.
No broad architecture approval gates remain for this capability. Physical columns, exact SSE event schemas, retryable-error classification, analytics-section response contracts, observability targets, and migration details still require acceptance in the implementation-ready specs.