docs: draft analytics engine capability
This commit is contained in:
153
docs/ANALYTICS-SPEC.md
Normal file
153
docs/ANALYTICS-SPEC.md
Normal file
@@ -0,0 +1,153 @@
|
|||||||
|
# Analytics Engine Capability Spec
|
||||||
|
|
||||||
|
- Status: Draft — not approved for implementation
|
||||||
|
- Capability: Analytics engine
|
||||||
|
- Depends on: Approved Data Foundation specification
|
||||||
|
- Last updated: 2026-08-20
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
|
||||||
|
Turn normalized ZarinPal payment facts into trustworthy, merchant-scoped analytics that explain payment performance, verification gaps, retry outcomes, processing time, fees, trends, and same-category position. Deterministic calculations are the product source of truth; later AI assistance may explain or recommend actions from sanitized aggregates but cannot define or alter metrics.
|
||||||
|
|
||||||
|
## Actors and consumers
|
||||||
|
|
||||||
|
- Authenticated merchant: sees detailed metrics and drill-downs only for its merchant boundary.
|
||||||
|
- Merchant API: requests versioned analytics sections and exposes typed results/events.
|
||||||
|
- Dashboard: renders sections independently as they become ready.
|
||||||
|
- Analytics worker: calculates idempotent section results from PostgreSQL and caches approved aggregates in Redis.
|
||||||
|
- Future AI adapter: post-MVP consumer of minimized, approved aggregate facts only.
|
||||||
|
|
||||||
|
## In scope
|
||||||
|
|
||||||
|
- Daily, Saturday–Friday weekly, Jalali-month, and Persian-season analytics in `Asia/Tehran`
|
||||||
|
- Overview, payment performance, fees, and peer-comparison sections
|
||||||
|
- Payment success and volume with correct `Paid`/`Verified` semantics
|
||||||
|
- Verification completion and unverified-paid gaps
|
||||||
|
- Retry recovery and attempt/session diagnostics
|
||||||
|
- Processing-time p50/p95 with sample and coverage
|
||||||
|
- Real merchant fee amounts and paid fee rate at session grain
|
||||||
|
- Current-period trends against an approved equivalent baseline
|
||||||
|
- Same-category merchant-weighted peer median and performance percentile
|
||||||
|
- Lazy, bounded parallel calculation with partial SSE delivery
|
||||||
|
- Versioned 30-day Redis cache and PostgreSQL revision invalidation
|
||||||
|
- Traceability from aggregate metrics to permitted merchant-owned evidence
|
||||||
|
|
||||||
|
## Out of scope for MVP
|
||||||
|
|
||||||
|
- Holiday or named-event reporting
|
||||||
|
- AI-generated recommendations or free-form LLM analysis
|
||||||
|
- Forecasting, causal claims, or automated campaign execution
|
||||||
|
- Arbitrary merchant-created formulas or cross-merchant filters
|
||||||
|
- Real-time streaming payment ingestion owned by this service
|
||||||
|
- Persisting calculated analytics as PostgreSQL business facts
|
||||||
|
- Public or unauthenticated analytics
|
||||||
|
|
||||||
|
## Canonical sections
|
||||||
|
|
||||||
|
| Section id | Responsibility | Minimum result |
|
||||||
|
|---|---|---|
|
||||||
|
| `overview` | Merchant-level outcome and trend summary | bank-paid sessions/amount, success rate, verification completion, selected period, prior-period comparison |
|
||||||
|
| `payment_performance` | Status, retry, and timing analysis | paid/verified/unverified/failed/reversed counts, retry recovery, init/verify p50/p95 with coverage |
|
||||||
|
| `fees` | Merchant fee analysis | paid fee amount, paid amount denominator, paid fee rate, period trend |
|
||||||
|
| `peer_comparison` | Privacy-bounded same-category benchmark | eligible peer count/session count, cohort median, direction-aware merchant percentile or typed suppression |
|
||||||
|
|
||||||
|
Every section uses a versioned typed contract and has independent `queued`, `calculating`, `ready`, `failed`, and where applicable `suppressed` states.
|
||||||
|
|
||||||
|
## Metric and period contract
|
||||||
|
|
||||||
|
- `METRIC-CATALOGUE.md` version `0.3.0` is canonical for formulas, units, grains, missing-data behavior, and peer direction.
|
||||||
|
- `Paid` and `Verified` both mean bank payment succeeded; only `Verified` is finalized. Never label `Paid` as failure.
|
||||||
|
- Amount and fee calculations count each eligible payment session once. Attempt rows cannot multiply session values.
|
||||||
|
- Rates are calculated from summed numerator and denominator components for the selected period, never averaged from daily percentages.
|
||||||
|
- Timing percentiles exclude null values and expose sample size and non-null coverage; missing time never becomes zero.
|
||||||
|
- Every result states the exact Tehran-local half-open period, Jalali label, unit, sample/denominator, source revision, metric-definition version, and calculation timestamp.
|
||||||
|
- Approved period types are one Tehran-local day, one Saturday–Friday week, one Jalali month, or one Persian season. Holiday/event dimensions do not exist in the MVP contract.
|
||||||
|
|
||||||
|
## Peer comparison contract
|
||||||
|
|
||||||
|
- Eligible peers are other merchants in the active merchant's category for the same selected period.
|
||||||
|
- Suppress the peer result unless at least 10 other merchants and 1,000 combined peer sessions contribute.
|
||||||
|
- Calculate each peer merchant's metric independently and take the median of those merchant values; do not transaction-weight the cohort.
|
||||||
|
- Return the cohort median and active merchant's performance percentile. Higher displayed percentile always means better, so processing-time and fee-rate ranking is inverted.
|
||||||
|
- Benchmark payment success, verification completion, retry recovery, init/verify p50/p95, and paid fee rate.
|
||||||
|
- Never return peer identities, individual values/ranks, rows, or arbitrary filters that enable differencing attacks.
|
||||||
|
|
||||||
|
## Calculation and cache lifecycle
|
||||||
|
|
||||||
|
1. An idempotent authenticated start request resolves merchant, selected period, merchant revision, cohort revision, and metric-definition version.
|
||||||
|
2. Equivalent requests map to one current analytics run and do not enqueue duplicate work.
|
||||||
|
3. Missing section results are enqueued together and calculated independently by the Go worker through Asynq `v0.26.0`, explicitly a pre-1.0 dependency pinned for compatibility.
|
||||||
|
4. Worker limits default to 16 jobs globally and four per merchant. All four core sections begin in bounded parallel and become visible independently.
|
||||||
|
5. Each attempt times out after 60 seconds. Transient failures receive at most two retries using exponential backoff of approximately one then two seconds plus jitter; deterministic failures are not retried.
|
||||||
|
6. Successful section results are published atomically to Redis under revisioned/versioned keys with a 30-day TTL. PostgreSQL facts and revision rows remain authoritative.
|
||||||
|
7. Relevant fact inserts advance the affected merchant and category/cohort revisions in PostgreSQL. New requests use new keys; old keys expire without scans.
|
||||||
|
8. Redis loss causes safe lazy recomputation. It never causes loss of payment facts or metric definitions.
|
||||||
|
|
||||||
|
## Partial delivery contract
|
||||||
|
|
||||||
|
- A separate authenticated SSE connection streams run/section state and each completed section immediately.
|
||||||
|
- SSE sends a heartbeat every 15 seconds and retains authorized replay events for one hour.
|
||||||
|
- Reconnect uses the last event id where retained; otherwise the client fetches the current run snapshot and reopens the stream.
|
||||||
|
- One failed section never hides ready siblings. Manual retry targets only an exhausted failed section and cannot clear successful results.
|
||||||
|
- Start limits are 10 requests/minute with burst 3; manual retry limits are 3 requests/10 minutes with burst 1; SSE limits are two concurrent connections/user and four/merchant. Independent user and merchant buckets return `429` with `Retry-After` before work is queued.
|
||||||
|
|
||||||
|
## Traceability and privacy
|
||||||
|
|
||||||
|
- Merchant-owned aggregate cards link to definitions and, where approved, tenant-scoped session/attempt evidence that reconciles to the aggregate.
|
||||||
|
- Payer-card tokens remain internal, opaque, merchant-scoped inputs and never appear in APIs, logs, exports, caches, peer results, or AI payloads.
|
||||||
|
- Redis values contain approved aggregates and trace metadata only, not raw payment rows or credentials.
|
||||||
|
- Cross-merchant calculations execute server-side and return only approved aggregate/suppressed contracts.
|
||||||
|
- Future AvalAI prompts may receive minimized aggregates only after a separate AI capability approval.
|
||||||
|
|
||||||
|
## Failure and edge behavior
|
||||||
|
|
||||||
|
- Zero, undefined denominator, unavailable source, peer suppression, calculation failure, and not-yet-ready are distinct typed states.
|
||||||
|
- Unknown future payment statuses are excluded from success numerator/denominator and reported through coverage evidence until classified.
|
||||||
|
- `Reversed` remains outside success and is displayed separately when non-zero.
|
||||||
|
- Empty approved periods return a valid empty result, not fabricated zero rates or a system failure.
|
||||||
|
- Section publication is atomic; partial/failed calculations never overwrite a valid ready cache entry.
|
||||||
|
- Database, Redis, queue, timeout, and authorization errors are distinguishable internally while merchant-facing errors remain safe and actionable.
|
||||||
|
|
||||||
|
## Observability
|
||||||
|
|
||||||
|
- Measure section queue delay, calculation duration, query duration, cache hit/miss, retry count, failure class, SSE delivery/reconnect, and suppression rate.
|
||||||
|
- Correlate start request, analytics run, section job, cache key version, and SSE event without logging sensitive merchant rows or payer tokens.
|
||||||
|
- Record PostgreSQL query plans during performance verification and add indexes only from measured access paths.
|
||||||
|
- Define service-level objectives and alert thresholds in the Platform Operations spec, not implicitly in worker code.
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- Every metric reconciles to an independent PostgreSQL query over normalized facts for representative merchants and all approved period types.
|
||||||
|
- Session-grain amount/fee totals remain unchanged when a session has multiple attempts.
|
||||||
|
- `Paid` plus `Verified` success, verification completion, retry recovery, and timing coverage match catalogue definitions.
|
||||||
|
- Saturday–Friday, Jalali-month, and Persian-season boundaries match `calendar_dates` and API half-open ranges.
|
||||||
|
- First access starts all missing sections once; ready sections stream independently and equivalent concurrent starts deduplicate.
|
||||||
|
- Cache hits avoid repeating source aggregation; merchant/cohort revision changes select fresh keys and recompute only missing sections.
|
||||||
|
- Redis flush causes correct cold-cache rebuilding without source-data loss.
|
||||||
|
- Timeouts, transient retry limits/backoff, failed-section isolation, manual retry, rate limits, heartbeat, replay, and snapshot fallback behave as specified.
|
||||||
|
- Peer results exclude the active merchant, use equal merchant weighting, obey both suppression thresholds, and never expose individual peers.
|
||||||
|
- Tenant-bound drill-downs reconcile to aggregates and cannot expose another merchant through parameter manipulation.
|
||||||
|
- No raw payer token, credential, cross-tenant row, or AvalAI secret enters results, logs, cache payloads, or browser data.
|
||||||
|
|
||||||
|
## Boundaries
|
||||||
|
|
||||||
|
- Always: use catalogue definitions, preserve grain, enforce tenant scope, expose period/unit/coverage/version, calculate idempotently, and keep results reproducible.
|
||||||
|
- Ask first: add/change a metric, period, peer cohort, threshold, cache key/version, drill-down, AI input, or cross-service interface.
|
||||||
|
- Never: average rates, multiply session money by attempts, classify `Paid` as failure, expose peer rows, trust client tenant ids, or let AI become the metric source of truth.
|
||||||
|
|
||||||
|
## Resolved decisions
|
||||||
|
|
||||||
|
- Normalized PostgreSQL facts are authoritative; Redis is a 30-day disposable analytics cache.
|
||||||
|
- Four sections calculate lazily and in bounded parallel, with partial authenticated SSE delivery.
|
||||||
|
- Reporting covers day, Iranian week, Jalali month, and Persian season only.
|
||||||
|
- Peer comparison uses same-category merchant-weighted medians and direction-aware percentiles with the approved suppression threshold.
|
||||||
|
- Payment lifecycle, retry, timing, and real-fee definitions follow `METRIC-CATALOGUE.md` and ADR 0013.
|
||||||
|
- Asynq `v0.26.0` is pinned and acknowledged as pre-1.0; jobs have bounded concurrency, timeouts, rate limits, and exponential-backoff retries.
|
||||||
|
|
||||||
|
## Open product decisions
|
||||||
|
|
||||||
|
- Default reporting period on first dashboard access, especially when the current period has no source data
|
||||||
|
- Default previous-period comparison behavior when an equivalent baseline is empty or incomplete
|
||||||
|
- Exact overview KPI order and which deterministic insight appears first
|
||||||
|
- Which tenant-scoped drill-down tables are MVP-visible versus definition-only
|
||||||
|
- Deterministic threshold/severity rules for surfacing actionable findings
|
||||||
Reference in New Issue
Block a user