zarinpal-analytics-backend/docs/PHYSICAL-SCHEMA-PROPOSAL.md

# PostgreSQL Physical Schema Proposal

- Status: Draft for decision batch
- Scope: Data foundation only
- Implementation authorization: Not granted

## Principles

- Normalize business entities and enforce relationships with foreign keys.
- Preserve source identifiers as text and use internal surrogate keys for joins.
- Keep payment session and payment attempt grains separate.
- Use UTC `timestamptz` for canonical instants and `bigint` for Rial amounts.
- Keep normalized PostgreSQL facts as source of truth; Redis analytics results are disposable and rebuildable.
- No initial table partitioning. Add indexes from approved access paths and measured query plans.
- The CSV importer is a one-time administrative bootstrap, not a production ingestion service.

## Proposed tables

### Provenance and reference

| Table | Purpose | Key constraints |
|---|---|---|
| `import_runs` | One bootstrap/recovery execution | unique source checksum + schema/importer policy; explicit lifecycle |
| `import_issues` | Aggregated validation/quality evidence | FK to import run; severity/rule/count/evidence |
| `merchant_categories` | Five source business categories | unique source category key |
| `psps` | Opaque PSP reference codes | unique source PSP code |
| `issuer_banks` | Opaque bank reference codes | unique source bank code |
| `calendar_dates` | Gregorian/Jalali day, week, month, and Persian-season reporting attributes | Gregorian date PK; unique Jalali date tuple |

Switch response codes remain nullable text on attempts until an authoritative description catalogue exists. Normalizing an undocumented 97.4%-missing field into a lookup table adds little value.

### Merchant identity

| Table | Purpose | Key constraints |
|---|---|---|
| `merchants` | Merchant profile and category membership | unique source merchant key; FK category |
| `terminals` | Merchant-owned payment terminals | unique source terminal key; FK merchant |
| `merchant_users` | Login identities | unique normalized username; FK merchant; password hash only |
| `auth_sessions` | Revocable authenticated sessions | hashed opaque session token; FK user; expiry/revocation |

Synthetic profile fields carry an explicit `profile_is_synthetic` marker. Credentials are generated outside Git.

### Payment facts

| Table | Grain and important fields | Key constraints/index direction |
|---|---|---|
| `payment_sessions` | One source session: merchant, terminal, amount in Rial, source status, created/expires instants, import provenance | unique source session key; FKs; merchant/time and terminal/time indexes |
| `payment_attempts` | One try: session, sequence, attempt status, PSP/bank, response, opaque payer token, verification type, timings, fee, attempt/verified/settled instants | unique `(payment_session_id, try_seq)`; FKs; session index; measured analytical indexes later |

`adjusted_fee` uses fixed precision and is labeled as an obfuscated relative value. The physical precision is finalized from exact decimal profiling before migration approval.

Both fact tables distinguish `bootstrap` from `zarinpal_live` origin. Bootstrap rows reference an import run; live rows do not invent one. Payment facts are append-only for the MVP; inserts advance database-managed analytics revisions.

### Analytics invalidation state

| Table | Grain | Purpose |
|---|---|---|
| `merchant_analytics_revisions` | one row per merchant | monotonically increasing source revision advanced by related fact inserts |
| `cohort_analytics_revisions` | one row per benchmark cohort/category | monotonically increasing peer-source revision advanced by relevant inserts |

No calculated dashboard result is stored in PostgreSQL. Analytics queries and Redis value contracts are finalized by the metric catalogue and API spec.

Inserts atomically increment affected revision rows through schema-owned database logic so direct upstream writes cannot bypass invalidation. Redis keys include merchant/cohort revisions, metric-definition version, and analytics-section id. Old revision keys expire under a configurable high TTL; invalidation does not scan or delete arbitrary keys. Redis values follow typed, versioned response schemas and contain aggregates only.

## Initial index policy

- Unique indexes implement source identities and natural relationship constraints.
- `payment_sessions (merchant_id, created_at DESC)` supports merchant period queries.
- `payment_sessions (terminal_id, created_at DESC)` supports terminal drill-down.
- `payment_attempts (payment_session_id, try_seq)` is unique and supports session traceability.
- Additional status, PSP, bank, payer-token, or timestamp indexes require an approved API query and `EXPLAIN (ANALYZE, BUFFERS)` evidence; do not index every CSV column.

## Deletion and mutation policy

- Payment/import provenance uses restrictive foreign keys; no cascading deletion of analytical facts from merchant/profile operations.
- Imported source facts are immutable after a completed import. Corrections create a new versioned recovery/import decision.
- Production payment facts are append-only in the MVP. Update/delete semantics require a future ADR and corresponding cache-invalidation rules.
- Synthetic display profiles and credentials can change independently from source facts.
- Redis analytics entries are disposable and rebuilt on a cache miss. Source/cohort and metric-definition revisions prevent stale keys from appearing current.

## Production population boundary

After the one-time CSV bootstrap, ZarinPal—not this project—writes future production payment records directly to the normalized PostgreSQL session/attempt tables. This project owns the schema/migration, insert-triggered analytics-revision, and read contract but not ZarinPal's write workflow. Compatibility, required fields, source origin, and append-only insertion semantics are versioned interfaces; recurring CSV ETL is prohibited.