zarinpal-analytics-backend/docs/PHYSICAL-SCHEMA-PROPOSAL.md

# PostgreSQL Physical Schema Proposal

- Status: Draft for decision batch
- Scope: Data foundation only
- Implementation authorization: Not granted

## Principles

- Normalize business entities and enforce relationships with foreign keys.
- Preserve source identifiers as text and use internal surrogate keys for joins.
- Keep payment session and payment attempt grains separate.
- Use UTC `timestamptz` for canonical instants and `bigint` for Rial amounts.
- Keep normalized facts as source of truth; analytics projections are rebuildable.
- No initial table partitioning. Add indexes from approved access paths and measured query plans.
- The CSV importer is a one-time administrative bootstrap, not a production ingestion service.

## Proposed tables

### Provenance and reference

| Table | Purpose | Key constraints |
|---|---|---|
| `import_runs` | One bootstrap/recovery execution | unique source checksum + schema/importer policy; explicit lifecycle |
| `import_issues` | Aggregated validation/quality evidence | FK to import run; severity/rule/count/evidence |
| `merchant_categories` | Five source business categories | unique source category key |
| `psps` | Opaque PSP reference codes | unique source PSP code |
| `issuer_banks` | Opaque bank reference codes | unique source bank code |
| `calendar_dates` | Gregorian/Jalali reporting attributes | Gregorian date PK; unique Jalali date tuple |
| `calendar_events` | Versioned holidays/events | stable event code + source version |
| `calendar_date_events` | Many-to-many event membership | unique date/event pair; FKs |

Switch response codes remain nullable text on attempts until an authoritative description catalogue exists. Normalizing an undocumented 97.4%-missing field into a lookup table adds little value.

### Merchant identity

| Table | Purpose | Key constraints |
|---|---|---|
| `merchants` | Merchant profile and category membership | unique source merchant key; FK category |
| `terminals` | Merchant-owned payment terminals | unique source terminal key; FK merchant |
| `merchant_users` | Login identities | unique normalized username; FK merchant; password hash only |
| `auth_sessions` | Revocable authenticated sessions | hashed opaque session token; FK user; expiry/revocation |

Synthetic profile fields carry an explicit `profile_is_synthetic` marker. Credentials are generated outside Git.

### Payment facts

| Table | Grain and important fields | Key constraints/index direction |
|---|---|---|
| `payment_sessions` | One source session: merchant, terminal, amount in Rial, source status, created/expires instants, import provenance | unique source session key; FKs; merchant/time and terminal/time indexes |
| `payment_attempts` | One try: session, sequence, attempt status, PSP/bank, response, opaque payer token, verification type, timings, fee, attempt/verified/settled instants | unique `(payment_session_id, try_seq)`; FKs; session index; measured analytical indexes later |

`adjusted_fee` uses fixed precision and is labeled as an obfuscated relative value. The physical precision is finalized from exact decimal profiling before migration approval.

Both fact tables distinguish `bootstrap` from `zarinpal_live` origin. Bootstrap rows reference an import run; live rows do not invent one. Live creation/update metadata forms the incremental analytics watermark contract, including status changes to an existing session.

### Rebuildable analytics projections

| Table family | Grain | Purpose |
|---|---|---|
| `merchant_day_metrics` | merchant + Tehran local date | Additive counts/sums and metric numerators/denominators |
| `merchant_day_*_breakdowns` | merchant + date + approved dimension | Only metric-catalogue breakdowns proven useful, such as PSP/failure groups |
| `merchant_period_*` | merchant + approved period/event | Non-additive or measured expensive metrics only |
| `cohort_period_*` | approved cohort + period | Suppressed peer benchmark inputs/outputs with provenance and freshness |

Projection names/columns are finalized by the analytics metric catalogue. Avoid a generic key/value metrics table because it weakens types, constraints, and discoverability.

Projection metadata records source watermark, calculation version, calculated time, and status. An incremental refresh runs at least every 15 minutes; a nightly reconciliation checks/rebuilds an authoritative window and advances freshness only after successful atomic publication.

## Initial index policy

- Unique indexes implement source identities and natural relationship constraints.
- `payment_sessions (merchant_id, created_at DESC)` supports merchant period queries.
- `payment_sessions (terminal_id, created_at DESC)` supports terminal drill-down.
- `payment_attempts (payment_session_id, try_seq)` is unique and supports session traceability.
- Projection primary keys start with merchant/cohort and period.
- Additional status, PSP, bank, payer-token, or timestamp indexes require an approved API query and `EXPLAIN (ANALYZE, BUFFERS)` evidence; do not index every CSV column.

## Deletion and mutation policy

- Payment/import provenance uses restrictive foreign keys; no cascading deletion of analytical facts from merchant/profile operations.
- Imported source facts are immutable after a completed import. Corrections create a new versioned recovery/import decision.
- Synthetic display profiles and credentials can change independently from source facts.
- Derived projections may be rebuilt and atomically replaced; freshness/provenance prevents stale data from appearing current.

## Production population boundary

After the one-time CSV bootstrap, ZarinPal—not this project—writes future production payment records directly to the normalized PostgreSQL session/attempt tables. This project owns the schema/migration and analytics contract but not ZarinPal's write workflow. Compatibility, required fields, source origin, and creation/update watermarks are versioned interfaces; recurring CSV ETL is prohibited.