zarinpal-analytics-backend/docs/PHYSICAL-SCHEMA-PROPOSAL.md

PostgreSQL Physical Schema Proposal

• Status: Draft for decision batch
• Scope: Data foundation only
• Implementation authorization: Not granted

Principles

• Normalize business entities and enforce relationships with foreign keys.
• Preserve source identifiers as text and use internal surrogate keys for joins.
• Keep payment session and payment attempt grains separate.
• Use UTC timestamptz for canonical instants and bigint for Rial amounts.
• Keep normalized PostgreSQL facts as source of truth; Redis analytics results are disposable and rebuildable.
• No initial table partitioning. Add indexes from approved access paths and measured query plans.
• The CSV importer is a one-time administrative bootstrap, not a production ingestion service.

Proposed tables

Provenance and reference

Table	Purpose	Key constraints
import_runs	One bootstrap/recovery execution	unique source checksum + schema/importer policy; explicit lifecycle
import_issues	Aggregated validation/quality evidence	FK to import run; severity/rule/count/evidence
merchant_categories	Five source business categories	unique source category key
psps	Opaque PSP reference codes	unique source PSP code
issuer_banks	Opaque bank reference codes	unique source bank code
calendar_dates	Gregorian/Jalali day, week, month, and Persian-season reporting attributes	Gregorian date PK; unique Jalali date tuple

Switch response codes remain nullable text on attempts until an authoritative description catalogue exists. Normalizing an undocumented 97.4%-missing field into a lookup table adds little value.

Merchant identity

Table	Purpose	Key constraints
merchants	Merchant profile and category membership	unique source merchant key; FK category
terminals	Merchant-owned payment terminals	unique source terminal key; FK merchant
merchant_users	Login identities	unique normalized username; FK merchant; password hash only
auth_sessions	Revocable authenticated sessions	hashed opaque session token; FK user; expiry/revocation

Synthetic profile fields carry an explicit profile_is_synthetic marker. Credentials are generated outside Git.

Payment facts

Table	Grain and important fields	Key constraints/index direction
payment_sessions	One source session: merchant, terminal, amount and merchant fee in Rial, source status, created/expires instants, import provenance	unique source session key; FKs; merchant/time and terminal/time indexes
payment_attempts	One try: session, sequence, attempt status, PSP/bank, response, opaque payer token, verification type, timings, attempt/verified/settled instants	unique (payment_session_id, try_seq); FKs; session index; measured analytical indexes later

Source adjusted_fee is a real merchant-fee concept for the product and is renamed fee_amount_rial. Profiling found the value identical across every repeated row of a session (150,450 repeated rows, zero conflicts), so it is stored once on payment_sessions as positive bigint Rial and never summed at attempt grain.

Both fact tables distinguish bootstrap from zarinpal_live origin. Bootstrap rows reference an import run; live rows do not invent one. Payment facts are append-only for the MVP; inserts advance database-managed analytics revisions.

Analytics invalidation state

Table	Grain	Purpose
merchant_analytics_revisions	one row per merchant	monotonically increasing source revision advanced by related fact inserts
cohort_analytics_revisions	one row per benchmark cohort/category	monotonically increasing peer-source revision advanced by relevant inserts

No calculated dashboard result is stored in PostgreSQL. Analytics queries and Redis value contracts are finalized by the metric catalogue and API spec.

Inserts atomically increment affected revision rows through schema-owned database logic so direct upstream writes cannot bypass invalidation. Redis keys include merchant/cohort revisions, metric-definition version, and analytics-section id. Old revision keys expire after 30 days; invalidation does not scan or delete arbitrary keys. Redis values follow typed, versioned response schemas and contain aggregates only.

Initial index policy

• Unique indexes implement source identities and natural relationship constraints.
• payment_sessions (merchant_id, created_at DESC) supports merchant period queries.
• payment_sessions (terminal_id, created_at DESC) supports terminal drill-down.
• payment_attempts (payment_session_id, try_seq) is unique and supports session traceability.
• Additional status, PSP, bank, payer-token, or timestamp indexes require an approved API query and EXPLAIN (ANALYZE, BUFFERS) evidence; do not index every CSV column.

Deletion and mutation policy

• Payment/import provenance uses restrictive foreign keys; no cascading deletion of analytical facts from merchant/profile operations.
• Imported source facts are immutable after a completed import. Corrections create a new versioned recovery/import decision.
• Production payment facts are append-only in the MVP. Update/delete semantics require a future ADR and corresponding cache-invalidation rules.
• Synthetic display profiles and credentials can change independently from source facts.
• Redis analytics entries are disposable and rebuilt on a cache miss. Source/cohort and metric-definition revisions prevent stale keys from appearing current.

Production population boundary

After the one-time CSV bootstrap, ZarinPal—not this project—writes future production payment records directly to the normalized PostgreSQL session/attempt tables. This project owns the schema/migration, insert-triggered analytics-revision, and read contract but not ZarinPal's write workflow. Compatibility, required fields, source origin, and append-only insertion semantics are versioned interfaces; recurring CSV ETL is prohibited.