zarinpal-analytics-backend/docs/PHYSICAL-SCHEMA-PROPOSAL.md

PostgreSQL Physical Schema Proposal

• Status: Draft for decision batch
• Scope: Data foundation only
• Implementation authorization: Not granted

Principles

• Normalize business entities and enforce relationships with foreign keys.
• Preserve source identifiers as text and use internal surrogate keys for joins.
• Keep payment session and payment attempt grains separate.
• Use UTC timestamptz for canonical instants and bigint for Rial amounts.
• Keep normalized facts as source of truth; analytics projections are rebuildable.
• No initial table partitioning. Add indexes from approved access paths and measured query plans.
• The CSV importer is a one-time administrative bootstrap, not a production ingestion service.

Proposed tables

Provenance and reference

Table	Purpose	Key constraints
import_runs	One bootstrap/recovery execution	unique source checksum + schema/importer policy; explicit lifecycle
import_issues	Aggregated validation/quality evidence	FK to import run; severity/rule/count/evidence
merchant_categories	Five source business categories	unique source category key
psps	Opaque PSP reference codes	unique source PSP code
issuer_banks	Opaque bank reference codes	unique source bank code
calendar_dates	Gregorian/Jalali day, week, month, and Persian-season reporting attributes	Gregorian date PK; unique Jalali date tuple

Switch response codes remain nullable text on attempts until an authoritative description catalogue exists. Normalizing an undocumented 97.4%-missing field into a lookup table adds little value.

Merchant identity

Table	Purpose	Key constraints
merchants	Merchant profile and category membership	unique source merchant key; FK category
terminals	Merchant-owned payment terminals	unique source terminal key; FK merchant
merchant_users	Login identities	unique normalized username; FK merchant; password hash only
auth_sessions	Revocable authenticated sessions	hashed opaque session token; FK user; expiry/revocation

Synthetic profile fields carry an explicit profile_is_synthetic marker. Credentials are generated outside Git.

Payment facts

Table	Grain and important fields	Key constraints/index direction
payment_sessions	One source session: merchant, terminal, amount in Rial, source status, created/expires instants, import provenance	unique source session key; FKs; merchant/time and terminal/time indexes
payment_attempts	One try: session, sequence, attempt status, PSP/bank, response, opaque payer token, verification type, timings, fee, attempt/verified/settled instants	unique (payment_session_id, try_seq); FKs; session index; measured analytical indexes later

adjusted_fee uses fixed precision and is labeled as an obfuscated relative value. The physical precision is finalized from exact decimal profiling before migration approval.

Both fact tables distinguish bootstrap from zarinpal_live origin. Bootstrap rows reference an import run; live rows do not invent one. Live creation/update metadata forms the incremental analytics watermark contract, including status changes to an existing session.

Rebuildable analytics projections

Table family	Grain	Purpose
merchant_day_metrics	merchant + Tehran local date	Additive counts/sums and metric numerators/denominators
merchant_day_*_breakdowns	merchant + date + approved dimension	Only metric-catalogue breakdowns proven useful, such as PSP/failure groups
merchant_period_*	merchant + approved day/week/month/season period	Non-additive or measured expensive metrics only
cohort_period_*	approved cohort + period	Suppressed peer benchmark inputs/outputs with provenance and freshness

Projection names/columns are finalized by the analytics metric catalogue. Avoid a generic key/value metrics table because it weakens types, constraints, and discoverability.

Projection metadata records source watermark, calculation version, calculated time, and status. An incremental refresh runs at least every 15 minutes; a nightly reconciliation checks/rebuilds an authoritative window and advances freshness only after successful atomic publication.

Initial index policy

• Unique indexes implement source identities and natural relationship constraints.
• payment_sessions (merchant_id, created_at DESC) supports merchant period queries.
• payment_sessions (terminal_id, created_at DESC) supports terminal drill-down.
• payment_attempts (payment_session_id, try_seq) is unique and supports session traceability.
• Projection primary keys start with merchant/cohort and period.
• Additional status, PSP, bank, payer-token, or timestamp indexes require an approved API query and EXPLAIN (ANALYZE, BUFFERS) evidence; do not index every CSV column.

Deletion and mutation policy

• Payment/import provenance uses restrictive foreign keys; no cascading deletion of analytical facts from merchant/profile operations.
• Imported source facts are immutable after a completed import. Corrections create a new versioned recovery/import decision.
• Synthetic display profiles and credentials can change independently from source facts.
• Derived projections may be rebuilt and atomically replaced; freshness/provenance prevents stale data from appearing current.

Production population boundary

After the one-time CSV bootstrap, ZarinPal—not this project—writes future production payment records directly to the normalized PostgreSQL session/attempt tables. This project owns the schema/migration and analytics contract but not ZarinPal's write workflow. Compatibility, required fields, source origin, and creation/update watermarks are versioned interfaces; recurring CSV ETL is prohibited.