starcoach-hackathon/zarinpal-analytics-backend
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(schema): define normalized postgres model

Browse Source
This commit is contained in:
Amirhossein Khalili
2026-08-20 18:23:37 +03:30
parent 08ac99a5a6
commit ee93460182
2 changed files with 484 additions and 54 deletions
Download Patch File Download Diff File Expand all files Collapse all files

209
docs/PHYSICAL-SCHEMA-ERD.html Normal file
View File

@@ -0,0 +1,209 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>ZarinPal Analytics — Physical Schema ERD</title>
<style>
:root {
color-scheme: light;
--paper: #f1f1f3;
--surface: #ffffff;
--ink: #19191a;
--muted: #6b6b73;
--soft: #8c8c94;
--rule: #d8d8dd;
--accent: #0a33ff;
--accent-tint: #e7ecff;
--yellow: #ffd400;
}
* { box-sizing: border-box; }
body {
margin: 0;
background: var(--paper);
color: var(--ink);
font-family: Tahoma, system-ui, sans-serif;
}
main { width: min(100% - 32px, 1320px); margin: 0 auto; padding: 48px 0 64px; }
header { max-width: 880px; margin-bottom: 40px; }
.eyebrow { margin: 0 0 8px; color: var(--accent); font: 700 12px/1.5 ui-monospace, monospace; letter-spacing: .12em; }
h1 { margin: 0; font-size: clamp(28px, 4vw, 40px); line-height: 1.2; }
.lede { margin: 16px 0 0; color: var(--muted); font-size: 16px; line-height: 1.7; }
figure { margin: 0 0 32px; padding: 24px; overflow-x: auto; background: var(--surface); border: 1px solid var(--rule); border-radius: 12px; }
figcaption { margin: 0 0 20px; font-size: 20px; font-weight: 700; }
svg { display: block; min-width: 1120px; width: 100%; height: auto; }
.edge { fill: none; stroke: var(--muted); stroke-width: 1.2; }
.edge-accent { stroke: var(--accent); }
.entity-mask { fill: var(--surface); }
.entity { fill: var(--surface); stroke: var(--ink); stroke-width: 1; }
.entity.focal { fill: var(--accent-tint); stroke: var(--accent); stroke-width: 1.2; }
.tag { fill: var(--yellow); }
.tag-text { fill: var(--ink); font: 700 8px/1 ui-monospace, monospace; letter-spacing: .08em; }
.name { fill: var(--ink); font: 700 12px/1 Tahoma, system-ui, sans-serif; }
.field { fill: var(--muted); font: 400 8px/1.2 ui-monospace, monospace; }
.cardinality { fill: var(--soft); font: 700 8px/1 ui-monospace, monospace; }
.divider { stroke: var(--rule); stroke-width: .8; }
.legend { display: flex; flex-wrap: wrap; gap: 12px 24px; margin-top: 16px; padding-top: 16px; border-top: 1px solid var(--rule); color: var(--muted); font: 12px/1.5 ui-monospace, monospace; }
.legend b { color: var(--ink); }
.note { margin-top: 24px; color: var(--muted); font-size: 13px; line-height: 1.7; }
@media (prefers-color-scheme: dark) {
:root { --paper: #0f0f10; --surface: #19191a; --ink: #ffffff; --muted: #d1d1d6; --soft: #a7a7af; --rule: #3a3a3d; --accent: #90aef4; --accent-tint: #282f42; }
.tag-text { fill: #19191a; }
}
</style>
</head>
<body>
<main>
<header>
<p class="eyebrow">DATA FOUNDATION / PHYSICAL ERD</p>
<h1>ZarinPal merchant analytics schema</h1>
<p class="lede">Two focused views keep the normalized payment model and operational state readable. The written physical-schema specification remains authoritative for every column, constraint, index, and lifecycle rule.</p>
</header>
<figure>
<figcaption>Payment facts and reference ownership</figcaption>
<svg viewBox="0 0 1280 640" role="img" aria-labelledby="payment-erd-title payment-erd-desc">
<title id="payment-erd-title">Payment facts physical entity relationships</title>
<desc id="payment-erd-desc">Merchant categories classify merchants, merchants own terminals, terminals own payment sessions, sessions contain payment attempts, and attempts reference PSP and issuer-bank codes. Bootstrap sessions reference an import run.</desc>
<path class="edge" d="M220 128 H280" />
<text class="cardinality" x="228" y="120">1</text><text class="cardinality" x="264" y="120">N</text>
<path class="edge" d="M460 128 H520" />
<text class="cardinality" x="468" y="120">1</text><text class="cardinality" x="504" y="120">N</text>
<path class="edge edge-accent" d="M700 128 H760" />
<text class="cardinality" x="708" y="120">1</text><text class="cardinality" x="744" y="120">N</text>
<path class="edge edge-accent" d="M960 144 H1020" />
<text class="cardinality" x="968" y="136">1</text><text class="cardinality" x="1004" y="136">N</text>
<path class="edge" d="M860 352 V272" />
<text class="cardinality" x="872" y="340">1</text><text class="cardinality" x="872" y="288">N</text>
<path class="edge" d="M1080 288 V376 H1016 V416" />
<text class="cardinality" x="1092" y="304">N</text><text class="cardinality" x="1028" y="404">1</text>
<path class="edge" d="M1140 288 V376 H1220 V416" />
<text class="cardinality" x="1152" y="304">N</text><text class="cardinality" x="1232" y="404">0..1</text>
<g transform="translate(40 80)">
<rect class="entity-mask" width="180" height="112" rx="8" /><rect class="entity" width="180" height="112" rx="8" />
<rect class="tag" x="12" y="12" width="48" height="16" rx="4" /><text class="tag-text" x="36" y="23" text-anchor="middle">ENTITY</text>
<text class="name" x="12" y="48">merchant_categories</text><line class="divider" x1="12" y1="60" x2="168" y2="60" />
<text class="field" x="12" y="80"># id bigint</text><text class="field" x="12" y="96">source_category_key UK</text>
</g>
<g transform="translate(280 72)">
<rect class="entity-mask" width="180" height="128" rx="8" /><rect class="entity" width="180" height="128" rx="8" />
<rect class="tag" x="12" y="12" width="48" height="16" rx="4" /><text class="tag-text" x="36" y="23" text-anchor="middle">ENTITY</text>
<text class="name" x="12" y="48">merchants</text><line class="divider" x1="12" y1="60" x2="168" y2="60" />
<text class="field" x="12" y="80"># id bigint</text><text class="field" x="12" y="96">→ category_id</text><text class="field" x="12" y="112">source_merchant_key UK</text>
</g>
<g transform="translate(520 64)">
<rect class="entity-mask" width="180" height="144" rx="8" /><rect class="entity" width="180" height="144" rx="8" />
<rect class="tag" x="12" y="12" width="48" height="16" rx="4" /><text class="tag-text" x="36" y="23" text-anchor="middle">ENTITY</text>
<text class="name" x="12" y="48">terminals</text><line class="divider" x1="12" y1="60" x2="168" y2="60" />
<text class="field" x="12" y="80"># id bigint</text><text class="field" x="12" y="96">→ merchant_id</text><text class="field" x="12" y="112">source_terminal_key UK</text><text class="field" x="12" y="128">verification_type</text>
</g>
<g transform="translate(760 48)">
<rect class="entity-mask" width="200" height="224" rx="8" /><rect class="entity focal" width="200" height="224" rx="8" />
<rect class="tag" x="12" y="12" width="48" height="16" rx="4" /><text class="tag-text" x="36" y="23" text-anchor="middle">ROOT</text>
<text class="name" x="12" y="48">payment_sessions</text><line class="divider" x1="12" y1="60" x2="188" y2="60" />
<text class="field" x="12" y="80"># id bigint</text><text class="field" x="12" y="96">→ merchant_id</text><text class="field" x="12" y="112">→ terminal_id</text><text class="field" x="12" y="128">source_session_key UK</text><text class="field" x="12" y="144">amount_rial</text><text class="field" x="12" y="160">fee_amount_rial</text><text class="field" x="12" y="176">session_status</text><text class="field" x="12" y="192">created_at / expires_at</text><text class="field" x="12" y="208">→ import_run_id nullable</text>
</g>
<g transform="translate(1020 32)">
<rect class="entity-mask" width="220" height="256" rx="8" /><rect class="entity" width="220" height="256" rx="8" />
<rect class="tag" x="12" y="12" width="48" height="16" rx="4" /><text class="tag-text" x="36" y="23" text-anchor="middle">ENTITY</text>
<text class="name" x="12" y="48">payment_attempts</text><line class="divider" x1="12" y1="60" x2="208" y2="60" />
<text class="field" x="12" y="80"># id bigint</text><text class="field" x="12" y="96">→ payment_session_id</text><text class="field" x="12" y="112">try_seq UK per session</text><text class="field" x="12" y="128">try_status</text><text class="field" x="12" y="144">→ psp_id</text><text class="field" x="12" y="160">switch_response_code?</text><text class="field" x="12" y="176">→ issuer_bank_id?</text><text class="field" x="12" y="192">payer_card_key?</text><text class="field" x="12" y="208">init/verify_time_ms?</text><text class="field" x="12" y="224">try_created_at</text><text class="field" x="12" y="240">verified/settled_at?</text>
</g>
<g transform="translate(760 352)">
<rect class="entity-mask" width="200" height="128" rx="8" /><rect class="entity" width="200" height="128" rx="8" />
<rect class="tag" x="12" y="12" width="48" height="16" rx="4" /><text class="tag-text" x="36" y="23" text-anchor="middle">ENTITY</text>
<text class="name" x="12" y="48">import_runs</text><line class="divider" x1="12" y1="60" x2="188" y2="60" />
<text class="field" x="12" y="80"># id bigint</text><text class="field" x="12" y="96">source_sha256 + versions</text><text class="field" x="12" y="112">status + reconciliation</text>
</g>
<g transform="translate(916 416)">
<rect class="entity-mask" width="200" height="112" rx="8" /><rect class="entity" width="200" height="112" rx="8" />
<rect class="tag" x="12" y="12" width="48" height="16" rx="4" /><text class="tag-text" x="36" y="23" text-anchor="middle">REF</text>
<text class="name" x="12" y="48">psps</text><line class="divider" x1="12" y1="60" x2="188" y2="60" />
<text class="field" x="12" y="80"># id bigint</text><text class="field" x="12" y="96">source_psp_code UK</text>
</g>
<g transform="translate(1120 416)">
<rect class="entity-mask" width="140" height="112" rx="8" /><rect class="entity" width="140" height="112" rx="8" />
<rect class="tag" x="12" y="12" width="40" height="16" rx="4" /><text class="tag-text" x="32" y="23" text-anchor="middle">REF</text>
<text class="name" x="12" y="48">issuer_banks</text><line class="divider" x1="12" y1="60" x2="128" y2="60" />
<text class="field" x="12" y="80"># id</text><text class="field" x="12" y="96">source_bank_code UK</text>
</g>
</svg>
<div class="legend"><span><b>#</b> primary key</span><span><b>→</b> foreign key</span><span><b>UK</b> unique key</span><span><b>?</b> nullable</span><span><b>Blue</b> payment aggregate root</span></div>
</figure>
<figure>
<figcaption>Identity, audit, reporting calendar, and cache revisions</figcaption>
<svg viewBox="0 0 1280 600" role="img" aria-labelledby="ops-erd-title ops-erd-desc">
<title id="ops-erd-title">Operational physical entity relationships</title>
<desc id="ops-erd-desc">Merchants own demo users and merchant analytics revisions; users own server-side sessions, 30-day authentication audit events, and 90-day analytics access audit events; categories own cohort revisions; the reporting calendar remains an independent date dimension.</desc>
<path class="edge edge-accent" d="M240 128 H320" /><text class="cardinality" x="248" y="120">1</text><text class="cardinality" x="304" y="120">N</text>
<path class="edge" d="M520 112 H600" /><text class="cardinality" x="528" y="104">1</text><text class="cardinality" x="584" y="104">N</text>
<path class="edge" d="M420 208 V320" /><text class="cardinality" x="432" y="224">1</text><text class="cardinality" x="432" y="308">N</text>
<path class="edge" d="M520 176 H560 V256 H1010 V224" /><text class="cardinality" x="528" y="168">1</text><text class="cardinality" x="1022" y="240">N</text>
<path class="edge" d="M140 192 V320" /><text class="cardinality" x="152" y="208">1</text><text class="cardinality" x="152" y="308">1</text>
<path class="edge" d="M820 384 H900" /><text class="cardinality" x="828" y="376">1</text><text class="cardinality" x="884" y="376">N</text>
<g transform="translate(40 64)">
<rect class="entity-mask" width="200" height="128" rx="8" /><rect class="entity focal" width="200" height="128" rx="8" />
<rect class="tag" x="12" y="12" width="48" height="16" rx="4" /><text class="tag-text" x="36" y="23" text-anchor="middle">ROOT</text>
<text class="name" x="12" y="48">merchants</text><line class="divider" x1="12" y1="60" x2="188" y2="60" />
<text class="field" x="12" y="80"># id bigint</text><text class="field" x="12" y="96">→ category_id</text><text class="field" x="12" y="112">source_merchant_key UK</text>
</g>
<g transform="translate(320 48)">
<rect class="entity-mask" width="200" height="160" rx="8" /><rect class="entity" width="200" height="160" rx="8" />
<rect class="tag" x="12" y="12" width="48" height="16" rx="4" /><text class="tag-text" x="36" y="23" text-anchor="middle">ENTITY</text>
<text class="name" x="12" y="48">merchant_users</text><line class="divider" x1="12" y1="60" x2="188" y2="60" />
<text class="field" x="12" y="80"># id bigint</text><text class="field" x="12" y="96">→ merchant_id</text><text class="field" x="12" y="112">username_normalized UK</text><text class="field" x="12" y="128">password_hash</text><text class="field" x="12" y="144">disabled_at?</text>
</g>
<g transform="translate(600 32)">
<rect class="entity-mask" width="220" height="176" rx="8" /><rect class="entity" width="220" height="176" rx="8" />
<rect class="tag" x="12" y="12" width="48" height="16" rx="4" /><text class="tag-text" x="36" y="23" text-anchor="middle">ENTITY</text>
<text class="name" x="12" y="48">auth_sessions</text><line class="divider" x1="12" y1="60" x2="208" y2="60" />
<text class="field" x="12" y="80"># id bigint</text><text class="field" x="12" y="96">→ merchant_user_id</text><text class="field" x="12" y="112">token_digest bytea UK</text><text class="field" x="12" y="128">idle_expires_at</text><text class="field" x="12" y="144">absolute_expires_at</text><text class="field" x="12" y="160">revoked_at?</text>
</g>
<g transform="translate(320 320)">
<rect class="entity-mask" width="220" height="160" rx="8" /><rect class="entity" width="220" height="160" rx="8" />
<rect class="tag" x="12" y="12" width="48" height="16" rx="4" /><text class="tag-text" x="36" y="23" text-anchor="middle">ENTITY</text>
<text class="name" x="12" y="48">auth_audit_events</text><line class="divider" x1="12" y1="60" x2="208" y2="60" />
<text class="field" x="12" y="80"># id bigint</text><text class="field" x="12" y="96">→ merchant_user_id?</text><text class="field" x="12" y="112">event_type / outcome</text><text class="field" x="12" y="128">occurred_at</text><text class="field" x="12" y="144">expires_at (30 days)</text>
</g>
<g transform="translate(900 48)">
<rect class="entity-mask" width="220" height="176" rx="8" /><rect class="entity" width="220" height="176" rx="8" />
<rect class="tag" x="12" y="12" width="48" height="16" rx="4" /><text class="tag-text" x="36" y="23" text-anchor="middle">ENTITY</text>
<text class="name" x="12" y="48">analytics_access_audit_events</text><line class="divider" x1="12" y1="60" x2="208" y2="60" />
<text class="field" x="12" y="80"># id bigint</text><text class="field" x="12" y="96">→ merchant_user_id + merchant_id</text><text class="field" x="12" y="112">event_type / result</text><text class="field" x="12" y="128">filter_fingerprint?</text><text class="field" x="12" y="144">row_count? / byte_count?</text><text class="field" x="12" y="160">expires_at (90 days)</text>
</g>
<g transform="translate(40 320)">
<rect class="entity-mask" width="200" height="128" rx="8" /><rect class="entity" width="200" height="128" rx="8" />
<rect class="tag" x="12" y="12" width="48" height="16" rx="4" /><text class="tag-text" x="36" y="23" text-anchor="middle">STATE</text>
<text class="name" x="12" y="48">merchant_analytics_revisions</text><line class="divider" x1="12" y1="60" x2="188" y2="60" />
<text class="field" x="12" y="80"># → merchant_id</text><text class="field" x="12" y="96">source_revision</text><text class="field" x="12" y="112">updated_at</text>
</g>
<g transform="translate(600 320)">
<rect class="entity-mask" width="220" height="128" rx="8" /><rect class="entity" width="220" height="128" rx="8" />
<rect class="tag" x="12" y="12" width="48" height="16" rx="4" /><text class="tag-text" x="36" y="23" text-anchor="middle">ROOT</text>
<text class="name" x="12" y="48">merchant_categories</text><line class="divider" x1="12" y1="60" x2="208" y2="60" />
<text class="field" x="12" y="80"># id bigint</text><text class="field" x="12" y="96">source_category_key UK</text><text class="field" x="12" y="112">title_fa</text>
</g>
<g transform="translate(900 320)">
<rect class="entity-mask" width="220" height="128" rx="8" /><rect class="entity" width="220" height="128" rx="8" />
<rect class="tag" x="12" y="12" width="48" height="16" rx="4" /><text class="tag-text" x="36" y="23" text-anchor="middle">STATE</text>
<text class="name" x="12" y="48">cohort_analytics_revisions</text><line class="divider" x1="12" y1="60" x2="208" y2="60" />
<text class="field" x="12" y="80"># → category_id</text><text class="field" x="12" y="96">source_revision</text><text class="field" x="12" y="112">updated_at</text>
</g>
<g transform="translate(900 480)">
<rect class="entity-mask" width="220" height="96" rx="8" /><rect class="entity" width="220" height="96" rx="8" />
<rect class="tag" x="12" y="12" width="48" height="16" rx="4" /><text class="tag-text" x="36" y="23" text-anchor="middle">DIM</text>
<text class="name" x="12" y="48">calendar_dates</text><line class="divider" x1="12" y1="60" x2="208" y2="60" /><text class="field" x="12" y="80"># gregorian_date + Jalali periods</text>
</g>
</svg>
<div class="legend"><span><b>Identity</b> is demo-sized and replaceable by SSO</span><span><b>Revision</b> rows select cache namespaces</span><span><b>Calendar</b> contains no holiday dimension</span></div>
</figure>
<p class="note">Typography note: this standalone backend document uses a system fallback to remain self-contained. The product UI uses the licensed Git-tracked Iran Yekan X FaNum variable webfont. Diagram palette follows the approved ZarinPal-inspired light/dark semantic tokens.</p>
</main>
</body>
</html>

329
docs/PHYSICAL-SCHEMA-PROPOSAL.md
View File

@@ -1,85 +1,306 @@
# PostgreSQL Physical Schema Proposal # PostgreSQL Physical Schema Specification
- Status: Draft for decision batch - Status: Decision-ready draft — stakeholder approval required
- Scope: Data foundation only - Scope: Data foundation, demo identity persistence, and analytics invalidation state
- Implementation authorization: Not granted - Implementation authorization: Not granted
- Last updated: 2026-08-20
- Visual companion: [`PHYSICAL-SCHEMA-ERD.html`](./PHYSICAL-SCHEMA-ERD.html)
## Principles ## Objective
- Normalize business entities and enforce relationships with foreign keys. Define the implementation-ready PostgreSQL shape for the normalized ZarinPal archive without writing migrations. The schema preserves source identity and import provenance, separates the 2,062,839 payment sessions from 1,949,353 real attempts, enforces tenant relationships with foreign keys, supports approved analytics and CSV-detail access paths, and leaves Redis as a disposable result cache.
- Preserve source identifiers as text and use internal surrogate keys for joins.
- Keep payment session and payment attempt grains separate.
- Use UTC `timestamptz` for canonical instants and `bigint` for Rial amounts.
- Keep normalized PostgreSQL facts as source of truth; Redis analytics results are disposable and rebuildable.
- No initial table partitioning. Add indexes from approved access paths and measured query plans.
- The CSV importer is a one-time administrative bootstrap, not a production ingestion service.
## Proposed tables ## Evidence and conventions
### Provenance and reference - The archive contains 2,213,289 source records, including 263,936 `NoAttempt` markers. A marker creates a session and no attempt row.
- Internal relational keys are `bigint GENERATED ALWAYS AS IDENTITY`. Source identifiers remain non-empty `text` with unique constraints because they are opaque even when they look numeric.
- Canonical instants are `timestamptz` in UTC. A `calendar_dates` dimension provides Tehran-local Gregorian/Jalali reporting attributes.
- Money is `bigint` Rial. `amount_rial` must be positive; `fee_amount_rial` may be zero in production and must not be multiplied by attempt count.
- Flexible upstream lifecycle/status codes remain non-empty `text`, not PostgreSQL enums. Analytics uses a versioned allowlist so a future well-formed status can be preserved and reported as unclassified instead of making direct upstream writes fail.
- All foreign keys are restrictive unless a table definition explicitly says otherwise. Payment and import facts are not cascade-deleted.
- Ent owns application entities and relationships; Atlas produces reviewed versioned SQL; pgx `COPY` is restricted to staging/import paths. This document defines the target contract, not migration SQL.
| Table | Purpose | Key constraints | ## Relationship invariants
1. A merchant belongs to exactly one category.
2. A terminal belongs to exactly one merchant.
3. A payment session belongs to exactly one merchant and one terminal; a composite foreign key proves that the terminal belongs to that merchant.
4. A payment attempt belongs to exactly one session and has a positive sequence unique within that session.
5. A bootstrap session belongs to exactly one completed import lineage; a live session has no synthetic import run.
6. Attempt provenance is inherited through its session rather than duplicated on every attempt.
7. PSP and issuer-bank codes are reference entities. Switch response codes remain nullable PSP-scoped text because no authoritative meaning catalogue was supplied.
8. Payer-card tokens are nullable, merchant-scoped through the session relationship, and never globally unique.
9. Verification method is terminal-level. A full archive scan found zero terminals with more than one `verify_type`, while one merchant used different methods across its terminals.
## Physical tables
### Import provenance
#### `import_runs`
| Column | Type/nullability | Constraint or meaning |
|---|---|---| |---|---|---|
| `import_runs` | One bootstrap/recovery execution | unique source checksum + schema/importer policy; explicit lifecycle | | `id` | `bigint` required | identity primary key |
| `import_issues` | Aggregated validation/quality evidence | FK to import run; severity/rule/count/evidence | | `source_sha256` | `text` required | lowercase 64-character SHA-256 hex |
| `merchant_categories` | Five source business categories | unique source category key | | `source_file_name` | `text` required | basename only; never an operator filesystem path |
| `psps` | Opaque PSP reference codes | unique source PSP code | | `source_schema_version` | `text` required | explicit supported input contract |
| `issuer_banks` | Opaque bank reference codes | unique source bank code | | `importer_revision` | `text` required | immutable code/build revision |
| `calendar_dates` | Gregorian/Jalali day, week, month, and Persian-season reporting attributes | Gregorian date PK; unique Jalali date tuple | | `status` | `text` required | `created`, `staging`, `validating`, `promoting`, `completed`, or `failed` |
| `observed_rows` | `bigint` required default `0` | source records read, non-negative |
| `accepted_source_rows` | `bigint` required default `0` | accepted source records, non-negative |
| `rejected_source_rows` | `bigint` required default `0` | rejected source records, non-negative |
| `created_sessions` | `bigint` required default `0` | promoted session rows, non-negative |
| `created_attempts` | `bigint` required default `0` | promoted real-attempt rows, non-negative |
| `failure_code` | `text` nullable | stable safe failure category |
| `failure_summary` | `text` nullable | bounded operator-safe summary without raw row data |
| `started_at` | `timestamptz` required | run start |
| `completed_at` | `timestamptz` nullable | terminal completion/failure time |
Switch response codes remain nullable text on attempts until an authoritative description catalogue exists. Their meaning is scoped by the attempt PSP, so the same raw code under different PSPs must never be assumed equivalent. Normalizing an undocumented 97.4%-missing field into a lookup table adds little value. A partial unique index on `(source_sha256, source_schema_version, importer_revision)` where `status = 'completed'` makes a completed import idempotent while permitting explicit retry after a failed run. Terminal state requires `completed_at`; non-terminal state forbids it.
### Merchant identity #### `import_issues`
| Table | Purpose | Key constraints | | Column | Type/nullability | Constraint or meaning |
|---|---|---| |---|---|---|
| `merchants` | Merchant profile and category membership | unique source merchant key; FK category | | `id` | `bigint` required | identity primary key |
| `terminals` | Merchant-owned payment terminals | unique source terminal key; FK merchant | | `import_run_id` | `bigint` required | restrictive FK to `import_runs` |
| `merchant_users` | Login identities | unique normalized username; FK merchant; password hash only | | `severity` | `text` required | `warning` or `fatal` |
| `auth_sessions` | Revocable demo authenticated sessions | unique SHA-256 digest of a 32-byte opaque token; FK user; last activity; idle/absolute expiry; revocation | | `rule_code` | `text` required | stable validation rule identifier |
| `affected_rows` | `bigint` required | positive aggregate count |
| `evidence` | `jsonb` required default `{}` | bounded aggregate examples; never payer tokens or credentials |
| `created_at` | `timestamptz` required | evidence time |
Synthetic profile fields carry an explicit `profile_is_synthetic` marker. Credentials are generated outside Git. Unique `(import_run_id, severity, rule_code)` keeps one aggregate record per rule. Index `(import_run_id, severity)` supports reports.
### Reference data
#### `merchant_categories`
`id bigint` identity PK; `source_category_key text` unique/non-empty; `title_fa text` non-empty; `created_at timestamptz` required.
#### `psps`
`id bigint` identity PK; `source_psp_code text` unique/non-empty; `created_at timestamptz` required. Codes are opaque and do not imply a public PSP identity.
#### `issuer_banks`
`id bigint` identity PK; `source_bank_code text` unique/non-empty; `created_at timestamptz` required. Codes are opaque and do not imply a public bank identity.
#### `calendar_dates`
| Column | Type/nullability | Constraint or meaning |
|---|---|---|
| `gregorian_date` | `date` required | primary key |
| `gregorian_year/month/day` | `smallint` required | validated Gregorian components |
| `jalali_year/month/day` | `smallint` required | month `1..12`, day valid for generated date |
| `iran_weekday` | `smallint` required | `0..6`, Saturday through Friday |
| `reporting_week_start` | `date` required | Saturday containing this date |
| `reporting_week_end_exclusive` | `date` required | following Saturday; equals start + 7 days |
| `persian_season` | `smallint` required | `1..4` |
Unique `(jalali_year, jalali_month, jalali_day)` prevents duplicate calendar mappings. Indexes on `(jalali_year, jalali_month, gregorian_date)`, `(jalali_year, persian_season, gregorian_date)`, and `(reporting_week_start, gregorian_date)` support approved periods. Holiday/named-event columns do not exist.
### Merchant and demo identity
#### `merchants`
| Column | Type/nullability | Constraint or meaning |
|---|---|---|
| `id` | `bigint` required | identity primary key |
| `source_merchant_key` | `text` required | unique/non-empty opaque source key |
| `category_id` | `bigint` required | restrictive FK to `merchant_categories` |
| `display_name` | `text` required | generated demo-facing name |
| `profile_is_synthetic` | `boolean` required | true for generated profile data |
| `created_at` | `timestamptz` required | creation time |
Index `category_id` supports peer cohort membership.
#### `terminals`
| Column | Type/nullability | Constraint or meaning |
|---|---|---|
| `id` | `bigint` required | identity primary key |
| `source_terminal_key` | `text` required | unique/non-empty opaque source key |
| `merchant_id` | `bigint` required | restrictive FK to `merchants` |
| `verification_type` | `text` required | `Automated` or `Manual` for the approved source contract |
| `created_at` | `timestamptz` required | creation time |
Unique `(id, merchant_id)` is the target of the session ownership FK. Index `merchant_id` supports merchant terminal lookup.
#### `merchant_users`
| Column | Type/nullability | Constraint or meaning |
|---|---|---|
| `id` | `bigint` required | identity primary key |
| `merchant_id` | `bigint` required | restrictive FK to `merchants` |
| `username_normalized` | `text` required | lowercase ASCII, globally unique |
| `password_hash` | `text` required | complete Argon2id encoded hash only |
| `is_demo` | `boolean` required default `true` | demo adapter marker |
| `disabled_at` | `timestamptz` nullable | disabled accounts cannot authenticate |
| `created_at/updated_at` | `timestamptz` required | lifecycle timestamps |
Unique `(id, merchant_id)` is the target of tenant-ownership audit FKs. A partial unique index on `merchant_id WHERE is_demo` enforces at most one seeded demo user per merchant without preventing a future non-demo identity migration.
#### `auth_sessions`
| Column | Type/nullability | Constraint or meaning |
|---|---|---|
| `id` | `bigint` required | identity primary key |
| `merchant_user_id` | `bigint` required | restrictive FK to `merchant_users` |
| `token_digest` | `bytea` required | unique 32-byte SHA-256 digest; never raw token |
| `created_at` | `timestamptz` required | creation time |
| `last_activity_at` | `timestamptz` required | coalesced activity time |
| `idle_expires_at` | `timestamptz` required | no more than 12 hours after approved activity |
| `absolute_expires_at` | `timestamptz` required | no more than 7 days after creation |
| `revoked_at` | `timestamptz` nullable | immediate invalidation time |
| `revocation_reason` | `text` nullable | safe bounded reason code |
Checks enforce `last_activity_at >= created_at`, `idle_expires_at <= absolute_expires_at`, and paired revocation fields. Unique `token_digest` is the lookup path. Index `(merchant_user_id, revoked_at, absolute_expires_at, created_at)` supports the two-active-session rule and revocation.
#### `auth_audit_events`
| Column | Type/nullability | Constraint or meaning |
|---|---|---|
| `id` | `bigint` required | identity primary key |
| `merchant_user_id` | `bigint` nullable | FK to `merchant_users`; null for an unresolved login subject |
| `event_type` | `text` required | stable approved security-event code |
| `outcome` | `text` required | safe outcome category |
| `correlation_id` | `text` nullable | request correlation only, never token/credential material |
| `occurred_at` | `timestamptz` required | event time |
| `expires_at` | `timestamptz` required | 30-day retention deadline |
Index `expires_at` supports bounded cleanup; `(merchant_user_id, occurred_at DESC)` supports tenant-safe investigation. Raw IP addresses, user agents, usernames attempted, passwords, hashes, and token values are prohibited.
#### `analytics_access_audit_events`
| Column | Type/nullability | Constraint or meaning |
|---|---|---|
| `id` | `bigint` required | identity primary key |
| `merchant_user_id` | `bigint` required | restrictive FK to `merchant_users` |
| `merchant_id` | `bigint` required | part of composite ownership FK with `merchant_user_id` |
| `event_type` | `text` required | approved analytics/detail/export action code |
| `result` | `text` required | stable completed/rejected/aborted outcome code |
| `period_id` | `text` nullable | approved canonical period identifier |
| `view_type` | `text` nullable | approved detail/export view code |
| `filter_fingerprint` | `bytea` nullable | 32-byte digest of canonical filters; never raw filter/cursor data |
| `row_count` | `bigint` nullable | non-negative completed export count only |
| `byte_count` | `bigint` nullable | non-negative completed export byte count only |
| `request_id` | `text` nullable | bounded safe correlation identifier |
| `occurred_at` | `timestamptz` required | action time |
| `expires_at` | `timestamptz` required | 90-day retention deadline |
The ownership FK guarantees that the user belongs to the recorded merchant. Checks enforce a 32-byte fingerprint when present and non-negative row/byte counts. Index `expires_at` supports automatic retention cleanup; `(merchant_id, occurred_at DESC)` and `(merchant_user_id, occurred_at DESC)` support operator-only investigation. Exported values, payer tokens, raw cursors/filters, credentials, user-controlled filenames, and peer data are prohibited. This table has no merchant-facing API or UI.
### Payment facts ### Payment facts
| Table | Grain and important fields | Key constraints/index direction | #### `payment_sessions`
| Column | Type/nullability | Constraint or meaning |
|---|---|---| |---|---|---|
| `payment_sessions` | One source session: merchant, terminal, amount and merchant fee in Rial, source status, created/expires instants, import provenance | unique source session key; FKs; merchant/time and terminal/time indexes | | `id` | `bigint` required | identity primary key |
| `payment_attempts` | One real try: session, positive sequence, attempt status, PSP/bank, PSP-scoped response, merchant-scoped payer token, verification type, timings, attempt/verified/settled instants | `try_seq >= 1`; unique `(payment_session_id, try_seq)`; FKs; session index; measured analytical indexes later | | `source_session_key` | `text` required | globally unique/non-empty opaque source key |
| `merchant_id` | `bigint` required | restrictive FK to `merchants` |
| `terminal_id` | `bigint` required | part of composite FK `(terminal_id, merchant_id)` to `terminals` |
| `amount_rial` | `bigint` required | positive session amount |
| `fee_amount_rial` | `bigint` required | non-negative real merchant fee, stored once |
| `session_status` | `text` required | non-empty source lifecycle code |
| `created_at` | `timestamptz` required | source Tehran-local value normalized to UTC |
| `expires_at` | `timestamptz` required | source `expire_in` normalized to UTC |
| `source_origin` | `text` required | `bootstrap` or `zarinpal_live` |
| `import_run_id` | `bigint` nullable | restrictive FK to `import_runs` |
A source row with `try_seq = 0` and `try_status = NoAttempt` materializes only `payment_sessions`; it does not create a fake attempt. Therefore a session has zero or more attempts, and zero-attempt sessions are found by the absence of related attempt rows. The origin check requires an import run for `bootstrap` and forbids one for `zarinpal_live`. Timestamp ordering is not enforced because source ordering anomalies are preserved as evidence rather than rewritten; expiry-before-creation, if found during exact validation, is reported according to the import specification.
Source `adjusted_fee` is a real merchant-fee concept for the product and is renamed `fee_amount_rial`. Profiling found the value identical across every repeated row of a session (150,450 repeated rows, zero conflicts), so it is stored once on `payment_sessions` as positive `bigint` Rial and never summed at attempt grain. Approved indexes:
Both fact tables distinguish `bootstrap` from `zarinpal_live` origin. Bootstrap rows reference an import run; live rows do not invent one. Payment facts are append-only for the MVP; inserts advance database-managed analytics revisions. - unique `source_session_key`
- `(merchant_id, created_at DESC, id DESC)` for period lists and keyset export
- `(terminal_id, created_at DESC, id DESC)` for terminal drill-down
- `(merchant_id, session_status, created_at DESC, id DESC)` for paid/unverified/failed detail views
### Analytics invalidation state #### `payment_attempts`
| Table | Grain | Purpose | | Column | Type/nullability | Constraint or meaning |
|---|---|---| |---|---|---|
| `merchant_analytics_revisions` | one row per merchant | monotonically increasing source revision advanced by related fact inserts | | `id` | `bigint` required | identity primary key |
| `cohort_analytics_revisions` | one row per benchmark cohort/category | monotonically increasing peer-source revision advanced by relevant inserts | | `payment_session_id` | `bigint` required | restrictive FK to `payment_sessions` |
| `try_seq` | `smallint` required | positive source sequence |
| `try_status` | `text` required | non-empty source lifecycle code; never `NoAttempt` |
| `psp_id` | `bigint` required | restrictive FK to `psps` |
| `switch_response_code` | `text` nullable | PSP-scoped raw code |
| `issuer_bank_id` | `bigint` nullable | restrictive FK to `issuer_banks` |
| `payer_card_key` | `text` nullable | unchanged restricted opaque token |
| `init_time_ms` | `integer` nullable | non-negative API duration |
| `verify_time_ms` | `integer` nullable | non-negative API duration |
| `try_created_at` | `timestamptz` required | source attempt creation normalized to UTC |
| `verified_at` | `timestamptz` nullable | preserved source instant |
| `settled_at` | `timestamptz` nullable | preserved source instant |
No calculated dashboard result is stored in PostgreSQL. Analytics queries and Redis value contracts are finalized by the metric catalogue and API spec. Unique `(payment_session_id, try_seq)` is the attempt identity and lookup index. A check requires `issuer_bank_id` and `payer_card_key` to be either both null or both populated, matching the authoritative data guide and observed missingness. No timestamp-order check is used because documented anomalies are retained. No standalone payer-token, bank, PSP, or status index is added until an approved measured query justifies it.
Inserts atomically increment affected revision rows through schema-owned database logic so direct upstream writes cannot bypass invalidation. Redis keys include merchant/cohort revisions, metric-definition version, and analytics-section id. Old revision keys expire after 30 days; invalidation does not scan or delete arbitrary keys. Redis values follow typed, versioned response schemas and contain aggregates only. ### Analytics invalidation
## Initial index policy #### `merchant_analytics_revisions`
- Unique indexes implement source identities and natural relationship constraints. `merchant_id bigint` PK/FK to `merchants`; `source_revision bigint` required default `0` and non-negative; `updated_at timestamptz` required.
- `payment_sessions (merchant_id, created_at DESC)` supports merchant period queries.
- `payment_sessions (terminal_id, created_at DESC)` supports terminal drill-down.
- `payment_attempts (payment_session_id, try_seq)` is unique and supports session traceability.
- Additional status, PSP, bank, payer-token, or timestamp indexes require an approved API query and `EXPLAIN (ANALYZE, BUFFERS)` evidence; do not index every CSV column.
## Deletion and mutation policy #### `cohort_analytics_revisions`
- Payment/import provenance uses restrictive foreign keys; no cascading deletion of analytical facts from merchant/profile operations. `category_id bigint` PK/FK to `merchant_categories`; `source_revision bigint` required default `0` and non-negative; `updated_at timestamptz` required.
- Imported source facts are immutable after a completed import. Corrections create a new versioned recovery/import decision.
- Production payment facts are append-only in the MVP. Update/delete semantics require a future ADR and corresponding cache-invalidation rules.
- Synthetic display profiles and credentials can change independently from source facts.
- Redis analytics entries are disposable and rebuilt on a cache miss. Source/cohort and metric-definition revisions prevent stale keys from appearing current.
## Production population boundary Revision values are cache namespace markers, not transaction counts. They may advance more than once in one business operation without changing correctness.
After the one-time CSV bootstrap, ZarinPal—not this project—writes future production payment records directly to the normalized PostgreSQL session/attempt tables. This project owns the schema/migration, insert-triggered analytics-revision, and read contract but not ZarinPal's write workflow. Compatibility, required fields, source origin, and append-only insertion semantics are versioned interfaces; recurring CSV ETL is prohibited. ## Database-managed revision contract
Direct ZarinPal inserts may bypass Go, so invalidation cannot rely only on application callbacks. Versioned PostgreSQL statement-level `AFTER INSERT` triggers with transition tables update each affected merchant and category once per insert statement:
- session insert: use merchant/category relationships from the inserted sessions;
- attempt insert: join inserted attempts to their sessions, merchants, and categories;
- update/delete triggers do not exist in MVP because payment facts are append-only.
This is deliberate schema-owned logic, named in migrations and covered by integration tests. Bulk promotion remains efficient because a multi-row `INSERT ... SELECT` advances each affected revision once, not once per imported row. Trigger failure aborts the fact insert; analytics can never observe a new fact under an old revision.
## Staging boundary
Importer staging tables are run-scoped, migration-owned technical structures rather than durable domain entities. They mirror raw columns plus `import_run_id`, `source_row_number`, and parse/validation metadata. They have no application Ent entity, no long-term API, and are truncated/dropped only by the explicit recovery lifecycle after the durable import report is complete. Promotion targets only the tables above.
## Deletion, mutation, and retention
- Imported and live payment facts are append-only in MVP. Corrections require a new specification/ADR and matching invalidation behavior.
- Merchant/category/terminal/import provenance referenced by facts cannot be deleted. Synthetic display names and password hashes may be updated through approved identity workflows.
- Session rows are revoked/expired, not hard-deleted during their security lifetime. Authentication audit events expire after 30 days; analytics-access and CSV-export audit events expire after 90 days. Cleanup is allowed only after `expires_at` and must not remove payment/import facts.
- Redis analytics entries are disposable and expire after 30 days; no calculated analytics table exists in PostgreSQL.
## Migration and verification requirements
- Atlas-generated versioned SQL is reviewed and committed; production startup never auto-migrates.
- Migration tests create the schema from zero and upgrade from every supported release state.
- Constraint tests cover terminal ownership, user/audit tenant ownership, no-attempt exclusion, attempt uniqueness, bootstrap/live provenance, paired card/bank nullability, session token digest length, audit fingerprint length, and revision-trigger atomicity.
- Query-plan verification uses production-shaped volumes and `EXPLAIN (ANALYZE, BUFFERS)` for approved analytics, detail pagination, CSV export, and session-authentication paths before adding further indexes.
- Initial fact tables are not partitioned. Partitioning requires measured operational need and a separate ADR.
## Acceptance criteria
- The physical schema represents all 2,062,839 sessions and exactly 1,949,353 real attempts without a fake `NoAttempt` attempt.
- Every session's terminal is provably owned by the same merchant through a database foreign key.
- Every source session key and `(session, try_seq)` pair is unique after promotion.
- Session-level amount, fee, merchant, terminal, status, creation, expiry, origin, and provenance are stored once.
- `verify_type` is stored once per terminal and reconciles to every source record for that terminal.
- Payer tokens remain unchanged and restricted; they are absent from uniqueness constraints, logs, exports, peer results, and AI inputs.
- Direct session/attempt inserts atomically select new merchant/cohort revision namespaces.
- Approved period, authentication, drill-down, and export access paths have an explicit initial index or a documented measure-first rule.
- Schema creation, FK/check constraints, trigger behavior, 30/90-day retention cleanup, and query plans are integration-tested before implementation approval can be considered complete.
## Boundaries
- Always: preserve source keys and nulls, enforce tenant ownership with FKs, store money at session grain, use UTC instants, version migrations, and test revision changes atomically.
- Ask first: add a table/column/index, denormalize tenant/category data, introduce partitioning, classify a new lifecycle status, or permit fact mutation/deletion.
- Never: create one wide CSV table, create a fake no-attempt row, duplicate session money on attempts, trust file order, store raw session tokens, or hide production migrations in startup.
## Decision batch for approval
1. Store terminal-invariant `verify_type` on `terminals`, not every payment attempt.
2. Preserve future non-empty lifecycle status values as text and classify them through the metric catalogue instead of using rigid PostgreSQL enums.
3. Use explicit statement-level insert triggers for merchant/cohort analytics revisions because direct ZarinPal writes can bypass Go.
4. Persist minimal 30-day authentication audit events and 90-day analytics/export audit events because the approved Identity and API specs require queryable operator evidence without raw payloads, cursors, credentials, or sensitive data.
5. Approve the table, constraint, index, and restrictive-deletion contract above as the migration target.