docs: correct payment and fee semantics

docs/DATA-FOUNDATION-SPEC.md

@@ -75,6 +75,7 @@ erDiagram
       bigint merchant_id FK
       bigint terminal_id FK
       bigint amount
+      bigint fee_amount_rial
       text session_status
       timestamptz created_at
       bigint import_run_id FK
@@ -89,7 +90,6 @@ erDiagram
       text issuer_bank_code
       text payer_card_key
       text verify_type
-      bigint adjusted_fee
       int init_time_ms
       int verify_time_ms
       timestamptz try_created_at
@@ -184,6 +184,7 @@ This contract applies only to the one-time challenge-data bootstrap and explicit
 - Daily, weekly, Jalali-month, and Persian-season analytics reconcile to deterministic source queries for additive metrics.
 - Rates are recomputed from their numerator and denominator over the requested period rather than averaged from daily percentages.
 - Non-additive metrics such as unique payer-token counts are never produced by summing daily distinct counts.
+- Source `adjusted_fee` is normalized once per session as `fee_amount_rial`; retry attempts never multiply fee totals.
 - First access idempotently creates/returns one run, starts all four core analytics sections in bounded parallel, and delivers each through authenticated SSE as soon as it is ready.
 - A second equivalent request reads current 30-day Redis entries without repeating source-fact aggregation.
 - Inserting a related payment fact advances only the affected merchant revision and any affected cohort revision; the next relevant request recomputes once and publishes atomically.
@@ -218,6 +219,6 @@ Resolved: holiday and named-event reporting are removed from scope. MVP reportin
 
 Resolved: reporting weeks follow the Iranian week: Saturday through Friday in `Asia/Tehran`, represented as half-open API/database ranges. See ADR 0012.
 
-Resolved: only source session status `Verified` is a definitely successful payment. Keep `Paid` separate until an authoritative ZarinPal definition changes the metric catalogue. See `METRIC-CATALOGUE.md`.
+Resolved: authoritative ZarinPal lifecycle guidance defines both `Paid` and `Verified` as successful bank payments. `Paid` means funds were deducted but merchant verification is incomplete; `Verified` is finalized. Success/paid-volume/recovery metrics include both while verification metrics keep them distinct. Source `adjusted_fee` is treated as the real merchant fee, normalized once per session, and shown in Rial. See ADR 0013 and `METRIC-CATALOGUE.md`.
 
 No broad architecture approval gates remain for this capability. Physical columns, exact SSE event schemas, retryable-error classification, analytics-section response contracts, observability targets, and migration details still require acceptance in the implementation-ready specs.

docs/DATA-PROFILING-REPORT.md

@@ -56,7 +56,7 @@ These are source codes, not Persian user-facing descriptions. Meanings and succe
 |---|---|---|
 | `try_seq` | 0–135 | `smallint`, non-negative |
 | `amount` | 1,000–2,000,000,000 | `bigint`, Rial |
-| `adjusted_fee` | 1,920–284,800 | fixed-precision numeric; obfuscated relative fee only |
+| `adjusted_fee` | 1,920–284,800 | positive integer merchant-fee value; confidentiality-adjusted in the challenge source and normalized as real `fee_amount_rial` per ADR 0013 |
 | `init_time_ms` | 44–59,905 | nullable integer milliseconds |
 | `verify_time_ms` | 35–58,553 | nullable integer milliseconds |
 | `expire_in` | timestamp strings | rename canonically to `expires_at`; it is not a duration |
@@ -68,7 +68,8 @@ No negative or zero amount/fee/timing values were observed. Source identifiers a
 - Each observed category id has one Persian title.
 - The quick scan found no merchant-to-category or terminal-to-owner conflict, but exact staging validation remains mandatory because session rows are not contiguous.
 - Merchant volume is highly skewed: median merchant has 78 attempts, p95 has 8,915, and the largest has 1,138,592. Unweighted merchant benchmarks and volume-weighted transaction benchmarks answer different questions and must be named separately.
-- One terminal/merchant/category and one amount/session relationship are proposed invariants, subject to exact staging validation before constraints are finalized.
+- One terminal/merchant/category, amount/session, and fee/session relationship are proposed invariants, subject to exact staging validation before constraints are finalized.
+- All 150,450 rows repeating a previously observed session repeated the same `adjusted_fee`; zero fee conflicts were found. The normalized model therefore stores the fee once per session rather than once per attempt.
 
 ## Timestamp quality
 
@@ -110,7 +111,6 @@ Populated `payer_card_key` values are opaque alphanumeric tokens 6–11 characte
 
 - Drop anomalous rows silently
 - Replace missing values with plausible defaults
-- Treat `adjusted_fee` as ZarinPal's real fee
+- Sum the repeated source fee at attempt grain
 - Depend on CSV row order
 - Claim payer tokens represent verified people
-

docs/METRIC-CATALOGUE.md

@@ -1,37 +1,89 @@
 # Analytics Metric Catalogue
 
-- Status: Draft — definitions are approved individually
-- Version: 0.1.0
+- Status: Draft — listed definitions approved; remaining peer weighting is open
+- Version: 0.2.0
 - Last updated: 2026-08-20
 
-Every merchant-facing number must identify its grain, formula, source fields, filters, time basis, unit, missing-data behavior, trace path, and definition version.
+Every merchant-facing number identifies its grain, formula, source fields, filters, Tehran-local period, unit, missing-data behavior, trace path, and definition version.
 
-## Confirmed successful sessions
+## Bank-paid sessions
 
-- Canonical id: `confirmed_successful_sessions`
+- Canonical id: `bank_paid_sessions`
 - Grain: distinct payment session
-- Formula: count payment sessions where source `session_status = 'Verified'`
+- Formula: count sessions where `session_status IN ('Paid', 'Verified')`
+- Meaning: the buyer completed the bank payment and funds were deducted
 - Unit: sessions
-- Time attribution: session `created_at`, converted to `Asia/Tehran` local reporting date
-- Missing behavior: `session_status` is required; unknown values are not counted and trigger quality evidence
-- Traceability: drill down to eligible session identifiers and their attempt histories within the authenticated merchant
-- Approved decision: `Paid` is not included; it remains a separate state until ZarinPal provides an authoritative definition
+- Time attribution: session `created_at` in `Asia/Tehran`
+- Traceability: merchant-scoped session identifiers and attempt histories
 
-## Paid but not confirmed as verified
+## Payment success rate
 
-- Canonical id: `paid_unverified_sessions`
-- Grain: distinct payment session
-- Formula: count payment sessions where source `session_status = 'Paid'`
+- Canonical id: `payment_success_rate`
+- Numerator: `bank_paid_sessions`
+- Denominator: sessions where `session_status IN ('Paid', 'Verified', 'Failed', 'Reversed')`
+- Formula: numerator / denominator; undefined when denominator is zero
+- Unknown statuses: excluded from both sides and disclosed as coverage/quality evidence until classified
+- Unit: percentage
+
+## Verified sessions
+
+- Canonical id: `verified_sessions`
+- Formula: count sessions where `session_status = 'Verified'`
+- Meaning: bank payment and merchant verification are complete; the payment is finalized
 - Unit: sessions
-- Presentation: separate operational state; never label as confirmed success or failure
-- Traceability: same merchant-scoped session/attempt drill-down
 
-## Open definitions
+## Unverified paid sessions
+
+- Canonical id: `unverified_paid_sessions`
+- Formula: count sessions where `session_status = 'Paid'`
+- Meaning: bank payment succeeded and funds were deducted, but merchant verification is incomplete
+- Presentation: actionable verification gap; never label as failure
+- Unit: sessions
+
+## Verification completion rate
+
+- Canonical id: `verification_completion_rate`
+- Formula: `verified_sessions / bank_paid_sessions`; undefined when no bank-paid sessions exist
+- Unit: percentage
+
+## Paid volume
+
+- Canonical id: `bank_paid_amount_rial`
+- Formula: sum session `amount` once where `session_status IN ('Paid', 'Verified')`
+- Unit: Rial
+- Subsets: `verified_amount_rial` uses `Verified`; `unverified_paid_amount_rial` uses `Paid`
+
+## Retried sessions and eventual payment
+
+- `retried_sessions`: count sessions containing more than one payment attempt
+- `eventually_paid_after_retry_sessions`: count retried sessions whose final session status is `Paid` or `Verified` and which contain a lower-`try_seq` attempt whose status is neither `Paid` nor `Verified`
+- `retry_recovery_rate`: `eventually_paid_after_retry_sessions / retried_sessions`; undefined when no retried sessions exist
+- Grain/unit: distinct sessions
+- Traceability: ordered merchant-scoped attempt history; never depend on CSV row order
+
+## Processing time
+
+- Report median (p50) and p95 separately for `init_time_ms` and `verify_time_ms`
+- Include only non-null values for the relevant field
+- Always show contributing sample count and coverage percentage
+- Do not replace missing timing with zero
+- Unit: milliseconds, with human-readable seconds allowed as an additional presentation
+
+## Merchant fee
+
+- Canonical field: `fee_amount_rial`, sourced from CSV `adjusted_fee` and stored once per session
+- `paid_fee_amount_rial`: sum session fee once where `session_status IN ('Paid', 'Verified')`
+- `paid_fee_rate`: `paid_fee_amount_rial / bank_paid_amount_rial`; undefined when paid amount is zero
+- Meaning/presentation: real merchant fee in Rial; do not label it obfuscated or relative-only in the product
+- Retry rule: never sum the repeated source value per attempt
+
+## Common behavior
+
+- Rates are recomputed from summed numerator/denominator components for every selected day, Saturday–Friday week, Jalali month, and Persian season.
+- Zero, undefined, missing, suppressed peer value, and calculation failure remain distinct states.
+- `Paid` and `Verified` are both bank-payment successes, but only `Verified` is finalized.
+- `Reversed` is not in the success numerator; display it separately when non-zero.
+
+## Open definition
 
-- Success-rate denominator and treatment of `Failed`/`Reversed`/`Paid`
-- Amount metric inclusion by session status
-- Retry and recovery definitions
-- Timing percentile definitions and missing coverage
-- Fee-ratio terminology for obfuscated `adjusted_fee`
 - Merchant-weighted versus transaction-weighted peer comparisons
-

docs/PHYSICAL-SCHEMA-PROPOSAL.md

@@ -44,10 +44,10 @@ Synthetic profile fields carry an explicit `profile_is_synthetic` marker. Creden
 
 | Table | Grain and important fields | Key constraints/index direction |
 |---|---|---|
-| `payment_sessions` | One source session: merchant, terminal, amount in Rial, source status, created/expires instants, import provenance | unique source session key; FKs; merchant/time and terminal/time indexes |
-| `payment_attempts` | One try: session, sequence, attempt status, PSP/bank, response, opaque payer token, verification type, timings, fee, attempt/verified/settled instants | unique `(payment_session_id, try_seq)`; FKs; session index; measured analytical indexes later |
+| `payment_sessions` | One source session: merchant, terminal, amount and merchant fee in Rial, source status, created/expires instants, import provenance | unique source session key; FKs; merchant/time and terminal/time indexes |
+| `payment_attempts` | One try: session, sequence, attempt status, PSP/bank, response, opaque payer token, verification type, timings, attempt/verified/settled instants | unique `(payment_session_id, try_seq)`; FKs; session index; measured analytical indexes later |
 
-`adjusted_fee` uses fixed precision and is labeled as an obfuscated relative value. The physical precision is finalized from exact decimal profiling before migration approval.
+Source `adjusted_fee` is a real merchant-fee concept for the product and is renamed `fee_amount_rial`. Profiling found the value identical across every repeated row of a session (150,450 repeated rows, zero conflicts), so it is stored once on `payment_sessions` as positive `bigint` Rial and never summed at attempt grain.
 
 Both fact tables distinguish `bootstrap` from `zarinpal_live` origin. Bootstrap rows reference an import run; live rows do not invent one. Payment facts are append-only for the MVP; inserts advance database-managed analytics revisions.
 

docs/decisions/0013-payment-lifecycle-and-fee-semantics.md

@@ -0,0 +1,35 @@
+# ADR 0013: Model bank-paid, verified, and merchant-fee semantics
+
+- Status: Accepted
+- Date: 2026-08-20
+
+## Context
+
+The source exposes session states `Paid`, `Verified`, `Failed`, and `Reversed`, but names alone did not establish business meaning. Authoritative clarification from ZarinPal's CTO defines a two-step lifecycle: the buyer first completes payment at the bank, then the merchant calls verification. `Paid` means the bank payment succeeded and funds were deducted but merchant verification is incomplete. `Verified` means both steps completed. `Reversed` is possible but extremely rare; the dataset contains one such session.
+
+The challenge field `adjusted_fee` was transformed/name-adjusted for participant confidentiality, but the product domain treats it as the merchant's real fee. Profiling found its value constant within every session: 150,450 repeated attempt rows and zero fee conflicts.
+
+## Decision
+
+Use these canonical populations:
+
+- bank-paid: session status in (`Paid`, `Verified`);
+- verified/finalized: session status `Verified`;
+- unverified paid: session status `Paid`;
+- unsuccessful current outcome: `Failed` or `Reversed` for the MVP success-rate denominator.
+
+Define payment success rate as bank-paid sessions divided by sessions in (`Paid`, `Verified`, `Failed`, `Reversed`). Keep unknown future statuses out of numerator/denominator and report their coverage until explicitly classified. Define verification completion rate as `Verified / (Paid + Verified)`.
+
+Paid amount is the session amount summed once for (`Paid`, `Verified`). Also expose verified amount and unverified-paid amount as explicit subsets. Never call `Paid` a failure.
+
+A retried session has more than one attempt. An eventually paid retry has final session status `Paid` or `Verified` and at least one lower-`try_seq` attempt whose status is neither `Paid` nor `Verified`; CSV row order is irrelevant.
+
+Rename source `adjusted_fee` to canonical `fee_amount_rial`, store it once on the session, and present it as a real Rial merchant fee. Paid fee totals sum it once for bank-paid sessions. Fee rate is `sum(paid session fee) / sum(paid session amount)` with numerator, denominator, period, and unit visible.
+
+## Consequences
+
+- Dashboard success combines `Paid` and `Verified`, while verification health separates them.
+- The unverified-paid count/amount is an actionable merchant discrepancy, not a failed-payment metric.
+- Retry recovery may end in either `Paid` or `Verified`.
+- Fee aggregation uses session grain and cannot multiply fees by retry count.
+- Unknown statuses require a metric-catalogue change rather than being guessed into success or failure.