docs: adopt CTO dataset semantics

This commit is contained in:
2026-08-20 17:29:26 +03:30
parent 5ef650dcf6
commit a70fc15517
9 changed files with 70 additions and 23 deletions

View File

@@ -13,7 +13,9 @@ The compressed CSV was scanned without modifying it. Counts and value distributi
| Measure | Observed |
|---|---:|
| CSV rows / payment attempts | 2,213,289 |
| CSV source records | 2,213,289 |
| `NoAttempt` session markers (not normalized attempts) | 263,936 |
| Actual attempt records (`try_seq >= 1`) | 1,949,353 |
| Distinct payment sessions | 2,062,839 |
| Merchants | 343 |
| Terminals | 348 |
@@ -21,7 +23,7 @@ The compressed CSV was scanned without modifying it. Counts and value distributi
| Distinct populated payer tokens | 402,173 |
| Source period | 2026-01-01 00:00:08 through 2026-06-30 23:59:55 |
The source row grain is a payment attempt. `(session_key, try_seq)` is the candidate source identity. Sessions and attempts must remain different domain concepts.
The source record grain is an attempt-or-no-attempt session record. `(session_key, try_seq)` is the candidate source-record identity. The CTO guide defines `(try_seq = 0, try_status = NoAttempt)` as a marker for a session with no recorded attempt; those 263,936 rows create sessions but not normalized attempts. Sessions and actual attempts must remain different domain concepts.
## Nullability
@@ -54,9 +56,9 @@ These are source codes, not Persian user-facing descriptions. Meanings and succe
| Source field | Observed range | Proposed canonical type/meaning |
|---|---|---|
| `try_seq` | 0–135 | `smallint`, non-negative |
| `try_seq` | 0–135 | source `smallint`; `0` is a no-attempt marker, normalized attempts require `1–135` |
| `amount` | 1,000–2,000,000,000 | `bigint`, Rial |
| `adjusted_fee` | 1,920–284,800 | positive integer merchant-fee value; confidentiality-adjusted in the challenge source and normalized as real `fee_amount_rial` per ADR 0013 |
| `adjusted_fee` | 1,920–284,800 | positive integer challenge stand-in, uniformly confidentiality-scaled; mapped to the product's canonical real-fee field `fee_amount_rial` per ADR 0013 |
| `init_time_ms` | 44–59,905 | nullable integer milliseconds |
| `verify_time_ms` | 35–58,553 | nullable integer milliseconds |
| `expire_in` | timestamp strings | rename canonically to `expires_at`; it is not a duration |
@@ -70,6 +72,7 @@ No negative or zero amount/fee/timing values were observed. Source identifiers a
- Merchant volume is highly skewed: median merchant has 78 attempts, p95 has 8,915, and the largest has 1,138,592. The MVP therefore uses a merchant-weighted median—one metric value per eligible peer merchant—to represent a typical same-category peer. Transaction-weighted peer aggregates are excluded because the largest merchants would dominate them.
- One terminal/merchant/category, amount/session, and fee/session relationship are proposed invariants, subject to exact staging validation before constraints are finalized.
- All 150,450 rows repeating a previously observed session repeated the same `adjusted_fee`; zero fee conflicts were found. The normalized model therefore stores the fee once per session rather than once per attempt.
- The CTO guide defines `psp_code` at attempt grain and namespaces `switch_response_code` by PSP; identical raw response codes from different PSPs are not semantically interchangeable.
## Timestamp quality
@@ -87,7 +90,7 @@ The file is not ordered reliably by session or `try_seq`. Sequential profiling o
## Payer-token evidence
Populated `payer_card_key` values are opaque alphanumeric tokens 6–11 characters long. They repeat within merchants and no token was observed across merchants. Under ADR 0005 they remain restricted internal values with unverified semantics. Their 53.3% missing rate prevents presenting distinct-token counts as complete customer counts.
Populated `payer_card_key` values are opaque alphanumeric tokens 6–11 characters long. The CTO guide confirms that equality is meaningful only within a merchant: the same card receives a different token at another merchant. Under ADR 0005 they remain restricted internal values. Their 53.3% missing rate prevents presenting distinct-token counts as complete customer counts, and they never establish a verified person.
## Import validation classification
@@ -95,7 +98,8 @@ Populated `payer_card_key` values are opaque alphanumeric tokens 6–11 characte
- Wrong/missing headers or unsupported schema version
- Unparseable required identifier, amount, or required timestamp
- Duplicate `(session_key, try_seq)`
- Duplicate source `(session_key, try_seq)`
- A `try_seq = 0` row whose status is not `NoAttempt`, or a `NoAttempt` row whose sequence is not zero
- One source session mapping to multiple merchants/terminals/categories/amounts after exact staging validation
- Terminal ownership/category conflicts
- Broken foreign-key construction or row-count reconciliation
@@ -113,4 +117,5 @@ Populated `payer_card_key` values are opaque alphanumeric tokens 6–11 characte
- Replace missing values with plausible defaults
- Sum the repeated source fee at attempt grain
- Depend on CSV row order
- Insert a `NoAttempt` marker as a normalized payment attempt
- Claim payer tokens represent verified people