docs: adopt CTO dataset semantics

This commit is contained in:
2026-08-20 17:29:26 +03:30
parent 5ef650dcf6
commit a70fc15517
9 changed files with 70 additions and 23 deletions

View File

@@ -5,23 +5,22 @@
## Context
The dataset does not document the semantics or generation method of `payer_card_key`. Profiling found 1,034,368 populated rows and 402,173 distinct values. Values are opaque alphanumeric tokens 6–11 characters long. They repeat within a merchant, enabling repeat-payer analysis, but no observed token appears under multiple merchants. Approximately 53% of rows are missing the value.
Initial profiling found 1,034,368 populated `payer_card_key` rows and 402,173 distinct values. Values are opaque alphanumeric tokens 6–11 characters long. The later CTO data guide confirms that the token pseudonymizes a buyer card within one merchant: the same card receives a different token at another merchant. Approximately 53% of rows are missing the value because card information is not returned when the bank stage does not complete.
Applying HMAC would preserve equality while adding secret storage, rotation, and re-import operations. Because the supplied value is already opaque and empirically merchant-scoped, that extra transformation provides limited benefit for this dataset and does not make the identifier anonymous.
## Decision
Retain the supplied `payer_card_key` unchanged as a nullable, restricted internal analytical field. Treat it only as an opaque merchant-scoped token with unverified upstream semantics.
Retain the supplied `payer_card_key` unchanged as a nullable, restricted internal analytical field. Equality is valid only within the owning merchant and may support covered repeat-card analysis.
Every lookup, grouping, or uniqueness calculation involving the token must include the merchant boundary. The token must never be returned by merchant-facing APIs, written to logs or traces, included in exports, rendered in the frontend, or sent to AvalAI/another third party. It must not be described as a card number, account number, or verified unique person.
Metrics derived from it must disclose source coverage and use language such as “شناسه پرداخت‌کننده در داده‌های موجود,” not claims about verified customers. A production source contract or evidence of reversibility/cross-merchant linkability triggers a new security review and ADR.
Metrics derived from it must disclose source coverage and use language such as “کارت‌های بازگشتی در داده‌های موجود,” not claims about verified customers or people. Evidence of reversibility or cross-merchant linkability triggers a new security review and ADR.
## Consequences
- Repeat-token frequency, retry, success, and approximate returning-payer analyses remain possible.
- Repeat-card-token frequency, retry, and success analyses are valid within one merchant for populated values.
- No new pseudonymization secret or rotation workflow is required for the hackathon dataset.
- Database access remains more privileged than dashboard/API access because the token is still linkable behavioral data.
- Missing coverage prevents treating distinct-token counts as complete payer counts.
- Tests must verify tenant scoping and assert that DTOs, logs, exports, and AI payloads cannot contain the field.

View File

@@ -7,7 +7,7 @@
The source exposes session states `Paid`, `Verified`, `Failed`, and `Reversed`, but names alone did not establish business meaning. Authoritative clarification from ZarinPal's CTO defines a two-step lifecycle: the buyer first completes payment at the bank, then the merchant calls verification. `Paid` means the bank payment succeeded and funds were deducted but merchant verification is incomplete. `Verified` means both steps completed. `Reversed` is possible but extremely rare; the dataset contains one such session.
The challenge field `adjusted_fee` was transformed/name-adjusted for participant confidentiality, but the product domain treats it as the merchant's real fee. Profiling found its value constant within every session: 150,450 repeated attempt rows and zero fee conflicts.
The CTO data guide states that challenge `adjusted_fee` is uniformly scaled for commercial confidentiality and is not the actual ZarinPal tariff; relative ordering, trends, and fee share remain valid. The approved product contract nevertheless treats the canonical production field as the merchant's real fee because accepted production data will supply the real value. Profiling found the challenge value constant within every session: 150,450 repeated rows and zero conflicts.
## Decision
@@ -24,7 +24,7 @@ Paid amount is the session amount summed once for (`Paid`, `Verified`). Also exp
A retried session has more than one attempt. An eventually paid retry has final session status `Paid` or `Verified` and at least one lower-`try_seq` attempt whose status is neither `Paid` nor `Verified`; CSV row order is irrelevant.
Rename source `adjusted_fee` to canonical `fee_amount_rial`, store it once on the session, and present it as a real Rial merchant fee. Paid fee totals sum it once for bank-paid sessions. Fee rate is `sum(paid session fee) / sum(paid session amount)` with numerator, denominator, period, and unit visible.
Map challenge `adjusted_fee` into canonical `fee_amount_rial` for the demo and require production writers to provide the real Rial fee under that canonical contract. Store it once on the session and present the canonical product value as the real merchant fee. Paid fee totals sum it once for bank-paid sessions. Fee rate is `sum(paid session fee) / sum(paid session amount)` with numerator, denominator, period, and unit visible.
## Consequences