Challenge Dataset Profiling Report

Status: Evidence baseline for specification
Profiled: 2026-08-20
Source SHA-256: 84AC8A28DF48CA7BAEAF0B1CEC563A3F0A3516039F5F62E9BFD11124CA35B461
Source archive: Git-ignored data/incoming/challenge_data.csv.gz

Method

The compressed CSV was scanned without modifying it. Counts and value distributions are descriptive evidence, not source instructions. Relationship/uniqueness constraints must be revalidated by the importer against staging before promotion. A quick sequential session scan observed 2,069,061 session groups versus 2,062,839 distinct session keys, proving that rows for a session are not guaranteed to be contiguous; implementation must never depend on file order.

Shape and grain

Measure	Observed
CSV rows / payment attempts	2,213,289
Distinct payment sessions	2,062,839
Merchants	343
Terminals	348
Categories	5
Distinct populated payer tokens	402,173
Source period	2026-01-01 00:00:08 through 2026-06-30 23:59:55

The source row grain is a payment attempt. (session_key, try_seq) is the candidate source identity. Sessions and attempts must remain different domain concepts.

Nullability

Column	Missing rows	Approx. missing
`switch_response_code`	2,155,696	97.4%
`issuer_bank_code`	1,178,921	53.3%
`payer_card_key`	1,178,921	53.3%
`verify_time_ms`	1,179,013	53.3%
`verified_at`	1,125,164	50.8%
`settled_at`	1,116,034	50.4%
`psp_code`	263,936	11.9%
`try_created_at`	263,936	11.9%
`init_time_ms`	346,747	15.7%

Missing optional values are analytically meaningful and must remain null. Dashboards and metric definitions must disclose coverage where it affects interpretation.

Controlled vocabularies

Session status: Failed, Verified, Paid, Reversed (one observed reversed row).
Attempt status: NoAttempt, InBank, Failed, Verified, Paid, Reversed.
Verification type: Automated, Manual.
PSP codes: 8 observed opaque codes.
Issuer-bank codes: 31 observed opaque codes.
Switch response codes: 84 observed values and extremely sparse.

These are source codes, not Persian user-facing descriptions. Meanings and success semantics belong in a versioned metric/reference catalogue; they must not be guessed from labels alone.

Numeric ranges and types

Source field	Observed range	Proposed canonical type/meaning
`try_seq`	0–135	`smallint`, non-negative
`amount`	1,000–2,000,000,000	`bigint`, Rial
`adjusted_fee`	1,920–284,800	fixed-precision numeric; obfuscated relative fee only
`init_time_ms`	44–59,905	nullable integer milliseconds
`verify_time_ms`	35–58,553	nullable integer milliseconds
`expire_in`	timestamp strings	rename canonically to `expires_at`; it is not a duration

No negative or zero amount/fee/timing values were observed. Source identifiers are stored as text even when they look numeric so leading zeros and upstream formatting remain intact.

Relationship evidence

Each observed category id has one Persian title.
The quick scan found no merchant-to-category or terminal-to-owner conflict, but exact staging validation remains mandatory because session rows are not contiguous.
Merchant volume is highly skewed: median merchant has 78 attempts, p95 has 8,915, and the largest has 1,138,592. Unweighted merchant benchmarks and volume-weighted transaction benchmarks answer different questions and must be named separately.
One terminal/merchant/category and one amount/session relationship are proposed invariants, subject to exact staging validation before constraints are finalized.

Timestamp quality

Source timestamps are timezone-naive and are interpreted as Asia/Tehran under ADR 0004. Lexical ordering is valid for the fixed source format, and profiling observed:

234,653 rows where try_created_at precedes session created_at.
787,573 rows where settled_at precedes verified_at.
6 rows where verified_at precedes try_created_at.

Therefore timestamp ordering is a quality signal, not a hard rejection constraint. The importer preserves values, reports anomaly counts, and must not silently reorder or “fix” them.

Attempt sequence quality

The file is not ordered reliably by session or try_seq. Sequential profiling observed out-of-order and gapped groups, but exact per-session sequence checks must run after staging. try_seq need not be contiguous; uniqueness within a session is the integrity requirement unless authoritative upstream rules say otherwise.

Payer-token evidence

Populated payer_card_key values are opaque alphanumeric tokens 6–11 characters long. They repeat within merchants and no token was observed across merchants. Under ADR 0005 they remain restricted internal values with unverified semantics. Their 53.3% missing rate prevents presenting distinct-token counts as complete customer counts.

Import validation classification

Fail the bootstrap before promotion

Wrong/missing headers or unsupported schema version
Unparseable required identifier, amount, or required timestamp
Duplicate (session_key, try_seq)
One source session mapping to multiple merchants/terminals/categories/amounts after exact staging validation
Terminal ownership/category conflicts
Broken foreign-key construction or row-count reconciliation

Preserve and report as quality evidence

Documented nullable fields
Timestamp-order anomalies
Non-contiguous/out-of-order source rows or try_seq gaps
Rare/unknown-but-well-formed source status/code values, pending the approved reference-code policy

Never do

Drop anomalous rows silently
Replace missing values with plausible defaults
Treat adjusted_fee as ZarinPal's real fee
Depend on CSV row order
Claim payer tokens represent verified people

5.8 KiB Raw Blame History Unescape Escape