diff --git a/AGENTS.md b/AGENTS.md index 78bbf10..542aab3 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -11,7 +11,7 @@ For every Go-related task, always invoke and follow the `golang-how-to` skill, t - Use the canonical domain terms in `CONTEXT.md`; update that glossary when a term's meaning is approved, without adding implementation details to it. - The backend is the authority for authentication, authorization, tenant isolation, metric definitions, and data provenance. - Every merchant-scoped query derives merchant identity from the authenticated server-side session; never trust a client-supplied merchant identifier as authorization. -- Cross-merchant results are anonymized aggregates only. Exclude the active merchant and suppress the benchmark unless at least 10 other merchants and 1,000 combined peer sessions contribute to the approved cohort/window. +- Cross-merchant results are anonymized same-category aggregates only. Exclude the active merchant and suppress the benchmark unless at least 10 other merchants and 1,000 combined peer sessions contribute in the selected period. Use one value per peer merchant to calculate the cohort median and direction-aware performance percentile; never transaction-weight by default. - Treat `payer_card_key` as an opaque, merchant-scoped source token with unverified upstream semantics. Retain it unchanged only for internal deterministic analytics; never expose it through merchant APIs, logs, exports, or AI inputs, and never describe it as a card number or verified person identifier. - Use integer minor currency units or an explicitly documented fixed-precision representation; never floating point for money. - Interpret timezone-naive challenge timestamps as `Asia/Tehran`, preserve their source provenance, and normalize canonical database/API timestamps to UTC `timestamptz`/ISO 8601. Any change to this assumption requires a new import version and ADR. diff --git a/CONTEXT.md b/CONTEXT.md index fcd66d7..d8662a6 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -70,4 +70,12 @@ One independently calculable and deliverable dashboard part. The MVP section ids ## Peer Cohort -An approved group of other merchants used for anonymized comparison. The active merchant is excluded, and a benchmark is suppressed unless at least 10 peers and 1,000 peer sessions contribute. +The other eligible merchants in the active merchant's category and selected reporting period, used for anonymized comparison. The active merchant is excluded, and a benchmark is suppressed unless at least 10 peers and 1,000 combined peer sessions contribute. + +## Merchant-Weighted Cohort Median + +The median of one independently calculated metric value per eligible peer merchant. Every merchant contributes one value regardless of transaction volume, so the result represents a typical peer rather than primarily representing the largest merchants. + +## Peer Performance Percentile + +The active merchant's position among eligible same-category peers for one metric and reporting period. A higher displayed percentile always means better performance; metrics where lower values are better are direction-adjusted before ranking. diff --git a/docs/DATA-FOUNDATION-SPEC.md b/docs/DATA-FOUNDATION-SPEC.md index 0bf564a..d4d69c3 100644 --- a/docs/DATA-FOUNDATION-SPEC.md +++ b/docs/DATA-FOUNDATION-SPEC.md @@ -196,6 +196,9 @@ This contract applies only to the one-time challenge-data bootstrap and explicit - Redis loss produces a cold-cache calculation rather than loss of source data. - Reporting-date rows covering the dataset range reconcile deterministically between Gregorian and Jalali day, week, month, and season attributes. - Every reporting week starts Saturday at `00:00:00` and ends Friday at `23:59:59.999…` in `Asia/Tehran`; API ranges use an equivalent half-open `[start, next_start)` interval. +- Peer comparisons use only other eligible merchants in the active merchant's category and the same selected reporting period. +- An available peer comparison contains the merchant-weighted cohort median and a direction-aware performance percentile. Each eligible peer contributes one metric value regardless of transaction volume; high-volume merchants cannot dominate the result. +- Peer outputs cover payment success, verification completion, retry recovery, processing-time p50/p95, and paid fee rate, and are suppressed unless at least 10 other merchants and 1,000 combined peer sessions contribute. ## Approval gates / unresolved decisions @@ -209,7 +212,7 @@ Resolved: interpret timezone-naive source timestamps as `Asia/Tehran`, normalize Resolved: retain `payer_card_key` unchanged as a restricted, merchant-scoped opaque source token for internal repeat-payer analytics. Do not HMAC it, expose it, log it, export it, or send it to AI providers. Its upstream semantics remain unverified. See ADR 0005. -Resolved: peer benchmarks exclude the active merchant and require at least 10 other merchants plus 1,000 combined peer sessions in the approved cohort/window. Otherwise the benchmark is suppressed as insufficient data; the merchant's own analytics remain available. See ADR 0006. +Resolved: peer benchmarks use other merchants in the active merchant's category for the same selected period, excluding the active merchant. Each peer contributes one independently calculated value to a merchant-weighted median, and the response includes a direction-aware merchant performance percentile. This represents a typical peer despite the dataset's extreme merchant-volume skew. At least 10 other merchants and 1,000 combined peer sessions are required; otherwise the benchmark is suppressed while the merchant's own analytics remain available. See ADR 0006. Resolved: use normalized PostgreSQL source-of-truth entities plus a reporting-date dimension, then compute versioned analytics sections lazily into Redis. Daily, weekly, Jalali-month, and Persian-season results use explicit date boundaries. Keep fact tables unpartitioned initially and introduce partitioning only from measured need. See ADR 0007. diff --git a/docs/DATA-PROFILING-REPORT.md b/docs/DATA-PROFILING-REPORT.md index b629231..bc41d51 100644 --- a/docs/DATA-PROFILING-REPORT.md +++ b/docs/DATA-PROFILING-REPORT.md @@ -67,7 +67,7 @@ No negative or zero amount/fee/timing values were observed. Source identifiers a - Each observed category id has one Persian title. - The quick scan found no merchant-to-category or terminal-to-owner conflict, but exact staging validation remains mandatory because session rows are not contiguous. -- Merchant volume is highly skewed: median merchant has 78 attempts, p95 has 8,915, and the largest has 1,138,592. Unweighted merchant benchmarks and volume-weighted transaction benchmarks answer different questions and must be named separately. +- Merchant volume is highly skewed: median merchant has 78 attempts, p95 has 8,915, and the largest has 1,138,592. The MVP therefore uses a merchant-weighted median—one metric value per eligible peer merchant—to represent a typical same-category peer. Transaction-weighted peer aggregates are excluded because the largest merchants would dominate them. - One terminal/merchant/category, amount/session, and fee/session relationship are proposed invariants, subject to exact staging validation before constraints are finalized. - All 150,450 rows repeating a previously observed session repeated the same `adjusted_fee`; zero fee conflicts were found. The normalized model therefore stores the fee once per session rather than once per attempt. diff --git a/docs/METRIC-CATALOGUE.md b/docs/METRIC-CATALOGUE.md index 56f3e36..e5b67ed 100644 --- a/docs/METRIC-CATALOGUE.md +++ b/docs/METRIC-CATALOGUE.md @@ -1,7 +1,7 @@ # Analytics Metric Catalogue -- Status: Draft — listed definitions approved; remaining peer weighting is open -- Version: 0.2.0 +- Status: Draft — listed definitions approved +- Version: 0.3.0 - Last updated: 2026-08-20 Every merchant-facing number identifies its grain, formula, source fields, filters, Tehran-local period, unit, missing-data behavior, trace path, and definition version. @@ -84,6 +84,13 @@ Every merchant-facing number identifies its grain, formula, source fields, filte - `Paid` and `Verified` are both bank-payment successes, but only `Verified` is finalized. - `Reversed` is not in the success numerator; display it separately when non-zero. -## Open definition +## Peer comparison -- Merchant-weighted versus transaction-weighted peer comparisons +- Cohort: eligible merchants in the active merchant's category for the same selected reporting period; always exclude the active merchant. +- Availability: return a typed suppressed state unless at least 10 other merchants and 1,000 combined peer payment sessions contribute. Suppression never hides the active merchant's own metrics. +- Weighting: calculate each eligible peer merchant's metric independently, then use the median of those merchant values. Every merchant contributes one value regardless of payment volume. +- Rationale: merchant volume is extremely skewed (median 78 attempts, p95 8,915, maximum 1,138,592). A transaction-weighted average would mostly describe the largest merchants, while a merchant-weighted median represents a typical peer and resists extreme values. +- Output: expose the cohort median and the active merchant's performance percentile; never expose an individual peer's identity, value, or rank. +- Percentile direction: a higher displayed percentile always means better performance. Use the natural direction for payment success, verification completion, and retry recovery; invert the ranking direction for processing time and paid fee rate, where lower is better. +- Benchmarked metrics: `payment_success_rate`, `verification_completion_rate`, `retry_recovery_rate`, `init_time_ms` p50/p95, `verify_time_ms` p50/p95, and `paid_fee_rate`. +- Periods: calculate benchmarks for the same selected day, Saturday–Friday week, Jalali month, or Persian season as the active merchant value. Never compare unlike periods. diff --git a/docs/decisions/0006-peer-benchmark-suppression.md b/docs/decisions/0006-peer-benchmark-suppression.md index 697125e..7cddfb3 100644 --- a/docs/decisions/0006-peer-benchmark-suppression.md +++ b/docs/decisions/0006-peer-benchmark-suppression.md @@ -5,24 +5,30 @@ ## Context -The product may compare a merchant's metrics with anonymized merchants in the same approved cohort. Comparisons built from very few businesses can reveal competitive information, and comparisons built from too few payment sessions are unstable. The five source categories contain 25, 38, 53, 69, and 158 merchants over the full dataset, so a useful minimum can be enforced without disabling category benchmarks globally. +The product may compare a merchant's metrics with anonymized merchants in the same category. Comparisons built from very few businesses can reveal competitive information, and comparisons built from too few payment sessions are unstable. The five source categories contain 25, 38, 53, 69, and 158 merchants over the full dataset, so a useful minimum can be enforced without disabling category benchmarks globally. + +Merchant payment volume is extremely skewed: the median merchant has 78 attempts, p95 has 8,915, and the largest has 1,138,592. A transaction-weighted aggregate would therefore be dominated by the largest merchants and would not describe a typical peer. ## Decision -Exclude the active merchant from every peer aggregate. Display a peer benchmark only when its selected cohort and reporting window contain both: +Use other eligible merchants in the active merchant's category as the peer cohort and exclude the active merchant from every peer aggregate. Display a peer benchmark only when its selected reporting window contains both: - at least 10 other eligible merchants; and - at least 1,000 combined peer payment sessions. If either condition fails, return a typed suppressed result rather than a numeric value. The UI explains that there is not enough data for a reliable comparison. Suppression applies only to peer comparisons; authenticated merchants retain access to their own permitted analytics. -Peer cohorts and reporting windows come from an approved allowlist. Merchant-facing clients cannot construct arbitrary cross-merchant filters that could isolate a competitor or recover values by differencing nearby queries. Individual peer identities, values, rows, and ranks are never returned. +For every approved metric, calculate one value per eligible peer merchant and use the median of those merchant values. Every merchant therefore has equal weight regardless of transaction volume. Return that cohort median plus the active merchant's performance percentile. A higher displayed percentile always means better performance, so ranking is direction-adjusted for processing time and fee rate, where lower values are better. + +Apply the comparison to payment success rate, verification completion rate, retry recovery rate, processing-time p50/p95, and paid fee rate for the same selected daily, Saturday–Friday weekly, Jalali-month, or Persian-season period. + +Categories and reporting windows come from an approved allowlist. Merchant-facing clients cannot construct arbitrary cross-merchant filters that could isolate a competitor or recover values by differencing nearby queries. Individual peer identities, values, rows, and ranks are never returned. ## Consequences - Benchmarks are less likely to expose a small set of competitors or amplify tiny samples. +- The benchmark represents a typical merchant in the category instead of allowing high-volume merchants to dominate it. - Narrow date windows may suppress otherwise valid category comparisons; the UI may suggest a broader approved period. - API contracts distinguish available, suppressed, and unavailable/error states. -- Tests cover both thresholds, exclusion of the active merchant, fixed cohort definitions, and attempts to vary filters around the threshold. +- Tests cover both thresholds, same-category membership, exclusion of the active merchant, equal per-merchant weighting, percentile direction, fixed period definitions, and attempts to vary filters around the threshold. - The thresholds are product policy, not a formal anonymity guarantee; changing them requires a reviewed ADR and security/privacy assessment. -