3.1 KiB
ADR 0006: Suppress under-sized peer benchmarks
- Status: Accepted
- Date: 2026-08-20
Context
The product may compare a merchant's metrics with anonymized merchants in the same category. Comparisons built from very few businesses can reveal competitive information, and comparisons built from too few payment sessions are unstable. The five source categories contain 25, 38, 53, 69, and 158 merchants over the full dataset, so a useful minimum can be enforced without disabling category benchmarks globally.
Merchant payment volume is extremely skewed: the median merchant has 78 attempts, p95 has 8,915, and the largest has 1,138,592. A transaction-weighted aggregate would therefore be dominated by the largest merchants and would not describe a typical peer.
Decision
Use other eligible merchants in the active merchant's category as the peer cohort and exclude the active merchant from every peer aggregate. Display a peer benchmark only when its selected reporting window contains both:
- at least 10 other eligible merchants; and
- at least 1,000 combined peer payment sessions.
If either condition fails, return a typed suppressed result rather than a numeric value. The UI explains that there is not enough data for a reliable comparison. Suppression applies only to peer comparisons; authenticated merchants retain access to their own permitted analytics.
For every approved metric, calculate one value per eligible peer merchant and use the median of those merchant values. Every merchant therefore has equal weight regardless of transaction volume. Return that cohort median plus the active merchant's performance percentile. A higher displayed percentile always means better performance, so ranking is direction-adjusted for processing time and fee rate, where lower values are better.
Apply the comparison to payment success rate, verification completion rate, retry recovery rate, processing-time p50/p95, and paid fee rate for the same selected daily, Saturday–Friday weekly, Jalali-month, or Persian-season period.
Categories and reporting windows come from an approved allowlist. Merchant-facing clients cannot construct arbitrary cross-merchant filters that could isolate a competitor or recover values by differencing nearby queries. Individual peer identities, values, rows, and ranks are never returned.
Consequences
- Benchmarks are less likely to expose a small set of competitors or amplify tiny samples.
- The benchmark represents a typical merchant in the category instead of allowing high-volume merchants to dominate it.
- Narrow date windows may suppress otherwise valid category comparisons; the UI may suggest a broader approved period.
- API contracts distinguish available, suppressed, and unavailable/error states.
- Tests cover both thresholds, same-category membership, exclusion of the active merchant, equal per-merchant weighting, percentile direction, fixed period definitions, and attempts to vary filters around the threshold.
- The thresholds are product policy, not a formal anonymity guarantee; changing them requires a reviewed ADR and security/privacy assessment.