# Implementation Plan: ZarinPal Merchant Analytics Backend - Status: Approved — Phase 4 implementation authorized 2026-08-20 - Capabilities: Data foundation, demo identity, analytics engine, and Merchant API - Approved contracts: physical schema, import workflow/validation, Identity, Analytics, metric catalogue `0.4.0`, API, and OpenAPI `1.0.0` - Task list: `tasks/todo.md` - Last updated: 2026-08-20 ## Overview Build the complete Go capability-oriented modular monolith needed by the dashboard: reviewed PostgreSQL migrations and normalized entities, the guarded one-time importer, demo merchant identity, deterministic analytics queries, Redis/Asynq orchestration, and the approved `/api/v1` REST/SSE/detail/CSV adapter. Deployment configuration remains owned by the deployment repository. ## Preconditions and boundaries - The approved Identity, Data Foundation, physical schema, importer, Analytics Engine, API, ADR, metric, and data-guide decisions remain authoritative. - Architecture and dependency injection are already approved: capability-oriented modular monolith with manual constructor wiring in thin `cmd/` composition roots. - The user approved this plan and task list and authorized implementation on 2026-08-20. - Implementation proceeds through `tasks/todo.md` in dependency order, starting with Task F1. - The API work depends on implementation-ready provider interfaces from Identity and Analytics. Internal provider signatures may be refined without changing the approved HTTP contract. - Any OpenAPI generator, validator, router, rate-limiter, or CSV dependency must be version-pinned, checked against official documentation, security-reviewed, and approved before addition. ## Architecture decisions - `docs/openapi.yaml` remains the public interface source of truth. Generated artifacts are reproducible outputs, never a competing hand-written contract. - HTTP handlers live in the Merchant API capability and depend on narrow consumer-owned Identity/Analytics interfaces. - Shared HTTP middleware owns request ids, request-size limits, safe Problem Details serialization, authentication, CSRF/origin enforcement, and no-store/security headers. - Merchant identity is never accepted from a path, query, header, request body, cursor, or event id. It is resolved from the current server-side session on every protected request. - JSON detail and CSV export share one canonical filter/authorization/projection policy so export cannot widen the authorized view. - SSE reauthorizes connection and replay, uses retained opaque event ids, and resets to a current snapshot when replay is unavailable. - Audit and observability record safe metadata and correlation ids, not payment rows, credentials, tokens, raw cursors, or peer data. ## Dependency graph ```text Go scaffold/tooling | v Ent model + reviewed Atlas migrations | +--> pgx staging importer --> validation/promotion/recovery | | | v | sole immutable dataset snapshot | +--> demo identity/session service | +--> calendar + metric query kernel | v Redis/Asynq run orchestration | v Approved OpenAPI --> HTTP boundary --> auth/period/run/SSE/detail/CSV | v audit + full verification ``` ## Ordered task index ### Foundation phase A: repository and schema - [ ] Task F1: Establish the Go workspace, command roots, configuration, and quality toolchain. - [ ] Task F2: Implement the normalized Ent model and reviewed Atlas migration baseline. ### Checkpoint F-A: Schema foundation - [ ] Clean migrations create the approved physical schema and ERD relationships without triggers or revision tables. - [ ] Build, unit-test, lint, vulnerability, and migration commands are reproducible. ### Foundation phase B: one-time dataset bootstrap - [ ] Task F3: Implement pinned archive preflight and bounded pgx staging. - [ ] Task F4: Implement set-based validation, atomic promotion, redacted reports, and recovery. - [ ] Task F5: Implement deterministic synthetic merchant profiles and secure demo-account seeding. ### Checkpoint F-B: Trusted dataset - [ ] The full pinned archive reconciles to approved counts and the second identical run is a no-op. - [ ] Fatal input, interruption, concurrency, and sensitive-data leak tests pass. ### Foundation phase C: identity and analytics providers - [ ] Task F6: Implement password authentication and revocable server-side merchant sessions. - [ ] Task F7: Implement calendar periods and the deterministic metric query kernel. - [ ] Task F8: Implement Redis/Asynq analytics runs, section cache, retries, and event history. - [ ] Task F9: Implement tenant-scoped detail queries and shared export projection. - [ ] Task F10: Stabilize provider interfaces and complete backend-provider integration checks. ### Checkpoint F-C: Provider readiness - [ ] Identity, periods, metrics, peer suppression, runs, events, detail, and export providers pass PostgreSQL/Redis integration tests. - [ ] Provider contracts expose the immutable dataset snapshot and no client-supplied merchant authority. ### API phase 1: Contract and HTTP boundary - [ ] Task 1: Establish the OpenAPI toolchain and conformance harness. - [ ] Task 2: Build the shared HTTP boundary and protected-request middleware. ### Checkpoint 1: Boundary readiness - [ ] OpenAPI validation/generation is reproducible from a clean checkout. - [ ] Shared Problem Details, request limits, request ids, cookie authentication, and CSRF/origin behavior pass focused tests. - [ ] Human review confirms no application handler can obtain tenant authority from client input. ### API phase 2: Core vertical slices - [ ] Task 3: Implement login, current-session, and logout endpoints. - [ ] Task 4: Implement approved reporting-period discovery. - [ ] Task 5: Implement analytics-run start and snapshot retrieval. ### Checkpoint 2: Core API flow - [ ] A merchant can authenticate, obtain an approved period, start one deduplicated run, and read its snapshot. - [ ] Cross-tenant ids are indistinguishable from missing ids. - [ ] OpenAPI examples and focused integration tests pass. ### API phase 3: Partial delivery and recovery - [ ] Task 6: Implement authenticated SSE delivery and recovery. - [ ] Task 7: Implement exhausted-section manual retry. ### Checkpoint 3: Asynchronous behavior - [ ] Independent section completion, heartbeat, replay, snapshot reset, revocation, and sibling-failure isolation pass integration tests. - [ ] Existing start/retry/SSE limits reject before queue or stream work. ### API phase 4: Evidence and export - [ ] Task 8: Implement cursor-paginated session detail views. - [ ] Task 9: Implement safeguarded Persian CSV export. ### Checkpoint 4: Reconciliation and data boundary - [ ] Each detail view reconciles with its approved aggregate. - [ ] JSON and CSV use identical tenant/filter predicates and approved field projections. - [ ] Payer tokens, credentials, peer rows, and internal keys are absent from responses, exports, logs, and traces. ### API phase 5: Operational hardening - [ ] Task 10: Publish the stable API problem-type catalogue. - [ ] Task 11: Add API audit and observability instrumentation. - [ ] Task 12: Complete contract, security, race, and performance verification. ### Checkpoint 5: Implementation-ready completion - [ ] All OpenAPI operations and documented status codes have passing contract tests. - [ ] `go test`, race detection, `go vet`, `golangci-lint`, and `govulncheck` pass using repository-approved commands. - [ ] Runtime checks cover tenant isolation, session revocation, SSE recovery, cursor integrity, export caps, formula neutralization, and sensitive-field absence. - [ ] Documentation matches runtime behavior and the full project Definition of Done is satisfied. - [ ] Human review approves completion before merge or deployment. ## Verification strategy ### Per task - Write tests that fail before the behavior exists and pass after it is implemented. - Run focused package tests for the changed slice. - For schema/import slices, compare migrations to the approved ERD, exercise rollback/recovery, and reconcile pinned archive counts without committing source data. - For metric slices, compare results to an independent SQL oracle across Tehran/Jalali boundaries, missing values, status semantics, peers, and large/skewed merchants. - For identity slices, exercise enumeration resistance, two-session enforcement, expiry/revocation, CSRF, throttling, tenant isolation, and sensitive-value absence. - For Redis/Asynq slices, exercise single-flight, independent partial completion, bounded retries, replay, rate limits, and safe cold recomputation after Redis loss. - Validate `docs/openapi.yaml` and regenerate/check generated artifacts without diff. - Run `go test ./...` and `go vet ./...` once the Go scaffold exists. - Inspect the staged diff and scan it for secrets and forbidden data fields before every commit. ### At checkpoints - Run `go test -race ./...`, `golangci-lint run`, and `govulncheck ./...` after the required tools are pinned by the approved scaffold/tooling task. - Exercise the actual HTTP server against PostgreSQL and Redis test dependencies for authentication, tenant isolation, analytics lifecycle, SSE, pagination, and export behavior. - Run OpenAPI request/response conformance tests for all examples, errors, media types, headers, and security requirements. - Verify query plans and bounded resource behavior for detail/export paths with representative merchant volumes. ## Parallelization - F2 follows F1. After F2, importer preflight/staging (F3), metric-query work (F7), and the non-credential portion of synthetic profile design (F5) may proceed independently. - F4 follows F3. Credential seeding completes only after F4, and session work (F6) follows F5. - Run orchestration (F8) and detail/export providers (F9) may proceed in parallel after their identity and metric dependencies are stable; F10 integrates them sequentially. - API contract tooling (Task 1) may proceed alongside provider work after F1, but protected endpoints wait for their named F-provider dependencies. - Tasks 3 and 4 may proceed in parallel only after Tasks 1–2 stabilize shared HTTP types and middleware. - Task 6 and the read-only parts of Task 8 may proceed in parallel after Task 5 and the Analytics provider interfaces stabilize. - Task 9 is sequential after Task 8 because CSV must reuse its authorization, filters, and projection. - Tasks 10–11 may proceed alongside later feature slices once stable error codes and audit events exist. - Task 12 is sequential after all feature slices and instrumentation are complete. ## Risks and mitigations | Risk | Impact | Mitigation | |---|---|---| | Import accepts corrupt/oversized data or partially promotes | Critical | Pinned preflight, bounded streaming, set-based validation, atomic promotion, exact reconciliation, fault-injection and recovery tests | | Physical schema or metrics drift from approved semantics | Critical | ERD/migration comparison, independent SQL oracle, boundary fixtures, catalogue-version evidence | | Demo authentication consumes effort or becomes production identity | Medium | Keep the bounded demo-only session scope and preserve the explicit future ZarinPal SSO replacement boundary | | Generated types drift from OpenAPI | High | Reproducible pinned generation plus clean-diff and contract-test gates | | Tenant scope leaks through ids/cursors/replay | Critical | Session-derived merchant context, ownership checks in providers, identical not-found policy, adversarial integration tests | | SSE behavior differs across proxy/browser boundaries | High | Test event framing, heartbeat, buffering headers, replay, reset, disconnect, and revocation through the deployed-style proxy path | | CSV widens detail access or leaks formulas/sensitive data | Critical | Reuse one canonical detail query/projection, preflight limits, formula neutralization, allowlist tests, abort-aware audit result | | Retry/idempotency races duplicate jobs | High | Atomic intent claim, canonical request hash, current-run deduplication, concurrency tests | | Large merchant queries exhaust API resources | High | Cursor pagination, fixed filters, timeouts, export preflight caps, measured plans, bounded streaming | | Error details expose internals | Medium | Central RFC 9457 mapper with stable allowlisted codes and negative leak tests | | Pre-1.0 queue behavior changes | High | Preserve approved Asynq `v0.26.0` pin and run retry/deduplication/recovery integration tests before any upgrade | ## Implementation gates, not product questions - Select and approve exact pinned OpenAPI generation/validation tooling before adding dependencies. - Confirm repository-wide build/test/lint command wrappers when the authorized Go scaffold is created. - Confirm the hosting mechanism that makes `https://zarinpal.amiirkhl.ir/docs/errors/{error-slug}` dereferenceable; the URI namespace and content responsibility are already approved. ## Product questions None. ## Approved feature-branch redesign API increment The user approved the merchant-dashboard redesign on 2026-08-20. The backend increment is additive and follows this dependency order: 1. Pure, tested Jalali boundary helpers and a dataset-bounded catalog of all complete supported period descriptors. 2. Authenticated merchant-terminal listing and optional terminal scope on analytics runs. 3. Bounded overview payment-history points for the selected scope. 4. Transaction listing filters and one merchant-owned transaction detail resource. 5. OpenAPI regeneration, local database smoke, and feature preview on port `8081`. No migration is required. All terminal and session identifiers are resolved under the merchant from the authenticated server session.