starcoach-hackathon/zarinpal-analytics-backend
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(plan): expand backend implementation roadmap

Browse Source
This commit is contained in:
Amirhossein Khalili
2026-08-20 19:00:07 +03:30
parent ec625252fb
commit b82ba0dc23
2 changed files with 277 additions and 51 deletions
Download Patch File Download Diff File Expand all files Collapse all files

116
tasks/plan.md
View File

@@ -1,18 +1,18 @@
# Implementation Plan: Merchant Analytics API
# Implementation Plan: ZarinPal Merchant Analytics Backend
- Status: Approved plan and task breakdown — implementation not authorized
- Capability: Merchant API
- Approved contract: `docs/API-SPEC.md` and `docs/openapi.yaml` version `1.0.0`
- Status: Expanded implementation plan ready for stakeholder review — implementation not authorized
- Capabilities: Data foundation, demo identity, analytics engine, and Merchant API
- Approved contracts: physical schema, import workflow/validation, Identity, Analytics, metric catalogue `0.4.0`, API, and OpenAPI `1.0.0`
- Task list: `tasks/todo.md`
- Last updated: 2026-08-20
## Overview
Implement the approved `/api/v1` REST, SSE, paginated-detail, and CSV interfaces as the HTTP adapter of the capability-oriented Go modular monolith. The API layer validates untrusted HTTP input, resolves the merchant from the server-side session, calls approved Identity/Analytics provider contracts, and serializes only OpenAPI-defined output. It does not own metric formulas, normalized payment facts, queue behavior, or deployment configuration.
Build the complete Go capability-oriented modular monolith needed by the dashboard: reviewed PostgreSQL migrations and normalized entities, the guarded one-time importer, demo merchant identity, deterministic analytics queries, Redis/Asynq orchestration, and the approved `/api/v1` REST/SSE/detail/CSV adapter. Deployment configuration remains owned by the deployment repository.
## Preconditions and boundaries
- The approved Identity, Data Foundation, Analytics Engine, API, ADR, metric, and data-guide decisions remain authoritative.
- The approved Identity, Data Foundation, physical schema, importer, Analytics Engine, API, ADR, metric, and data-guide decisions remain authoritative.
- Architecture and dependency injection are already approved: capability-oriented modular monolith with manual constructor wiring in thin `cmd/` composition roots.
- This plan does not authorize `go.mod`, packages, generated files, dependencies, migrations, containers, or executable code.
- Implementation starts only after the plan and task list are approved and the user separately authorizes implementation.
@@ -32,38 +32,67 @@ Implement the approved `/api/v1` REST, SSE, paginated-detail, and CSV interfaces
## Dependency graph
```text
Approved OpenAPI 1.0.0 + backend scaffold
|
v
Contract toolchain and HTTP boundary
|
+---------+----------+
| |
v v
Identity adapter Analytics provider adapters
| |
v +---------+---------+
Auth endpoints Periods Runs/snapshot
|
+------+------+
v v
SSE Section retry
Analytics provider adapters
|
v
Detail JSON pagination
|
v
CSV export
|
v
Audit, observability, and full conformance
Go scaffold/tooling
|
v
Ent model + reviewed Atlas migrations
|
+--> pgx staging importer --> validation/promotion/recovery
| |
| v
| sole immutable dataset snapshot
|
+--> demo identity/session service
|
+--> calendar + metric query kernel
|
v
Redis/Asynq run orchestration
|
v
Approved OpenAPI --> HTTP boundary --> auth/period/run/SSE/detail/CSV
|
v
audit + full verification
```
## Ordered task index
### Phase 1: Contract and HTTP boundary
### Foundation phase A: repository and schema
- [ ] Task F1: Establish the Go workspace, command roots, configuration, and quality toolchain.
- [ ] Task F2: Implement the normalized Ent model and reviewed Atlas migration baseline.
### Checkpoint F-A: Schema foundation
- [ ] Clean migrations create the approved physical schema and ERD relationships without triggers or revision tables.
- [ ] Build, unit-test, lint, vulnerability, and migration commands are reproducible.
### Foundation phase B: one-time dataset bootstrap
- [ ] Task F3: Implement pinned archive preflight and bounded pgx staging.
- [ ] Task F4: Implement set-based validation, atomic promotion, redacted reports, and recovery.
- [ ] Task F5: Implement deterministic synthetic merchant profiles and secure demo-account seeding.
### Checkpoint F-B: Trusted dataset
- [ ] The full pinned archive reconciles to approved counts and the second identical run is a no-op.
- [ ] Fatal input, interruption, concurrency, and sensitive-data leak tests pass.
### Foundation phase C: identity and analytics providers
- [ ] Task F6: Implement password authentication and revocable server-side merchant sessions.
- [ ] Task F7: Implement calendar periods and the deterministic metric query kernel.
- [ ] Task F8: Implement Redis/Asynq analytics runs, section cache, retries, and event history.
- [ ] Task F9: Implement tenant-scoped detail queries and shared export projection.
- [ ] Task F10: Stabilize provider interfaces and complete backend-provider integration checks.
### Checkpoint F-C: Provider readiness
- [ ] Identity, periods, metrics, peer suppression, runs, events, detail, and export providers pass PostgreSQL/Redis integration tests.
- [ ] Provider contracts expose the immutable dataset snapshot and no client-supplied merchant authority.
### API phase 1: Contract and HTTP boundary
- [ ] Task 1: Establish the OpenAPI toolchain and conformance harness.
- [ ] Task 2: Build the shared HTTP boundary and protected-request middleware.
@@ -74,7 +103,7 @@ Analytics provider adapters
- [ ] Shared Problem Details, request limits, request ids, cookie authentication, and CSRF/origin behavior pass focused tests.
- [ ] Human review confirms no application handler can obtain tenant authority from client input.
### Phase 2: Core vertical slices
### API phase 2: Core vertical slices
- [ ] Task 3: Implement login, current-session, and logout endpoints.
- [ ] Task 4: Implement approved reporting-period discovery.
@@ -86,7 +115,7 @@ Analytics provider adapters
- [ ] Cross-tenant ids are indistinguishable from missing ids.
- [ ] OpenAPI examples and focused integration tests pass.
### Phase 3: Partial delivery and recovery
### API phase 3: Partial delivery and recovery
- [ ] Task 6: Implement authenticated SSE delivery and recovery.
- [ ] Task 7: Implement exhausted-section manual retry.
@@ -96,7 +125,7 @@ Analytics provider adapters
- [ ] Independent section completion, heartbeat, replay, snapshot reset, revocation, and sibling-failure isolation pass integration tests.
- [ ] Existing start/retry/SSE limits reject before queue or stream work.
### Phase 4: Evidence and export
### API phase 4: Evidence and export
- [ ] Task 8: Implement cursor-paginated session detail views.
- [ ] Task 9: Implement safeguarded Persian CSV export.
@@ -107,7 +136,7 @@ Analytics provider adapters
- [ ] JSON and CSV use identical tenant/filter predicates and approved field projections.
- [ ] Payer tokens, credentials, peer rows, and internal keys are absent from responses, exports, logs, and traces.
### Phase 5: Operational hardening
### API phase 5: Operational hardening
- [ ] Task 10: Publish the stable API problem-type catalogue.
- [ ] Task 11: Add API audit and observability instrumentation.
@@ -127,6 +156,10 @@ Analytics provider adapters
- Write tests that fail before the behavior exists and pass after it is implemented.
- Run focused package tests for the changed slice.
- For schema/import slices, compare migrations to the approved ERD, exercise rollback/recovery, and reconcile pinned archive counts without committing source data.
- For metric slices, compare results to an independent SQL oracle across Tehran/Jalali boundaries, missing values, status semantics, peers, and large/skewed merchants.
- For identity slices, exercise enumeration resistance, two-session enforcement, expiry/revocation, CSRF, throttling, tenant isolation, and sensitive-value absence.
- For Redis/Asynq slices, exercise single-flight, independent partial completion, bounded retries, replay, rate limits, and safe cold recomputation after Redis loss.
- Validate `docs/openapi.yaml` and regenerate/check generated artifacts without diff.
- Run `go test ./...` and `go vet ./...` once the Go scaffold exists.
- Inspect the staged diff and scan it for secrets and forbidden data fields before every commit.
@@ -140,6 +173,10 @@ Analytics provider adapters
## Parallelization
- F2 follows F1. After F2, importer preflight/staging (F3), metric-query work (F7), and the non-credential portion of synthetic profile design (F5) may proceed independently.
- F4 follows F3. Credential seeding completes only after F4, and session work (F6) follows F5.
- Run orchestration (F8) and detail/export providers (F9) may proceed in parallel after their identity and metric dependencies are stable; F10 integrates them sequentially.
- API contract tooling (Task 1) may proceed alongside provider work after F1, but protected endpoints wait for their named F-provider dependencies.
- Tasks 3 and 4 may proceed in parallel only after Tasks 1–2 stabilize shared HTTP types and middleware.
- Task 6 and the read-only parts of Task 8 may proceed in parallel after Task 5 and the Analytics provider interfaces stabilize.
- Task 9 is sequential after Task 8 because CSV must reuse its authorization, filters, and projection.
@@ -150,6 +187,9 @@ Analytics provider adapters
| Risk | Impact | Mitigation |
|---|---|---|
| Import accepts corrupt/oversized data or partially promotes | Critical | Pinned preflight, bounded streaming, set-based validation, atomic promotion, exact reconciliation, fault-injection and recovery tests |
| Physical schema or metrics drift from approved semantics | Critical | ERD/migration comparison, independent SQL oracle, boundary fixtures, catalogue-version evidence |
| Demo authentication consumes effort or becomes production identity | Medium | Keep the bounded demo-only session scope and preserve the explicit future ZarinPal SSO replacement boundary |
| Generated types drift from OpenAPI | High | Reproducible pinned generation plus clean-diff and contract-test gates |
| Tenant scope leaks through ids/cursors/replay | Critical | Session-derived merchant context, ownership checks in providers, identical not-found policy, adversarial integration tests |
| SSE behavior differs across proxy/browser boundaries | High | Test event framing, heartbeat, buffering headers, replay, reset, disconnect, and revocation through the deployed-style proxy path |