zarinpal-analytics-deployment/tasks/todo.md

Platform Operations implementation tasks

• Status: Planned and ready for stakeholder review; implementation not authorized
• Canonical plan: tasks/plan.md
• Adaptation baseline: docs/QLOCKIFY-ADAPTATION-CONTRACT.md

Task 0: Approve the implementation plan

Description: Confirm that the Qlockify adaptation contract, dependency order, task boundaries, and verification gates are the correct basis for later implementation.

Acceptance criteria:

• Platform Operations specification and ADRs 0001–0003 are approved.
• Qlockify flow/environment mapping is documented without real runtime values.
• Stakeholder approves this plan/task breakdown and separately authorizes implementation when ready.

Verification: Documentation diff review and confirmation that no implementation artifact was created.

Dependencies: None

Files likely touched: documentation only

Estimated scope: Small

Task 1: Add the deployment environment contract

Description: Add the root deployment example and redaction-safe validation for the Qlockify-shaped root/backend/frontend environment layout. Backend and frontend examples remain owned by their repositories and are consumed without duplication.

Acceptance criteria:

• The root example contains only deployment-owned safe defaults/blank placeholders; every service-owned variable has one documented owner.
• Real .env files stay ignored, mode 0600, excluded from backup, and absent from rendered/logged output.
• Validation rejects missing, unknown, malformed, or client-exposed secret variables for the selected operation.

Verification: Placeholder render, negative validation matrix, Git-ignore/permission checks, and staged secret scan.

Dependencies: Task 0 plus explicit implementation authorization

Files likely touched: .env.example, validation helper/test, documentation; maximum 5 files

Estimated scope: Medium

Task 2: Add PostgreSQL and Redis Compose foundation

Description: Define pinned private data services, health checks, named volumes, authentication, resource limits, bounded logs, and Redis AOF behavior.

Acceptance criteria:

• PostgreSQL and Redis become healthy using the approved pins and memory limits.
• Neither publishes a host port; only approved services can join the private data network.
• Redis uses AOF everysec, noeviction, 192-MB maxmemory, and remains excluded from backup.

Verification: Compose validation, health/port/network/volume/resource inspection, restart and Redis-loss tests.

Dependencies: Task 1

Files likely touched: compose.yaml, data config, focused validation/test files; maximum 5 files

Estimated scope: Medium

Task 3: Add backend runtime boundaries

Description: Wire API and worker services plus one-off migrate/import/admin profiles from the same recorded backend checkout/image lineage.

Acceptance criteria:

• API and worker use private networks, health/readiness, graceful shutdown, and approved resource limits.
• Migrate/import/admin are explicit one-off profiles and never run during ordinary startup.
• Worker uses pre-1.0 Asynq v0.26.0 with concurrency 16 globally and 4 per merchant.

Verification: Compose profile/startup/failure/shutdown tests and service command-contract review.

Dependencies: Task 2 and the backend-owned command contract

Files likely touched: compose.yaml, backend image definition, lifecycle tests/docs; maximum 5 files

Estimated scope: Medium

Task 4: Add frontend production build

Description: Build and serve the direct frontend checkout using its ordinary static context and same-origin API configuration.

Acceptance criteria:

• Build consumes frontend commit 64f5ac2 or later and its tracked font/license normally.
• Static sentinel, SPA fallback, and emitted font resource/content type pass.
• Deployment Git contains no font/license copy, fetch, credentials, or alternate asset path.

Verification: Production build/static-server checks plus repository-policy scan.

Dependencies: Task 1 and the frontend-owned build contract

Files likely touched: frontend image definition, compose.yaml, focused health/tests; maximum 5 files

Estimated scope: Medium

Checkpoint A: Runtime foundation

• Tasks 1–4 pass from clean nested checkouts with placeholder test values.
• Normal startup cannot invoke migration/import/admin.
• No secrets, runtime state, raw data, or duplicated font assets are tracked.

Task 5: Adapt Qlockify Nginx routing

Description: Adapt the Qlockify template/entrypoint for the single canonical domain, default-host rejection, frontend/API routing, security headers, health, caching, and SSE.

Acceptance criteria:

• Only ports 80/443 publish; HTTP redirects exactly to zarinpal.amiirkhl.ir; unknown hosts reject.
• API and frontend routing/fallback do not overlap and public readiness remains generic.
• Exact SSE route preserves auth/event ids and streams heartbeats without buffering, cache, compression, or premature timeout.

Verification: nginx -t, route/header/default-host tests, static/font test, and streamed SSE fixture.

Dependencies: Tasks 3–4

Files likely touched: Nginx template, image/entrypoint, focused tests; maximum 5 files

Estimated scope: Medium

Task 6: Adapt Qlockify Certbot lifecycle

Description: Adapt Qlockify's HTTP-01 bootstrap, pinned Certbot service, certificate watcher, renewal, and Nginx reload for the single domain.

Acceptance criteria:

• Missing production certificate uses only the bounded bootstrap certificate path until issuance.
• Staging issuance, production issuance, renewal, and validated Nginx reload pass.
• No alternate domain, local production certificate mode, private key, or ACME state enters Git.

Verification: Compose/entrypoint tests, staging ACME drill, certificate name/expiry/protocol inspection.

Dependencies: Task 5

Files likely touched: Certbot entrypoint/config, compose.yaml, focused tests/docs; maximum 5 files

Estimated scope: Medium

Task 7: Add guarded first-start orchestration

Description: Add the explicit first-server path that starts data services, migrates, imports/reconciles the pinned CSV, and activates public services only after success.

Acceptance criteria:

• Input is exact/read-only and migration/import/reconciliation failures prevent public activation.
• A completed identical import is a safe no-op; recovery never guesses a partial CSV position.
• Ordinary deploy/start/reboot paths never invoke first-start, importer, or admin jobs.

Verification: Success, checksum/schema/count, interruption/recovery, idempotency, and ordinary-start negative tests.

Dependencies: Tasks 2–6 and approved backend importer commands

Files likely touched: scripts/first-start.sh, Compose profile, focused tests/docs; maximum 5 files

Estimated scope: Medium

Checkpoint B: Edge and first-start

• Nginx/TLS/static/API/SSE checks pass.
• First-start is fail-closed and ordinary startup never imports.
• Only Nginx is externally reachable.

Task 8: Add release records and operation lock

Description: Define ignored current/previous/timestamped release records and one exclusive lock shared by deploy, rollback, restore, migration, and import as required.

Acceptance criteria:

• Record captures three SHAs, local image ids, migration, backup, operator, health, outcome, and previous release without secrets.
• Invalid/incomplete records cannot become active and current/previous updates are atomic.
• Conflicting operations fail safely without stale-lock ambiguity.

Verification: Schema/parser tests, invalid-record matrix, lock contention/stale-owner tests, and redaction scan.

Dependencies: Task 1

Files likely touched: release schema/helper/tests/docs; maximum 5 files

Estimated scope: Medium

Task 9: Adapt Qlockify manual deployment

Description: Adapt Qlockify's deploy structure to verify three manually pulled repositories, build/migrate/replace services, observe health, and activate one coherent release.

Acceptance criteria:

• Script performs no Git fetch/pull/reset and rejects wrong path/remote/branch/SHA/dirty state.
• Build, migration, readiness, font, worker, or SSE failure leaves the active release unchanged and records evidence.
• Successful deployment records all three SHAs and passes a 15-minute postflight observation before activation.

Verification: ShellCheck, disposable Git fixtures, Compose mocks/integration, and induced failure matrix.

Dependencies: Tasks 5–8

Files likely touched: scripts/deploy.sh, release helper, focused tests/runbook; maximum 5 files

Estimated scope: Medium

Task 10: Add three-hash rollback

Description: Select the previous release record, verify its three revisions, rebuild, and run the normal postflight without automatic database down-migration.

Acceptance criteria:

• Exact previous hashes and clean repository guards precede checkout/build.
• Migration incompatibility stops rollback with the current stack preserved.
• Normal application rollback completes within 15 minutes and is recorded as a new event.

Verification: Git fixture tests, incompatible-migration negative test, and timed rollback drill.

Dependencies: Task 9

Files likely touched: rollback script/helper/tests/runbook; maximum 5 files

Estimated scope: Medium

Checkpoint C: Release safety

• Dirty/diverged/manual-pull/build/migration/readiness failures preserve the active stack.
• Active and previous releases map to exact three-repository revisions.
• Timed rollback meets its target without a down migration.

Task 11: Adapt Qlockify PostgreSQL backup

Description: Retain Qlockify's matching-client plain-SQL dump flow while packaging only PostgreSQL plus a non-secret release/migration/checksum manifest.

Acceptance criteria:

• Dump nonzero exit, empty output, invalid header, or checksum failure prevents package success.
• Package contains only plain SQL and the approved non-secret manifest/checksums.
• .env, secrets, Redis, TLS, CSV, credentials, logs, monitoring, font/license, and media are absent.

Verification: Fixture database dump, package listing/header/checksum checks, and forbidden-content scan.

Dependencies: Tasks 2, 8

Files likely touched: scripts/backup.sh, manifest helper/tests/runbook; maximum 5 files

Estimated scope: Medium

Task 12: Adapt encrypted ParsPack upload and retention

Description: Retain Qlockify's OpenSSL AES-256-CBC PBKDF2-200k and rclone structure while keeping three encrypted local and seven encrypted remote verified backups.

Acceptance criteria:

• Upload uses a private temporary key and is finalized only after remote verification.
• Failed encryption/upload/verification preserves all previous copies and no plaintext backup remains durably.
• Retention removes only exact stale targets after new-copy verification and never deletes the last good copy.

Verification: Fake S3/ParsPack-compatible fixture, failure/retention matrix, checksum and plaintext-residue checks.

Dependencies: Task 11

Files likely touched: upload script, retention helper/tests/runbook; maximum 5 files

Estimated scope: Medium

Task 13: Add guarded local restore

Description: Restore an explicitly selected verified backup into a new empty target using psql -X --set ON_ERROR_STOP=on and independent production guards.

Acceptance criteria:

• Provenance/checksum/header/version/free-space/exact-empty-target checks precede mutation.
• Every missing production factor aborts before writers stop or a database is changed.
• Restore never changes .env, Redis, TLS, CSV, credentials, logs, or frontend assets.

Verification: Disposable database restore, negative guard matrix, partial-SQL failure, and exclusion checks.

Dependencies: Task 11

Files likely touched: scripts/restore.sh, guard/helper/tests/runbook; maximum 5 files

Estimated scope: Medium

Task 14: Adapt encrypted S3 restore handoff

Description: Download an explicit encrypted object, verify/decrypt it in a restrictive temporary directory, and hand it to the guarded local restore.

Acceptance criteria:

• Production requires an explicit object id; an implicit latest cannot select a production backup.
• Remote identity/size/checksum and decrypted package checks pass before restore invocation.
• Temporary encrypted/plaintext material is removed only from the exact validated temporary directory.

Verification: Fake S3 object-selection/tamper tests, decryption failure tests, and guarded handoff rehearsal.

Dependencies: Tasks 12–13

Files likely touched: scripts/restore-from-s3.sh, focused tests/runbook; maximum 4 files

Estimated scope: Medium

Checkpoint D: Recovery

• Daily schedule/3-local/7-remote retention behaves safely.
• Forbidden content is absent from every backup package.
• Clean restore passes schema/count/tenant/login/cold-cache/worker/SSE validation within two hours.

Task 15: Bootstrap and accept the MVP server

Description: Execute the reviewed runbooks on the prepared Ubuntu server, fill runtime values, perform first-start, activate the release, and rehearse rollback/restore.

Acceptance criteria:

• Host access/firewall/swap/Docker/DNS/TLS and only-80/443 exposure checks pass.
• Migration/import/accounts/tenant isolation/worker/SSE/frontend font and manual health/log checks pass.
• Three-hash release, rollback, encrypted remote backup, and clean restore meet their time/recovery targets.

Verification: Signed non-secret bootstrap/MVP acceptance record.

Dependencies: Checkpoints A–D and a prepared server/runtime values

Files likely touched: ignored server evidence/release records only

Estimated scope: Operational session

Checkpoint E: MVP ready

• Every MVP acceptance criterion passes.
• Active/previous releases and backup/restore are proven.
• Stakeholder explicitly opens the observability phase.

Task 16: Add private Prometheus collection

Description: Add post-MVP application, worker/SSE, PostgreSQL/Redis, host/container, backup/restore, and TLS collection with bounded labels and retention.

Acceptance criteria:

• Required signals and target health pass within approved resource/retention limits.
• Prometheus/exporters are private and not routed by Nginx.
• Merchant/user/request/card identifiers are absent from metric labels.

Verification: Target/signal/cardinality/resource tests and sensitive-label scan.

Dependencies: Checkpoint E

Files likely touched: split into focused Prometheus/exporter slices of no more than 5 files

Estimated scope: Multiple Medium slices

Task 17: Add private Grafana dashboards and alerts

Description: Add post-MVP dashboards, release correlation, actionable symptom alerts, runbook links, and tested operator delivery.

Acceptance criteria:

• Service, worker/SSE, data, host, backup, and TLS dashboards answer the approved operator questions.
• Every page/ticket alert is actionable, bounded, linked to a runbook, and tested through an induced failure.
• Grafana remains private and contains no sensitive labels, logs, or embedded credentials.

Verification: Dashboard review, induced failure, alert test-fire, access and redaction checks.

Dependencies: Task 16

Files likely touched: split into focused dashboard/alert slices of no more than 5 files

Estimated scope: Multiple Medium slices

Final observability checkpoint before AI

• Prometheus/Grafana acceptance criteria pass after MVP.
• Alert delivery and runbook ownership are proven.
• Stakeholder explicitly approves beginning AI implementation.
• No CI/CD pipeline or automated deployment system was introduced.