docs: plan platform implementation tasks

tasks/todo.md

@@ -1,230 +1,348 @@
 # Platform Operations implementation tasks
 
-- Status: Ready for stakeholder review — implementation not authorized
+- Status: Planned and ready for stakeholder review; implementation not authorized
 - Canonical plan: `tasks/plan.md`
+- Adaptation baseline: `docs/QLOCKIFY-ADAPTATION-CONTRACT.md`
 
-## Task 0: Resolve implementation inputs and preserve the implementation gate
+## Task 0: Approve the implementation plan
 
-**Description:** Use the approved manual Qlockify-style specification to collect the concrete values and service contracts required for later implementation while preserving the separate explicit build-authorization gate.
+**Description:** Confirm that the Qlockify adaptation contract, dependency order, task boundaries, and verification gates are the correct basis for later implementation.
 
 **Acceptance criteria:**
-- [x] Revised Platform Operations spec is explicitly approved.
-- [x] ADRs 0001–0003 match the approved topology, manual release, and font decisions.
-- [ ] Production connection values, protected secrets, repository URLs, service commands/routes, and responsible operators are recorded outside public Git where required.
-- [ ] Stakeholder explicitly authorizes creation of deployment implementation artifacts.
+- [x] Platform Operations specification and ADRs 0001–0003 are approved.
+- [x] Qlockify flow/environment mapping is documented without real runtime values.
+- [ ] Stakeholder approves this plan/task breakdown and separately authorizes implementation when ready.
 
-**Verification:** Approved specification, clean documentation review, resolved slice inputs, and explicit build authorization.
+**Verification:** Documentation diff review and confirmation that no implementation artifact was created.
 
 **Dependencies:** None
 
-**Files likely touched:** spec and ADRs only
+**Files likely touched:** documentation only
 
 **Estimated scope:** Small
 
-## Task 1: Define safe configuration and release records
+## Task 1: Add the deployment environment contract
 
-**Description:** Define tracked placeholders, external protected secrets, deployment locking, and non-secret active/previous release records.
+**Description:** Add the root deployment example and redaction-safe validation for the Qlockify-shaped root/backend/frontend environment layout. Backend and frontend examples remain owned by their repositories and are consumed without duplication.
 
 **Acceptance criteria:**
-- [ ] Release record contains three SHAs, local image ids, migration, backup, operator, and health evidence.
-- [ ] Configuration examples contain no real or plausible secret values.
-- [ ] Font provenance uses frontend commit/build only; no second asset path exists.
+- [ ] The root example contains only deployment-owned safe defaults/blank placeholders; every service-owned variable has one documented owner.
+- [ ] Real `.env` files stay ignored, mode `0600`, excluded from backup, and absent from rendered/logged output.
+- [ ] Validation rejects missing, unknown, malformed, or client-exposed secret variables for the selected operation.
 
-**Verification:** Schema/format tests, secret scan, invalid-record tests.
+**Verification:** Placeholder render, negative validation matrix, Git-ignore/permission checks, and staged secret scan.
 
-**Dependencies:** Task 0
+**Dependencies:** Task 0 plus explicit implementation authorization
 
-**Files likely touched:** examples/schema/tests/docs, maximum 5 files
+**Files likely touched:** `.env.example`, validation helper/test, documentation; maximum 5 files
 
 **Estimated scope:** Medium
 
-## Task 2: Implement PostgreSQL and Redis Compose foundation
+## Task 2: Add PostgreSQL and Redis Compose foundation
 
-**Description:** Add pinned PostgreSQL 18 and Redis 7 Alpine, private data network, volumes, authentication, health, resources, and approved Redis AOF choice.
+**Description:** Define pinned private data services, health checks, named volumes, authentication, resource limits, bounded logs, and Redis AOF behavior.
 
 **Acceptance criteria:**
-- [ ] Both services become healthy with explicit limits/tags.
-- [ ] Neither publishes a host port.
-- [ ] Redis is excluded from backup and loss does not affect PostgreSQL authority.
+- [ ] PostgreSQL and Redis become healthy using the approved pins and memory limits.
+- [ ] Neither publishes a host port; only approved services can join the private data network.
+- [ ] Redis uses AOF `everysec`, `noeviction`, 192-MB maxmemory, and remains excluded from backup.
 
-**Verification:** Compose validation, health, port/network, volume, Redis-loss tests.
+**Verification:** Compose validation, health/port/network/volume/resource inspection, restart and Redis-loss tests.
 
 **Dependencies:** Task 1
 
-**Files likely touched:** Compose/config/tests, maximum 5 files
+**Files likely touched:** `compose.yaml`, data config, focused validation/test files; maximum 5 files
 
 **Estimated scope:** Medium
 
-## Task 3: Implement backend, worker, and one-off boundaries
+## Task 3: Add backend runtime boundaries
 
-**Description:** Wire API/worker plus separate migration/import/admin commands and graceful lifecycle.
+**Description:** Wire API and worker services plus one-off migrate/import/admin profiles from the same recorded backend checkout/image lineage.
 
 **Acceptance criteria:**
-- [ ] Migration completes before API/worker readiness.
-- [ ] Normal startup never runs import/admin; import uses one exact read-only path.
-- [ ] `first-start.sh` starts data services, migrates, imports and reconciles the pinned archive, then starts public services only after success.
-- [ ] Worker is private/event-driven and uses pinned pre-1.0 Asynq `v0.26.0`.
+- [ ] API and worker use private networks, health/readiness, graceful shutdown, and approved resource limits.
+- [ ] Migrate/import/admin are explicit one-off profiles and never run during ordinary startup.
+- [ ] Worker uses pre-1.0 Asynq `v0.26.0` with concurrency 16 globally and 4 per merchant.
 
-**Verification:** First-start, ordinary-startup, migration/import failure, checksum/count reconciliation, idempotent no-op, and shutdown integration tests.
+**Verification:** Compose profile/startup/failure/shutdown tests and service command-contract review.
 
-**Dependencies:** Task 2 and approved backend commands
+**Dependencies:** Task 2 and the backend-owned command contract
 
-**Files likely touched:** Compose/service wrappers/tests, maximum 5 files per slice
+**Files likely touched:** `compose.yaml`, backend image definition, lifecycle tests/docs; maximum 5 files
 
 **Estimated scope:** Medium
 
-## Task 4: Implement frontend build and font health
+## Task 4: Add frontend production build
 
-**Description:** Build the nested frontend checkout normally and verify static SPA/font output without deployment copies or injection.
+**Description:** Build and serve the direct frontend checkout using its ordinary static context and same-origin API configuration.
 
 **Acceptance criteria:**
-- [ ] Build uses recorded frontend commit at/after `64f5ac2`.
-- [ ] Expected font resource returns 200 with correct content type.
-- [ ] Deployment repository contains no font binary, `FontLicense.txt`, fetch, credential, or alternate path.
+- [ ] Build consumes frontend commit `64f5ac2` or later and its tracked font/license normally.
+- [ ] Static sentinel, SPA fallback, and emitted font resource/content type pass.
+- [ ] Deployment Git contains no font/license copy, fetch, credentials, or alternate asset path.
 
-**Verification:** Build/static health plus repository-policy scan.
+**Verification:** Production build/static-server checks plus repository-policy scan.
 
-**Dependencies:** Task 1 and approved frontend build contract
+**Dependencies:** Task 1 and the frontend-owned build contract
 
-**Files likely touched:** frontend deployment wrapper/service/tests, maximum 5 files
+**Files likely touched:** frontend image definition, `compose.yaml`, focused health/tests; maximum 5 files
 
 **Estimated scope:** Medium
 
-## Task 5: Implement Nginx and automatic TLS
+## Checkpoint A: Runtime foundation
 
-**Description:** Adapt Qlockify's Nginx/Certbot pattern for canonical routing, Let's Encrypt, API/frontend separation, exact SSE behavior, security headers, and health.
+- [ ] Tasks 1–4 pass from clean nested checkouts with placeholder test values.
+- [ ] Normal startup cannot invoke migration/import/admin.
+- [ ] No secrets, runtime state, raw data, or duplicated font assets are tracked.
+
+## Task 5: Adapt Qlockify Nginx routing
+
+**Description:** Adapt the Qlockify template/entrypoint for the single canonical domain, default-host rejection, frontend/API routing, security headers, health, caching, and SSE.
 
 **Acceptance criteria:**
-- [ ] Only 80/443 are public; unknown hosts reject and canonical redirect is exact.
-- [ ] Staging/production issuance and renewal/reload pass.
-- [ ] SSE heartbeat/reconnect streams without buffering/cache/compression.
+- [ ] Only ports 80/443 publish; HTTP redirects exactly to `zarinpal.amiirkhl.ir`; unknown hosts reject.
+- [ ] API and frontend routing/fallback do not overlap and public readiness remains generic.
+- [ ] Exact SSE route preserves auth/event ids and streams heartbeats without buffering, cache, compression, or premature timeout.
 
-**Verification:** `nginx -t`, external TLS/header/routing checks, streamed SSE test.
+**Verification:** `nginx -t`, route/header/default-host tests, static/font test, and streamed SSE fixture.
 
-**Dependencies:** Tasks 3–4 and approved routes/CSP
+**Dependencies:** Tasks 3–4
 
-**Files likely touched:** Nginx/Certbot assets/tests, maximum 5 files per slice
+**Files likely touched:** Nginx template, image/entrypoint, focused tests; maximum 5 files
 
 **Estimated scope:** Medium
 
-## Checkpoint A: Runtime MVP foundation
+## Task 6: Adapt Qlockify Certbot lifecycle
 
-- [ ] Tasks 1–5 pass from clean nested checkouts.
-- [ ] Only Nginx is public; migration/import/admin/font/SSE boundaries behave correctly.
-- [ ] No secret/raw data/runtime/font duplication enters deployment Git.
-
-## Task 6: Adapt Qlockify manual deploy and rollback
-
-**Description:** Modify Qlockify's deploy flow for ZarinPal paths/services and three-hash release records while keeping interactive Git pulls as a separate operator prerequisite and removing unconditional destructive Git synchronization.
+**Description:** Adapt Qlockify's HTTP-01 bootstrap, pinned Certbot service, certificate watcher, renewal, and Nginx reload for the single domain.
 
 **Acceptance criteria:**
-- [ ] Operator manually fast-forwards all three repositories to latest `origin/main`; the deployment command performs no pull and records all three hashes before build.
-- [ ] Dirty/diverged checkout, build, migration, or health failure preserves the active release; a manual pull failure prevents deployment from starting.
-- [ ] Previous hashes rebuild/restore within 15 minutes without automatic down migration.
+- [ ] Missing production certificate uses only the bounded bootstrap certificate path until issuance.
+- [ ] Staging issuance, production issuance, renewal, and validated Nginx reload pass.
+- [ ] No alternate domain, local production certificate mode, private key, or ACME state enters Git.
 
-**Verification:** Script tests plus induced staging failures and timed rollback drill.
+**Verification:** Compose/entrypoint tests, staging ACME drill, certificate name/expiry/protocol inspection.
 
-**Dependencies:** Checkpoint A
+**Dependencies:** Task 5
 
-**Files likely touched:** deploy script/release helper/tests/docs, maximum 5 files per slice
+**Files likely touched:** Certbot entrypoint/config, `compose.yaml`, focused tests/docs; maximum 5 files
 
 **Estimated scope:** Medium
 
-## Task 7: Adapt Qlockify encrypted backup/upload
+## Task 7: Add guarded first-start orchestration
 
-**Description:** Reuse Qlockify's PostgreSQL dump, OpenSSL, rclone upload, and retention flow with ZarinPal paths/services/credentials and a non-secret manifest.
+**Description:** Add the explicit first-server path that starts data services, migrates, imports/reconciles the pinned CSV, and activates public services only after success.
 
 **Acceptance criteria:**
-- [ ] Package contents, plain-SQL header/checksums, encryption, and remote verification gate success.
-- [ ] Exactly three verified encrypted local and seven remote backups are retained.
-- [ ] `.env`, secrets, Redis, TLS, CSV, credentials, logs, monitoring, font, and license are excluded.
+- [ ] Input is exact/read-only and migration/import/reconciliation failures prevent public activation.
+- [ ] A completed identical import is a safe no-op; recovery never guesses a partial CSV position.
+- [ ] Ordinary deploy/start/reboot paths never invoke first-start, importer, or admin jobs.
 
-**Verification:** Fixture database/object-store tests, exclusion inspection, failed-upload/cleanup tests.
+**Verification:** Success, checksum/schema/count, interruption/recovery, idempotency, and ordinary-start negative tests.
 
-**Dependencies:** Tasks 1–3 and approved backup inputs
+**Dependencies:** Tasks 2–6 and approved backend importer commands
 
-**Files likely touched:** adapted backup/upload scripts/tests/docs, maximum 5 files per slice
+**Files likely touched:** `scripts/first-start.sh`, Compose profile, focused tests/docs; maximum 5 files
 
 **Estimated scope:** Medium
 
-## Task 8: Adapt Qlockify guarded restore
+## Checkpoint B: Edge and first-start
 
-**Description:** Reuse Qlockify download/decrypt/restore flow while requiring explicit backup id, exact destination, empty target, and all production guards.
+- [ ] Nginx/TLS/static/API/SSE checks pass.
+- [ ] First-start is fail-closed and ordinary startup never imports.
+- [ ] Only Nginx is externally reachable.
+
+## Task 8: Add release records and operation lock
+
+**Description:** Define ignored current/previous/timestamped release records and one exclusive lock shared by deploy, rollback, restore, migration, and import as required.
 
 **Acceptance criteria:**
-- [ ] Every absent/mismatched production factor aborts before mutation.
-- [ ] No environment/secret/Redis restoration occurs.
-- [ ] Clean restore passes schema/count/tenant/login/cold-cache/worker/SSE checks within two hours.
+- [ ] Record captures three SHAs, local image ids, migration, backup, operator, health, outcome, and previous release without secrets.
+- [ ] Invalid/incomplete records cannot become active and current/previous updates are atomic.
+- [ ] Conflicting operations fail safely without stale-lock ambiguity.
 
-**Verification:** Disposable-database tests, guard negative matrix, timed rehearsal.
+**Verification:** Schema/parser tests, invalid-record matrix, lock contention/stale-owner tests, and redaction scan.
 
-**Dependencies:** Task 7
+**Dependencies:** Task 1
 
-**Files likely touched:** adapted restore scripts/tests/docs, maximum 5 files per slice
+**Files likely touched:** release schema/helper/tests/docs; maximum 5 files
 
 **Estimated scope:** Medium
 
-## Task 9: Bootstrap and accept MVP server
+## Task 9: Adapt Qlockify manual deployment
 
-**Description:** Follow the reviewed bootstrap runbook, run the guarded first-start migration/import/application sequence, perform separate demo setup, and capture MVP evidence.
+**Description:** Adapt Qlockify's deploy structure to verify three manually pulled repositories, build/migrate/replace services, observe health, and activate one coherent release.
 
 **Acceptance criteria:**
-- [ ] Access/firewall/Docker/DNS/TLS/deployment boundaries pass.
-- [ ] Three-hash release, migration, import, accounts, tenant isolation, worker/SSE, and font pass.
-- [ ] MVP manual health/log/disk/certificate/queue checks and encrypted backup/restore pass.
+- [ ] Script performs no Git fetch/pull/reset and rejects wrong path/remote/branch/SHA/dirty state.
+- [ ] Build, migration, readiness, font, worker, or SSE failure leaves the active release unchanged and records evidence.
+- [ ] Successful deployment records all three SHAs and passes a 15-minute postflight observation before activation.
 
-**Verification:** Completed bootstrap/acceptance record.
+**Verification:** ShellCheck, disposable Git fixtures, Compose mocks/integration, and induced failure matrix.
 
-**Dependencies:** Tasks 6–8 and approved server/window
+**Dependencies:** Tasks 5–8
 
-**Files likely touched:** non-sensitive release/evidence records only
+**Files likely touched:** `scripts/deploy.sh`, release helper, focused tests/runbook; maximum 5 files
+
+**Estimated scope:** Medium
+
+## Task 10: Add three-hash rollback
+
+**Description:** Select the previous release record, verify its three revisions, rebuild, and run the normal postflight without automatic database down-migration.
+
+**Acceptance criteria:**
+- [ ] Exact previous hashes and clean repository guards precede checkout/build.
+- [ ] Migration incompatibility stops rollback with the current stack preserved.
+- [ ] Normal application rollback completes within 15 minutes and is recorded as a new event.
+
+**Verification:** Git fixture tests, incompatible-migration negative test, and timed rollback drill.
+
+**Dependencies:** Task 9
+
+**Files likely touched:** rollback script/helper/tests/runbook; maximum 5 files
+
+**Estimated scope:** Medium
+
+## Checkpoint C: Release safety
+
+- [ ] Dirty/diverged/manual-pull/build/migration/readiness failures preserve the active stack.
+- [ ] Active and previous releases map to exact three-repository revisions.
+- [ ] Timed rollback meets its target without a down migration.
+
+## Task 11: Adapt Qlockify PostgreSQL backup
+
+**Description:** Retain Qlockify's matching-client plain-SQL dump flow while packaging only PostgreSQL plus a non-secret release/migration/checksum manifest.
+
+**Acceptance criteria:**
+- [ ] Dump nonzero exit, empty output, invalid header, or checksum failure prevents package success.
+- [ ] Package contains only plain SQL and the approved non-secret manifest/checksums.
+- [ ] `.env`, secrets, Redis, TLS, CSV, credentials, logs, monitoring, font/license, and media are absent.
+
+**Verification:** Fixture database dump, package listing/header/checksum checks, and forbidden-content scan.
+
+**Dependencies:** Tasks 2, 8
+
+**Files likely touched:** `scripts/backup.sh`, manifest helper/tests/runbook; maximum 5 files
+
+**Estimated scope:** Medium
+
+## Task 12: Adapt encrypted ParsPack upload and retention
+
+**Description:** Retain Qlockify's OpenSSL AES-256-CBC PBKDF2-200k and rclone structure while keeping three encrypted local and seven encrypted remote verified backups.
+
+**Acceptance criteria:**
+- [ ] Upload uses a private temporary key and is finalized only after remote verification.
+- [ ] Failed encryption/upload/verification preserves all previous copies and no plaintext backup remains durably.
+- [ ] Retention removes only exact stale targets after new-copy verification and never deletes the last good copy.
+
+**Verification:** Fake S3/ParsPack-compatible fixture, failure/retention matrix, checksum and plaintext-residue checks.
+
+**Dependencies:** Task 11
+
+**Files likely touched:** upload script, retention helper/tests/runbook; maximum 5 files
+
+**Estimated scope:** Medium
+
+## Task 13: Add guarded local restore
+
+**Description:** Restore an explicitly selected verified backup into a new empty target using `psql -X --set ON_ERROR_STOP=on` and independent production guards.
+
+**Acceptance criteria:**
+- [ ] Provenance/checksum/header/version/free-space/exact-empty-target checks precede mutation.
+- [ ] Every missing production factor aborts before writers stop or a database is changed.
+- [ ] Restore never changes `.env`, Redis, TLS, CSV, credentials, logs, or frontend assets.
+
+**Verification:** Disposable database restore, negative guard matrix, partial-SQL failure, and exclusion checks.
+
+**Dependencies:** Task 11
+
+**Files likely touched:** `scripts/restore.sh`, guard/helper/tests/runbook; maximum 5 files
+
+**Estimated scope:** Medium
+
+## Task 14: Adapt encrypted S3 restore handoff
+
+**Description:** Download an explicit encrypted object, verify/decrypt it in a restrictive temporary directory, and hand it to the guarded local restore.
+
+**Acceptance criteria:**
+- [ ] Production requires an explicit object id; an implicit `latest` cannot select a production backup.
+- [ ] Remote identity/size/checksum and decrypted package checks pass before restore invocation.
+- [ ] Temporary encrypted/plaintext material is removed only from the exact validated temporary directory.
+
+**Verification:** Fake S3 object-selection/tamper tests, decryption failure tests, and guarded handoff rehearsal.
+
+**Dependencies:** Tasks 12–13
+
+**Files likely touched:** `scripts/restore-from-s3.sh`, focused tests/runbook; maximum 4 files
+
+**Estimated scope:** Medium
+
+## Checkpoint D: Recovery
+
+- [ ] Daily schedule/3-local/7-remote retention behaves safely.
+- [ ] Forbidden content is absent from every backup package.
+- [ ] Clean restore passes schema/count/tenant/login/cold-cache/worker/SSE validation within two hours.
+
+## Task 15: Bootstrap and accept the MVP server
+
+**Description:** Execute the reviewed runbooks on the prepared Ubuntu server, fill runtime values, perform first-start, activate the release, and rehearse rollback/restore.
+
+**Acceptance criteria:**
+- [ ] Host access/firewall/swap/Docker/DNS/TLS and only-80/443 exposure checks pass.
+- [ ] Migration/import/accounts/tenant isolation/worker/SSE/frontend font and manual health/log checks pass.
+- [ ] Three-hash release, rollback, encrypted remote backup, and clean restore meet their time/recovery targets.
+
+**Verification:** Signed non-secret bootstrap/MVP acceptance record.
+
+**Dependencies:** Checkpoints A–D and a prepared server/runtime values
+
+**Files likely touched:** ignored server evidence/release records only
 
 **Estimated scope:** Operational session
 
-## Checkpoint B: MVP ready
+## Checkpoint E: MVP ready
 
 - [ ] Every MVP acceptance criterion passes.
-- [ ] Active/previous three-hash releases and rollback are ready.
-- [ ] Clean restore meets RPO/RTO.
-- [ ] Stakeholder explicitly confirms the core project is ready, opening the observability phase.
+- [ ] Active/previous releases and backup/restore are proven.
+- [ ] Stakeholder explicitly opens the observability phase.
 
-## Task 10: Add Prometheus and private exporters
+## Task 16: Add private Prometheus collection
 
-**Description:** Add post-MVP collection for application, worker/SSE, PostgreSQL/Redis, host/container, backup/restore, and TLS signals.
+**Description:** Add post-MVP application, worker/SSE, PostgreSQL/Redis, host/container, backup/restore, and TLS collection with bounded labels and retention.
 
 **Acceptance criteria:**
-- [ ] Required signals appear with bounded labels and retention.
-- [ ] Prometheus/exporters are private and resource-limited.
+- [ ] Required signals and target health pass within approved resource/retention limits.
+- [ ] Prometheus/exporters are private and not routed by Nginx.
 - [ ] Merchant/user/request/card identifiers are absent from metric labels.
 
-**Verification:** Target health, signal/cardinality/resource tests.
+**Verification:** Target/signal/cardinality/resource tests and sensitive-label scan.
 
-**Dependencies:** Checkpoint B
+**Dependencies:** Checkpoint E
 
-**Files likely touched:** split into <=5-file monitoring slices
+**Files likely touched:** split into focused Prometheus/exporter slices of no more than 5 files
 
 **Estimated scope:** Multiple Medium slices
 
-## Task 11: Add Grafana dashboards and alerts
+## Task 17: Add private Grafana dashboards and alerts
 
-**Description:** Add private dashboards, release correlation, actionable symptom alerts, runbook links, and delivery testing.
+**Description:** Add post-MVP dashboards, release correlation, actionable symptom alerts, runbook links, and tested operator delivery.
 
 **Acceptance criteria:**
-- [ ] Service, worker/SSE, data, host, backup, and TLS dashboards answer the spec's operator questions.
-- [ ] Only page/ticket alerts exist and each is actionable.
-- [ ] Induced failures alert and can be diagnosed without source archaeology.
+- [ ] Service, worker/SSE, data, host, backup, and TLS dashboards answer the approved operator questions.
+- [ ] Every page/ticket alert is actionable, bounded, linked to a runbook, and tested through an induced failure.
+- [ ] Grafana remains private and contains no sensitive labels, logs, or embedded credentials.
 
-**Verification:** Dashboard review, induced failure, alert test-fire, redaction check.
+**Verification:** Dashboard review, induced failure, alert test-fire, access and redaction checks.
 
-**Dependencies:** Task 10
+**Dependencies:** Task 16
 
-**Files likely touched:** split into <=5-file dashboard/alert slices
+**Files likely touched:** split into focused dashboard/alert slices of no more than 5 files
 
 **Estimated scope:** Multiple Medium slices
 
 ## Final observability checkpoint before AI
 
-- [ ] Prometheus/Grafana phase meets its acceptance criteria.
+- [ ] Prometheus/Grafana acceptance criteria pass after MVP.
 - [ ] Alert delivery and runbook ownership are proven.
 - [ ] Stakeholder explicitly approves beginning AI implementation.
 - [ ] No CI/CD pipeline or automated deployment system was introduced.