# Implementation Plan: Platform Operations - Status: Approved — Phase 4 implementation authorized 2026-08-20 - Capability: Platform operations - Task list target: `tasks/todo.md` - Planning baseline: `docs/QLOCKIFY-ADAPTATION-CONTRACT.md` - Last updated: 2026-08-20 ## Outcome Adapt the read-only Qlockify deployment repository into a manual, production-shaped ZarinPal Analytics deployment for `zarinpal.amiirkhl.ir`. Planning does not wait for a server IP, live DNS, real credentials, or named operators. Tracked artifacts use safe defaults and blank secret placeholders; real values are supplied only when an operation needs them. There is no CI/CD pipeline, runner, registry, automated Git synchronization, or private font injection. Prometheus/Grafana remain a required post-MVP, pre-AI phase. The user authorized implementation of the tasks below on 2026-08-20. ## Dependency graph ```text approved spec + Qlockify adaptation contract + explicit build authorization | +--> environment examples/validation --> data-service Compose | | | | +--> release record/lock +--> API/worker/one-offs | | +--> frontend build --------------------------------+ | Nginx routing --> Certbot/TLS | first-start --> deploy --> rollback | | +-----------+--> backup/upload | guarded restore | first-server MVP gate | Prometheus --> Grafana | AI may begin ``` ## Fixed architecture decisions - Preserve ADR 0001's three independent repositories and direct ignored `backend/` and `frontend/` checkouts. - Follow ADR 0002: the operator pulls all repositories interactively before deployment; scripts verify/record but do not pull or reset Git. - Follow ADR 0003/frontend commit `64f5ac2`: consume the tracked font/license through the normal frontend build context. - Pin `postgres:18.4-alpine3.24`, `redis:7.4.10-alpine3.21`, `nginx:1.29.8-alpine3.23`, and `certbot/certbot:v5.7.0`. - Keep PostgreSQL authoritative. Redis uses AOF `everysec`/`noeviction` but remains disposable and excluded from backup. - Keep API, worker, migrate, importer, and admin commands separate; Asynq stays explicitly pre-1.0 at `v0.26.0`. - Reuse Qlockify's root/backend/frontend `.env` shape and backup/restore script family with the differences recorded in the adaptation contract. - Use structured logs and manual operator checks for MVP; add private Prometheus/Grafana only after MVP acceptance and before AI. ## Phase 0: planning baseline - The Platform Operations spec, ADRs 0001–0003, Qlockify adaptation contract, bootstrap runbook, and backup/restore runbook define the approved behavior. - Runtime values such as IP, DNS access, email, ParsPack credentials, and encryption passphrase are deliberately deferred to the operation that uses them. - Backend/frontend command and variable names are integrated when their implementation contracts exist; earlier independent deployment work is not blocked. - Stakeholder approval and explicit implementation authorization were recorded on 2026-08-20. Checkpoint: the plan and tasks are approved; no implementation artifact exists yet. ## Phase 1: configuration and runtime foundation 1. Create the deployment-root example and redaction-safe validation for the Qlockify-shaped three-file layout; consume backend/frontend-owned examples without duplicating them. 2. Implement private PostgreSQL/Redis services, named volumes, health checks, authentication, resource limits, and bounded logs. 3. Add API/worker plus separate migration/import/admin profiles from one backend revision. 4. Add the ordinary frontend static build and font/static health checks. Checkpoint: Compose renders from placeholder test values; only intended networks exist; ordinary startup cannot invoke migration/import/admin; no secret/font/runtime artifact is tracked. ## Phase 2: edge and first-start lifecycle 5. Adapt Qlockify Nginx routing for the single canonical domain, API, frontend, readiness, and exact SSE behavior. 6. Adapt Qlockify Certbot issuance/renewal/reload for the single domain and pinned image. 7. Add guarded first-start orchestration for data readiness, migration, one-time import/reconciliation, and first public activation. Checkpoint: routing/TLS configuration validates, first-start fails closed before public activation, and later ordinary startup never imports. ## Phase 3: releases and rollback 8. Define the ignored release-record format and exclusive operation lock. 9. Adapt Qlockify's deploy flow to verify manually pulled clean revisions, build/migrate/replace services, observe health, and activate one three-hash release. 10. Implement prior-release checkout/rebuild rollback with migration compatibility guards. Checkpoint: induced checkout/build/migration/readiness failures preserve the active stack; a prior application release is recoverable within 15 minutes under normal conditions. ## Phase 4: backup and restore 11. Adapt Qlockify's plain-SQL PostgreSQL backup to emit only database data plus a non-secret manifest/checksums. 12. Adapt its OpenSSL/rclone upload and encrypted 3-local/7-remote retention flow for the ZarinPal ParsPack prefix. 13. Harden local restore with explicit backup, clean-target, compatibility, and production confirmation guards. 14. Adapt S3 download/decrypt to verify an explicit remote object and hand off to the guarded restore. Checkpoint: excluded content inspection passes, failed upload/retention cannot delete the last good copy, and a clean rehearsal meets RPO 24 hours/RTO two hours. ## Phase 5: first-server MVP acceptance 15. Follow the bootstrap runbook on the prepared Ubuntu server, fill real environment values, deploy/import once, verify the application, rehearse rollback/restore, and record evidence. Checkpoint: all MVP criteria pass. The stakeholder explicitly marks MVP ready before monitoring work begins. ## Phase 6: required observability before AI 16. Add private Prometheus and approved exporters/instrumentation with bounded labels, retention, and resource use. 17. Add private Grafana dashboards, release correlation, symptom-based alerts, runbook links, and tested delivery. Checkpoint: an induced failure is detected and diagnosed from telemetry; the stakeholder approves the observability gate before AI begins. ## Verification strategy - Documentation/planning: Markdown diff checks, source mapping review, secret-pattern scan, and no implementation files. - Configuration: placeholder-only rendering, missing/invalid/unknown variable tests, permission/exclusion checks, and no resolved-value output. - Compose: configuration validation, service health, network/port inspection, resource/log limits, profiles, and restart/Redis-loss behavior. - Shell operations: ShellCheck plus disposable fixture tests for exact paths, locks, dirty/diverged Git, failure cleanup, retention, and restore guard matrices. - Edge: `nginx -t`, canonical/unknown-host behavior, security headers, TLS issuance/renewal, static/font behavior, API routing, and streamed SSE checks. - Release/recovery: induced deployment failures, timed prior-hash rollback, backup inspection, remote verification, and clean-database restore rehearsal. - MVP/observability: manual acceptance record first; monitoring target/cardinality/dashboard/alert tests only after the MVP gate. ## Risks and mitigations | Risk | Impact | Mitigation | |---|---|---| | Qlockify-specific behavior is copied blindly | Wrong services or unsafe operations | File-by-file adaptation matrix and explicit retain/change/drop review | | Environment ownership drifts | Secret mismatch or accidental client exposure | Root/backend/frontend ownership contract; one owner per variable; redaction-safe validation | | Manual releases select inconsistent source | Broken stack | Interactive pulls first, one lock, three clean SHA checks, one release record | | Server-side builds vary over time | Difficult reproduction | Pin infrastructure/dependency inputs and record three SHAs plus local image ids | | Migration prevents rollback | Extended outage | Expand/contract sequencing, preflight backup, no automatic down migration | | Backup contains secrets or is unusable | Disclosure or data loss | PostgreSQL-only package, encryption/checksums, explicit exclusions, clean restore rehearsal | | Redis loss interrupts analytics | Temporary missing results | AOF `everysec`, PostgreSQL authority, idempotent reconciliation and cold-cache tests | | 2-GB server exhausts memory | OOM/restarts | Approved per-service limits, 2-GB swap, bounded imports/logs, capacity upgrade when measured | | Monitoring is postponed too far | Weak diagnosis before AI | Hard MVP-to-observability-to-AI phase gate | ## Parallel and sequential work - After environment/paths are fixed, PostgreSQL/Redis foundation, frontend build, Nginx fixtures, and backup fixtures can be prepared in separate focused slices. - Compose integration, migration/import, release activation, rollback, production restore, DNS/TLS cutover, and phase gates remain sequential. - Backend/frontend repositories define their application commands and variables; deployment consumes those contracts without editing those repositories in this workstream. ## Completion - Every authorized task in `tasks/todo.md` passes its acceptance and verification. - The active stack maps to exact deployment/backend/frontend commits and can roll back safely. - Only Nginx publishes application ports; secrets/runtime/raw/font copies remain outside deployment Git. - Encrypted off-host backup and clean restore meet their RPO/RTO. - Prometheus/Grafana acceptance passes after MVP and before AI.