zarinpal-analytics-deployment/tasks/plan.md

Implementation Plan: Platform Operations

• Status: Planned from draft specification; implementation blocked pending explicit approval
• Capability: Platform operations
• Task list target: tasks/todo.md
• Last updated: 2026-08-20

Overview

Implement the approved Platform Operations contract as a single-server, production-shaped Docker Compose platform for zarinpal.amiirkhl.ir. The deployment repository remains the only operational coordinator; backend and frontend source stay in their own repositories and publish immutable images. Work proceeds from decisions and validation contracts through topology, edge, lifecycle, release automation, recovery, and end-to-end proof.

No task below is authorized merely because it appears in this plan. The Platform Operations spec must be approved and its blocking decisions resolved first.

Dependency graph

approve spec + resolve platform versions/host/runner/backup decisions
        |
        +--> release manifest + configuration/secret schemas
        |        |
        |        +--> Compose networks/data services
        |        |       |
        |        |       +--> migration/import/admin lifecycle
        |        |       +--> API/worker/frontend services
        |        |
        |        +--> image/build contracts in service repositories
        |                |
        |                +--> cross-repository candidate events
        |
        +--> Nginx/TLS/routing contract implementation
        |
        +--> observability baseline
        |
        +--> backup --> guarded restore --> restore rehearsal
        |
        +--> host bootstrap validation
                         |
                         v
                full release/rollback/DR acceptance

Architecture decisions

• Preserve ADR 0001's three-repository topology.
• Build application images once in service CI and deploy immutable digests through ADR 0002 release manifests.
• Consume the licensed font/license through the ordinary frontend context per accepted ADR 0003 and frontend commit 64f5ac2; no deployment injection path.
• Keep PostgreSQL authoritative and backed up. Redis uses routine AOF persistence but remains excluded from disaster-recovery backup.
• Separate API, worker, migration, one-time import, and administrative commands. No recurring CSV or scheduled application worker exists.
• Use expand/contract migrations and application-image rollback; never automatic database down migrations.
• Keep Nginx as the only public entry point and monitoring/operator access private.

Phases and checkpoints

Phase 0: approval and evidence

• Resolve host/provider/capacity, container privilege model, forge/runner/registry versions, exact image versions/digests, resource budgets, API health/routes, backup encryption/provider, certificate-renewal method, and operator ownership.
• Approve docs/PLATFORM-OPERATIONS-SPEC.md and ADR 0002. ADR 0003 is already accepted.
• Obtain approved machine-readable backend/frontend interface and runtime command contracts.

Checkpoint: no implementation file exists before the approval record and decisions are complete.

Phase 1: release and configuration foundation

• Define and validate the release-manifest schema, active/previous release records, exact three-repository/image provenance, and compatibility fields.
• Define tracked safe configuration examples plus the external host secret-file contract.
• Add policy tests that reject floating images, unintended ports, secret-like values, missing health checks, or font injection paths.

Checkpoint: a synthetic manifest/config validates offline and contains no secret values or mutable references.

Phase 2: runtime topology and edge

• Implement isolated Compose networks, named volumes, PostgreSQL/Redis health/resources, private application services, and monitoring profile.
• Implement one-off migration, import, and administrative profiles without coupling them to normal startup.
• Implement canonical-domain Nginx/TLS, SPA/API separation, exact SSE route behavior, security headers, limits, and health routing.
• Add focused topology/routing tests before deployment automation.

Checkpoint: local/integration inspection proves only Nginx publishes 80/443, startup gates are correct, and SSE is not buffered.

Phase 3: service artifact and lifecycle coordination

• Coordinate backend image commands/health/labels and frontend static/font health with the owning repositories through approved changes there; do not duplicate their source here.
• Implement deployment locks, preflight, migration, service replacement, postflight, observation, and application rollback using release manifests.
• Implement Redis-loss reconciliation acceptance with backend/worker owners, preserving Asynq v0.26.0 as a pinned pre-1.0 dependency.

Checkpoint: an induced migration/readiness/font/worker failure preserves or restores the prior release.

Phase 4: CI/CD and provenance

• Add deployment-repository validation and production-release workflows.
• Coordinate service-repository workflows that publish immutable images and send candidate events only after required checks.
• Enforce protected environment approval, serialized release, registry digest retention, SBOM/provenance/vulnerability gates, and secret isolation.

Checkpoint: PRs never deploy; backend/frontend candidate events cannot directly mutate production; one manifest controls a tested release and rollback.

Phase 5: observability and recovery

• Implement bounded structured container logging, Prometheus/exporters/Grafana, dashboards, and symptom alerts with runbook links.
• Implement encrypted logical PostgreSQL backup, remote verification, three-local/seven-remote retention, and telemetry.
• Implement exact-destination guarded restore into an empty destination and application validation.
• Rehearse restore and record achieved RPO/RTO.

Checkpoint: induced failures are diagnosable from telemetry, backup age alert fires, and clean restore succeeds within two hours.

Phase 6: bootstrap and full acceptance

• Convert the reviewed first-server checklist into only the minimum approved automation, retaining manual safety/identity gates.
• Bootstrap the selected host, stage DNS/TLS, deploy a release, run one-time import/credentials, and complete alert/backup/restore handoff.
• Execute every measurable criterion from the Platform Operations spec and retain non-sensitive evidence.

Checkpoint: stakeholder review confirms release, rollback, Redis-loss, backup/restore, exposure, SLO, and operator-boundary evidence.

Risks and mitigations

Risk	Impact	Mitigation
Single-host failure	Full outage	Immutable rebuild path, encrypted off-host PostgreSQL backup, two-hour restore rehearsal, documented DNS/host replacement
Cross-repository race	Incompatible release	Candidate events only, serialized deployment lock, exact manifest, compatibility metadata, one coordinator
Migration blocks rollback	Extended outage/data risk	Expand/contract, measured locks, fresh verified backup, no automatic down migration
Redis loss drops in-flight queue/cache	Partial analytics interruption	AOF for routine restart, PostgreSQL authority, idempotent stale-run reconciliation, cold-cache tests
SSE proxy buffering/timeouts	Dashboard appears stuck	Exact route, buffering/compression/cache off, heartbeat-aware timeout, integration test
Docker privilege compromise	Host root compromise	Approve rootless/dedicated model, restricted machine identity/SSH, no daemon TCP exposure, minimal host users
Secrets leak through Compose/log/backup	Credential/data compromise	File-mounted least privilege, redaction tests, no env in backup, rotate-first incident rule
Backup exists but is unusable	Data loss beyond RTO	Archive/checksum/remote verification plus monthly clean restore
Font missing/divergent	Broken approved typography/license trail	Frontend-owned tracked inputs at 64f5ac2+, build/static smoke, digest provenance, no second path
Monitoring overload/noise	Lost diagnostics or ignored alerts	Bounded labels/retention, symptom alerts, test-fire, only page/ticket severities

Parallelization after contracts are fixed

• Safe in parallel: Nginx focused tests, backup/restore tooling, observability dashboards, and CI validation, provided the shared manifest/config schemas are merged first.
• Sequential: schema migrations, production deploy/rollback, database restore, DNS/TLS cutover, and one-time import.
• Coordinated across repositories: backend commands/health/migration metadata, frontend image/font health, and candidate-event payload. Providers define contracts before consumers implement them.

Completion gate

• Every task in tasks/todo.md meets acceptance and verification evidence.
• All spec acceptance criteria pass on a production-shaped host.
• No unresolved decision remains hidden in a default or script.
• Active/previous release manifests, SLO dashboards, alerts, backup, restore rehearsal, and operator handoff are complete.
• Backend/frontend repositories contain only their approved coordinated changes; the deployment repository contains no application source, raw data, secrets, credentials, TLS keys, backups, database volumes, or duplicate font/license files.