starcoach-hackathon/zarinpal-analytics-deployment
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: replan manual platform delivery

Browse Source
This commit is contained in:
Amirhossein Khalili
2026-08-20 18:35:36 +03:30
parent 789f60e2ca
commit ec7a7753f6
2 changed files with 215 additions and 244 deletions
Download Patch File Download Diff File Expand all files Collapse all files

172
tasks/plan.md
View File

@@ -1,139 +1,125 @@
# Implementation Plan: Platform Operations
- Status: Planned from draft specification; implementation blocked pending explicit approval
- Status: Planned from draft revision; implementation blocked pending explicit spec approval
- Capability: Platform operations
- Task list target: `tasks/todo.md`
- Last updated: 2026-08-20
## Overview
## Confirmed implementation direction
Implement the approved Platform Operations contract as a single-server, production-shaped Docker Compose platform for `zarinpal.amiirkhl.ir`. The deployment repository remains the only operational coordinator; backend and frontend source stay in their own repositories and publish immutable images. Work proceeds from decisions and validation contracts through topology, edge, lifecycle, release automation, recovery, and end-to-end proof.
Build a Qlockify-style, single-Ubuntu-server platform for `zarinpal.amiirkhl.ir`. A trusted Docker-group deployment user manually pulls the latest `main` in the deployment, backend, and frontend repositories; records all three commit hashes; builds locally with Docker Compose; migrates; verifies health; and records the release. Rollback restores the previous hashes and rebuilds.
No task below is authorized merely because it appears in this plan. The Platform Operations spec must be approved and its blocking decisions resolved first.
There is no CI/CD pipeline, runner, private image registry, or automated cross-repository trigger. PostgreSQL backup/restore adapts Qlockify's existing scripts with minimal project changes and excludes environment/secrets. Prometheus/Grafana are a required second phase after MVP readiness and before AI work.
No task below is authorized merely because it appears here.
## Dependency graph
```text
approve spec + resolve platform versions/host/runner/backup decisions
approve revised spec + resolve server/runtime contracts
|
+--> release manifest + configuration/secret schemas
+--> safe config + manual release-record contract
| |
| +--> Compose networks/data services
| +--> Compose data/app services
| | |
| | +--> migration/import/admin lifecycle
| | +--> API/worker/frontend services
| | +--> migrate/import/admin separation
| | +--> manual deploy/rollback
| |
| +--> image/build contracts in service repositories
| |
| +--> cross-repository candidate events
| +--> Nginx/Certbot routing and TLS
|
+--> Nginx/TLS/routing contract implementation
+--> Qlockify backup adaptation --> guarded restore --> rehearsal
|
+--> observability baseline
|
+--> backup --> guarded restore --> restore rehearsal
|
+--> host bootstrap validation
|
v
full release/rollback/DR acceptance
+--> first-server bootstrap --> MVP acceptance
|
v
Prometheus/Grafana phase
|
v
AI may begin
```
## Architecture decisions
- Preserve ADR 0001's three-repository topology.
- Build application images once in service CI and deploy immutable digests through ADR 0002 release manifests.
- Consume the licensed font/license through the ordinary frontend context per accepted ADR 0003 and frontend commit `64f5ac2`; no deployment injection path.
- Keep PostgreSQL authoritative and backed up. Redis uses routine AOF persistence but remains excluded from disaster-recovery backup.
- Separate API, worker, migration, one-time import, and administrative commands. No recurring CSV or scheduled application worker exists.
- Use expand/contract migrations and application-image rollback; never automatic database down migrations.
- Keep Nginx as the only public entry point and monitoring/operator access private.
- Preserve ADR 0001's three independent repositories and ignored nested service checkouts.
- Follow accepted ADR 0002: latest `main`, record three hashes, build manually on the server, store active/previous release records.
- Follow accepted ADR 0003 and frontend commit `64f5ac2`: ordinary frontend context only for font/license.
- Use PostgreSQL 18 and Redis 7 Alpine with explicit supported tags.
- Keep PostgreSQL authoritative and backed up. Redis remains excluded from disaster-recovery backup.
- Separate API, worker, migration, one-time CSV import, and administrative processes.
- Keep Asynq pinned to pre-1.0 `v0.26.0`.
- Use automatic Let's Encrypt/Certbot TLS.
- Treat structured logs/health/manual checks as MVP; add Prometheus/Grafana after MVP and before AI.
## Phases and checkpoints
## Phase 0: approve exact contracts
### Phase 0: approval and evidence
- Approve the revised Platform Operations spec.
- Resolve server/provider/capacity/user/path, image tags, resource limits, repository URLs/commands/health routes, backup endpoint/credentials/schedule, Redis AOF choice, release-record format, operator/secret escrow, and MVP-ready/observability gates.
- Obtain the backend command contract for API, worker, migration, import, admin, liveness, readiness, and Redis-loss recovery.
- Obtain the frontend build/static health contract.
- Resolve host/provider/capacity, container privilege model, forge/runner/registry versions, exact image versions/digests, resource budgets, API health/routes, backup encryption/provider, certificate-renewal method, and operator ownership.
- Approve `docs/PLATFORM-OPERATIONS-SPEC.md` and ADR 0002. ADR 0003 is already accepted.
- Obtain approved machine-readable backend/frontend interface and runtime command contracts.
Checkpoint: no implementation file exists before explicit approval and resolved inputs.
Checkpoint: no implementation file exists before the approval record and decisions are complete.
## Phase 1: configuration and Compose foundation
### Phase 1: release and configuration foundation
- Add safe tracked configuration examples and validate protected external secret paths.
- Define non-secret active/previous release records containing the three SHAs and health/migration evidence.
- Implement isolated Compose networks, PostgreSQL/Redis volumes/health/resources, backend/worker/frontend services, and one-off migration/import/admin profiles.
- Add focused validation for ports, startup ordering, font build presence, and forbidden tracked/runtime artifacts.
- Define and validate the release-manifest schema, active/previous release records, exact three-repository/image provenance, and compatibility fields.
- Define tracked safe configuration examples plus the external host secret-file contract.
- Add policy tests that reject floating images, unintended ports, secret-like values, missing health checks, or font injection paths.
Checkpoint: clean local/server validation proves only Nginx will publish ports and normal startup cannot trigger import/admin.
Checkpoint: a synthetic manifest/config validates offline and contains no secret values or mutable references.
## Phase 2: Nginx, TLS, and lifecycle
### Phase 2: runtime topology and edge
- Adapt Qlockify Nginx/Certbot structure for `zarinpal.amiirkhl.ir`.
- Implement canonical redirect/default-host rejection, API/frontend separation, exact SSE route, security headers, caching, and health behavior.
- Implement manual deploy/release/rollback from Qlockify's script structure without unconditional destructive Git reset.
- Exercise failed pull/build/migration/readiness/font/worker conditions while preserving the active release.
- Implement isolated Compose networks, named volumes, PostgreSQL/Redis health/resources, private application services, and monitoring profile.
- Implement one-off migration, import, and administrative profiles without coupling them to normal startup.
- Implement canonical-domain Nginx/TLS, SPA/API separation, exact SSE route behavior, security headers, limits, and health routing.
- Add focused topology/routing tests before deployment automation.
Checkpoint: manual deployment records the three SHAs, passes TLS/health/SSE checks, and a prior-hash rebuild rollback completes within 15 minutes.
Checkpoint: local/integration inspection proves only Nginx publishes 80/443, startup gates are correct, and SSE is not buffered.
## Phase 3: backup, restore, and first-server acceptance
### Phase 3: service artifact and lifecycle coordination
- Minimally adapt Qlockify backup/upload/restore/S3-restore scripts for ZarinPal paths/services/credentials.
- Remove Qlockify environment-file backup/restore; include PostgreSQL archive plus non-secret manifest/checksums only.
- Add exact destination and production confirmation guards.
- Bootstrap the server, deploy the MVP, perform one-time import/credential setup, create an encrypted remote backup, and rehearse clean restore within two hours.
- Coordinate backend image commands/health/labels and frontend static/font health with the owning repositories through approved changes there; do not duplicate their source here.
- Implement deployment locks, preflight, migration, service replacement, postflight, observation, and application rollback using release manifests.
- Implement Redis-loss reconciliation acceptance with backend/worker owners, preserving Asynq `v0.26.0` as a pinned pre-1.0 dependency.
Checkpoint: every MVP acceptance criterion passes through manual health/log/disk/certificate/queue/backup checks.
Checkpoint: an induced migration/readiness/font/worker failure preserves or restores the prior release.
## Phase 4: required observability before AI
### Phase 4: CI/CD and provenance
- Add Prometheus, Grafana, and minimum private exporters/instrumentation.
- Add service/release, worker/SSE, PostgreSQL/Redis, host/container, backup/restore, and TLS dashboards.
- Add bounded labels, retention, symptom-based alerts, runbook links, and tested notification delivery.
- Validate the observability acceptance gate before any AI implementation starts.
- Add deployment-repository validation and production-release workflows.
- Coordinate service-repository workflows that publish immutable images and send candidate events only after required checks.
- Enforce protected environment approval, serialized release, registry digest retention, SBOM/provenance/vulnerability gates, and secret isolation.
Checkpoint: PRs never deploy; backend/frontend candidate events cannot directly mutate production; one manifest controls a tested release and rollback.
### Phase 5: observability and recovery
- Implement bounded structured container logging, Prometheus/exporters/Grafana, dashboards, and symptom alerts with runbook links.
- Implement encrypted logical PostgreSQL backup, remote verification, three-local/seven-remote retention, and telemetry.
- Implement exact-destination guarded restore into an empty destination and application validation.
- Rehearse restore and record achieved RPO/RTO.
Checkpoint: induced failures are diagnosable from telemetry, backup age alert fires, and clean restore succeeds within two hours.
### Phase 6: bootstrap and full acceptance
- Convert the reviewed first-server checklist into only the minimum approved automation, retaining manual safety/identity gates.
- Bootstrap the selected host, stage DNS/TLS, deploy a release, run one-time import/credentials, and complete alert/backup/restore handoff.
- Execute every measurable criterion from the Platform Operations spec and retain non-sensitive evidence.
Checkpoint: stakeholder review confirms release, rollback, Redis-loss, backup/restore, exposure, SLO, and operator-boundary evidence.
Checkpoint: an induced failure is detected, correlated to the three-hash release, diagnosed from telemetry, and alerts reach the responsible operator.
## Risks and mitigations
| Risk | Impact | Mitigation |
|---|---|---|
| Single-host failure | Full outage | Immutable rebuild path, encrypted off-host PostgreSQL backup, two-hour restore rehearsal, documented DNS/host replacement |
| Cross-repository race | Incompatible release | Candidate events only, serialized deployment lock, exact manifest, compatibility metadata, one coordinator |
| Migration blocks rollback | Extended outage/data risk | Expand/contract, measured locks, fresh verified backup, no automatic down migration |
| Redis loss drops in-flight queue/cache | Partial analytics interruption | AOF for routine restart, PostgreSQL authority, idempotent stale-run reconciliation, cold-cache tests |
| SSE proxy buffering/timeouts | Dashboard appears stuck | Exact route, buffering/compression/cache off, heartbeat-aware timeout, integration test |
| Docker privilege compromise | Host root compromise | Approve rootless/dedicated model, restricted machine identity/SSH, no daemon TCP exposure, minimal host users |
| Secrets leak through Compose/log/backup | Credential/data compromise | File-mounted least privilege, redaction tests, no env in backup, rotate-first incident rule |
| Backup exists but is unusable | Data loss beyond RTO | Archive/checksum/remote verification plus monthly clean restore |
| Font missing/divergent | Broken approved typography/license trail | Frontend-owned tracked inputs at `64f5ac2+`, build/static smoke, digest provenance, no second path |
| Monitoring overload/noise | Lost diagnostics or ignored alerts | Bounded labels/retention, symptom alerts, test-fire, only page/ticket severities |
| Manual deploy selects inconsistent source | Broken stack | One deployment lock, latest three `main` checkouts recorded before build, one release record |
| Dirty checkout is destroyed | Lost operator/user work | Fail on dirty/divergent state; no unconditional `git reset --hard` |
| Server-side build differs over time | Harder reproduction | Pin base/dependency inputs in service repos, record Git hashes/local image ids/build time |
| Migration blocks rollback | Extended outage | Expand/contract, measured locks, fresh backup, no automatic down migration |
| Redis loss drops queue/cache | Partial analytics interruption | Optional AOF, PostgreSQL authority, idempotent reconciliation/cold-cache tests |
| SSE buffering/timeouts | Dashboard appears stuck | Exact Nginx route, buffering/compression/cache off, heartbeat/reconnect test |
| Docker-group compromise | Host-root compromise | Dedicated key-only trusted account, restricted SSH, no daemon TCP exposure |
| Backup leaks credentials | Wider compromise | Remove `.env` backup/restore, encrypt, separate passphrase, archive exclusion tests |
| Backup is unusable | Data loss beyond RTO | Checksums/listing/remote verification plus clean restore rehearsal |
| Delayed monitoring hides degradation | Slower MVP incident diagnosis | MVP health/log/operator checklist; mandatory Prometheus/Grafana before AI |
## Parallelization after contracts are fixed
## Parallelization after approval
- Safe in parallel: Nginx focused tests, backup/restore tooling, observability dashboards, and CI validation, provided the shared manifest/config schemas are merged first.
- Sequential: schema migrations, production deploy/rollback, database restore, DNS/TLS cutover, and one-time import.
- Coordinated across repositories: backend commands/health/migration metadata, frontend image/font health, and candidate-event payload. Providers define contracts before consumers implement them.
- Safe in parallel: focused Nginx tests, backup adaptation, release-record validation, and later dashboards after shared paths/contracts are fixed.
- Sequential: deployment, migration, production restore, DNS/TLS cutover, one-time import, and MVP-to-observability phase gate.
- Cross-repository coordination: backend commands/health and frontend build/font behavior are defined by their owners before deployment consumes them; no automated pipeline is introduced.
## Completion gate
## Completion
- Every task in `tasks/todo.md` meets acceptance and verification evidence.
- All spec acceptance criteria pass on a production-shaped host.
- No unresolved decision remains hidden in a default or script.
- Active/previous release manifests, SLO dashboards, alerts, backup, restore rehearsal, and operator handoff are complete.
- Backend/frontend repositories contain only their approved coordinated changes; the deployment repository contains no application source, raw data, secrets, credentials, TLS keys, backups, database volumes, or duplicate font/license files.
- Every authorized task in `tasks/todo.md` passes its acceptance and verification.
- The active stack maps to exact deployment/backend/frontend commits.
- Rollback, Redis-loss recovery, backup, restore, import separation, and exposure checks pass.
- Prometheus/Grafana acceptance passes before AI begins.
- Deployment Git contains no application source, secrets, raw data, credentials, certificates, backups, volumes, or duplicate font/license assets.