starcoach-hackathon/zarinpal-analytics-deployment
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: plan platform implementation tasks

Browse Source
This commit is contained in:
Amirhossein Khalili
2026-08-20 19:07:36 +03:30
parent 21085c7405
commit eec3709533
2 changed files with 329 additions and 191 deletions
Download Patch File Download Diff File Expand all files Collapse all files

188
tasks/plan.md
View File

@@ -1,126 +1,146 @@
# Implementation Plan: Platform Operations
- Status: Implementation plan ready for stakeholder review — implementation not authorized
- Status: Ready for stakeholder review; implementation not authorized
- Capability: Platform operations
- Task list target: `tasks/todo.md`
- Planning baseline: `docs/QLOCKIFY-ADAPTATION-CONTRACT.md`
- Last updated: 2026-08-20
## Confirmed implementation direction
## Outcome
Build a Qlockify-style, single-Ubuntu-server platform for `zarinpal.amiirkhl.ir`. A trusted Docker-group deployment user manually pulls the latest `main` in the deployment, backend, and frontend repositories; records all three commit hashes; builds locally with Docker Compose; migrates; verifies health; and records the release. Rollback restores the previous hashes and rebuilds.
Adapt the read-only Qlockify deployment repository into a manual, production-shaped ZarinPal Analytics deployment for `zarinpal.amiirkhl.ir`. Planning does not wait for a server IP, live DNS, real credentials, or named operators. Tracked artifacts use safe defaults and blank secret placeholders; real values are supplied only when an operation needs them.
There is no CI/CD pipeline, runner, private image registry, or automated cross-repository trigger. PostgreSQL backup/restore adapts Qlockify's existing scripts with minimal project changes and excludes environment/secrets. Prometheus/Grafana are a required second phase after MVP readiness and before AI work.
There is no CI/CD pipeline, runner, registry, automated Git synchronization, or private font injection. Prometheus/Grafana remain a required post-MVP, pre-AI phase.
No task below is authorized merely because it appears here.
No task below authorizes implementation.
## Dependency graph
```text
resolve concrete server/runtime inputs + authorize build
|
+--> safe config + manual release-record contract
| |
| +--> Compose data/app services
| | |
| | +--> migrate/import/admin separation
| | +--> manual deploy/rollback
| |
| +--> Nginx/Certbot routing and TLS
|
+--> Qlockify backup adaptation --> guarded restore --> rehearsal
|
+--> first-server bootstrap --> MVP acceptance
|
v
Prometheus/Grafana phase
|
v
AI may begin
approved spec + Qlockify adaptation contract + explicit build authorization
|
+--> environment examples/validation --> data-service Compose
| | |
| +--> release record/lock +--> API/worker/one-offs
| |
+--> frontend build --------------------------------+
|
Nginx routing --> Certbot/TLS
|
first-start --> deploy --> rollback
| |
+-----------+--> backup/upload
|
guarded restore
|
first-server MVP gate
|
Prometheus --> Grafana
|
AI may begin
```
## Architecture decisions
## Fixed architecture decisions
- Preserve ADR 0001's three independent repositories and ignored nested service checkouts.
- Follow accepted ADR 0002: latest `main`, record three hashes, build manually on the server, store active/previous release records.
- Follow accepted ADR 0003 and frontend commit `64f5ac2`: ordinary frontend context only for font/license.
- Use PostgreSQL 18 and Redis 7 Alpine with explicit supported tags.
- Keep PostgreSQL authoritative and backed up. Redis remains excluded from disaster-recovery backup.
- Separate API, worker, migration, one-time CSV import, and administrative processes.
- Keep Asynq pinned to pre-1.0 `v0.26.0`.
- Use automatic Let's Encrypt/Certbot TLS.
- Treat structured logs/health/manual checks as MVP; add Prometheus/Grafana after MVP and before AI.
- Preserve ADR 0001's three independent repositories and direct ignored `backend/` and `frontend/` checkouts.
- Follow ADR 0002: the operator pulls all repositories interactively before deployment; scripts verify/record but do not pull or reset Git.
- Follow ADR 0003/frontend commit `64f5ac2`: consume the tracked font/license through the normal frontend build context.
- Pin `postgres:18.4-alpine3.24`, `redis:7.4.10-alpine3.21`, `nginx:1.29.8-alpine3.23`, and `certbot/certbot:v5.7.0`.
- Keep PostgreSQL authoritative. Redis uses AOF `everysec`/`noeviction` but remains disposable and excluded from backup.
- Keep API, worker, migrate, importer, and admin commands separate; Asynq stays explicitly pre-1.0 at `v0.26.0`.
- Reuse Qlockify's root/backend/frontend `.env` shape and backup/restore script family with the differences recorded in the adaptation contract.
- Use structured logs and manual operator checks for MVP; add private Prometheus/Grafana only after MVP acceptance and before AI.
## Phase 0: resolve implementation inputs and authorize the build
## Phase 0: planning baseline
- Treat the approved Platform Operations spec and ADRs 0001–0003 as the implementation baseline.
- Obtain the production IP/DNS state, repository URLs, actual protected environment values, ParsPack endpoint/bucket/prefix/credentials, and named operator/secret-escrow contacts.
- Obtain the backend command contract for API, worker, migration, import, admin, liveness, readiness, and Redis-loss recovery.
- Obtain the frontend build/static health contract.
- Obtain explicit authorization before creating deployment implementation artifacts.
- The Platform Operations spec, ADRs 0001–0003, Qlockify adaptation contract, bootstrap runbook, and backup/restore runbook define the approved behavior.
- Runtime values such as IP, DNS access, email, ParsPack credentials, and encryption passphrase are deliberately deferred to the operation that uses them.
- Backend/frontend command and variable names are integrated when their implementation contracts exist; earlier independent deployment work is not blocked.
- Implementation starts only after a separate explicit stakeholder authorization.
Checkpoint: no implementation file exists before explicit build authorization and the inputs needed by its slice are resolved.
Checkpoint: the plan and tasks are approved; no implementation artifact exists yet.
## Phase 1: configuration and Compose foundation
## Phase 1: configuration and runtime foundation
- Add safe tracked configuration examples and validate protected external secret paths.
- Define non-secret active/previous release records containing the three SHAs and health/migration evidence.
- Implement isolated Compose networks, PostgreSQL/Redis volumes/health/resources, backend/worker/frontend services, and one-off migration/import/admin profiles.
- Add focused validation for ports, startup ordering, font build presence, and forbidden tracked/runtime artifacts.
1. Create the deployment-root example and redaction-safe validation for the Qlockify-shaped three-file layout; consume backend/frontend-owned examples without duplicating them.
2. Implement private PostgreSQL/Redis services, named volumes, health checks, authentication, resource limits, and bounded logs.
3. Add API/worker plus separate migration/import/admin profiles from one backend revision.
4. Add the ordinary frontend static build and font/static health checks.
Checkpoint: clean local/server validation proves only Nginx will publish ports and normal startup cannot trigger import/admin.
Checkpoint: Compose renders from placeholder test values; only intended networks exist; ordinary startup cannot invoke migration/import/admin; no secret/font/runtime artifact is tracked.
## Phase 2: Nginx, TLS, and lifecycle
## Phase 2: edge and first-start lifecycle
- Adapt Qlockify Nginx/Certbot structure for `zarinpal.amiirkhl.ir`.
- Implement canonical redirect/default-host rejection, API/frontend separation, exact SSE route, security headers, caching, and health behavior.
- Implement manual deploy/release/rollback from Qlockify's script structure without unconditional destructive Git reset.
- Exercise failed pull/build/migration/readiness/font/worker conditions while preserving the active release.
5. Adapt Qlockify Nginx routing for the single canonical domain, API, frontend, readiness, and exact SSE behavior.
6. Adapt Qlockify Certbot issuance/renewal/reload for the single domain and pinned image.
7. Add guarded first-start orchestration for data readiness, migration, one-time import/reconciliation, and first public activation.
Checkpoint: manual deployment records the three SHAs, passes TLS/health/SSE checks, and a prior-hash rebuild rollback completes within 15 minutes.
Checkpoint: routing/TLS configuration validates, first-start fails closed before public activation, and later ordinary startup never imports.
## Phase 3: backup, restore, and first-server acceptance
## Phase 3: releases and rollback
- Minimally adapt Qlockify backup/upload/restore/S3-restore scripts for ZarinPal paths/services/credentials.
- Remove Qlockify environment-file backup/restore; include PostgreSQL archive plus non-secret manifest/checksums only.
- Add exact destination and production confirmation guards.
- Bootstrap the server with `first-start.sh`: migrate, import/reconcile the pinned archive, then activate public services; perform separate credential setup, encrypted remote backup, and a clean restore rehearsal within two hours.
8. Define the ignored release-record format and exclusive operation lock.
9. Adapt Qlockify's deploy flow to verify manually pulled clean revisions, build/migrate/replace services, observe health, and activate one three-hash release.
10. Implement prior-release checkout/rebuild rollback with migration compatibility guards.
Checkpoint: every MVP acceptance criterion passes through manual health/log/disk/certificate/queue/backup checks.
Checkpoint: induced checkout/build/migration/readiness failures preserve the active stack; a prior application release is recoverable within 15 minutes under normal conditions.
## Phase 4: required observability before AI
## Phase 4: backup and restore
- Add Prometheus, Grafana, and minimum private exporters/instrumentation.
- Add service/release, worker/SSE, PostgreSQL/Redis, host/container, backup/restore, and TLS dashboards.
- Add bounded labels, retention, symptom-based alerts, runbook links, and tested notification delivery.
- Validate the observability acceptance gate before any AI implementation starts.
11. Adapt Qlockify's plain-SQL PostgreSQL backup to emit only database data plus a non-secret manifest/checksums.
12. Adapt its OpenSSL/rclone upload and encrypted 3-local/7-remote retention flow for the ZarinPal ParsPack prefix.
13. Harden local restore with explicit backup, clean-target, compatibility, and production confirmation guards.
14. Adapt S3 download/decrypt to verify an explicit remote object and hand off to the guarded restore.
Checkpoint: an induced failure is detected, correlated to the three-hash release, diagnosed from telemetry, and alerts reach the responsible operator.
Checkpoint: excluded content inspection passes, failed upload/retention cannot delete the last good copy, and a clean rehearsal meets RPO 24 hours/RTO two hours.
## Phase 5: first-server MVP acceptance
15. Follow the bootstrap runbook on the prepared Ubuntu server, fill real environment values, deploy/import once, verify the application, rehearse rollback/restore, and record evidence.
Checkpoint: all MVP criteria pass. The stakeholder explicitly marks MVP ready before monitoring work begins.
## Phase 6: required observability before AI
16. Add private Prometheus and approved exporters/instrumentation with bounded labels, retention, and resource use.
17. Add private Grafana dashboards, release correlation, symptom-based alerts, runbook links, and tested delivery.
Checkpoint: an induced failure is detected and diagnosed from telemetry; the stakeholder approves the observability gate before AI begins.
## Verification strategy
- Documentation/planning: Markdown diff checks, source mapping review, secret-pattern scan, and no implementation files.
- Configuration: placeholder-only rendering, missing/invalid/unknown variable tests, permission/exclusion checks, and no resolved-value output.
- Compose: configuration validation, service health, network/port inspection, resource/log limits, profiles, and restart/Redis-loss behavior.
- Shell operations: ShellCheck plus disposable fixture tests for exact paths, locks, dirty/diverged Git, failure cleanup, retention, and restore guard matrices.
- Edge: `nginx -t`, canonical/unknown-host behavior, security headers, TLS issuance/renewal, static/font behavior, API routing, and streamed SSE checks.
- Release/recovery: induced deployment failures, timed prior-hash rollback, backup inspection, remote verification, and clean-database restore rehearsal.
- MVP/observability: manual acceptance record first; monitoring target/cardinality/dashboard/alert tests only after the MVP gate.
## Risks and mitigations
| Risk | Impact | Mitigation |
|---|---|---|
| Manual deploy selects inconsistent source | Broken stack | One deployment lock, latest three `main` checkouts recorded before build, one release record |
| Dirty checkout is destroyed | Lost operator/user work | Fail on dirty/divergent state; no unconditional `git reset --hard` |
| Server-side build differs over time | Harder reproduction | Pin base/dependency inputs in service repos, record Git hashes/local image ids/build time |
| Migration blocks rollback | Extended outage | Expand/contract, measured locks, fresh backup, no automatic down migration |
| Redis loss drops queue/cache | Partial analytics interruption | AOF `everysec`, PostgreSQL authority, idempotent reconciliation/cold-cache tests |
| SSE buffering/timeouts | Dashboard appears stuck | Exact Nginx route, buffering/compression/cache off, heartbeat/reconnect test |
| Docker-group compromise | Host-root compromise | Restricted key-only access to trusted `ubuntu` account, no daemon TCP exposure |
| Backup leaks credentials | Wider compromise | Remove `.env` backup/restore, encrypt, separate passphrase, archive exclusion tests |
| Backup is unusable | Data loss beyond RTO | Checksums/listing/remote verification plus clean restore rehearsal |
| Delayed monitoring hides degradation | Slower MVP incident diagnosis | MVP health/log/operator checklist; mandatory Prometheus/Grafana before AI |
| Qlockify-specific behavior is copied blindly | Wrong services or unsafe operations | File-by-file adaptation matrix and explicit retain/change/drop review |
| Environment ownership drifts | Secret mismatch or accidental client exposure | Root/backend/frontend ownership contract; one owner per variable; redaction-safe validation |
| Manual releases select inconsistent source | Broken stack | Interactive pulls first, one lock, three clean SHA checks, one release record |
| Server-side builds vary over time | Difficult reproduction | Pin infrastructure/dependency inputs and record three SHAs plus local image ids |
| Migration prevents rollback | Extended outage | Expand/contract sequencing, preflight backup, no automatic down migration |
| Backup contains secrets or is unusable | Disclosure or data loss | PostgreSQL-only package, encryption/checksums, explicit exclusions, clean restore rehearsal |
| Redis loss interrupts analytics | Temporary missing results | AOF `everysec`, PostgreSQL authority, idempotent reconciliation and cold-cache tests |
| 2-GB server exhausts memory | OOM/restarts | Approved per-service limits, 2-GB swap, bounded imports/logs, capacity upgrade when measured |
| Monitoring is postponed too far | Weak diagnosis before AI | Hard MVP-to-observability-to-AI phase gate |
## Parallelization after approval
## Parallel and sequential work
- Safe in parallel: focused Nginx tests, backup adaptation, release-record validation, and later dashboards after shared paths/contracts are fixed.
- Sequential: deployment, migration, production restore, DNS/TLS cutover, one-time import, and MVP-to-observability phase gate.
- Cross-repository coordination: backend commands/health and frontend build/font behavior are defined by their owners before deployment consumes them; no automated pipeline is introduced.
- After environment/paths are fixed, PostgreSQL/Redis foundation, frontend build, Nginx fixtures, and backup fixtures can be prepared in separate focused slices.
- Compose integration, migration/import, release activation, rollback, production restore, DNS/TLS cutover, and phase gates remain sequential.
- Backend/frontend repositories define their application commands and variables; deployment consumes those contracts without editing those repositories in this workstream.
## Completion
- Every authorized task in `tasks/todo.md` passes its acceptance and verification.
- The active stack maps to exact deployment/backend/frontend commits.
- Rollback, Redis-loss recovery, backup, restore, import separation, and exposure checks pass.
- Prometheus/Grafana acceptance passes before AI begins.
- Deployment Git contains no application source, secrets, raw data, credentials, certificates, backups, volumes, or duplicate font/license assets.
- The active stack maps to exact deployment/backend/frontend commits and can roll back safely.
- Only Nginx publishes application ports; secrets/runtime/raw/font copies remain outside deployment Git.
- Encrypted off-host backup and clean restore meet their RPO/RTO.
- Prometheus/Grafana acceptance passes after MVP and before AI.