Compare commits

4 Commits

7 changed files with 67 additions and 23 deletions

View File

@@ -13,6 +13,12 @@ REDIS_PASSWORD=change-me-with-a-different-random-hex-value
# Absolute host path to the immutable first-import archive.
IMPORT_FILE=/srv/zarinpal-data/challenge_data.csv.gz
# Reviewed calendar bootstrap input and non-secret audit identities.
CALENDAR_FILE=/srv/zarinpal-data/reviewed-calendar.json
CALENDAR_CURRENT_JALALI_YEAR=1405
CALENDAR_REVIEWER=change-me-independent-reviewer
CALENDAR_OPERATOR=change-me-platform-operator
# Optional backup settings. These are reserved for the backup/restore scripts.
S3_BACKUP_BUCKET=
S3_BACKUP_PREFIX=backup/zarinpal-analytics

View File

@@ -16,7 +16,7 @@ zarinpal-analytics-deployment/ # this repository
└── (deployment assets created through the authorized task list)
```
The production hostname is `zarinpal.amiirkhl.ir`. Caddy routes `/api/*` and `/healthz` to the Go API and every other path to the static frontend. PostgreSQL, Redis, the worker, and all one-off commands remain on the private Compose data network.
The production hostname is `zarinpal.amiirkhl.ir`. Caddy routes `/api/*` and `/healthz` to the Go API and every other path to the static frontend. PostgreSQL, Redis, the worker, and all one-off commands remain on the private Compose data network; the worker additionally joins a dedicated bridge network for outbound AI-provider access without joining the shared Caddy proxy network.
## Required server layout
@@ -38,11 +38,11 @@ cp frontend/.env.production.example frontend/.env
chmod 600 .env backend/.env frontend/.env
```
Replace every `change-me` value. Generate URL-safe secrets with `openssl rand -hex 32`. The root file owns the PostgreSQL and Redis passwords; Compose injects their connection settings into the Go containers. The frontend file contains public build-time switches only.
Replace every `change-me` value. Generate URL-safe secrets with `openssl rand -hex 32`. The root file owns the PostgreSQL and Redis passwords plus the reviewed-calendar path and non-secret audit identities; Compose injects their connection settings into the Go containers. The frontend file contains public build-time switches only.
## First production start
Before adding/reloading the Caddy route, validate and build the stack, import the immutable dataset, create the demo account, and only then start the public upstreams:
Before adding/reloading the Caddy route, place the independently reviewed calendar JSON at `CALENDAR_FILE`, make it read-only, and confirm that its `sourceChecksum` is not a review placeholder. Then validate and build the stack, import the immutable dataset, bootstrap the reviewed calendar, create the demo account, and only then start the public upstreams:
```bash
docker compose config --quiet
@@ -50,6 +50,7 @@ docker compose build backend frontend
docker compose up -d postgres redis
docker compose --profile tools run --rm migrate
docker compose --profile import run --rm importer
docker compose --profile tools run --rm calendar
docker compose --profile tools run --rm admin
docker compose up -d backend worker frontend
docker compose ps
@@ -68,18 +69,19 @@ git -C frontend pull --ff-only origin main
bash scripts/deploy.sh
```
Normal deploys run migrations but never run the importer or admin command.
Normal deploys run migrations but never run the importer, calendar bootstrap, or admin command.
## Re-running one-off tools
`IMPORT_FILE` in the root `.env` is the absolute host path of the immutable `challenge_data.csv.gz`. If recovery requires the idempotent importer or the demo account must be reset, run them explicitly:
`IMPORT_FILE` in the root `.env` is the absolute host path of the immutable `challenge_data.csv.gz`; `CALENDAR_FILE` is the absolute path of the reviewed JSON catalog. If recovery requires an idempotent data/catalog bootstrap or the demo account must be reset, run the applicable command explicitly:
```bash
docker compose --profile import run --rm importer
docker compose --profile tools run --rm calendar
docker compose --profile tools run --rm admin
```
Both commands are explicit one-offs. Re-running the importer relies on the backend's checksum-based idempotency guard.
All three commands are explicit one-offs. Calendar bootstrap resumes the same reviewed version safely and rejects placeholder checksums, conflicting metadata, retired versions, or replacement of a different active catalog. Future catalog rotation must use the separately reviewed import, approve, and activate workflow.
## Operations

View File

@@ -56,10 +56,13 @@ services:
- /bin/sh
- -ec
- |
chown redis:redis /data
exec /usr/bin/setpriv --reuid redis --regid redis --clear-groups \
/bin/sh -ec '
umask 077
printf 'appendonly yes\nappendfsync everysec\nmaxmemory 192mb\nmaxmemory-policy noeviction\nrequirepass %s\n' "$$REDIS_PASSWORD" > /tmp/redis.conf
chown redis:redis /data /tmp/redis.conf
exec su-exec redis redis-server /tmp/redis.conf
printf "appendonly yes\nappendfsync everysec\nmaxmemory 192mb\nmaxmemory-policy noeviction\nrequirepass %s\n" "$$REDIS_PASSWORD" > /data/redis.conf
exec redis-server /data/redis.conf
'
volumes:
- redis_data:/data
networks:
@@ -113,6 +116,10 @@ services:
condition: service_healthy
networks:
- data
- egress
dns:
- 1.1.1.1
- 8.8.8.8
healthcheck:
test: ["CMD-SHELL", "kill -0 1"]
interval: 10s
@@ -185,9 +192,35 @@ services:
- data
profiles: ["tools"]
calendar:
<<: *backend-runtime
entrypoint: ["/app/calendar"]
command:
- -action
- bootstrap
- -file
- /calendar/reviewed.json
- -current-jalali-year
- "${CALENDAR_CURRENT_JALALI_YEAR:-0}"
- -reviewer
- "${CALENDAR_REVIEWER:-}"
- -actor
- "${CALENDAR_OPERATOR:-}"
restart: "no"
depends_on:
postgres:
condition: service_healthy
volumes:
- ${CALENDAR_FILE:-/srv/zarinpal-data/reviewed-calendar.json}:/calendar/reviewed.json:ro
networks:
- data
profiles: ["tools"]
networks:
data:
internal: true
egress:
driver: bridge
caddy_proxy:
external: true
name: ${CADDY_NETWORK:-caddy_proxy}

View File

@@ -11,7 +11,8 @@ RUN CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/api ./c
&& CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/worker ./cmd/worker \
&& CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/migrate ./cmd/migrate \
&& CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/importer ./cmd/importer \
&& CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/admin ./cmd/admin
&& CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/admin ./cmd/admin \
&& CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/calendar ./cmd/calendar
FROM alpine:3.22

View File

@@ -133,6 +133,7 @@ Nginx is the only public application entry point. The worker is event-driven and
| `redis` | Queue, cache, rate-limit, replay coordination | `data` | named AOF volume, excluded from backup | none |
| `migrate` | Versioned schema migration | `data` | none | one-off only |
| `import` | One-time CSV bootstrap/recovery | `data` | exact read-only input bind | one-off profile only |
| `calendar` | Reviewed catalog bootstrap with audited state transitions | `data` | reviewed read-only JSON bind | one-off profile only |
| `admin` | Demo credentials and approved maintenance | `data` | explicit output bind if requested | one-off profile only |
Post-MVP adds `prometheus`, `grafana`, and approved exporters on a separate private monitoring network. Grafana binds only to loopback/private operator access unless a later ADR approves a protected public route.
@@ -238,7 +239,7 @@ Compose health/dependency conditions prevent common startup races, but the manua
### One-time CSV import
- Import is an explicit operator command/profile, never normal startup, cron, a scheduler, watcher, or deploy trigger.
- On the first server release only, tracked `scripts/first-start.sh` orchestrates PostgreSQL/Redis readiness, the one-off migration, the importer profile, reconciliation, and then public application startup. Later ordinary deploys/restarts never invoke it.
- On the first server release only, tracked `scripts/first-start.sh` orchestrates PostgreSQL/Redis readiness, the one-off migration, the importer profile, reconciliation, reviewed calendar bootstrap, and then public application startup. Later ordinary deploys/restarts never invoke it.
- The operator provides an exact resolved path outside Git; only that file is mounted read-only.
- Import cannot overlap restore, backup, or another import and starts only after migration succeeds.
- The backend importer pins the approved checksum/schema/filename/counts, enforces resource caps, streams data, reconciles counts, removes temporary raw staging, and records incomplete failure safely according to the approved backend import specifications.

View File

@@ -11,11 +11,11 @@ The production host already runs a dedicated Caddy Compose project that owns por
ZarinPal Analytics connects its frontend and API containers to the existing external `caddy_proxy` Docker network using the stable aliases `zarinpal-frontend` and `zarinpal-backend`. Caddy routes `zarinpal.amiirkhl.ir` to those aliases. This Compose project publishes no host ports and owns no TLS material.
PostgreSQL, Redis, the worker, migration, importer, and admin services remain on an internal data network. Only the API additionally joins `caddy_proxy`; the frontend joins only `caddy_proxy`. Caddy preserves the complete `/api` path and disables response buffering for proxied API traffic so authenticated SSE streams are delivered promptly.
PostgreSQL, Redis, the worker, migration, importer, and admin services remain on an internal data network. The worker additionally joins a dedicated bridge egress network so approved external AI-provider calls and DNS resolution can succeed without attaching it to the shared proxy network. Only the API additionally joins `caddy_proxy`; the frontend joins only `caddy_proxy`. Caddy preserves the complete `/api` path and disables response buffering for proxied API traffic so authenticated SSE streams are delivered promptly.
## Consequences
- The Caddy deployment must exist first and create the external `caddy_proxy` network.
- Caddy configuration and this deployment must be released together for first activation.
- Nginx/Certbot-specific requirements in the earlier platform documents are superseded by this ADR. Their routing, TLS, and port-ownership intent is retained and implemented by the shared Caddy service.
- The shared proxy network is a trusted infrastructure boundary; no database, Redis, worker, or administrative service is attached to it.
- The shared proxy network is a trusted infrastructure boundary; no database, Redis, worker, or administrative service is attached to it. Worker internet access is isolated through the dedicated `egress` network.

View File

@@ -95,15 +95,16 @@ If issuance fails, stop repeated production requests, inspect DNS/CAA/firewall/c
1. Acquire the production deployment lock and record the operator/release id.
2. Place the approved `challenge_data.csv.gz` at the exact Git-ignored deployment `data/incoming/` path. Verify regular-file/read-only properties, size, and pinned SHA-256; do not copy it into an image.
3. Invoke tracked `scripts/first-start.sh`. It starts PostgreSQL and Redis, waits with bounded backoff, applies the one-off migration, runs the one-shot importer with a read-only mount, and reconciles the pinned counts before starting public services.
4. Review the JSON/Markdown import report and durable import-run/issues metadata. Fatal validation, resource, migration, or reconciliation failure must exit nonzero before backend/frontend/Nginx activation.
5. Verify backend and worker liveness/readiness, worker heartbeat, Asynq `v0.26.0` compatibility, queue limits, and absence of unintended published ports.
6. Verify the frontend static sentinel plus the emitted Iran Yekan font resource. The source font/license remain frontend-owned; no deployment-side injection or copy occurs.
7. Verify Nginx public generic health, canonical redirect, security headers, login page, SSE behavior, and that only 80/443 are public.
8. Run demo credential generation as a separate explicit administrative job. Store one-time plaintext output outside Git/logs/backups.
9. Authenticate as at least two merchant accounts, verify different dashboards and cross-tenant isolation, then remove or escrow plaintext credential output.
10. Verify MVP operator checks: Compose health/restarts, structured logs, disk, certificate, queue/Redis state, and backup destination access.
11. Record the active three-hash release only after all postflight checks pass.
3. Place the independently reviewed calendar JSON at the absolute `CALENDAR_FILE` path. Verify it is a regular read-only file, record its checksum and reviewer, and reject candidate data whose `sourceChecksum` still contains a review placeholder.
4. Invoke tracked `scripts/first-start.sh`. It starts PostgreSQL and Redis, waits with bounded backoff, applies the one-off migration, runs the one-shot importer and reconciliation, bootstraps the reviewed calendar through its audited `DRAFT`/`APPROVED`/`ACTIVE` states, and only then starts public services.
5. Review the JSON/Markdown import report, durable import-run/issues metadata, active calendar version, and calendar audit records. Fatal validation, resource, migration, import, reconciliation, calendar, or scheduling failure must exit nonzero before backend/frontend/Nginx activation.
6. Verify backend and worker liveness/readiness, worker heartbeat, Asynq `v0.26.0` compatibility, queue limits, and absence of unintended published ports.
7. Verify the frontend static sentinel plus the emitted Iran Yekan font resource. The source font/license remain frontend-owned; no deployment-side injection or copy occurs.
8. Verify Nginx public generic health, canonical redirect, security headers, login page, SSE behavior, and that only 80/443 are public.
9. Run demo credential generation as a separate explicit administrative job. Store one-time plaintext output outside Git/logs/backups.
10. Authenticate as at least two merchant accounts, verify different dashboards and cross-tenant isolation, then remove or escrow plaintext credential output.
11. Verify MVP operator checks: Compose health/restarts, structured logs, disk, certificate, queue/Redis state, and backup destination access.
12. Record the active three-hash release only after all postflight checks pass.
Failure before the active-release switch restores the prior/empty known state. Do not improvise a database down migration.