4 Commits

7 changed files with 67 additions and 23 deletions

View File

@@ -13,6 +13,12 @@ REDIS_PASSWORD=change-me-with-a-different-random-hex-value
# Absolute host path to the immutable first-import archive. # Absolute host path to the immutable first-import archive.
IMPORT_FILE=/srv/zarinpal-data/challenge_data.csv.gz IMPORT_FILE=/srv/zarinpal-data/challenge_data.csv.gz
# Reviewed calendar bootstrap input and non-secret audit identities.
CALENDAR_FILE=/srv/zarinpal-data/reviewed-calendar.json
CALENDAR_CURRENT_JALALI_YEAR=1405
CALENDAR_REVIEWER=change-me-independent-reviewer
CALENDAR_OPERATOR=change-me-platform-operator
# Optional backup settings. These are reserved for the backup/restore scripts. # Optional backup settings. These are reserved for the backup/restore scripts.
S3_BACKUP_BUCKET= S3_BACKUP_BUCKET=
S3_BACKUP_PREFIX=backup/zarinpal-analytics S3_BACKUP_PREFIX=backup/zarinpal-analytics

View File

@@ -16,7 +16,7 @@ zarinpal-analytics-deployment/ # this repository
└── (deployment assets created through the authorized task list) └── (deployment assets created through the authorized task list)
``` ```
The production hostname is `zarinpal.amiirkhl.ir`. Caddy routes `/api/*` and `/healthz` to the Go API and every other path to the static frontend. PostgreSQL, Redis, the worker, and all one-off commands remain on the private Compose data network. The production hostname is `zarinpal.amiirkhl.ir`. Caddy routes `/api/*` and `/healthz` to the Go API and every other path to the static frontend. PostgreSQL, Redis, the worker, and all one-off commands remain on the private Compose data network; the worker additionally joins a dedicated bridge network for outbound AI-provider access without joining the shared Caddy proxy network.
## Required server layout ## Required server layout
@@ -38,11 +38,11 @@ cp frontend/.env.production.example frontend/.env
chmod 600 .env backend/.env frontend/.env chmod 600 .env backend/.env frontend/.env
``` ```
Replace every `change-me` value. Generate URL-safe secrets with `openssl rand -hex 32`. The root file owns the PostgreSQL and Redis passwords; Compose injects their connection settings into the Go containers. The frontend file contains public build-time switches only. Replace every `change-me` value. Generate URL-safe secrets with `openssl rand -hex 32`. The root file owns the PostgreSQL and Redis passwords plus the reviewed-calendar path and non-secret audit identities; Compose injects their connection settings into the Go containers. The frontend file contains public build-time switches only.
## First production start ## First production start
Before adding/reloading the Caddy route, validate and build the stack, import the immutable dataset, create the demo account, and only then start the public upstreams: Before adding/reloading the Caddy route, place the independently reviewed calendar JSON at `CALENDAR_FILE`, make it read-only, and confirm that its `sourceChecksum` is not a review placeholder. Then validate and build the stack, import the immutable dataset, bootstrap the reviewed calendar, create the demo account, and only then start the public upstreams:
```bash ```bash
docker compose config --quiet docker compose config --quiet
@@ -50,6 +50,7 @@ docker compose build backend frontend
docker compose up -d postgres redis docker compose up -d postgres redis
docker compose --profile tools run --rm migrate docker compose --profile tools run --rm migrate
docker compose --profile import run --rm importer docker compose --profile import run --rm importer
docker compose --profile tools run --rm calendar
docker compose --profile tools run --rm admin docker compose --profile tools run --rm admin
docker compose up -d backend worker frontend docker compose up -d backend worker frontend
docker compose ps docker compose ps
@@ -68,18 +69,19 @@ git -C frontend pull --ff-only origin main
bash scripts/deploy.sh bash scripts/deploy.sh
``` ```
Normal deploys run migrations but never run the importer or admin command. Normal deploys run migrations but never run the importer, calendar bootstrap, or admin command.
## Re-running one-off tools ## Re-running one-off tools
`IMPORT_FILE` in the root `.env` is the absolute host path of the immutable `challenge_data.csv.gz`. If recovery requires the idempotent importer or the demo account must be reset, run them explicitly: `IMPORT_FILE` in the root `.env` is the absolute host path of the immutable `challenge_data.csv.gz`; `CALENDAR_FILE` is the absolute path of the reviewed JSON catalog. If recovery requires an idempotent data/catalog bootstrap or the demo account must be reset, run the applicable command explicitly:
```bash ```bash
docker compose --profile import run --rm importer docker compose --profile import run --rm importer
docker compose --profile tools run --rm calendar
docker compose --profile tools run --rm admin docker compose --profile tools run --rm admin
``` ```
Both commands are explicit one-offs. Re-running the importer relies on the backend's checksum-based idempotency guard. All three commands are explicit one-offs. Calendar bootstrap resumes the same reviewed version safely and rejects placeholder checksums, conflicting metadata, retired versions, or replacement of a different active catalog. Future catalog rotation must use the separately reviewed import, approve, and activate workflow.
## Operations ## Operations

View File

@@ -56,10 +56,13 @@ services:
- /bin/sh - /bin/sh
- -ec - -ec
- | - |
umask 077 chown redis:redis /data
printf 'appendonly yes\nappendfsync everysec\nmaxmemory 192mb\nmaxmemory-policy noeviction\nrequirepass %s\n' "$$REDIS_PASSWORD" > /tmp/redis.conf exec /usr/bin/setpriv --reuid redis --regid redis --clear-groups \
chown redis:redis /data /tmp/redis.conf /bin/sh -ec '
exec su-exec redis redis-server /tmp/redis.conf umask 077
printf "appendonly yes\nappendfsync everysec\nmaxmemory 192mb\nmaxmemory-policy noeviction\nrequirepass %s\n" "$$REDIS_PASSWORD" > /data/redis.conf
exec redis-server /data/redis.conf
'
volumes: volumes:
- redis_data:/data - redis_data:/data
networks: networks:
@@ -113,6 +116,10 @@ services:
condition: service_healthy condition: service_healthy
networks: networks:
- data - data
- egress
dns:
- 1.1.1.1
- 8.8.8.8
healthcheck: healthcheck:
test: ["CMD-SHELL", "kill -0 1"] test: ["CMD-SHELL", "kill -0 1"]
interval: 10s interval: 10s
@@ -185,9 +192,35 @@ services:
- data - data
profiles: ["tools"] profiles: ["tools"]
calendar:
<<: *backend-runtime
entrypoint: ["/app/calendar"]
command:
- -action
- bootstrap
- -file
- /calendar/reviewed.json
- -current-jalali-year
- "${CALENDAR_CURRENT_JALALI_YEAR:-0}"
- -reviewer
- "${CALENDAR_REVIEWER:-}"
- -actor
- "${CALENDAR_OPERATOR:-}"
restart: "no"
depends_on:
postgres:
condition: service_healthy
volumes:
- ${CALENDAR_FILE:-/srv/zarinpal-data/reviewed-calendar.json}:/calendar/reviewed.json:ro
networks:
- data
profiles: ["tools"]
networks: networks:
data: data:
internal: true internal: true
egress:
driver: bridge
caddy_proxy: caddy_proxy:
external: true external: true
name: ${CADDY_NETWORK:-caddy_proxy} name: ${CADDY_NETWORK:-caddy_proxy}

View File

@@ -11,7 +11,8 @@ RUN CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/api ./c
&& CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/worker ./cmd/worker \ && CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/worker ./cmd/worker \
&& CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/migrate ./cmd/migrate \ && CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/migrate ./cmd/migrate \
&& CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/importer ./cmd/importer \ && CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/importer ./cmd/importer \
&& CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/admin ./cmd/admin && CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/admin ./cmd/admin \
&& CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/calendar ./cmd/calendar
FROM alpine:3.22 FROM alpine:3.22

View File

@@ -133,6 +133,7 @@ Nginx is the only public application entry point. The worker is event-driven and
| `redis` | Queue, cache, rate-limit, replay coordination | `data` | named AOF volume, excluded from backup | none | | `redis` | Queue, cache, rate-limit, replay coordination | `data` | named AOF volume, excluded from backup | none |
| `migrate` | Versioned schema migration | `data` | none | one-off only | | `migrate` | Versioned schema migration | `data` | none | one-off only |
| `import` | One-time CSV bootstrap/recovery | `data` | exact read-only input bind | one-off profile only | | `import` | One-time CSV bootstrap/recovery | `data` | exact read-only input bind | one-off profile only |
| `calendar` | Reviewed catalog bootstrap with audited state transitions | `data` | reviewed read-only JSON bind | one-off profile only |
| `admin` | Demo credentials and approved maintenance | `data` | explicit output bind if requested | one-off profile only | | `admin` | Demo credentials and approved maintenance | `data` | explicit output bind if requested | one-off profile only |
Post-MVP adds `prometheus`, `grafana`, and approved exporters on a separate private monitoring network. Grafana binds only to loopback/private operator access unless a later ADR approves a protected public route. Post-MVP adds `prometheus`, `grafana`, and approved exporters on a separate private monitoring network. Grafana binds only to loopback/private operator access unless a later ADR approves a protected public route.
@@ -238,7 +239,7 @@ Compose health/dependency conditions prevent common startup races, but the manua
### One-time CSV import ### One-time CSV import
- Import is an explicit operator command/profile, never normal startup, cron, a scheduler, watcher, or deploy trigger. - Import is an explicit operator command/profile, never normal startup, cron, a scheduler, watcher, or deploy trigger.
- On the first server release only, tracked `scripts/first-start.sh` orchestrates PostgreSQL/Redis readiness, the one-off migration, the importer profile, reconciliation, and then public application startup. Later ordinary deploys/restarts never invoke it. - On the first server release only, tracked `scripts/first-start.sh` orchestrates PostgreSQL/Redis readiness, the one-off migration, the importer profile, reconciliation, reviewed calendar bootstrap, and then public application startup. Later ordinary deploys/restarts never invoke it.
- The operator provides an exact resolved path outside Git; only that file is mounted read-only. - The operator provides an exact resolved path outside Git; only that file is mounted read-only.
- Import cannot overlap restore, backup, or another import and starts only after migration succeeds. - Import cannot overlap restore, backup, or another import and starts only after migration succeeds.
- The backend importer pins the approved checksum/schema/filename/counts, enforces resource caps, streams data, reconciles counts, removes temporary raw staging, and records incomplete failure safely according to the approved backend import specifications. - The backend importer pins the approved checksum/schema/filename/counts, enforces resource caps, streams data, reconciles counts, removes temporary raw staging, and records incomplete failure safely according to the approved backend import specifications.

View File

@@ -11,11 +11,11 @@ The production host already runs a dedicated Caddy Compose project that owns por
ZarinPal Analytics connects its frontend and API containers to the existing external `caddy_proxy` Docker network using the stable aliases `zarinpal-frontend` and `zarinpal-backend`. Caddy routes `zarinpal.amiirkhl.ir` to those aliases. This Compose project publishes no host ports and owns no TLS material. ZarinPal Analytics connects its frontend and API containers to the existing external `caddy_proxy` Docker network using the stable aliases `zarinpal-frontend` and `zarinpal-backend`. Caddy routes `zarinpal.amiirkhl.ir` to those aliases. This Compose project publishes no host ports and owns no TLS material.
PostgreSQL, Redis, the worker, migration, importer, and admin services remain on an internal data network. Only the API additionally joins `caddy_proxy`; the frontend joins only `caddy_proxy`. Caddy preserves the complete `/api` path and disables response buffering for proxied API traffic so authenticated SSE streams are delivered promptly. PostgreSQL, Redis, the worker, migration, importer, and admin services remain on an internal data network. The worker additionally joins a dedicated bridge egress network so approved external AI-provider calls and DNS resolution can succeed without attaching it to the shared proxy network. Only the API additionally joins `caddy_proxy`; the frontend joins only `caddy_proxy`. Caddy preserves the complete `/api` path and disables response buffering for proxied API traffic so authenticated SSE streams are delivered promptly.
## Consequences ## Consequences
- The Caddy deployment must exist first and create the external `caddy_proxy` network. - The Caddy deployment must exist first and create the external `caddy_proxy` network.
- Caddy configuration and this deployment must be released together for first activation. - Caddy configuration and this deployment must be released together for first activation.
- Nginx/Certbot-specific requirements in the earlier platform documents are superseded by this ADR. Their routing, TLS, and port-ownership intent is retained and implemented by the shared Caddy service. - Nginx/Certbot-specific requirements in the earlier platform documents are superseded by this ADR. Their routing, TLS, and port-ownership intent is retained and implemented by the shared Caddy service.
- The shared proxy network is a trusted infrastructure boundary; no database, Redis, worker, or administrative service is attached to it. - The shared proxy network is a trusted infrastructure boundary; no database, Redis, worker, or administrative service is attached to it. Worker internet access is isolated through the dedicated `egress` network.

View File

@@ -95,15 +95,16 @@ If issuance fails, stop repeated production requests, inspect DNS/CAA/firewall/c
1. Acquire the production deployment lock and record the operator/release id. 1. Acquire the production deployment lock and record the operator/release id.
2. Place the approved `challenge_data.csv.gz` at the exact Git-ignored deployment `data/incoming/` path. Verify regular-file/read-only properties, size, and pinned SHA-256; do not copy it into an image. 2. Place the approved `challenge_data.csv.gz` at the exact Git-ignored deployment `data/incoming/` path. Verify regular-file/read-only properties, size, and pinned SHA-256; do not copy it into an image.
3. Invoke tracked `scripts/first-start.sh`. It starts PostgreSQL and Redis, waits with bounded backoff, applies the one-off migration, runs the one-shot importer with a read-only mount, and reconciles the pinned counts before starting public services. 3. Place the independently reviewed calendar JSON at the absolute `CALENDAR_FILE` path. Verify it is a regular read-only file, record its checksum and reviewer, and reject candidate data whose `sourceChecksum` still contains a review placeholder.
4. Review the JSON/Markdown import report and durable import-run/issues metadata. Fatal validation, resource, migration, or reconciliation failure must exit nonzero before backend/frontend/Nginx activation. 4. Invoke tracked `scripts/first-start.sh`. It starts PostgreSQL and Redis, waits with bounded backoff, applies the one-off migration, runs the one-shot importer and reconciliation, bootstraps the reviewed calendar through its audited `DRAFT`/`APPROVED`/`ACTIVE` states, and only then starts public services.
5. Verify backend and worker liveness/readiness, worker heartbeat, Asynq `v0.26.0` compatibility, queue limits, and absence of unintended published ports. 5. Review the JSON/Markdown import report, durable import-run/issues metadata, active calendar version, and calendar audit records. Fatal validation, resource, migration, import, reconciliation, calendar, or scheduling failure must exit nonzero before backend/frontend/Nginx activation.
6. Verify the frontend static sentinel plus the emitted Iran Yekan font resource. The source font/license remain frontend-owned; no deployment-side injection or copy occurs. 6. Verify backend and worker liveness/readiness, worker heartbeat, Asynq `v0.26.0` compatibility, queue limits, and absence of unintended published ports.
7. Verify Nginx public generic health, canonical redirect, security headers, login page, SSE behavior, and that only 80/443 are public. 7. Verify the frontend static sentinel plus the emitted Iran Yekan font resource. The source font/license remain frontend-owned; no deployment-side injection or copy occurs.
8. Run demo credential generation as a separate explicit administrative job. Store one-time plaintext output outside Git/logs/backups. 8. Verify Nginx public generic health, canonical redirect, security headers, login page, SSE behavior, and that only 80/443 are public.
9. Authenticate as at least two merchant accounts, verify different dashboards and cross-tenant isolation, then remove or escrow plaintext credential output. 9. Run demo credential generation as a separate explicit administrative job. Store one-time plaintext output outside Git/logs/backups.
10. Verify MVP operator checks: Compose health/restarts, structured logs, disk, certificate, queue/Redis state, and backup destination access. 10. Authenticate as at least two merchant accounts, verify different dashboards and cross-tenant isolation, then remove or escrow plaintext credential output.
11. Record the active three-hash release only after all postflight checks pass. 11. Verify MVP operator checks: Compose health/restarts, structured logs, disk, certificate, queue/Redis state, and backup destination access.
12. Record the active three-hash release only after all postflight checks pass.
Failure before the active-release switch restores the prior/empty known state. Do not improvise a database down migration. Failure before the active-release switch restores the prior/empty known state. Do not improvise a database down migration.