zarinpal-analytics-backend/README.md

ZarinPal Merchant Analytics — Backend

Go backend for merchant authentication, one-time payment-data ingestion, deterministic analytics, Redis-coordinated background jobs, and optional AI-assisted insights.

What runs in this repository

Command	Purpose	Run mode
go run ./cmd/migrate	Apply versioned PostgreSQL migrations	One-off, before the first start and after updates
go run ./cmd/importer	Import the immutable challenge_data.csv.gz dataset	One-off bootstrap/recovery command
go run ./cmd/admin	Create or reset the demo login from DEMO_USERNAME and DEMO_PASSWORD	One-off, after importing data
go run ./cmd/api	Start the HTTP API on HTTP_ADDR	Long-running
go run ./cmd/worker	Process queued growth reports, page insights, and growth-coach jobs	Long-running alongside the API
go run ./cmd/scheduler	Enqueue due growth-coach work when AI_COACH_ENABLED=true	Optional long-running process
go run ./cmd/calendar --help	Import, approve, or activate reviewed calendar catalogs	Administrative/optional
go run ./cmd/admin-reset-ai	Reset AI-related administrative state	Administrative/recovery only

The importer is checksum-idempotent: importing the same archive again is a safe no-op. It is not a file watcher or scheduled ETL process. The worker does not replace the API; both processes should run at the same time.

Local setup

Prerequisites

• Go 1.24 or newer
• PostgreSQL 18 (a compatible recent PostgreSQL release may also work)
• Redis 7.4
• Git
• The source dataset as a gzip-compressed CSV

PostgreSQL stores application and analytics data. Redis is disposable coordination/cache state used by the API and worker. Both services must be reachable before starting the backend.

1. Start PostgreSQL and Redis

Use locally installed services, or start development-only containers:

docker run --name zpa-postgres \
  -e POSTGRES_USER=postgres \
  -e POSTGRES_PASSWORD=change-me \
  -e POSTGRES_DB=zarinpal \
  -p 127.0.0.1:5432:5432 \
  -d postgres:18.4-alpine3.24

docker run --name zpa-redis \
  -p 127.0.0.1:6379:6379 \
  -d redis:7.4.10-alpine3.21 redis-server --appendonly yes

On later runs, restart them with:

docker start zpa-postgres zpa-redis

These ports bind to localhost only. Do not expose PostgreSQL or Redis directly to the internet.

2. Configure the backend

From the repository root:

cp .env.example .env

PowerShell equivalent:

Copy-Item .env.example .env

Edit .env and replace all placeholder values. For the documented local demo login, use:

DATABASE_URL=postgresql://postgres:change-me@localhost:5432/zarinpal?sslmode=disable
REDIS_ADDR=localhost:6379
REDIS_PASSWORD=
DEMO_USERNAME=merchant-0001
DEMO_PASSWORD=merchant-demo-2026
SESSION_SECRET=<at-least-32-random-characters>
CURSOR_SECRET=<a-different-at-least-32-character-secret>
FRONTEND_ORIGIN=http://127.0.0.1:5173
COOKIE_SECURE=false

Generate independent secrets, for example with openssl rand -hex 32. The Go commands read process environment variables; they do not automatically load .env.

Load .env in Bash before running commands:

set -a
source .env
set +a

For PowerShell, load the file into the current terminal:

Get-Content .env | ForEach-Object {
  if ($_ -match '^\s*([^#][^=]*)=(.*)$') {
    [Environment]::SetEnvironmentVariable($matches[1].Trim(), $matches[2], 'Process')
  }
}

Keep .env private and uncommitted. AI features are optional and disabled in the example. If enabled, the provider key belongs only in the backend environment—never in the frontend.

3. Add the dataset

Place the archive at the configured IMPORT_PATH, which defaults to:

data/incoming/challenge_data.csv.gz

The dataset directory is Git-ignored. Do not commit the archive, database dumps, import output, or credentials.

4. Prepare the database and demo user

Run these commands from the repository root and in this order:

go mod download
go run ./cmd/migrate
go run ./cmd/importer
go run ./cmd/admin

cmd/admin idempotently creates or resets the configured demo user and attaches it to the imported merchant with the largest session population. Passwords are stored as Argon2id hashes and are not printed by the command.

Local demo credentials:

Username: merchant-0001
Password: merchant-demo-2026

These credentials work only when .env contains the same values before cmd/admin runs. They are public development credentials; change them for every internet-accessible deployment.

5. Start the worker and API

Open two terminals, load .env in both, and run:

# Terminal 1: background jobs
go run ./cmd/worker

# Terminal 2: HTTP API
go run ./cmd/api

The API is available at http://127.0.0.1:8080. Start the frontend separately at http://127.0.0.1:5173; its Vite server proxies /api to this API.

If the growth-coach scheduler is enabled, run it in a third terminal:

go run ./cmd/scheduler

Verification and common commands

curl http://127.0.0.1:8080/healthz
go test ./...
go vet ./...
go build ./cmd/...

The equivalent Make targets are make migrate, make import, make seed, make worker, make api, make test, make vet, and make build.

Production deployment on a remote server

Deploy the backend and frontend together through the zarinpal-analytics-deployment repository. Its Compose stack runs PostgreSQL, Redis, the API, worker, one-off tools, and the static frontend. A separate shared Caddy instance owns ports 80/443, TLS, and domain routing.

The examples below use analytics.example.com; replace it with your domain. Replace repository URL placeholders with your own template/fork URLs.

1. Prepare the server and DNS access

1. Provision an Ubuntu server with a public IP and enough disk/RAM for the dataset and PostgreSQL.
2. Install Git, Docker Engine, and the Docker Compose plugin.
3. Allow SSH only from trusted addresses and allow inbound TCP 80 and 443.
4. Do not publish PostgreSQL port 5432 or Redis port 6379.

2. Install the shared Caddy router

Use the Caddy deployment repository to install the main router:

git clone https://git.amiirkhl.ir/Interanet/caddy-deployment.git ~/caddy-deployment
cd ~/caddy-deployment

Follow that repository's README to start Caddy. Ensure its Compose configuration creates/uses an external Docker network named caddy_proxy and attaches the Caddy container to it. If the repository does not create the network, create it once with docker network create caddy_proxy before starting Caddy.

3. Clone the three application repositories

The deployment script expects the backend and frontend as independent nested checkouts:

git clone <deployment-repository-url> ~/zarinpal-analytics-deployment
cd ~/zarinpal-analytics-deployment
git clone <backend-repository-url> backend
git clone <frontend-repository-url> frontend

All three repositories must be on a clean main branch matching origin/main.

4. Create production environment files

cp .env.example .env
cp backend/.env.production.example backend/.env
cp frontend/.env.production.example frontend/.env
chmod 600 .env backend/.env frontend/.env

Then:

• Replace every change-me value.
• Set FRONTEND_ORIGIN=https://analytics.example.com in backend/.env.
• Keep COOKIE_SECURE=true.
• Generate different PostgreSQL, Redis, session, and cursor secrets with openssl rand -hex 32.
• Set a new production DEMO_PASSWORD; do not reuse merchant-demo-2026 on a public server.
• Leave AI flags off unless a backend-only provider key, model, and positive budgets are configured.
• Put no secret in frontend/.env; all PUBLIC_* values are embedded in the browser build.

5. Place the import archive

Copy the immutable archive to a private host path and make IMPORT_FILE in the deployment .env point to it:

sudo install -d -m 750 -o "$USER" -g "$USER" /srv/zarinpal-data
cp /path/to/challenge_data.csv.gz /srv/zarinpal-data/challenge_data.csv.gz
chmod 440 /srv/zarinpal-data/challenge_data.csv.gz

6. Perform the first application start

From ~/zarinpal-analytics-deployment:

docker compose config --quiet
docker compose build backend frontend
docker compose up -d postgres redis
docker compose --profile tools run --rm migrate
docker compose --profile import run --rm importer
docker compose --profile tools run --rm admin
docker compose up -d backend worker frontend
docker compose ps

Wait until postgres, redis, backend, worker, and frontend are healthy. Inspect failures with:

docker compose logs --tail=200 postgres redis backend worker frontend

7. Connect the domain through Caddy

Create an A record for analytics.example.com pointing to the server's public IPv4 address. Add an AAAA record only when IPv6 is configured and allowed through the firewall.

Add a site to the Caddy deployment's Caddyfile (or its equivalent templated configuration):

analytics.example.com {
    @backend path /api/* /healthz

    handle @backend {
        reverse_proxy zarinpal-backend:8080 {
            flush_interval -1
        }
    }

    handle {
        reverse_proxy zarinpal-frontend:8080
    }
}

Validate and reload Caddy using the commands documented by the Caddy repository. Caddy will obtain and renew HTTPS certificates automatically after public DNS resolves and ports 80/443 reach the server.

Verify:

curl -I https://analytics.example.com/
curl https://analytics.example.com/healthz

Then sign in with the production values of DEMO_USERNAME and DEMO_PASSWORD and verify that dashboard requests complete and the worker logs show no repeated failures.

8. Deploy later updates

cd ~/zarinpal-analytics-deployment
git pull --ff-only origin main
git -C backend pull --ff-only origin main
git -C frontend pull --ff-only origin main
bash scripts/deploy.sh

Normal updates apply migrations and restart the application, but do not rerun the importer or reset the demo password. Run those one-off profiles explicitly only when intended.

Never use docker compose down -v in production: -v deletes the PostgreSQL and Redis volumes. Back up PostgreSQL and rehearse restoration before treating the deployment as production-ready.

API and architecture documentation

• docs/openapi.yaml is the OpenAPI 3.1 contract used to generate frontend types.
• docs/API-SPEC.md documents authentication, tenant isolation, SSE, pagination, and exports.
• docs/IMPORT-WORKFLOW-SPEC.md documents the import boundary.
• docs/METRIC-CATALOGUE.md defines analytics metrics.