feat: add idempotent calendar bootstrap

Bootstrap independently reviewed catalogs through audited import, approval, and initial activation transitions, then resume insight scheduling safely.
docs: document setup and deployment
2026-08-23 02:24:16 +08:00 · 2026-08-23 01:40:45 +08:00
7 changed files with 957 additions and 80 deletions
--- a/README.md
+++ b/README.md
@@ -1,54 +1,321 @@
 # ZarinPal Merchant Analytics — Backend

-Go backend for merchant identity, dataset ingestion, deterministic analytics, authenticated APIs, and the bounded AvalAI-assisted growth analyst.
+Go backend for merchant authentication, one-time payment-data ingestion, deterministic analytics, Redis-coordinated background jobs, and optional AI-assisted insights.

-Current state: the local MVP implements database migration, streaming dataset import,
-secure demo identity, deterministic dashboard analytics, merchant-scoped details/CSV,
-the authenticated HTTP API, and a Redis-connected worker boundary.
+## What runs in this repository

-## Local development
+| Command                        | Purpose                                                                 | Run mode                                          |
+| ------------------------------ | ----------------------------------------------------------------------- | ------------------------------------------------- |
+| `go run ./cmd/migrate`         | Apply versioned PostgreSQL migrations                                   | One-off, before the first start and after updates |
+| `go run ./cmd/importer`        | Import the immutable `challenge_data.csv.gz` dataset                    | One-off bootstrap/recovery command                |
+| `go run ./cmd/admin`           | Create or reset the demo login from `DEMO_USERNAME` and `DEMO_PASSWORD` | One-off, after importing data                     |
+| `go run ./cmd/api`             | Start the HTTP API on `HTTP_ADDR`                                       | Long-running                                      |
+| `go run ./cmd/worker`          | Process queued growth reports, page insights, and growth-coach jobs     | Long-running alongside the API                    |
+| `go run ./cmd/scheduler`       | Enqueue due growth-coach work when `AI_COACH_ENABLED=true`              | Optional long-running process                     |
+| `go run ./cmd/calendar --help` | Bootstrap or rotate independently reviewed calendar catalogs           | Administrative/optional                           |
+| `go run ./cmd/admin-reset-ai`  | Reset AI-related administrative state                                   | Administrative/recovery only                      |

-Copy `.env.example` to the ignored `.env`, load its variables in your shell, and run:
+The importer is checksum-idempotent: importing the same archive again is a safe no-op. It is not a file watcher or scheduled ETL process. The worker does not replace the API; both processes should run at the same time.

-```text
-go run ./cmd/migrate
-go run ./cmd/importer
-go run ./cmd/admin
-go run ./cmd/worker
-go run ./cmd/api
+## Local setup
+
+### Prerequisites
+
+- Go 1.24 or newer
+- PostgreSQL 18 (a compatible recent PostgreSQL release may also work)
+- Redis 7.4
+- Git
+- The source dataset as a gzip-compressed CSV
+
+PostgreSQL stores application and analytics data. Redis is disposable coordination/cache state used by the API and worker. Both services must be reachable before starting the backend.
+
+### 1. Start PostgreSQL and Redis
+
+Use locally installed services, or start development-only containers:
+
+```bash
+docker run --name zpa-postgres \
+  -e POSTGRES_USER=postgres \
+  -e POSTGRES_PASSWORD=change-me \
+  -e POSTGRES_DB=zarinpal \
+  -p 127.0.0.1:5432:5432 \
+  -d postgres:18.4-alpine3.24
+
+docker run --name zpa-redis \
+  -p 127.0.0.1:6379:6379 \
+  -d redis:7.4.10-alpine3.21 redis-server --appendonly yes
 ```

-`cmd/admin` idempotently creates or resets `DEMO_USERNAME` using `DEMO_PASSWORD`
-and assigns it to the imported merchant with the largest session population. It
-never prints the password or stores it outside the Argon2id hash. The local API
-uses the `zpa_session` HttpOnly cookie when `COOKIE_SECURE=false`; secure
-deployments use the contract's `__Host-zpa_session` cookie.
+On later runs, restart them with:

-## Local dataset input
+```bash
+docker start zpa-postgres zpa-redis
+```

-Place the provided archive at:
+These ports bind to localhost only. Do not expose PostgreSQL or Redis directly to the internet.
+
+### 2. Configure the backend
+
+From the repository root:
+
+```bash
+cp .env.example .env
+```
+
+PowerShell equivalent:
+
+```powershell
+Copy-Item .env.example .env
+```
+
+Edit `.env` and replace all placeholder values. For the documented local demo login, use:
+
+```dotenv
+DATABASE_URL=postgresql://postgres:change-me@localhost:5432/zarinpal?sslmode=disable
+REDIS_ADDR=localhost:6379
+REDIS_PASSWORD=
+DEMO_USERNAME=merchant-0001
+DEMO_PASSWORD=merchant-demo-2026
+SESSION_SECRET=<at-least-32-random-characters>
+CURSOR_SECRET=<a-different-at-least-32-character-secret>
+FRONTEND_ORIGIN=http://127.0.0.1:5173
+COOKIE_SECURE=false
+```
+
+Generate independent secrets, for example with `openssl rand -hex 32`. The Go commands read process environment variables; they do not automatically load `.env`.
+
+Load `.env` in Bash before running commands:
+
+```bash
+set -a
+source .env
+set +a
+```
+
+For PowerShell, load the file into the current terminal:
+
+```powershell
+Get-Content .env | ForEach-Object {
+  if ($_ -match '^\s*([^#][^=]*)=(.*)$') {
+    [Environment]::SetEnvironmentVariable($matches[1].Trim(), $matches[2], 'Process')
+  }
+}
+```
+
+Keep `.env` private and uncommitted. AI features are optional and disabled in the example. If enabled, the provider key belongs only in the backend environment—never in the frontend.
+
+### 3. Add the dataset
+
+Place the archive at the configured `IMPORT_PATH`, which defaults to:

 ```text
 data/incoming/challenge_data.csv.gz
 ```

-The entire `data/incoming/` payload is ignored by Git. The one-time bootstrap importer will read from this location (or an explicitly supplied path), stream the compressed CSV, validate it, and write an import manifest containing the source checksum, schema version, row counts, quality results, and importer revision. It is an administrative bootstrap/recovery command, not a scheduled ingestion job. The hackathon MVP does not mutate payment facts after this import; any future ZarinPal live-write integration is outside this project's current scope. The raw file and resulting PostgreSQL data are not committed.
+The dataset directory is Git-ignored. Do not commit the archive, database dumps, import output, or credentials.

-The proposed first-server Docker flow and Bash entrypoint are specified in [`docs/IMPORT-WORKFLOW-SPEC.md`](docs/IMPORT-WORKFLOW-SPEC.md). Implementation is authorized and must follow the dependency order in [`tasks/todo.md`](tasks/todo.md).
+### 4. Prepare the database and demo user

-## Planned runnable boundaries
+Run these commands from the repository root and in this order:

-One repository may expose several small `cmd/` entry points while sharing domain packages under `internal/`:
+```bash
+go mod download
+go run ./cmd/migrate
+go run ./cmd/importer
+```

- HTTP API
- dataset importer
- database migration command
- HTTP API with authenticated SSE analytics delivery
- on-demand analytics worker that calculates dashboard sections in parallel and caches versioned results in Redis
+When calendar-dependent growth features are enabled, bootstrap an independently reviewed external catalog after the importer and before `cmd/admin`/application startup:

-The approved package-layout and manual-constructor decisions are implemented through the scaffold and provider tasks.
+```bash
+go run ./cmd/calendar -action bootstrap -file /path/to/reviewed-calendar.json -current-jalali-year 1405 -reviewer reviewer-id -actor operator-id
+```

-## Merchant API contract
+The command rejects bundled review candidates, resumes the same reviewed version safely, and refuses to replace a different active version. Catalog rotation continues to use the explicit import, approve, and activate actions. Finish setup by creating the demo user:

- [`docs/API-SPEC.md`](docs/API-SPEC.md) defines the reviewable MVP capability, security boundaries, SSE recovery, detail pagination, and CSV-export policy.
- [`docs/openapi.yaml`](docs/openapi.yaml) is the approved OpenAPI 3.1 interface source of truth for generated frontend types.
+```bash
+go run ./cmd/admin
+```
+
+`cmd/admin` idempotently creates or resets the configured demo user and attaches it to the imported merchant with the largest session population. Passwords are stored as Argon2id hashes and are not printed by the command.
+
+Local demo credentials:
+
+```text
+Username: merchant-0001
+Password: merchant-demo-2026
+```
+
+These credentials work only when `.env` contains the same values before `cmd/admin` runs. They are public development credentials; change them for every internet-accessible deployment.
+
+### 5. Start the worker and API
+
+Open two terminals, load `.env` in both, and run:
+
+```bash
+# Terminal 1: background jobs
+go run ./cmd/worker
+
+# Terminal 2: HTTP API
+go run ./cmd/api
+```
+
+The API is available at `http://127.0.0.1:8080`. Start the frontend separately at `http://127.0.0.1:5173`; its Vite server proxies `/api` to this API.
+
+If the growth-coach scheduler is enabled, run it in a third terminal:
+
+```bash
+go run ./cmd/scheduler
+```
+
+## Verification and common commands
+
+```bash
+curl http://127.0.0.1:8080/healthz
+go test ./...
+go vet ./...
+go build ./cmd/...
+```
+
+The equivalent Make targets are `make migrate`, `make import`, `make seed`, `make worker`, `make api`, `make test`, `make vet`, and `make build`.
+
+## Production deployment on a remote server
+
+Deploy the backend and frontend together through the `zarinpal-analytics-deployment` repository. Its Compose stack runs PostgreSQL, Redis, the API, worker, one-off tools, and the static frontend. A separate shared Caddy instance owns ports 80/443, TLS, and domain routing.
+
+The examples below use `analytics.example.com`; replace it with your domain. Replace repository URL placeholders with your own template/fork URLs.
+
+### 1. Prepare the server and DNS access
+
+1. Provision an Ubuntu server with a public IP and enough disk/RAM for the dataset and PostgreSQL.
+2. Install Git, Docker Engine, and the Docker Compose plugin.
+3. Allow SSH only from trusted addresses and allow inbound TCP 80 and 443.
+4. Do not publish PostgreSQL port 5432 or Redis port 6379.
+
+### 2. Install the shared Caddy router
+
+Use the [Caddy deployment repository](https://git.amiirkhl.ir/Interanet/caddy-deployment.git) to install the main router:
+
+```bash
+git clone https://git.amiirkhl.ir/Interanet/caddy-deployment.git ~/caddy-deployment
+cd ~/caddy-deployment
+```
+
+Follow that repository's README to start Caddy. Ensure its Compose configuration creates/uses an external Docker network named `caddy_proxy` and attaches the Caddy container to it. If the repository does not create the network, create it once with `docker network create caddy_proxy` before starting Caddy.
+
+### 3. Clone the three application repositories
+
+The deployment script expects the backend and frontend as independent nested checkouts:
+
+```bash
+git clone <deployment-repository-url> ~/zarinpal-analytics-deployment
+cd ~/zarinpal-analytics-deployment
+git clone <backend-repository-url> backend
+git clone <frontend-repository-url> frontend
+```
+
+All three repositories must be on a clean `main` branch matching `origin/main`.
+
+### 4. Create production environment files
+
+```bash
+cp .env.example .env
+cp backend/.env.production.example backend/.env
+cp frontend/.env.production.example frontend/.env
+chmod 600 .env backend/.env frontend/.env
+```
+
+Then:
+
+- Replace every `change-me` value.
+- Set `FRONTEND_ORIGIN=https://analytics.example.com` in `backend/.env`.
+- Keep `COOKIE_SECURE=true`.
+- Generate different PostgreSQL, Redis, session, and cursor secrets with `openssl rand -hex 32`.
+- Set a new production `DEMO_PASSWORD`; do not reuse `merchant-demo-2026` on a public server.
+- Leave AI flags off unless a backend-only provider key, model, and positive budgets are configured.
+- Put no secret in `frontend/.env`; all `PUBLIC_*` values are embedded in the browser build.
+
+### 5. Place the import archive
+
+Copy the immutable archive to a private host path and make `IMPORT_FILE` in the deployment `.env` point to it:
+
+```bash
+sudo install -d -m 750 -o "$USER" -g "$USER" /srv/zarinpal-data
+cp /path/to/challenge_data.csv.gz /srv/zarinpal-data/challenge_data.csv.gz
+chmod 440 /srv/zarinpal-data/challenge_data.csv.gz
+```
+
+### 6. Perform the first application start
+
+From `~/zarinpal-analytics-deployment`:
+
+```bash
+docker compose config --quiet
+docker compose build backend frontend
+docker compose up -d postgres redis
+docker compose --profile tools run --rm migrate
+docker compose --profile import run --rm importer
+docker compose --profile tools run --rm calendar
+docker compose --profile tools run --rm admin
+docker compose up -d backend worker frontend
+docker compose ps
+```
+
+Wait until `postgres`, `redis`, `backend`, `worker`, and `frontend` are healthy. Inspect failures with:
+
+```bash
+docker compose logs --tail=200 postgres redis backend worker frontend
+```
+
+### 7. Connect the domain through Caddy
+
+Create an `A` record for `analytics.example.com` pointing to the server's public IPv4 address. Add an `AAAA` record only when IPv6 is configured and allowed through the firewall.
+
+Add a site to the Caddy deployment's `Caddyfile` (or its equivalent templated configuration):
+
+```caddyfile
+analytics.example.com {
+    @backend path /api/* /healthz
+
+    handle @backend {
+        reverse_proxy zarinpal-backend:8080 {
+            flush_interval -1
+        }
+    }
+
+    handle {
+        reverse_proxy zarinpal-frontend:8080
+    }
+}
+```
+
+Validate and reload Caddy using the commands documented by the Caddy repository. Caddy will obtain and renew HTTPS certificates automatically after public DNS resolves and ports 80/443 reach the server.
+
+Verify:
+
+```bash
+curl -I https://analytics.example.com/
+curl https://analytics.example.com/healthz
+```
+
+Then sign in with the production values of `DEMO_USERNAME` and `DEMO_PASSWORD` and verify that dashboard requests complete and the worker logs show no repeated failures.
+
+### 8. Deploy later updates
+
+```bash
+cd ~/zarinpal-analytics-deployment
+git pull --ff-only origin main
+git -C backend pull --ff-only origin main
+git -C frontend pull --ff-only origin main
+bash scripts/deploy.sh
+```
+
+Normal updates apply migrations and restart the application, but do not rerun the importer or reset the demo password. Run those one-off profiles explicitly only when intended.
+
+Never use `docker compose down -v` in production: `-v` deletes the PostgreSQL and Redis volumes. Back up PostgreSQL and rehearse restoration before treating the deployment as production-ready.
+
+## API and architecture documentation
+
+- [`docs/openapi.yaml`](docs/openapi.yaml) is the OpenAPI 3.1 contract used to generate frontend types.
+- [`docs/API-SPEC.md`](docs/API-SPEC.md) documents authentication, tenant isolation, SSE, pagination, and exports.
+- [`docs/IMPORT-WORKFLOW-SPEC.md`](docs/IMPORT-WORKFLOW-SPEC.md) documents the import boundary.
+- [`docs/METRIC-CATALOGUE.md`](docs/METRIC-CATALOGUE.md) defines analytics metrics.
--- a/cmd/calendar/main.go
+++ b/cmd/calendar/main.go
@@ -3,90 +3,185 @@ package main
 import (
 	"context"
 	"encoding/json"
+	"errors"
 	"flag"
 	"fmt"
+	"io"
 	"log/slog"
 	"os"
 	"os/signal"
+	"strings"

 	"git.amiirkhl.ir/starcoach-hackathon/zarinpal-analytics-backend/internal/config"
 	"git.amiirkhl.ir/starcoach-hackathon/zarinpal-analytics-backend/internal/database"
 	"git.amiirkhl.ir/starcoach-hackathon/zarinpal-analytics-backend/internal/eventcatalog"
 	"git.amiirkhl.ir/starcoach-hackathon/zarinpal-analytics-backend/internal/insight"
+	"github.com/jackc/pgx/v5/pgxpool"
 )

-func main() {
-	action := flag.String("action", "", "import, approve, or activate")
-	filePath := flag.String("file", "", "reviewed JSON catalog for import")
-	version := flag.String("version", "", "catalog version for approve or activate")
-	actor := flag.String("actor", "", "audited reviewer or operator identity")
-	currentJalaliYear := flag.Int("current-jalali-year", 0, "current Jalali year used for coverage validation")
-	flag.Parse()
+type options struct {
+	action            string
+	filePath          string
+	version           string
+	actor             string
+	reviewer          string
+	currentJalaliYear int
+}

+func main() {
+	os.Exit(realMain(os.Args[1:]))
+}
+
+func realMain(arguments []string) int {
 	logger := slog.New(slog.NewJSONHandler(os.Stdout, nil))
 	ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt)
 	defer cancel()
+	if err := run(ctx, arguments, logger); err != nil {
+		if errors.Is(err, flag.ErrHelp) {
+			fmt.Fprint(os.Stdout, calendarUsage)
+			return 0
+		}
+		logger.Error("calendar command failed", "error", err)
+		return 1
+	}
+	return 0
+}
+
+const calendarUsage = `Usage: calendar -action <import|approve|activate|bootstrap> [options]
+
+  -file PATH                  reviewed JSON catalog for import or bootstrap
+  -current-jalali-year YEAR  current Jalali year for coverage validation
+  -version VERSION           catalog version for approve or activate
+  -reviewer ID               independent reviewer identity for bootstrap
+  -actor ID                  approval or activation operator identity
+`
+
+func run(ctx context.Context, arguments []string, logger *slog.Logger) error {
+	command, err := parseOptions(arguments)
+	if err != nil {
+		return err
+	}
+	var input eventcatalog.Import
+	if command.action == "import" || command.action == "bootstrap" {
+		input, err = readImport(command.filePath)
+		if err != nil {
+			return err
+		}
+		if err := eventcatalog.ValidateImport(input, command.currentJalaliYear); err != nil {
+			return fmt.Errorf("validating calendar catalog: %w", err)
+		}
+	}
+
 	cfg, err := config.Load()
 	if err != nil {
-		fail(logger, "configuration failed", err)
+		return fmt.Errorf("loading configuration: %w", err)
 	}
 	pool, err := database.Open(ctx, cfg.DatabaseURL)
 	if err != nil {
-		fail(logger, "database connection failed", err)
+		return fmt.Errorf("connecting to database: %w", err)
 	}
 	defer pool.Close()
 	repository := eventcatalog.NewRepository(pool)

-	switch *action {
+	switch command.action {
 	case "import":
-		if *filePath == "" || *currentJalaliYear == 0 {
-			fail(logger, "invalid import arguments", fmt.Errorf("file and current-jalali-year are required"))
-		}
-		file, err := os.Open(*filePath)
-		if err != nil {
-			fail(logger, "opening calendar catalog failed", err)
-		}
-		defer file.Close()
-		var input eventcatalog.Import
-		decoder := json.NewDecoder(file)
-		decoder.DisallowUnknownFields()
-		if err := decoder.Decode(&input); err != nil {
-			fail(logger, "decoding calendar catalog failed", err)
-		}
-		if err := eventcatalog.ValidateImport(input, *currentJalaliYear); err != nil {
-			fail(logger, "calendar validation failed", err)
-		}
 		if err := repository.ImportDraft(ctx, input); err != nil {
-			fail(logger, "calendar import failed", err)
+			return fmt.Errorf("importing calendar catalog: %w", err)
 		}
 		logger.Info("calendar draft imported", "version", input.Version, "events", len(input.Events))
 	case "approve":
-		if *version == "" || *actor == "" {
-			fail(logger, "invalid approval arguments", fmt.Errorf("version and actor are required"))
+		if err := repository.Approve(ctx, command.version, command.actor); err != nil {
+			return fmt.Errorf("approving calendar catalog: %w", err)
 		}
-		if err := repository.Approve(ctx, *version, *actor); err != nil {
-			fail(logger, "calendar approval failed", err)
-		}
-		logger.Info("calendar approved", "version", *version, "reviewer", *actor)
+		logger.Info("calendar approved", "version", command.version, "reviewer", command.actor)
 	case "activate":
-		if *version == "" || *actor == "" {
-			fail(logger, "invalid activation arguments", fmt.Errorf("version and actor are required"))
+		if err := repository.Activate(ctx, command.version, command.actor); err != nil {
+			return fmt.Errorf("activating calendar catalog: %w", err)
 		}
-		if err := repository.Activate(ctx, *version, *actor); err != nil {
-			fail(logger, "calendar activation failed", err)
-		}
-		service := insight.NewService(insight.NewRepository(pool), insight.NewPostgreSQLBuilder(pool))
-		created, err := service.EnqueueAll(ctx, "CALENDAR")
+		created, err := scheduleInsights(ctx, pool)
 		if err != nil {
-			fail(logger, "calendar refresh scheduling failed", err)
+			return err
 		}
-		logger.Info("calendar activated", "version", *version, "actor", *actor, "created_jobs", created)
-	default:
-		fail(logger, "invalid action", fmt.Errorf("action must be import, approve, or activate"))
+		logger.Info("calendar activated", "version", command.version, "actor", command.actor, "created_jobs", created)
+	case "bootstrap":
+		result, err := eventcatalog.BootstrapCatalog(ctx, repository, input, command.currentJalaliYear, command.reviewer, command.actor, func(ctx context.Context) (int, error) {
+			return scheduleInsights(ctx, pool)
+		})
+		if err != nil {
+			return fmt.Errorf("bootstrapping calendar catalog: %w", err)
+		}
+		logger.Info("calendar bootstrap complete", "version", result.Version, "status", result.Status, "transitions", result.Transitions, "no_op", result.NoOp, "created_jobs", result.CreatedJobs)
 	}
+	return nil
 }

-func fail(logger *slog.Logger, message string, err error) {
-	logger.Error(message, "error", err)
-	os.Exit(1)
+func parseOptions(arguments []string) (options, error) {
+	var result options
+	flags := flag.NewFlagSet("calendar", flag.ContinueOnError)
+	flags.SetOutput(io.Discard)
+	flags.StringVar(&result.action, "action", "", "import, approve, activate, or bootstrap")
+	flags.StringVar(&result.filePath, "file", "", "reviewed JSON catalog for import or bootstrap")
+	flags.StringVar(&result.version, "version", "", "catalog version for approve or activate")
+	flags.StringVar(&result.actor, "actor", "", "audited reviewer or activation operator identity")
+	flags.StringVar(&result.reviewer, "reviewer", "", "independent reviewer identity for bootstrap")
+	flags.IntVar(&result.currentJalaliYear, "current-jalali-year", 0, "current Jalali year used for coverage validation")
+	if err := flags.Parse(arguments); err != nil {
+		return options{}, fmt.Errorf("parsing calendar arguments: %w", err)
+	}
+	if flags.NArg() != 0 {
+		return options{}, errors.New("calendar: positional arguments are not supported")
+	}
+	result.action = strings.TrimSpace(result.action)
+	result.filePath = strings.TrimSpace(result.filePath)
+	result.version = strings.TrimSpace(result.version)
+	result.actor = strings.TrimSpace(result.actor)
+	result.reviewer = strings.TrimSpace(result.reviewer)
+	switch result.action {
+	case "import":
+		if result.filePath == "" || result.currentJalaliYear == 0 {
+			return options{}, errors.New("calendar: file and current-jalali-year are required for import")
+		}
+	case "approve", "activate":
+		if result.version == "" || result.actor == "" {
+			return options{}, fmt.Errorf("calendar: version and actor are required for %s", result.action)
+		}
+	case "bootstrap":
+		if result.filePath == "" || result.currentJalaliYear == 0 || result.reviewer == "" || result.actor == "" {
+			return options{}, errors.New("calendar: file, current-jalali-year, reviewer, and actor are required for bootstrap")
+		}
+	default:
+		return options{}, errors.New("calendar: action must be import, approve, activate, or bootstrap")
+	}
+	return result, nil
+}
+
+func readImport(path string) (eventcatalog.Import, error) {
+	file, err := os.Open(path)
+	if err != nil {
+		return eventcatalog.Import{}, fmt.Errorf("opening calendar catalog: %w", err)
+	}
+	defer file.Close()
+	var input eventcatalog.Import
+	decoder := json.NewDecoder(file)
+	decoder.DisallowUnknownFields()
+	if err := decoder.Decode(&input); err != nil {
+		return eventcatalog.Import{}, fmt.Errorf("decoding calendar catalog: %w", err)
+	}
+	var extra any
+	if err := decoder.Decode(&extra); !errors.Is(err, io.EOF) {
+		if err == nil {
+			return eventcatalog.Import{}, errors.New("decoding calendar catalog: trailing JSON document")
+		}
+		return eventcatalog.Import{}, fmt.Errorf("decoding calendar catalog trailing data: %w", err)
+	}
+	return input, nil
+}
+
+func scheduleInsights(ctx context.Context, pool *pgxpool.Pool) (int, error) {
+	service := insight.NewService(insight.NewRepository(pool), insight.NewPostgreSQLBuilder(pool))
+	created, err := service.EnqueueAll(ctx, "CALENDAR")
+	if err != nil {
+		return 0, fmt.Errorf("enqueueing calendar insights: %w", err)
+	}
+	return created, nil
 }
--- a/cmd/calendar/main_test.go
+++ b/cmd/calendar/main_test.go
@@ -0,0 +1,72 @@
+package main
+
+import (
+	"errors"
+	"flag"
+	"os"
+	"path/filepath"
+	"testing"
+)
+
+func TestParseOptionsSupportsHelp(t *testing.T) {
+	t.Parallel()
+
+	if _, err := parseOptions([]string{"--help"}); !errors.Is(err, flag.ErrHelp) {
+		t.Fatalf("error=%v, want flag.ErrHelp", err)
+	}
+}
+
+func TestParseOptionsRequiresBootstrapInputs(t *testing.T) {
+	t.Parallel()
+
+	tests := []struct {
+		name string
+		args []string
+	}{
+		{name: "file", args: []string{"-action", "bootstrap", "-current-jalali-year", "1405", "-reviewer", "reviewer", "-actor", "operator"}},
+		{name: "current year", args: []string{"-action", "bootstrap", "-file", "/calendar/reviewed.json", "-reviewer", "reviewer", "-actor", "operator"}},
+		{name: "reviewer", args: []string{"-action", "bootstrap", "-file", "/calendar/reviewed.json", "-current-jalali-year", "1405", "-actor", "operator"}},
+		{name: "operator", args: []string{"-action", "bootstrap", "-file", "/calendar/reviewed.json", "-current-jalali-year", "1405", "-reviewer", "reviewer"}},
+	}
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			t.Parallel()
+			if _, err := parseOptions(test.args); err == nil {
+				t.Fatal("expected missing bootstrap input error")
+			}
+		})
+	}
+
+	options, err := parseOptions([]string{"-action", "bootstrap", "-file", "/calendar/reviewed.json", "-current-jalali-year", "1405", "-reviewer", "reviewer", "-actor", "operator"})
+	if err != nil {
+		t.Fatal(err)
+	}
+	if options.action != "bootstrap" || options.currentJalaliYear != 1405 || options.reviewer != "reviewer" || options.actor != "operator" {
+		t.Fatalf("options=%+v", options)
+	}
+}
+
+func TestReadImportRejectsUnknownFieldsAndTrailingDocuments(t *testing.T) {
+	t.Parallel()
+
+	directory := t.TempDir()
+	tests := []struct {
+		name string
+		json string
+	}{
+		{name: "unknown field", json: `{"version":"test","unexpected":true}`},
+		{name: "trailing document", json: `{"version":"test"} {}`},
+	}
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			t.Parallel()
+			path := filepath.Join(directory, test.name+".json")
+			if err := os.WriteFile(path, []byte(test.json), 0o600); err != nil {
+				t.Fatal(err)
+			}
+			if _, err := readImport(path); err == nil {
+				t.Fatal("expected strict JSON error")
+			}
+		})
+	}
+}
--- a/internal/eventcatalog/bootstrap.go
+++ b/internal/eventcatalog/bootstrap.go
@@ -0,0 +1,140 @@
+package eventcatalog
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"strings"
+)
+
+var (
+	ErrCatalogNotFound = errors.New("event catalog: version not found")
+	ErrCatalogConflict = errors.New("event catalog: bootstrap conflict")
+)
+
+type CatalogStatus string
+
+const (
+	CatalogStatusDraft    CatalogStatus = "DRAFT"
+	CatalogStatusApproved CatalogStatus = "APPROVED"
+	CatalogStatusActive   CatalogStatus = "ACTIVE"
+	CatalogStatusRetired  CatalogStatus = "RETIRED"
+)
+
+type CatalogState struct {
+	Version                 string
+	Status                  CatalogStatus
+	CoverageStartJalaliYear int
+	CoverageEndJalaliYear   int
+	SourceURI               string
+	SourceVersion           string
+	SourceChecksum          string
+	ImportedBy              string
+}
+
+type BootstrapResult struct {
+	Version     string
+	Status      CatalogStatus
+	Transitions []string
+	NoOp        bool
+	CreatedJobs int
+}
+
+type bootstrapStore interface {
+	CatalogState(context.Context, string) (CatalogState, error)
+	ActiveCatalogState(context.Context) (CatalogState, error)
+	ImportDraft(context.Context, Import) error
+	Approve(context.Context, string, string) error
+	ActivateInitial(context.Context, string, string) error
+}
+
+type BootstrapScheduler func(context.Context) (int, error)
+
+func BootstrapCatalog(ctx context.Context, store bootstrapStore, input Import, currentJalaliYear int, reviewer, operator string, schedule BootstrapScheduler) (BootstrapResult, error) {
+	reviewer = strings.TrimSpace(reviewer)
+	operator = strings.TrimSpace(operator)
+	if err := ValidateImport(input, currentJalaliYear); err != nil {
+		return BootstrapResult{}, err
+	}
+	if err := ValidateReviewableChecksum(input.SourceChecksum); err != nil {
+		return BootstrapResult{}, err
+	}
+	if err := ValidateApproval(input.ImportedBy, reviewer); err != nil {
+		return BootstrapResult{}, err
+	}
+	if operator == "" {
+		return BootstrapResult{}, errors.New("event catalog: activation operator is required")
+	}
+	if schedule == nil {
+		return BootstrapResult{}, errors.New("event catalog: bootstrap scheduler is required")
+	}
+
+	active, err := store.ActiveCatalogState(ctx)
+	if err != nil && !errors.Is(err, ErrCatalogNotFound) {
+		return BootstrapResult{}, fmt.Errorf("loading active calendar catalog: %w", err)
+	}
+	if active.Version != "" && active.Version != input.Version {
+		return BootstrapResult{}, fmt.Errorf("%w: active version %q differs from requested version %q", ErrCatalogConflict, active.Version, input.Version)
+	}
+
+	imported := false
+	state, err := store.CatalogState(ctx, input.Version)
+	if errors.Is(err, ErrCatalogNotFound) {
+		if err := store.ImportDraft(ctx, input); err != nil {
+			return BootstrapResult{}, fmt.Errorf("importing calendar draft: %w", err)
+		}
+		state = stateFromImport(input, CatalogStatusDraft)
+		imported = true
+	} else if err != nil {
+		return BootstrapResult{}, fmt.Errorf("loading calendar catalog version: %w", err)
+	} else if !state.matches(input) {
+		return BootstrapResult{}, fmt.Errorf("%w: version %q metadata does not match the reviewed file", ErrCatalogConflict, input.Version)
+	}
+
+	result := BootstrapResult{Version: input.Version}
+	if imported {
+		result.Transitions = append(result.Transitions, "IMPORTED")
+	}
+	if state.Status == CatalogStatusDraft && active.Version == "" {
+		if err := store.Approve(ctx, input.Version, reviewer); err != nil {
+			return BootstrapResult{}, fmt.Errorf("approving calendar catalog: %w", err)
+		}
+		result.Transitions = append(result.Transitions, "APPROVED")
+		state.Status = CatalogStatusApproved
+	}
+	if state.Status == CatalogStatusApproved && active.Version == "" {
+		if err := store.ActivateInitial(ctx, input.Version, operator); err != nil {
+			return BootstrapResult{}, fmt.Errorf("activating calendar catalog: %w", err)
+		}
+		result.Transitions = append(result.Transitions, "ACTIVATED")
+		state.Status = CatalogStatusActive
+	}
+	if state.Status != CatalogStatusActive {
+		return BootstrapResult{}, fmt.Errorf("%w: version %q has status %q", ErrCatalogConflict, input.Version, state.Status)
+	}
+	created, err := schedule(ctx)
+	if err != nil {
+		return BootstrapResult{}, fmt.Errorf("scheduling calendar refresh: %w", err)
+	}
+	result.Status = CatalogStatusActive
+	result.CreatedJobs = created
+	result.NoOp = len(result.Transitions) == 0 && created == 0
+	return result, nil
+}
+
+func stateFromImport(input Import, status CatalogStatus) CatalogState {
+	return CatalogState{
+		Version:                 input.Version,
+		Status:                  status,
+		CoverageStartJalaliYear: input.CoverageStartJalaliYear,
+		CoverageEndJalaliYear:   input.CoverageEndJalaliYear,
+		SourceURI:               input.SourceURI,
+		SourceVersion:           input.SourceVersion,
+		SourceChecksum:          input.SourceChecksum,
+		ImportedBy:              input.ImportedBy,
+	}
+}
+
+func (state CatalogState) matches(input Import) bool {
+	return state == stateFromImport(input, state.Status)
+}
--- a/internal/eventcatalog/bootstrap_integration_test.go
+++ b/internal/eventcatalog/bootstrap_integration_test.go
@@ -0,0 +1,87 @@
+package eventcatalog
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"os"
+	"testing"
+	"time"
+
+	"github.com/jackc/pgx/v5/pgxpool"
+)
+
+func TestRepositoryBootstrapStateAndInitialActivation(t *testing.T) {
+	pool := eventCatalogTestPool(t)
+	repository := NewRepository(pool)
+	ctx := t.Context()
+	prefix := fmt.Sprintf("bootstrap-%d", time.Now().UnixNano())
+	first := catalogImportForIntegration(prefix + "-first")
+	second := catalogImportForIntegration(prefix + "-second")
+	t.Cleanup(func() {
+		_, _ = pool.Exec(context.Background(), `DELETE FROM calendar_catalog_versions WHERE version IN ($1,$2)`, first.Version, second.Version)
+	})
+
+	if err := repository.ImportDraft(ctx, first); err != nil {
+		t.Fatal(err)
+	}
+	state, err := repository.CatalogState(ctx, first.Version)
+	if err != nil || state.Status != CatalogStatusDraft || !state.matches(first) {
+		t.Fatalf("draft state=%+v error=%v", state, err)
+	}
+	if err := repository.Approve(ctx, first.Version, "reviewer"); err != nil {
+		t.Fatal(err)
+	}
+	if err := repository.ActivateInitial(ctx, first.Version, "operator"); err != nil {
+		t.Fatal(err)
+	}
+	active, err := repository.ActiveCatalogState(ctx)
+	if err != nil || active.Version != first.Version || active.Status != CatalogStatusActive {
+		t.Fatalf("active state=%+v error=%v", active, err)
+	}
+	var auditCount int
+	if err := pool.QueryRow(ctx, `SELECT count(*) FROM calendar_catalog_audit a JOIN calendar_catalog_versions v ON v.id=a.catalog_version_id WHERE v.version=$1`, first.Version).Scan(&auditCount); err != nil || auditCount != 3 {
+		t.Fatalf("first audit count=%d error=%v", auditCount, err)
+	}
+
+	if err := repository.ImportDraft(ctx, second); err != nil {
+		t.Fatal(err)
+	}
+	if err := repository.Approve(ctx, second.Version, "reviewer"); err != nil {
+		t.Fatal(err)
+	}
+	if err := repository.ActivateInitial(ctx, second.Version, "operator"); !errors.Is(err, ErrCatalogConflict) {
+		t.Fatalf("activation error=%v, want ErrCatalogConflict", err)
+	}
+	state, err = repository.CatalogState(ctx, second.Version)
+	if err != nil || state.Status != CatalogStatusApproved {
+		t.Fatalf("conflicting state=%+v error=%v", state, err)
+	}
+	if err := pool.QueryRow(ctx, `SELECT count(*) FROM calendar_catalog_audit a JOIN calendar_catalog_versions v ON v.id=a.catalog_version_id WHERE v.version=$1`, second.Version).Scan(&auditCount); err != nil || auditCount != 2 {
+		t.Fatalf("second audit count=%d error=%v", auditCount, err)
+	}
+}
+
+func eventCatalogTestPool(t *testing.T) *pgxpool.Pool {
+	t.Helper()
+	databaseURL := os.Getenv("TEST_DATABASE_URL")
+	if databaseURL == "" {
+		t.Skip("TEST_DATABASE_URL is not configured")
+	}
+	pool, err := pgxpool.New(context.Background(), databaseURL)
+	if err != nil {
+		t.Fatal(err)
+	}
+	t.Cleanup(pool.Close)
+	return pool
+}
+
+func catalogImportForIntegration(version string) Import {
+	input := validCoveredImport()
+	input.Version = version
+	input.SourceChecksum = "sha256:" + fmt.Sprintf("%064x", len(version))
+	for index := range input.Events {
+		input.Events[index].ID = fmt.Sprintf("%s-event-%d", version, index)
+	}
+	return input
+}
--- a/internal/eventcatalog/bootstrap_test.go
+++ b/internal/eventcatalog/bootstrap_test.go
@@ -0,0 +1,156 @@
+package eventcatalog
+
+import (
+	"context"
+	"errors"
+	"reflect"
+	"testing"
+)
+
+func TestBootstrapCatalogTransitionsToActive(t *testing.T) {
+	t.Parallel()
+
+	store := &bootstrapStoreFake{}
+	createdJobs := 7
+	result, err := BootstrapCatalog(t.Context(), store, validCoveredImport(), 1405, "reviewer", "operator", func(context.Context) (int, error) {
+		return createdJobs, nil
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+	if want := []string{"IMPORTED", "APPROVED", "ACTIVATED"}; !reflect.DeepEqual(result.Transitions, want) {
+		t.Fatalf("transitions=%v, want %v", result.Transitions, want)
+	}
+	if result.Status != CatalogStatusActive || result.NoOp || result.CreatedJobs != createdJobs {
+		t.Fatalf("result=%+v", result)
+	}
+}
+
+func TestBootstrapCatalogResumesAndNoOps(t *testing.T) {
+	t.Parallel()
+
+	tests := []struct {
+		name        string
+		status      CatalogStatus
+		transitions []string
+	}{
+		{name: "draft", status: CatalogStatusDraft, transitions: []string{"APPROVED", "ACTIVATED"}},
+		{name: "approved", status: CatalogStatusApproved, transitions: []string{"ACTIVATED"}},
+		{name: "active", status: CatalogStatusActive},
+	}
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			t.Parallel()
+			input := validCoveredImport()
+			store := &bootstrapStoreFake{state: stateFromImport(input, test.status), active: stateFromImport(input, CatalogStatusActive)}
+			if test.status != CatalogStatusActive {
+				store.active = CatalogState{}
+			}
+			result, err := BootstrapCatalog(t.Context(), store, input, 1405, "reviewer", "operator", func(context.Context) (int, error) { return 0, nil })
+			if err != nil {
+				t.Fatal(err)
+			}
+			if !reflect.DeepEqual(result.Transitions, test.transitions) {
+				t.Fatalf("transitions=%v, want %v", result.Transitions, test.transitions)
+			}
+			if result.NoOp != (test.status == CatalogStatusActive) {
+				t.Fatalf("no_op=%t", result.NoOp)
+			}
+		})
+	}
+}
+
+func TestBootstrapCatalogFailsClosed(t *testing.T) {
+	t.Parallel()
+
+	input := validCoveredImport()
+	tests := []struct {
+		name   string
+		state  CatalogState
+		active CatalogState
+		mutate func(*Import)
+	}{
+		{name: "retired version", state: stateFromImport(input, CatalogStatusRetired)},
+		{name: "different active version", active: CatalogState{Version: "newer", Status: CatalogStatusActive}},
+		{name: "checksum mismatch", state: stateFromImport(input, CatalogStatusDraft), mutate: func(input *Import) { input.SourceChecksum = "different" }},
+		{name: "metadata mismatch", state: stateFromImport(input, CatalogStatusDraft), mutate: func(input *Import) { input.SourceVersion = "different" }},
+	}
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			t.Parallel()
+			candidate := input
+			if test.mutate != nil {
+				test.mutate(&candidate)
+			}
+			store := &bootstrapStoreFake{state: test.state, active: test.active}
+			_, err := BootstrapCatalog(t.Context(), store, candidate, 1405, "reviewer", "operator", func(context.Context) (int, error) { return 0, nil })
+			if !errors.Is(err, ErrCatalogConflict) {
+				t.Fatalf("error=%v, want ErrCatalogConflict", err)
+			}
+			if len(store.transitions) != 0 {
+				t.Fatalf("unexpected transitions=%v", store.transitions)
+			}
+		})
+	}
+}
+
+func TestBootstrapCatalogRejectsInvalidReviewAndRetriesScheduling(t *testing.T) {
+	t.Parallel()
+
+	input := validCoveredImport()
+	input.SourceChecksum = "review-required-before-activation"
+	store := &bootstrapStoreFake{}
+	if _, err := BootstrapCatalog(t.Context(), store, input, 1405, "reviewer", "operator", func(context.Context) (int, error) { return 0, nil }); err == nil {
+		t.Fatal("expected placeholder checksum error")
+	}
+
+	input = validCoveredImport()
+	store = &bootstrapStoreFake{state: stateFromImport(input, CatalogStatusActive), active: stateFromImport(input, CatalogStatusActive)}
+	wantErr := errors.New("scheduling failed")
+	if _, err := BootstrapCatalog(t.Context(), store, input, 1405, "reviewer", "operator", func(context.Context) (int, error) { return 0, wantErr }); !errors.Is(err, wantErr) {
+		t.Fatalf("error=%v, want scheduling error", err)
+	}
+	result, err := BootstrapCatalog(t.Context(), store, input, 1405, "reviewer", "operator", func(context.Context) (int, error) { return 2, nil })
+	if err != nil || result.CreatedJobs != 2 || result.NoOp {
+		t.Fatalf("result=%+v error=%v", result, err)
+	}
+}
+
+type bootstrapStoreFake struct {
+	state       CatalogState
+	active      CatalogState
+	transitions []string
+}
+
+func (f *bootstrapStoreFake) CatalogState(context.Context, string) (CatalogState, error) {
+	if f.state.Version == "" {
+		return CatalogState{}, ErrCatalogNotFound
+	}
+	return f.state, nil
+}
+
+func (f *bootstrapStoreFake) ActiveCatalogState(context.Context) (CatalogState, error) {
+	if f.active.Version == "" {
+		return CatalogState{}, ErrCatalogNotFound
+	}
+	return f.active, nil
+}
+
+func (f *bootstrapStoreFake) ImportDraft(_ context.Context, input Import) error {
+	f.state = stateFromImport(input, CatalogStatusDraft)
+	f.transitions = append(f.transitions, "IMPORTED")
+	return nil
+}
+
+func (f *bootstrapStoreFake) Approve(_ context.Context, _ string, _ string) error {
+	f.state.Status = CatalogStatusApproved
+	f.transitions = append(f.transitions, "APPROVED")
+	return nil
+}
+
+func (f *bootstrapStoreFake) ActivateInitial(_ context.Context, _ string, _ string) error {
+	f.state.Status = CatalogStatusActive
+	f.active = f.state
+	f.transitions = append(f.transitions, "ACTIVATED")
+	return nil
+}
--- a/internal/eventcatalog/import.go
+++ b/internal/eventcatalog/import.go
@@ -60,6 +60,26 @@ type Repository struct{ pool *pgxpool.Pool }

 func NewRepository(pool *pgxpool.Pool) *Repository { return &Repository{pool: pool} }

+func (r *Repository) CatalogState(ctx context.Context, version string) (CatalogState, error) {
+	return r.catalogState(ctx, `SELECT version,status,coverage_start_jalali_year,coverage_end_jalali_year,source_uri,source_version,source_checksum,imported_by FROM calendar_catalog_versions WHERE version=$1`, version)
+}
+
+func (r *Repository) ActiveCatalogState(ctx context.Context) (CatalogState, error) {
+	return r.catalogState(ctx, `SELECT version,status,coverage_start_jalali_year,coverage_end_jalali_year,source_uri,source_version,source_checksum,imported_by FROM calendar_catalog_versions WHERE status='ACTIVE'`)
+}
+
+func (r *Repository) catalogState(ctx context.Context, query string, arguments ...any) (CatalogState, error) {
+	var state CatalogState
+	err := r.pool.QueryRow(ctx, query, arguments...).Scan(&state.Version, &state.Status, &state.CoverageStartJalaliYear, &state.CoverageEndJalaliYear, &state.SourceURI, &state.SourceVersion, &state.SourceChecksum, &state.ImportedBy)
+	if errors.Is(err, pgx.ErrNoRows) {
+		return CatalogState{}, ErrCatalogNotFound
+	}
+	if err != nil {
+		return CatalogState{}, fmt.Errorf("querying calendar catalog state: %w", err)
+	}
+	return state, nil
+}
+
 func (r *Repository) ImportDraft(ctx context.Context, input Import) error {
 	if err := ValidateEvents(input.Events); err != nil {
 		return err
@@ -169,3 +189,43 @@ func (r *Repository) Activate(ctx context.Context, version, actor string) error
 	}
 	return tx.Commit(ctx)
 }
+
+func (r *Repository) ActivateInitial(ctx context.Context, version, actor string) error {
+	if strings.TrimSpace(actor) == "" {
+		return errors.New("event catalog: activation operator is required")
+	}
+	tx, err := r.pool.BeginTx(ctx, pgx.TxOptions{})
+	if err != nil {
+		return fmt.Errorf("starting initial calendar activation: %w", err)
+	}
+	defer func() { _ = tx.Rollback(context.WithoutCancel(ctx)) }()
+	if _, err := tx.Exec(ctx, `SELECT pg_advisory_xact_lock(hashtextextended('calendar-catalog-activation',0))`); err != nil {
+		return fmt.Errorf("locking initial calendar activation: %w", err)
+	}
+	var activeVersion string
+	err = tx.QueryRow(ctx, `SELECT version FROM calendar_catalog_versions WHERE status='ACTIVE'`).Scan(&activeVersion)
+	if err == nil {
+		if activeVersion == version {
+			return tx.Commit(ctx)
+		}
+		return fmt.Errorf("%w: active version %q differs from requested version %q", ErrCatalogConflict, activeVersion, version)
+	}
+	if !errors.Is(err, pgx.ErrNoRows) {
+		return fmt.Errorf("loading active calendar catalog: %w", err)
+	}
+	var id int64
+	var sourceChecksum string
+	if err := tx.QueryRow(ctx, `SELECT id,source_checksum FROM calendar_catalog_versions WHERE version=$1 AND status='APPROVED' FOR UPDATE`, version).Scan(&id, &sourceChecksum); err != nil {
+		return fmt.Errorf("loading approved calendar catalog: %w", err)
+	}
+	if err := ValidateReviewableChecksum(sourceChecksum); err != nil {
+		return err
+	}
+	if _, err := tx.Exec(ctx, `UPDATE calendar_catalog_versions SET status='ACTIVE',activated_at=now() WHERE id=$1`, id); err != nil {
+		return fmt.Errorf("activating calendar catalog: %w", err)
+	}
+	if _, err := tx.Exec(ctx, `INSERT INTO calendar_catalog_audit(catalog_version_id,action,actor) VALUES($1,'ACTIVATED',$2)`, id, actor); err != nil {
+		return fmt.Errorf("auditing calendar activation: %w", err)
+	}
+	return tx.Commit(ctx)
+}
Author	SHA1	Message	Date
Amirhossein Khalili	5673b60ba6	feat: add idempotent calendar bootstrap Bootstrap independently reviewed catalogs through audited import, approval, and initial activation transitions, then resume insight scheduling safely.	2026-08-23 02:24:16 +08:00
Amirhossein Khalili	72238e7e87	docs: document setup and deployment	2026-08-23 01:40:45 +08:00