diff --git a/docs/GROWTH-ANALYST-SPEC.md b/docs/GROWTH-ANALYST-SPEC.md new file mode 100644 index 0000000..e7a77ee --- /dev/null +++ b/docs/GROWTH-ANALYST-SPEC.md @@ -0,0 +1,61 @@ +# Growth Analyst Capability Spec + +- Status: Approved — implementation authorized 2026-08-21 +- Capability: Read-only AI-assisted growth analytics +- Public contract: `docs/openapi.yaml` +- UI contract: frontend `docs/GROWTH-ANALYST-UX-SPEC.md` + +## Objective + +Give an authenticated merchant a trustworthy 90-day Persian growth outlook: historical event wins, upcoming similar occasions, likely low-performance windows, privacy-safe peer context, and compact campaign briefs. PostgreSQL and deterministic Go calculations own every number; AvalAI may only prioritize approved evidence and produce qualitative copy. + +## Deterministic analytics contract + +- Aggregate session count and paid Rial amount by Tehran-local day for the authenticated merchant and optional owned terminal. `Paid` and `Verified` are successful; money remains integer Rial at session grain. +- Forecast begins the day after the latest complete imported dataset date and covers 90 days. Count and amount are modeled independently from weekday-aware rolling baselines, bounded trend adjustment, catalogued event uplift, and empirical residual P10/P50/P90 values. +- Rolling-origin backtests classify usable output as `HIGH` or `MEDIUM`; other results return `INSUFFICIENT_DATA` and do not call AvalAI. +- Historical event effect compares matching weekdays from the preceding eight non-event weeks. At least four observations and 100 baseline sessions are required. A material win/decline is at least 20%. +- Upcoming similarity prefers an exact event family, then catalog tag similarity of at least 0.60. Return at most five opportunities and three risks. A risk requires a forecast at least 20% below a baseline with at least 100 sessions. +- Merchant scope may include peer context only under the existing threshold of 10 other same-category merchants and 1,000 combined peer sessions, with one equally weighted value per merchant. Terminal scope always suppresses peers. +- The versioned catalog stores Persian title, Gregorian date range, family, tags, source, supported merchant categories, and official/commercial classification. The initial reviewed catalog covers the imported history and the following 90 days. + +## AI boundary + +- AvalAI is backend-only and receives minimized aggregate evidence, never raw transactions, payer keys, credentials, arbitrary merchant ids, SQL, or peer rows. +- The consumer-owned `GrowthAnalyticsTools` boundary exposes only `get_growth_context`, `get_event_evidence`, `get_peer_context`, and `get_campaign_playbook`. Scope is injected from the authenticated job, not accepted from model arguments. +- Use `POST /v1/responses`, `store:false`, strict function schemas, strict structured output, `parallel_tool_calls:false`, no more than four tool calls and two model turns, 1,600 output tokens, a 30-second deadline, and one transient retry. +- Treat model output as untrusted. Accept only known evidence ids and qualitative Persian fields. Reject numeric claims, unsafe language, unknown references, refusal/incomplete output, or schema drift and use deterministic Persian fallback copy. +- Skip AvalAI when no material opportunity/risk exists or confidence is insufficient. AI failure produces `PARTIAL`, never an empty report. + +## Persistence, caching, jobs, and limits + +- PostgreSQL is authoritative for reports, revisions, deterministic evidence, validated AI copy, usage metadata, and idempotency. Redis is disposable hot cache, lock, rate-counter, and budget state. +- A fingerprint includes merchant/terminal scope, import watermark, forecast date/horizon, catalog, analytics, prompt/schema, and model versions. A matching successful report is reused without an AI call. +- Keep the current matching report indefinitely and superseded/manual-refresh revisions for 90 days. Concurrent identical requests collapse to one job. +- States are `QUEUED`, `ANALYZING`, `GENERATING`, `READY`, `PARTIAL`, and `FAILED`. Preserve the previous successful report during refresh/failure. +- Defaults: create 10/user/minute; five uncached scopes/merchant/Tehran day; one merchant-wide manual refresh/Tehran day; four concurrent generations globally; explicit production daily request/token budgets; one active generation per fingerprint. +- Budget exhaustion returns deterministic `PARTIAL` output. Only an unavailable manual refresh returns `429` with `nextRefreshAt`/`Retry-After`. +- Structured logs include safe report/version/status/cache/usage/latency fields without prompts, transaction values, usernames, or sensitive identifiers. + +## Public API + +- `POST /api/v1/analytics/growth-reports` accepts optional `terminalId` and `refresh`, requires CSRF plus `Idempotency-Key`, and returns an existing or queued report resource. +- `GET /api/v1/analytics/growth-reports/latest?terminalId=...` returns the latest authorized scope report. +- `GET /api/v1/analytics/growth-reports/{reportId}` returns an owned report; absent and cross-tenant ids are indistinguishable. +- `GET /api/v1/analytics/growth-reports/{reportId}/events` is authenticated SSE with stable ids, heartbeat, replay/snapshot recovery, and terminal status. +- Responses include deterministic series/ranges/confidence/evidence, opportunities, risks, peer result or suppression reason, validated briefs, cache metadata, and next refresh time. + +## Boundaries and acceptance + +- Always derive merchant identity from the server session, validate terminal ownership, use parameterized SQL/context deadlines, keep numbers deterministic, and preserve useful prior output on failure. +- Never add chat, memory, notifications, automated campaigns/actions, runtime web search, raw-data tools, or model-authored numerical predictions. +- Tests cover event/baseline/forecast/backtest/risk/peer logic, fingerprints, Tehran refresh boundaries, authorization, persistence, locks/deduplication, budgets, AI schemas/rejections/timeouts, fallback, SSE, and sensitive-field absence. +- Feature is disabled by default behind `AI_GROWTH_ENABLED`. Production enablement requires explicit AvalAI key/model/request budget/token budget configuration. + +## Sources + +- AvalAI Responses: https://docs.avalai.ir/en/api-reference/responses +- AvalAI function calling: https://docs.avalai.ir/en/guides/function-calling?api-mode=responses +- AvalAI structured outputs: https://docs.avalai.ir/en/guides/structured-outputs?api-mode=responses +- University of Tehran Iranian calendar: https://calendar.ut.ac.ir/ + diff --git a/docs/decisions/0015-deterministic-ai-growth-reports.md b/docs/decisions/0015-deterministic-ai-growth-reports.md new file mode 100644 index 0000000..8f5c569 --- /dev/null +++ b/docs/decisions/0015-deterministic-ai-growth-reports.md @@ -0,0 +1,20 @@ +# ADR 0015: Deterministic analytics with bounded AI interpretation + +- Status: Accepted +- Date: 2026-08-21 + +## Context + +Growth forecasting, event uplift, low-period detection, and peer comparisons must be reproducible, testable, privacy-safe, and available when an AI provider is unavailable. AvalAI adds value for semantic prioritization and natural Persian campaign guidance, but must not become the source of financial analytics. + +## Decision + +Use PostgreSQL plus deterministic Go analytics as the sole authority for numbers and evidence. Run a bounded backend-only AvalAI Responses workflow over sanitized aggregates, validate its structured qualitative output, and persist report snapshots by a versioned data fingerprint. Use Redis only for hot caching, coordination, budgets, and rate limits. Defer MCP; keep a narrow internal tool interface that an MCP adapter may wrap only when multiple independent agent clients exist. + +## Consequences + +- The growth page remains useful with AI disabled, unavailable, rate-limited, or rejected. +- Repeated page visits do not create paid calls while the fingerprint is unchanged. +- Algorithm, catalog, prompt, schema, model, or import changes intentionally invalidate the report. +- The backend owns more deterministic forecasting and validation code, but the result is auditable and tenant-safe. + diff --git a/tasks/ai-growth-analyst.md b/tasks/ai-growth-analyst.md new file mode 100644 index 0000000..8be9a2b --- /dev/null +++ b/tasks/ai-growth-analyst.md @@ -0,0 +1,11 @@ +# AI Growth Analyst Implementation Tasks + +- [ ] G1: Add approved spec, ADR, OpenAPI resources, event catalog, report schema, and reversible migration. +- [ ] G2: Add pure tested daily aggregation, event effect, similarity, 90-day forecast, residual ranges, backtest confidence, risk, and peer-suppression logic. +- [ ] G3: Add PostgreSQL report repository, fingerprint/idempotency, Redis hot cache/locks/budgets, and Tehran-day limits. +- [ ] G4: Add bounded worker state machine and sanitized `GrowthAnalyticsTools` boundary. +- [ ] G5: Add AvalAI Responses client, strict tool/final schemas, output validation, retry/timeout, usage metadata, and deterministic Persian fallbacks. +- [ ] G6: Add authenticated create/latest/get/SSE handlers with terminal ownership, CSRF, idempotency, replay, and `429` refresh semantics. +- [ ] G7: Wire configuration/composition roots, feature flag, local environment templates, retention cleanup, safe logs, and operational documentation. +- [ ] G8: Run unit/integration/contract/security/race/vet/build verification and real local PostgreSQL/Redis/API smoke without sending an AvalAI request unless a user-provided key is configured. +