docs: add readme architecture diagrams

This commit is contained in:
2026-06-21 11:10:18 +03:30
parent dd9567deb5
commit bd358f5e48
20 changed files with 65 additions and 49 deletions

View File

@@ -1,4 +1,4 @@
# Backend README
# Backend
This backend is a minimal production-aware PostgreSQL job queue implemented with Django and Django REST Framework.
@@ -23,6 +23,8 @@ All queue behavior is implemented in service functions under `jobs/services.py`.
## High-Level Architecture
![prompt for generating an svg image for the backend high-level architecture of a minimal Django REST Framework job queue: PostgreSQL at the center storing jobs_job and jobs_jobevent, Django REST API for create/list/retry/stats, separate Django worker command using the same service functions, N worker threads configured by environment variables, and demo handlers; use clean labeled boxes and arrows, no extra services like Redis or queue table](../assets/images/backend/high-level-architecture.png)
```text
PostgreSQL
|
@@ -91,6 +93,8 @@ This is an at-least-once queue. A worker can perform a side effect and then cras
### Statuses
![prompt for generating an svg image for a deterministic job queue state machine with states queued, running, succeeded, failed; show valid transitions queued to running on claim, running to succeeded on completion, running to queued on retry or lease timeout, running to failed when attempts are exhausted, failed to queued on manual retry; include notes that invalid transitions are rejected by service ownership checks and database constraints](../assets/images/backend/job-state-machine.png)
`Job.status` is intentionally small:
```text
@@ -114,6 +118,8 @@ Unsupported transitions are rejected by service-level ownership checks and by da
### Ownership
![prompt for generating an svg image for job ownership and stale worker protection in a PostgreSQL job queue: Worker A claims attempt 1 with locked_by and locked_until, lease expires, cleanup requeues, Worker B claims attempt 2, Worker A tries to complete attempt 1 and the database update affects zero rows because attempts and locked_by no longer match; use worker lanes, database row snapshots, and a clear rejected stale completion marker](../assets/images/backend/ownership-stale-worker.png)
A worker owns a job only when all of these are true:
```text
@@ -143,6 +149,8 @@ This is one of the most important correctness properties in the project.
## Database Model
![prompt for generating an svg image for an ERD of a minimal two-table job queue database with jobs_job and jobs_jobevent; jobs_job has uuid id, type, payload, status, priority, available_at, attempts, max_attempts, idempotency_key, locked_by, locked_until, result, timestamps; jobs_jobevent has id, job_id foreign key, type, attempt, worker_id, message, data, created_at; show one-to-many relationship from jobs_job to jobs_jobevent and highlight that there is no queue table or worker table](../assets/images/backend/database-erd.png)
### `Job`
`Job` is the source of truth for queue state.
@@ -240,6 +248,8 @@ This allows many jobs with no idempotency key while preventing duplicate client-
## Indexes
![prompt for generating an svg image for PostgreSQL indexes used by the job queue: a claim path using partial index job_claim_idx on queued jobs ordered by available_at, priority descending, created_at, id; a timeout cleanup path using job_timeout_idx on running locked_until; and event lookup indexes for job timeline and global cursor feed; use three grouped panels with query arrows](../assets/images/backend/database-indexes.png)
### Claim Index
```text
@@ -405,6 +415,8 @@ Table jobs_jobevent {
## Claiming Algorithm
![prompt for generating an svg image for atomic job claiming with SELECT FOR UPDATE SKIP LOCKED: multiple worker threads query queued available jobs, one locks and updates a row to running, other workers skip locked rows and claim different jobs, then a claimed event is inserted; show a short transaction boundary around selection and update, and show handler execution outside the transaction](../assets/images/backend/skip-locked-claim.png)
The worker claims a job in a short transaction:
```python
@@ -471,6 +483,8 @@ Then a `succeeded` event is inserted.
## Failure And Retry Algorithm
![prompt for generating an svg image for automatic exponential backoff retry in a job queue: running job fails, if attempts remain it returns to queued with available_at delayed by base * 2^(attempt-1), emits retry_scheduled, then later is claimed again; if max attempts is reached it transitions to failed; include a small backoff ladder 5s, 10s, 20s up to max 300s](../assets/images/backend/exponential-backoff.png)
On handler failure, the worker calls `fail_job`.
If attempts remain:
@@ -515,6 +529,8 @@ max delay -> 300 seconds
## Lease Renewal
![prompt for generating an svg image for lease renewal in long-running jobs: worker owns a running job with locked_until, handler periodically emits progress and renews the lease, locked_until moves forward only when worker_id and attempt match; show a contrasting timeout demo job that does not renew and becomes eligible for cleanup](../assets/images/backend/lease-renewal.png)
Long-running handlers can renew their lease:
```python
@@ -540,6 +556,8 @@ The demo `demo.timeout` handler intentionally does not renew leases so the UI ca
## Expired Lease Cleanup
![prompt for generating an svg image for expired lease cleanup: worker cleanup loop scans running jobs where locked_until is in the past, locks expired rows with SKIP LOCKED, requeues them with retry backoff when attempts remain or marks failed when attempts are exhausted, and emits timeout_requeued or failed events; use a flowchart with two decision branches](../assets/images/backend/expired-lease-cleanup.png)
Every worker thread periodically runs cleanup:
```text
@@ -570,6 +588,8 @@ There is no separate cron process. This is intentional for the interview version
## Worker Runtime
![prompt for generating an svg image for the worker runtime loop: start worker runner, spawn JOB_WORKER_THREADS threads, each thread periodically cleanup expired leases, claim next job, sleep if none, execute handler, renew lease/progress during work, then complete or retry/fail, and stop claiming new jobs on SIGINT or SIGTERM; use a loop diagram with clear labels](../assets/images/backend/worker-runtime-loop.png)
Run workers with:
```powershell
@@ -643,6 +663,8 @@ Example:
## API
![prompt for generating an svg image for the backend API surface of the job queue: health endpoint, jobs create/list/detail/retry/stats endpoints, job events endpoint, global cursor-paginated event feed, schema/docs endpoints; group endpoints by health, jobs, events, documentation; use REST method badges and arrows to jobs_job/jobs_jobevent tables](../assets/images/backend/api-surface.png)
Base URL in local development:
```text
@@ -1094,4 +1116,3 @@ POST /api/jobs/{id}/retry/
GET /api/schema/
GET /api/docs/
```