Files

Amirhossein Khalili 2b0b8cb041 chore: configure django debug toolbar

2026-06-21 15:03:09 +03:30

33 KiB

Raw Blame History

Backend

This backend is a minimal production-aware PostgreSQL job queue implemented with Django and Django REST Framework.

The project intentionally avoids building a general queue platform. There is no Queue model, no Worker model, no Execution model, no Redis abstraction, and no public worker claim endpoint. The important interview requirement is implemented directly:

deterministic job state transitions
valid database state enforced by constraints
safe concurrent claiming with PostgreSQL row locks
retries with exponential backoff
worker leases and expired-lease recovery
append-only execution events for UI/demo visibility
environment-configured multi-threaded workers

The backend has only two queue-domain tables:

jobs_job
jobs_jobevent

All queue behavior is implemented in service functions under jobs/services.py.

High-Level Architecture

PostgreSQL
  |
  | stores valid queue state and append-only events
  |
Django REST API
  |
  | creates jobs, lists jobs, lists events, retries failed jobs, exposes stats
  |
Worker command
  |
  | imports the same backend service code
  | runs N worker threads configured by environment variables
  |
Demo handlers

The API process and worker process use the same codebase. In Docker Compose they are separate services using the same image:

backend -> python manage.py migrate && python manage.py runserver 0.0.0.0:8000
worker  -> python manage.py run_job_workers

The worker does not call HTTP endpoints to claim jobs. It imports claim_next_job, complete_job, fail_job, renew_lease, and cleanup_expired_jobs from jobs/services.py.

Design Scope

Included:

PostgreSQL-backed internal queue
one logical internal queue
priority scheduling
delayed eligibility through available_at
idempotency keys for job creation
deterministic retries with exponential backoff
worker leases
lease renewal for long-running handlers
expired lease cleanup
graceful worker shutdown
DRF API for jobs/events/stats
cursor pagination for global events
limit-offset pagination for jobs
Django Unfold admin

Explicitly not included:

queue CRUD
queue database table
worker database table
execution database table
cancellation state
batch claiming
Redis implementation
cron service
WebSockets/PostgreSQL LISTEN/NOTIFY
exactly-once execution guarantee

This is an at-least-once queue. A worker can perform a side effect and then crash before marking the job as succeeded. For real external work, handlers must be idempotent.

Core Queue Semantics

Statuses

Job.status is intentionally small:

queued
running
succeeded
failed

Supported transitions:

queued  -> running
running -> succeeded
running -> queued      retry after handler failure or lease expiry
running -> failed      attempts exhausted
failed  -> queued      manual retry

Unsupported transitions are rejected by service-level ownership checks and by database shape constraints.

Ownership

A worker owns a job only when all of these are true:

status = running
locked_by = worker_id
attempts = attempt_number_that_worker_claimed
locked_until is in the future

Completion, failure, progress, and lease renewal all verify the worker still owns the same attempt. This prevents a stale worker from overwriting the result of a later attempt.

Example stale worker scenario:

1. Worker A claims job attempt 1.
2. Worker A hangs.
3. Lease expires.
4. Cleanup requeues the job.
5. Worker B claims attempt 2.
6. Worker A wakes up and tries to mark attempt 1 succeeded.
7. The update affects zero rows because attempts no longer matches.

This is one of the most important correctness properties in the project.

Database Model

`Job`

Job is the source of truth for queue state.

Important fields:

Field	Purpose
`id`	UUID primary key.
`type`	Handler routing key, for example `demo.success`.
`payload`	JSON input for the handler.
`status`	One of `queued`, `running`, `succeeded`, `failed`.
`priority`	Higher priority is claimed first. Default is `50`.
`available_at`	Earliest time the job can be claimed. Used for delayed jobs and retry backoff.
`attempts`	Number of claim attempts already made.
`max_attempts`	Maximum allowed attempts before final failure.
`idempotency_key`	Optional unique key to prevent duplicate job creation.
`locked_by`	Ephemeral worker identity for the current running attempt.
`locked_until`	Lease expiry time for the current running attempt.
`last_error`	Latest failure reason.
`result`	JSON result for succeeded jobs.
`finished_at`	Required for terminal statuses.

`JobEvent`

JobEvent is an append-only audit/event timeline for demo visibility and debugging.

Events are not the source of truth for queue state. The Job row is the source of truth. Events explain how the job reached its current state.

Supported event types:

created
claimed
progress
lease_renewed
succeeded
retry_scheduled
failed
timeout_requeued
manual_retry

Database Constraints

The model enforces valid job shape in the database.

Valid Status Shape

queued jobs:

locked_by is null
locked_until is null
finished_at is null

running jobs:

locked_by is not null
locked_until is not null
finished_at is null

succeeded and failed jobs:

locked_by is null
locked_until is null
finished_at is not null

Attempts Constraint

attempts >= 0
max_attempts >= 1
attempts <= max_attempts

Idempotency Constraint

idempotency_key is unique only when present:

unique idempotency_key where idempotency_key is not null

This allows many jobs with no idempotency key while preventing duplicate client-created jobs when a key is supplied.

Indexes

Claim Index

job_claim_idx
fields: available_at, -priority, created_at, id
condition: status = queued

This supports the hot claim query:

WHERE status = queued
  AND available_at <= now()
  AND attempts < max_attempts
ORDER BY priority DESC, available_at ASC, created_at ASC, id ASC

Timeout Index

job_timeout_idx
fields: locked_until
condition: status = running

This supports expired lease cleanup.

Event Indexes

job_event_job_id_idx: job, id
job_event_id_idx: id

These support job detail timelines and global cursor-paginated event feeds.

ERD DBML

Copy this DBML into dbdiagram.io to generate the ERD.

Project minimal_job_queue {
  database_type: "PostgreSQL"
  Note: '''
    Minimal PostgreSQL-backed job queue.

    The queue domain intentionally has two tables:
    - jobs_job: current queue state and lock ownership
    - jobs_jobevent: append-only audit/event timeline

    There is no queue table, worker table, execution table, or task table.
  '''
}

Enum job_status {
  queued
  running
  succeeded
  failed
}

Enum job_event_type {
  created
  claimed
  progress
  lease_renewed
  succeeded
  retry_scheduled
  failed
  timeout_requeued
  manual_retry
}

Table jobs_job {
  id uuid [pk, not null, note: "UUID primary key generated by Django uuid.uuid4"]

  type varchar(120) [not null, note: "Handler routing key, for example demo.success"]
  payload jsonb [not null, default: `'{}'::jsonb`, note: "Handler input payload"]

  status job_status [not null, default: "queued", note: "queued | running | succeeded | failed"]
  priority smallint [not null, default: 50, note: "Higher priority is claimed first"]
  available_at timestamptz [not null, note: "Earliest time the job can be claimed"]

  attempts integer [not null, default: 0, note: "Number of attempts already claimed"]
  max_attempts integer [not null, default: 3, note: "Maximum attempts before final failure"]

  idempotency_key varchar(255) [unique, note: "Nullable unique key. Real DB uses a partial unique index where not null."]

  locked_by varchar(255) [note: "Ephemeral worker id for current running attempt"]
  locked_until timestamptz [note: "Lease expiry for current running attempt"]

  last_error text [not null, default: "", note: "Latest handler or lease error"]
  result jsonb [note: "JSON result after success"]

  created_at timestamptz [not null, note: "Created timestamp"]
  updated_at timestamptz [not null, note: "Updated timestamp"]
  finished_at timestamptz [note: "Set only for succeeded/failed jobs"]

  indexes {
    status [name: "jobs_job_status_idx"]
    available_at [name: "jobs_job_available_at_idx"]
    created_at [name: "jobs_job_created_at_idx"]
    (available_at, priority, created_at, id) [name: "job_claim_idx", note: "Partial index in PostgreSQL: WHERE status = 'queued'. priority is DESC in Django migration."]
    locked_until [name: "job_timeout_idx", note: "Partial index in PostgreSQL: WHERE status = 'running'."]
    idempotency_key [name: "job_idempotency_key_unique", unique, note: "Partial unique index in PostgreSQL: WHERE idempotency_key IS NOT NULL."]
  }

  Note: '''
    Check constraints implemented in Django/PostgreSQL:

    job_status_valid_shape:
      queued:
        locked_by IS NULL
        locked_until IS NULL
        finished_at IS NULL

      running:
        locked_by IS NOT NULL
        locked_until IS NOT NULL
        finished_at IS NULL

      succeeded/failed:
        locked_by IS NULL
        locked_until IS NULL
        finished_at IS NOT NULL

    job_attempts_valid:
      attempts >= 0
      max_attempts >= 1
      attempts <= max_attempts
  '''
}

Table jobs_jobevent {
  id bigint [pk, increment, not null]
  job_id uuid [not null, ref: > jobs_job.id]

  type job_event_type [not null, note: "created | claimed | progress | lease_renewed | succeeded | retry_scheduled | failed | timeout_requeued | manual_retry"]
  attempt integer [not null, default: 0]
  worker_id varchar(255) [note: "Worker identity that emitted the event, if applicable"]
  message text [not null, default: ""]
  data jsonb [not null, default: `'{}'::jsonb`]
  created_at timestamptz [not null]

  indexes {
    type [name: "jobs_jobevent_type_idx"]
    created_at [name: "jobs_jobevent_created_at_idx"]
    (job_id, id) [name: "job_event_job_id_idx"]
    id [name: "job_event_id_idx"]
  }

  Note: '''
    Append-only audit timeline.
    Job state is not derived from this table.
    The jobs_job row remains the source of truth.
  '''
}

Claiming Algorithm

The worker claims a job in a short transaction:

queryset = Job.objects.filter(
    status=Job.Status.QUEUED,
    available_at__lte=now,
    attempts__lt=F("max_attempts"),
).order_by("-priority", "available_at", "created_at", "id")

queryset = queryset.select_for_update(skip_locked=True)
job = queryset.first()

Then the selected job is updated:

status       = running
attempts     = attempts + 1
locked_by    = worker_id
locked_until = now + lease_seconds
last_error   = ""
result       = null

And a claimed event is inserted.

Why SKIP LOCKED matters:

multiple worker threads can query at the same time
one worker locks a candidate row
other workers skip locked rows instead of blocking
each job attempt is claimed by at most one worker

The handler runs outside the claim transaction. The database row is not held locked while work is executing.

Completion Algorithm

Success is persisted only if the worker still owns the same attempt:

WHERE id = job_id
  AND status = running
  AND locked_by = worker_id
  AND attempts = attempt

If the update affects zero rows, the worker lost ownership. The service returns None and does not force the job to succeeded.

On success:

status       = succeeded
result       = handler result
locked_by    = null
locked_until = null
finished_at  = now

Then a succeeded event is inserted.

Failure And Retry Algorithm

On handler failure, the worker calls fail_job.

If attempts remain:

running -> queued
available_at = now + backoff_delay
locked_by = null
locked_until = null
last_error = error message
event = retry_scheduled

If attempts are exhausted:

running -> failed
locked_by = null
locked_until = null
finished_at = now
last_error = error message
event = failed

Backoff is deterministic:

delay_seconds = min(base_seconds * 2 ^ (attempt - 1), max_backoff_seconds)

With defaults:

attempt 1 -> 5 seconds
attempt 2 -> 10 seconds
attempt 3 -> 20 seconds
...
max delay -> 300 seconds

Lease Renewal

Long-running handlers can renew their lease:

context.renew()

Lease renewal succeeds only if:

id = job_id
status = running
locked_by = worker_id
attempts = attempt

This means a stale worker cannot renew an old attempt after the job has already been requeued and claimed by another worker.

The demo demo.slow handler renews leases while it sleeps.

The demo demo.timeout handler intentionally does not renew leases so the UI can demonstrate lease expiry and requeue/failure behavior.

Expired Lease Cleanup

Every worker thread periodically runs cleanup:

status = running
locked_until < now

Expired jobs are locked with select_for_update(skip_locked=True).

If attempts remain:

running -> queued
available_at = now + backoff_delay
event = timeout_requeued

If attempts are exhausted:

running -> failed
event = failed

There is no separate cron process. This is intentional for the interview version: the worker process owns both execution and lease cleanup.

Worker Runtime

Run workers with:

python manage.py run_job_workers

In Docker:

docker compose up worker

The worker runner:

starts JOB_WORKER_THREADS Python threads
gives each thread its own generated worker id
periodically cleans expired leases
claims one eligible job
executes the matching handler
marks the job succeeded, retry scheduled, or failed
sleeps when no job is available
stops claiming new jobs on SIGINT/SIGTERM

Worker id format:

hostname:pid:thread-number:short-random-id

Example:

638b98e627f5:1:thread-1:4e3b5893

Environment Variables

Django/PostgreSQL

Variable	Default	Purpose
`DEBUG`	`true`	Enables debug mode and permissive CORS in local development.
`ENABLE_DJANGO_DEBUG_TOOLBAR`	`true` when `DEBUG=true`	Enables Django Debug Toolbar for local backend requests.
`DJANGO_INTERNAL_IPS`	`127.0.0.1,localhost,host.docker.internal`	Internal IP allowlist used by Django Debug Toolbar.
`DJANGO_SECRET_KEY`	local dev key	Django secret key. Override outside local development.
`DJANGO_ALLOWED_HOSTS`	`localhost,127.0.0.1,0.0.0.0`	Allowed hosts.
`DJANGO_CORS_ALLOWED_ORIGINS`	`http://localhost:5173,http://127.0.0.1:5173`	Frontend origins.
`POSTGRES_DB`	`job_queue`	Database name.
`POSTGRES_USER`	`postgres`	Database user.
`POSTGRES_PASSWORD`	`postgres`	Database password.
`POSTGRES_HOST`	`localhost`	Database host. In Docker this is `postgres`.
`POSTGRES_PORT`	`5432`	Database port.
`POSTGRES_CONN_MAX_AGE`	`0`	Django persistent DB connection age.
`TIME_ZONE`	`Asia/Tehran`	Django timezone.

Worker

Variable	Default	Purpose
`JOB_WORKER_THREADS`	`4`	Number of worker threads in the worker process.
`JOB_WORKER_POLL_INTERVAL_MS`	`500`	Sleep interval when no job is claimable.
`JOB_WORKER_LEASE_SECONDS`	`30`	Lock lease duration for a running attempt.
`JOB_WORKER_CLEANUP_INTERVAL_SECONDS`	`5`	How often each worker thread checks expired leases.
`JOB_WORKER_CLEANUP_BATCH_SIZE`	`100`	Max expired leases cleaned in one cleanup pass.
`JOB_WORKER_BACKOFF_BASE_SECONDS`	`5`	Base retry backoff.
`JOB_WORKER_BACKOFF_MAX_SECONDS`	`300`	Maximum retry delay.
`JOB_WORKER_MAX_ATTEMPTS_DEFAULT`	`3`	Default max attempts for jobs that do not specify one.
`JOB_WORKER_LOG_LEVEL`	`INFO`	Worker terminal log level. Use `DEBUG` for detailed polling/claim/execution logs.

API

Base URL in local development:

http://localhost:8000/api

Health

GET /api/health/

Response:

{
  "ok": true,
  "database": "ok"
}

If the database cannot be reached:

{
  "ok": false,
  "database": "error",
  "detail": "..."
}

Create Job

POST /api/jobs/

Body:

{
  "type": "demo.success",
  "payload": {
    "sleep_seconds": 1
  },
  "priority": 50,
  "available_at": null,
  "max_attempts": 3,
  "idempotency_key": "optional-client-key"
}

Behavior:

creates a queued job
sets available_at to now when omitted/null
emits a created event
returns 201 Created for a new job
returns 200 OK with the existing job when idempotency_key already exists

List Jobs

GET /api/jobs/?limit=25&offset=0&status=queued&type=demo

Pagination:

limit-offset pagination
default limit: 25
max limit: 100

Filters:

Query param	Behavior
`status`	Exact status match.
`type`	Case-insensitive substring match against job type.

Response:

{
  "count": 53,
  "next": "http://localhost:8000/api/jobs/?limit=25&offset=25",
  "previous": null,
  "results": [
    {
      "id": "8421aca2-7d0c-484c-9c19-e7e324a39e21",
      "type": "demo.success",
      "payload": {
        "sleep_seconds": 1
      },
      "status": "queued",
      "priority": 50,
      "available_at": "2026-06-21T02:33:01.725170+03:30",
      "attempts": 0,
      "max_attempts": 3,
      "idempotency_key": null,
      "locked_by": null,
      "locked_until": null,
      "last_error": "",
      "result": null,
      "created_at": "2026-06-21T02:32:46.176967+03:30",
      "updated_at": "2026-06-21T02:32:46.176967+03:30",
      "finished_at": null
    }
  ]
}

Get Job Detail

GET /api/jobs/{id}/

Returns the same job shape as the list result item.

Get Events For One Job

GET /api/jobs/{id}/events/

Returns events ordered by ascending id:

[
  {
    "id": 1,
    "job": "8421aca2-7d0c-484c-9c19-e7e324a39e21",
    "type": "created",
    "attempt": 0,
    "worker_id": null,
    "message": "Job created",
    "data": {
      "type": "demo.success"
    },
    "created_at": "2026-06-21T02:32:46.176967+03:30"
  }
]

Global Event Feed

GET /api/job-events/?limit=50&type=failed&job_id={uuid}

Pagination:

cursor pagination
ordering: newest first by id
default page size: 50
max page size: 200

Filters:

Query param	Behavior
`type`	Exact event type match.
`job_id`	Events for one job.

Response:

{
  "next": "http://localhost:8000/api/job-events/?cursor=cD05NzM%3D&limit=50",
  "previous": null,
  "results": [
    {
      "id": 973,
      "job": "8421aca2-7d0c-484c-9c19-e7e324a39e21",
      "type": "failed",
      "attempt": 3,
      "worker_id": "638b98e627f5:1:thread-1:4e3b5893",
      "message": "Intentional demo failure",
      "data": {
        "error": "Intentional demo failure"
      },
      "created_at": "2026-06-21T02:33:20.600665+03:30"
    }
  ]
}

Cursor pagination is used for the event feed because events are append-only and live. Offset pagination can drift when new rows are inserted while the user is paging.

Job Stats

GET /api/jobs/stats/

Response:

{
  "total": 53,
  "by_status": {
    "queued": 0,
    "running": 0,
    "succeeded": 41,
    "failed": 12
  },
  "overdue_running": 0,
  "retries_pending": 0,
  "oldest_queued_at": null
}

Retry Failed Job

POST /api/jobs/{id}/retry/

Only failed jobs can be manually retried.

Behavior:

failed -> queued
attempts = 0
available_at = now
lock fields cleared
last_error cleared
result cleared
finished_at cleared
event = manual_retry

Built-In Demo Job Types

Handlers are registered in jobs/handlers.py.

Type	Behavior
`demo.success`	Sleeps briefly, renews lease, emits progress, succeeds.
`demo.fail`	Emits progress and raises an intentional error.
`demo.slow`	Sleeps longer, renews lease, emits progress, succeeds.
`demo.timeout`	Sleeps without lease renewal to demonstrate expired lease cleanup.
`demo.flaky`	Fails until a configured attempt, then succeeds.

Example payloads:

{
  "sleep_seconds": 3
}

{
  "error": "Intentional demo failure"
}

{
  "fail_until_attempt": 2
}

Django Admin

The project uses django-unfold for the Django admin UI.

URL:

http://localhost:8000/admin/

Create an admin user:

docker compose exec backend python manage.py createsuperuser

The admin is useful for inspecting raw job state, event rows, locks, retry timestamps, and failed jobs.

Local Development

Install dependencies in the backend environment:

cd backend
python -m venv .venv
.\.venv\Scripts\python.exe -m pip install -r requirements.txt

Run migrations:

.\.venv\Scripts\python.exe manage.py migrate

Run API:

.\.venv\Scripts\python.exe manage.py runserver 0.0.0.0:8000

Run workers:

.\.venv\Scripts\python.exe manage.py run_job_workers

Run the full Docker stack from the repository root:

docker compose up --build

Rebuild only backend and worker:

docker compose up -d --build backend worker

Stop all Docker services:

docker compose down

Testing

Run backend checks:

cd backend
.\.venv\Scripts\python.exe -m ruff check .
$env:TEST_DATABASE_ENGINE='sqlite'; .\.venv\Scripts\python.exe -m pytest

Run backend tests with coverage:

cd backend
$env:TEST_DATABASE_ENGINE='sqlite'; .\.venv\Scripts\python.exe -m pytest --cov --cov-report=term-missing

The test suite covers:

successful claim
concurrent claim prevention on PostgreSQL
deterministic priority and available_at ordering
idempotent create
retry scheduling with exponential backoff
exhausted attempts becoming failed
expired lease cleanup
stale worker cannot complete old attempt
lease renewal ownership checks
manual retry
worker shutdown flag
jobs limit-offset pagination
global event cursor pagination

The PostgreSQL-specific concurrent claim test is skipped under SQLite because SKIP LOCKED is PostgreSQL behavior.

Assumptions And Simplifications

This backend intentionally chooses a small, defensible design for the interview task. The following are explicit assumptions and simplifications:

Single internal queue: there is no Queue table because queue routing is not part of the assignment. Job selection is controlled by priority, available_at, and deterministic ordering.
Ephemeral workers: there is no Worker table because workers self-identify through generated locked_by values and JobEvent.worker_id. Persisting worker rows would require registration, heartbeat cleanup, and stale-row management.
No execution table: JobEvent is enough for demo visibility and debugging. A larger production system may keep immutable per-attempt execution rows for duration metrics, artifacts, and richer attempt history.
No cancellation state: cancellation is omitted to keep the state machine small and focused on queued, running, succeeded, and failed.
No batch claiming: workers claim one job at a time. This keeps correctness and tests easier to reason about, although batch claiming can improve throughput.
No WebSockets or PostgreSQL LISTEN/NOTIFY: the UI uses polling and cursor pagination for deterministic demo behavior.
No authentication: the API is local/demo focused. Production would require authentication, authorization, and role-based controls for write/operator actions.
No retention policy: JobEvent grows forever in this version. Production should archive, partition, or delete old events.
Clock assumption: lease expiry depends on database and application clocks being reasonably synchronized.
Handler idempotency assumption: external side effects must tolerate retries because delivery is at least once.

Engineering Concepts Demonstrated

The implementation is small, but it deliberately demonstrates production-relevant backend concepts:

transactional state machine
row-level locking
SELECT ... FOR UPDATE SKIP LOCKED
partial indexes for hot queue queries
database check constraints for valid row shape
idempotency key for duplicate create protection
ownership guard using locked_by + attempts
lease-based failure recovery
automatic exponential backoff
at-least-once delivery semantics
cursor pagination for append-only event feeds
limit-offset pagination for bounded job lists
append-only audit log
graceful worker shutdown
environment-based runtime configuration

What Would Change For A Larger Production System

The current design is intentionally minimal. A larger production queue would likely add:

Redis or a dedicated broker for very high throughput and broader distributed workloads.
Authentication and authorization for all write endpoints and operator actions.
Cancellation or cooperative stop states for jobs that should no longer run.
Dead-letter metadata or a dead-letter inspection view for permanently failed jobs.
Metrics, alerting, and dashboards for queue depth, job age, throughput, failures, and retries.
Event retention, archival, or PostgreSQL partitioning for JobEvent.
A dedicated execution table if detailed per-attempt history is required.
Queue pause, drain, and resume controls for operational maintenance.
Batch claiming if single-job claims become a throughput bottleneck.

Operational Notes

At-Least-Once Delivery

This queue provides at-least-once execution.

It prevents two active workers from owning the same job attempt. It does not guarantee that a handler's external side effects happen exactly once.

Example:

1. Worker performs an external API call.
2. Worker process crashes before marking the job succeeded.
3. Lease expires.
4. Another worker retries the job.

The external API call may happen twice unless the handler is idempotent.

Why PostgreSQL

PostgreSQL is used because the assignment focuses on deterministic transactional state and safe concurrent claiming.

SELECT ... FOR UPDATE SKIP LOCKED gives a simple and reliable concurrency primitive for a small internal queue:

transactional claim
row-level ownership
no separate broker
easy inspection through SQL/admin/UI

For a high-throughput distributed production queue, Redis-backed tools such as BullMQ or Sidekiq are often better. This project intentionally stays PostgreSQL-first for interview clarity and fewer moving parts.

Why No Queue Model

There is one internal logical queue. type, priority, and available_at are enough for this demo.

Adding queue CRUD would add API surface and database complexity without improving the core correctness demonstration.

Why No Worker Model

Workers are ephemeral. A worker id is generated at process/thread startup and stored only in locked_by and events.

A worker table would require registration, heartbeat cleanup, stale row management, and admin decisions that are not needed for this assignment.

Why No Execution Model

Job.attempts stores the current attempt count, and JobEvent stores the audit timeline.

That is enough for this interview implementation. A separate immutable JobExecution table would be useful if the system needed detailed duration metrics, per-attempt artifacts, or long-term attempt history beyond events.

Important Files

File	Purpose
`config/settings.py`	Django, database, DRF, Unfold, and worker settings.
`jobs/models.py`	`Job` and `JobEvent` models, constraints, indexes.
`jobs/services.py`	Queue state machine and transactional service functions.
`jobs/handlers.py`	Demo job handlers.
`jobs/worker.py`	Threaded worker runner.
`jobs/management/commands/run_job_workers.py`	Worker management command and logging setup.
`jobs/views.py`	DRF API views.
`jobs/serializers.py`	API serializers.
`jobs/admin.py`	Django Unfold admin registration.
`jobs/tests/`	Backend tests.

Current API Summary

GET  /api/health/
GET  /api/config/
POST /api/jobs/
GET  /api/jobs/?limit=25&offset=0&status=queued&type=demo
GET  /api/jobs/{id}/
GET  /api/jobs/{id}/events/
GET  /api/job-events/?limit=50&type=failed&job_id={uuid}
GET  /api/jobs/stats/
POST /api/jobs/{id}/retry/
GET  /api/schema/
GET  /api/docs/

33 KiB Raw Blame History

Backend

High-Level Architecture

Design Scope

Core Queue Semantics

Statuses

Ownership

Database Model

Job

JobEvent

Database Constraints

Valid Status Shape

Attempts Constraint

Idempotency Constraint

Indexes

Claim Index

Timeout Index

Event Indexes

ERD DBML

Claiming Algorithm

Completion Algorithm

Failure And Retry Algorithm

Lease Renewal

Expired Lease Cleanup

Worker Runtime

Environment Variables

Django/PostgreSQL

Worker

API

Health

Create Job

List Jobs

Get Job Detail

Get Events For One Job

Global Event Feed

Job Stats

Retry Failed Job

Built-In Demo Job Types

Django Admin

Local Development

Testing

Assumptions And Simplifications

Engineering Concepts Demonstrated

What Would Change For A Larger Production System

Operational Notes

At-Least-Once Delivery

Why PostgreSQL

Why No Queue Model

Why No Worker Model

Why No Execution Model

Important Files

Current API Summary

33 KiB

Raw Blame History

`Job`

`JobEvent`