2.5 KiB
ADR 0003: Ent application persistence with pgx bulk ingestion
- Status: Accepted
- Date: 2026-08-20
Context
The backend requires an ORM-backed relational model with explicit entity relationships and a repeatable path from schema definitions to an ERD. The source archive contains more than 2.2 million payment-attempt rows, making row-by-row ORM insertion unsuitable for ingestion. Production schema changes must remain reviewable and reversible rather than occurring implicitly at process startup.
Decision
Use Ent for application persistence, relationship modeling, and type-safe access within capability adapters. Keep Ent-generated artifacts and the schema definitions that produce them in Git according to the future scaffold specification.
Use pgx directly for the importer’s PostgreSQL staging boundary. Feed CopyFrom with a streaming CopyFromSource (or equivalent bounded batches) so decompressed CSV rows are not accumulated in memory. Validate and promote staged data through explicit transactions/steps described by the import spec; bulk loading must not bypass reconciliation or data-quality rules.
Use Atlas, Ent's supported migration engine, to generate/plan versioned SQL migrations from the reviewed Ent schema. Review and commit the SQL to Git, then apply it through a dedicated deployment migration step. Atlas is build/deployment tooling, not a runtime service. Ent automatic schema creation is allowed only in disposable development/test contexts if the later implementation spec permits it; it must never run as part of production application startup.
Consequences
- Domain-oriented code gains typed entities, edges, predicates, and compile-time feedback.
- Engineers must still inspect generated SQL, query plans, indexes, transaction boundaries, and possible N+1 behavior; the ORM is not a performance or correctness boundary.
- The importer owns a deliberate lower-level pgx path and cannot reuse ordinary Ent writes for convenience.
- Both Ent and pgx require compatible pool/transaction ownership and context propagation; adapters must not hide unbounded work.
- Atlas-generated versioned migration files—not runtime diffs—are the production contract. CI validates migration integrity and drift before deployment.
References
- Ent versioned migrations: https://entgo.io/docs/versioned/intro/
- Atlas versioned migrations: https://atlasgo.io/concepts/declarative-vs-versioned#versioned-migrations
- pgx v5
CopyFrom: https://pkg.go.dev/github.com/jackc/pgx/v5#Conn.CopyFrom