Skip to content

fix: postgres 18 compatibility + root docker-compose + ETL-tuned pg config#2

Merged
amgdy merged 6 commits into
mainfrom
copilot/validate-dev-container-build-run
Apr 16, 2026
Merged

fix: postgres 18 compatibility + root docker-compose + ETL-tuned pg config#2
amgdy merged 6 commits into
mainfrom
copilot/validate-dev-container-build-run

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 16, 2026

Three related issues with the Docker/Compose setup: PG18 breaks on wal_level=minimal without max_wal_senders=0, PG18 changed the expected volume mount path, and there was no compose file for running the production app end-to-end.

PostgreSQL 18 breaking changes (both compose files)

  • max_wal_senders default changed 0→10 in PG18 — setting wal_level=minimal now requires explicitly adding -c max_wal_senders=0, otherwise PG crashes on startup:
    FATAL: WAL streaming ("max_wal_senders" > 0) requires "wal_level" to be "replica" or "logical"
    
  • Volume mount path changed — PG18 manages its own versioned subdirectory (18/main) and expects the volume at /var/lib/postgresql, not /var/lib/postgresql/data

ETL-optimised postgres config

Tuned for the bulk GitHub API → star-schema upsert workload (both compose files):

Setting Value Rationale
shared_buffers 256MB Dimension table cache hits during repeated upserts
work_mem 16MB Avoids disk spill per upsert hash-join/sort op
maintenance_work_mem 64MB Faster index rebuilds post bulk-insert batch
effective_cache_size 512MB Planner prefers index scans on dimension tables
max_wal_size 512MB Caps WAL between checkpoints
checkpoint_timeout 15min Prevents mid-import checkpoint stalls
default_statistics_target 200 Better post-ANALYZE plans on skewed dims
shm_size 512m Matches shared_buffers

New root docker-compose.yml

Adds a compose file for running the full production stack locally (previously only the devcontainer compose existed, which runs sleep infinity):

docker compose up --build   # builds app/Dockerfile, starts PG, opens :3000

ADMIN_PASSWORD defaults to changeme123 — override via .env or environment variable before use.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Postgres 18 compatibility fixes and a new root Compose setup to run the app + database locally, with Postgres settings tuned for the ETL/upsert workload.

Changes:

  • Added a root docker-compose.yml to build/run the production Docker image alongside Postgres.
  • Updated Postgres 18 Compose settings (volume mount path + max_wal_senders=0 with wal_level=minimal) and applied ETL-oriented Postgres tuning in both compose files.
  • Simplified .devcontainer/docker-compose.yml by removing deploy.resources and wiring services onto an explicit network.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
docker-compose.yml New end-to-end local Compose stack (app + Postgres 18) with ETL-tuned Postgres config and healthchecks.
.devcontainer/docker-compose.yml Aligns devcontainer Postgres 18 config and healthcheck with the new tuning/compatibility requirements; adds explicit devnet network.

Comment thread docker-compose.yml Outdated
Comment thread docker-compose.yml
@amgdy amgdy marked this pull request as ready for review April 16, 2026 11:30
@amgdy amgdy merged commit ae9fab4 into main Apr 16, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants