Skip to content

fix(build): prevent Redis connections during Docker collectstatic build step#52011

Draft
mendral-app[bot] wants to merge 1 commit intomasterfrom
mendral/fix-redis-connection-during-collectstatic
Draft

fix(build): prevent Redis connections during Docker collectstatic build step#52011
mendral-app[bot] wants to merge 1 commit intomasterfrom
mendral/fix-redis-connection-during-collectstatic

Conversation

@mendral-app
Copy link
Contributor

@mendral-app mendral-app bot commented Mar 23, 2026

Summary

  • Two separate call sites attempt live Redis connections when Django initialises during python manage.py collectstatic in the Docker image build layer, producing redis.exceptions.ConnectionError: Error 111 connecting to localhost:6379. Connection refused. tracebacks on every build (both linux/amd64 and linux/arm64). The errors are caught so builds don't fail, but the noise is unnecessary.

Root cause

Call site 1 — posthog/apps.py:106: PostHogConfig.ready() calls queue_sync_hog_function_templates() unconditionally for all non-test invocations. That function immediately does r.setnx(lock_key, 1) to acquire a distributed Redis lock before queuing a Celery task. No Redis is available during docker build.

Call site 2 — products/notebooks/backend/kernel_runtime.py:1564: KernelRuntimeService was instantiated at module level, which registers an atexit hook that calls shutdown_all() → acquires a Redis lock on process exit. The module is imported transitively via NotebooksConfig.ready()api/notebook.pykernel_runtime.py. The atexit fires when collectstatic finishes.

Fix

  1. Guard queue_sync_hog_function_templates() with the STATIC_COLLECTION env var already set by the Dockerfile for this step.
  2. Replace the module-level KernelRuntimeService() singleton with a lazy getter — the instance (and its atexit/signal hooks) is only created on first real use, not at import time.

Fixes: https://app.mendral.com/insights/01KMD080H98VRD7A9GER5RHWDY


Note

Created by Mendral. Tag @mendral-app with feedback or questions.

@mendral-app mendral-app bot requested a review from webjunkie March 23, 2026 21:10
@webjunkie
Copy link
Contributor

@mendral-app you already changed the same call site?


<<<<<<< mendral/fix-redis-connection-during-collectstatic
        # Skip during Docker image builds (collectstatic) — no Redis available at build time
        if not settings.TEST and not os.environ.get("STATIC_COLLECTION"):
=======
        # Skip during collectstatic (STATIC_COLLECTION=1 in Dockerfile) — no Redis available at build time
        if not settings.TEST and not settings.STATIC_COLLECTION:
>>>>>>> master
            queue_sync_hog_function_templates()

…ld step

Two call sites attempt live Redis connections when Django initialises
during `python manage.py collectstatic` in the Docker build layer:

1. PostHogConfig.ready() calls queue_sync_hog_function_templates() which
   immediately does r.setnx() to acquire a distributed lock.
   Guard with the STATIC_COLLECTION env var already set by the Dockerfile.

2. KernelRuntimeService is instantiated at module level in kernel_runtime.py,
   registering an atexit hook that calls shutdown_all() → acquires a Redis
   lock on process exit (i.e. when collectstatic finishes).
   Replace the module-level singleton with a lazy getter so the instance
   (and its atexit hook) is only created on first real use.

Both errors were caught and logged so builds did not fail, but they
produced noisy ConnectionError tracebacks on every image build.
@mendral-app mendral-app bot force-pushed the mendral/fix-redis-connection-during-collectstatic branch from 0381909 to 61f9912 Compare March 23, 2026 21:14
@mendral-app
Copy link
Contributor Author

mendral-app bot commented Mar 23, 2026

Good catch @webjunkie! Yes, master already added settings.STATIC_COLLECTION for that call site, so my change to os.environ.get("STATIC_COLLECTION") was redundant and conflicting. I've rebased the branch onto master and dropped my apps.py change — the PR now only contains the kernel_runtime.py lazy initialization fix (which master doesn't have yet).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant