fix(build): prevent Redis connections during Docker collectstatic build step#52011
Draft
mendral-app[bot] wants to merge 1 commit intomasterfrom
Draft
fix(build): prevent Redis connections during Docker collectstatic build step#52011mendral-app[bot] wants to merge 1 commit intomasterfrom
mendral-app[bot] wants to merge 1 commit intomasterfrom
Conversation
Contributor
|
@mendral-app you already changed the same call site? |
…ld step Two call sites attempt live Redis connections when Django initialises during `python manage.py collectstatic` in the Docker build layer: 1. PostHogConfig.ready() calls queue_sync_hog_function_templates() which immediately does r.setnx() to acquire a distributed lock. Guard with the STATIC_COLLECTION env var already set by the Dockerfile. 2. KernelRuntimeService is instantiated at module level in kernel_runtime.py, registering an atexit hook that calls shutdown_all() → acquires a Redis lock on process exit (i.e. when collectstatic finishes). Replace the module-level singleton with a lazy getter so the instance (and its atexit hook) is only created on first real use. Both errors were caught and logged so builds did not fail, but they produced noisy ConnectionError tracebacks on every image build.
0381909 to
61f9912
Compare
Contributor
Author
|
Good catch @webjunkie! Yes, master already added |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
python manage.py collectstaticin the Docker image build layer, producingredis.exceptions.ConnectionError: Error 111 connecting to localhost:6379. Connection refused.tracebacks on every build (bothlinux/amd64andlinux/arm64). The errors are caught so builds don't fail, but the noise is unnecessary.Root cause
Call site 1 —
posthog/apps.py:106:PostHogConfig.ready()callsqueue_sync_hog_function_templates()unconditionally for all non-test invocations. That function immediately doesr.setnx(lock_key, 1)to acquire a distributed Redis lock before queuing a Celery task. No Redis is available duringdocker build.Call site 2 —
products/notebooks/backend/kernel_runtime.py:1564:KernelRuntimeServicewas instantiated at module level, which registers anatexithook that callsshutdown_all()→ acquires a Redis lock on process exit. The module is imported transitively viaNotebooksConfig.ready()→api/notebook.py→kernel_runtime.py. The atexit fires whencollectstaticfinishes.Fix
queue_sync_hog_function_templates()with theSTATIC_COLLECTIONenv var already set by the Dockerfile for this step.KernelRuntimeService()singleton with a lazy getter — the instance (and its atexit/signal hooks) is only created on first real use, not at import time.Fixes: https://app.mendral.com/insights/01KMD080H98VRD7A9GER5RHWDY
Note
Created by Mendral. Tag @mendral-app with feedback or questions.