fix: clean up temp dirs and close fd leaks in Helm, Kustomize, Cuelang - #1551
fix: clean up temp dirs and close fd leaks in Helm, Kustomize, Cuelang#1551kaizeenn wants to merge 1 commit into
Conversation
…omize, Cuelang Temp directories created with tempfile.mkdtemp() were never removed, and file descriptors returned by tempfile.mkstemp() were discarded without being closed. On large compilation runs this caused disk-space and file- descriptor exhaustion. Changes: - helm.py compile_file: replace mkdtemp() with TemporaryDirectory() context manager so the rendered-chart temp dir is always deleted. - helm.py render_chart: close the mkstemp() fd immediately with os.close(fd), then unlink the temp file after reading it. - helm.py write_helm_values_file: close the mkstemp() fd immediately with os.close(fd) before opening the file for writing. - kustomize.py compile_file: replace two mkdtemp() calls with a single nested TemporaryDirectory() context manager; consolidate the two try/except blocks into one. - cuelang.py compile_file: replace mkdtemp() with TemporaryDirectory() context manager. Fixes kapicorp#1509 Signed-off-by: kaizeenn <khairil0153@gmail.com>
There was a problem hiding this comment.
The mkdtemp → TemporaryDirectory() conversions and both fd closes look correct, and the kustomize except KustomizeTemplateError: raise ahead of the generic handler nicely avoids the double-wrap. One inline nit on exception-safety below; non-blocking.
Two related leaks I found while reviewing, both pre-existing and out of scope here, were opened as separate PRs rather than expanding this one:
- #1555
write_helm_values_file()'s temp file is never unlinked by either caller. The fd half is covered here; the file lifecycle isn't. Complementary to this PR, it does touchcompile_file, so expect a small conflict with whichever of us merges second. - #1554 unrelated to #1509:
cuelang.pyraisesKustomizeTemplateError(copy-paste) whenCuelangTemplateErroralready exists inkapitan/errors.py.
Nice cleanup overall.
| f.seek(0) | ||
| return (f.read(), error_message) | ||
| content = f.read() | ||
| os.unlink(helm_output) |
There was a problem hiding this comment.
os.unlink here isn't exception-safe if helm_cli or f.read() above raises; this line is skipped and the temp file leaks (the same bug class this PR targets). Wrap in try/finally, or let a context manager delete on exit:
with tempfile.NamedTemporaryFile(suffix=".helm_output.yml", mode="w+") as f:
error_message = helm_cli(helm_path, args, stdout=f)
f.seek(0)
return (f.read(), error_message)Low real-world risk since helm_cli returns the error rather than raising non-blocking.
|
@kaizeenn thanks for sending this in. However this approach seems to repeat the same formula in the mentioned input types. Perhaps we can abstract this in the base class or have common generic functions to avoid duplication? |
Summary
Fixes #1509.
Three input compilers created temp directories with
tempfile.mkdtemp()and never cleaned them up. Twotempfile.mkstemp()calls discarded the returned file descriptor (_) without closing it. On large or long-running compilation jobs this causes disk-space exhaustion and file-descriptor leaks.Changes
kapitan/inputs/helm.pycompile_file: replacemkdtemp()withtempfile.TemporaryDirectory()context manager — rendered-chart dir is now always deleted on exit, even if an exception is raised.render_chart: close themkstemp()fd immediately withos.close(fd), thenos.unlink()the temp file after its contents are read.write_helm_values_file: close themkstemp()fd immediately withos.close(fd)before opening the file for writing.kapitan/inputs/kustomize.pycompile_file: replace two separatemkdtemp()calls with a singlewith tempfile.TemporaryDirectory() as temp_dir, tempfile.TemporaryDirectory() as output_dir:block; consolidate the previously splittry/exceptblocks into one, re-raisingKustomizeTemplateErrordirectly so it is not double-wrapped.kapitan/inputs/cuelang.pycompile_file: replacemkdtemp()withtempfile.TemporaryDirectory()context manager.Testing
All pre-existing test failures (helm/kustomize/cuelang suites require
helm/kustomize/cuebinaries not present in CI) are identical before and after this change — no regressions introduced.