fix(python): default show_graph physical plan to the streaming engine - #28826
Open
binaydhakal wants to merge 2 commits into
Open
fix(python): default show_graph physical plan to the streaming engine#28826binaydhakal wants to merge 2 commits into
show_graph physical plan to the streaming engine#28826binaydhakal wants to merge 2 commits into
Conversation
`LazyFrame.show_graph(plan_stage="physical")` uses the default `auto` engine, which plans and executes via the streaming engine. But its `plan_engine` is the literal "auto", so the physical-plan branch fell through to the in-memory renderer and produced a different graph from `engine="streaming"`. Resolve "auto" to "streaming" for plan rendering so the two match. Also raise `ValueError` (not `TypeError`) for an invalid `plan_stage`, since it is an invalid value of a valid type. Closes pola-rs#28802
binaydhakal
requested review from
MarcoGorelli,
alexander-beedie,
c-peters and
ritchie46
as code owners
August 14, 2026 21:52
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #28826 +/- ##
==========================================
- Coverage 81.55% 81.55% -0.01%
==========================================
Files 1888 1888
Lines 267844 267845 +1
Branches 3257 3257
==========================================
- Hits 218445 218441 -4
- Misses 48555 48560 +5
Partials 844 844 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #28802
Problem
LazyFrame.show_graph(plan_stage="physical")uses the defaultautoengine, which plans (and, by default, executes) via the streaming engine. But_AutoEngine.plan_engineis the literal"auto", soshow_graph's physical-plan branch never matched== "streaming"and fell through to the in-memory renderer. The result: the default plot differed fromengine="streaming", even thoughautoruns the streaming plan.Fix
Inside
show_graph, resolveauto→streamingwhen choosing how to build and render the plan, so the default matches the engine that actually runs. The change is scoped toshow_graph:_AutoEngine.plan_engineis intentionally not touched, becausecollect/explainread it to select optimization flags, and broadening it there disables in-memory-only optimizations (e.g. join collapsing) forauto— verified locally that doing so regressestests/unit/lazyframe/test_optimizations.py::test_collapse_joins.As a drive-by (per @TNieuwdorp's note on the issue), an invalid
plan_stagenow raisesValueErrorinstead ofTypeError— it's an invalid value of a valid type.Tests
test_show_graph_phys_auto_defaults_to_streaming— asserts theautophysical plot equalsstreaming's and differs fromin-memory's. Fails onmain, passes with this change.test_show_graph_invalid_stageupdated to expectValueError.Verified locally (polars built from this branch):
collect/explainoutputs are unchanged across all three engines, and the fulltests/unit/lazyframe/suite passes apart from 4 pre-existing serde/pickle failures unrelated to this change.