Skip to content

fix(python): default show_graph physical plan to the streaming engine - #28826

Open
binaydhakal wants to merge 2 commits into
pola-rs:mainfrom
binaydhakal:fix-show-graph-auto-streaming
Open

fix(python): default show_graph physical plan to the streaming engine#28826
binaydhakal wants to merge 2 commits into
pola-rs:mainfrom
binaydhakal:fix-show-graph-auto-streaming

Conversation

@binaydhakal

Copy link
Copy Markdown

Closes #28802

Problem

LazyFrame.show_graph(plan_stage="physical") uses the default auto engine, which plans (and, by default, executes) via the streaming engine. But _AutoEngine.plan_engine is the literal "auto", so show_graph's physical-plan branch never matched == "streaming" and fell through to the in-memory renderer. The result: the default plot differed from engine="streaming", even though auto runs the streaming plan.

import polars as pl
lf = pl.LazyFrame({"a": [1, 2, 3]})
lf.show_graph(plan_stage="physical")                     # in-memory plan (wrong)
lf.show_graph(plan_stage="physical", engine="streaming") # streaming plan

Fix

Inside show_graph, resolve autostreaming when choosing how to build and render the plan, so the default matches the engine that actually runs. The change is scoped to show_graph: _AutoEngine.plan_engine is intentionally not touched, because collect/explain read it to select optimization flags, and broadening it there disables in-memory-only optimizations (e.g. join collapsing) for auto — verified locally that doing so regresses tests/unit/lazyframe/test_optimizations.py::test_collapse_joins.

As a drive-by (per @TNieuwdorp's note on the issue), an invalid plan_stage now raises ValueError instead of TypeError — it's an invalid value of a valid type.

Tests

  • test_show_graph_phys_auto_defaults_to_streaming — asserts the auto physical plot equals streaming's and differs from in-memory's. Fails on main, passes with this change.
  • test_show_graph_invalid_stage updated to expect ValueError.

Verified locally (polars built from this branch): collect/explain outputs are unchanged across all three engines, and the full tests/unit/lazyframe/ suite passes apart from 4 pre-existing serde/pickle failures unrelated to this change.

`LazyFrame.show_graph(plan_stage="physical")` uses the default `auto`
engine, which plans and executes via the streaming engine. But its
`plan_engine` is the literal "auto", so the physical-plan branch fell
through to the in-memory renderer and produced a different graph from
`engine="streaming"`. Resolve "auto" to "streaming" for plan rendering
so the two match.

Also raise `ValueError` (not `TypeError`) for an invalid `plan_stage`,
since it is an invalid value of a valid type.

Closes pola-rs#28802
@github-actions github-actions Bot added A-streaming Related to the streaming engine fix Bug fix python Related to Python Polars first-contribution First contribution by user title needs formatting labels Aug 14, 2026
@codecov

codecov Bot commented Aug 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.55%. Comparing base (9b5d73f) to head (08d5897).

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #28826      +/-   ##
==========================================
- Coverage   81.55%   81.55%   -0.01%     
==========================================
  Files        1888     1888              
  Lines      267844   267845       +1     
  Branches     3257     3257              
==========================================
- Hits       218445   218441       -4     
- Misses      48555    48560       +5     
  Partials      844      844              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-streaming Related to the streaming engine first-contribution First contribution by user fix Bug fix python Related to Python Polars title needs formatting

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LazyFrame.show_graph() does not yet default to streaming in 2.0

1 participant