Support `pl.Expr.to_physical/hash` by mroeschke · Pull Request #23313 · rapidsai/cudf

mroeschke · 2026-07-17T01:24:47Z

Description

https://docs.pola.rs/api/python/dev/reference/expressions/api/polars.Expr.to_physical.html
https://docs.pola.rs/api/python/dev/reference/expressions/api/polars.Expr.hash.html

Checklist

I am familiar with the Contributing Guidelines.
New or existing tests cover these changes.
The documentation is up to date with these changes.

…physical

coderabbitai · 2026-07-17T01:33:03Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 546bd9f1-a417-40cb-9722-a53e2a64a135

📥 Commits

Reviewing files that changed from the base of the PR and between 3fb62e8 and 65ea3e2.

📒 Files selected for processing (1)

python/pylibcudf/pylibcudf/unary.pyx

🚧 Files skipped from review as they are similar to previous changes (1)

python/pylibcudf/pylibcudf/unary.pyx

📝 Walkthrough

Summary by CodeRabbit

New Features
- Added expression support for hash (seedable, 64-bit output) and to_physical for supported date/datetime/duration types.
- Introduced bit_cast to convert columns to a target data type while preserving ordering/null-order metadata and column name.
Bug Fixes
- to_physical now raises a clear NotImplementedError for nested logical dtypes (e.g., list/struct containing date types).
Tests
- Added GPU coverage for to_physical (positive and nested-logical negative cases) and improved hash tests for schema correctness, determinism, and seed sensitivity.

Walkthrough

Adds hash and to_physical support to cuDF-Polars unary expressions, including libcudf bit-casting bindings, nested-type validation, metadata preservation, hashing, and GPU test coverage.

Changes

Unary expression support

Layer / File(s)	Summary
Bit-cast binding `python/pylibcudf/.../column_view.pxd`, `python/pylibcudf/pylibcudf/unary.*`	Exposes libcudf `bit_cast` through Cython declarations, typing, exports, and a device-column wrapper.
Unary hash and physical conversion `python/cudf_polars/cudf_polars/dsl/expressions/unary.py`	Registers `hash` and `to_physical` as pointwise functions, validates nested logical types, computes xxhash values, and applies physical bit-casts while preserving column metadata.
Unary behavior validation `python/cudf_polars/tests/expressions/test_numeric_unaryops.py`	Tests physical conversions, unsupported nested logical types, hash seeds, deterministic results, and UInt64 output schemas.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

rapidsai/cudf#23212 — Extends UnaryFunction with additional unary operations.
rapidsai/cudf#23250 — Covers related unary hash support and changes to .hash() tests.
rapidsai/cudf#23284 — Extends the same UnaryFunction machinery with additional unary operations.

Suggested reviewers: galipremsagar, rjzamora

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely summarizes the main change: adding support for `pl.Expr.to_physical` and `pl.Expr.hash`.
Description check	✅ Passed	The description is directly related to the changeset and references the affected Polars APIs and issue.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

coderabbitai

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@python/cudf_polars/cudf_polars/dsl/expressions/unary.py`:
- Around line 491-497: Update the unary hash handling in the name == "hash"
branch to honor all provided Polars seeds instead of silently using only the
first; since plc.hashing.xxhash_64 accepts one seed, implement the required
multi-seed behavior or explicitly raise NotImplementedError for multiple seeds,
while preserving single-seed hashing.

In `@python/cudf_polars/tests/expressions/test_numeric_unaryops.py`:
- Around line 219-232: Update test_hash to validate collected hash values and
null handling by replacing the schema-only assertion with
assert_gpu_result_equal(q, engine=engine). Preserve both parametrized seed
variants and the existing query inputs.

In `@python/pylibcudf/pylibcudf/unary.pyx`:
- Around line 175-191: Expand the bit_cast docstring to document its
bit-preserving conversion behavior, the input Column, target DataType, optional
stream and memory resource parameters, and the returned Column. Follow the
surrounding public wrapper docstring conventions and describe the result without
changing the implementation.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 9f971490-e674-441f-ab50-123b3ff57d23

📥 Commits

Reviewing files that changed from the base of the PR and between 08bd466 and 670352e.

📒 Files selected for processing (5)

python/cudf_polars/cudf_polars/dsl/expressions/unary.py
python/cudf_polars/tests/expressions/test_numeric_unaryops.py
python/pylibcudf/pylibcudf/libcudf/column/column_view.pxd
python/pylibcudf/pylibcudf/unary.pyi
python/pylibcudf/pylibcudf/unary.pyx

TomAugspurger · 2026-07-17T13:44:33Z

+    "dtype",
+    [
+        pl.Date,
+        pl.Datetime("us"),


Do we support pl.Time too? https://docs.pola.rs/api/python/dev/reference/expressions/api/polars.Expr.to_physical.html mentions it.

Also Categorical.

We don't support either types (we raise during translation)

…physical

coderabbitai

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

python/cudf_polars/cudf_polars/dsl/expressions/unary.py (1)
491-497: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Pass all Expr.hash seeds through explicitly

hash(tuple(self.options)) collapses seed, seed_1, seed_2, and seed_3 into one Python hash, so this cannot preserve Polars’ seed semantics. Forward the four seeds directly to xxhash_64 with the documented defaults, or raise NotImplementedError for unsupported arities.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@python/cudf_polars/cudf_polars/dsl/expressions/unary.py` around lines 491 -
497, The Expr hash implementation in the name == "hash" branch incorrectly
combines all options into one Python hash. Extract and forward seed, seed_1,
seed_2, and seed_3 individually to plc.hashing.xxhash_64 using Polars’
documented defaults, and raise NotImplementedError for unsupported seed arities.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@python/cudf_polars/cudf_polars/dsl/expressions/unary.py`:
- Around line 491-497: The Expr hash implementation in the name == "hash" branch
incorrectly combines all options into one Python hash. Extract and forward seed,
seed_1, seed_2, and seed_3 individually to plc.hashing.xxhash_64 using Polars’
documented defaults, and raise NotImplementedError for unsupported seed arities.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 5e356c2e-1f5c-4a68-9a74-731919834e83

📥 Commits

Reviewing files that changed from the base of the PR and between 670352e and 3fb62e8.

📒 Files selected for processing (2)

python/cudf_polars/cudf_polars/dsl/expressions/unary.py
python/cudf_polars/tests/expressions/test_numeric_unaryops.py

🚧 Files skipped from review as they are similar to previous changes (1)

python/cudf_polars/tests/expressions/test_numeric_unaryops.py

mroeschke added 4 commits July 9, 2026 01:55

Support cudf-polars to_physical expression

5320a68

Merge remote-tracking branch 'upstream/main' into cudf_polars/enh/to_…

44d1d5f

…physical

Improve to_physical implementation

0ed29e8

Implement pl.Expr.hash

670352e

mroeschke self-assigned this Jul 17, 2026

mroeschke added the improvement Improvement / enhancement to an existing function label Jul 17, 2026

mroeschke requested a review from a team as a code owner July 17, 2026 01:24

mroeschke added the non-breaking Non-breaking change label Jul 17, 2026

mroeschke requested a review from a team as a code owner July 17, 2026 01:24

mroeschke requested review from galipremsagar and rjzamora July 17, 2026 01:24

github-actions Bot added Python Affects Python cuDF API. cudf-polars Issues specific to cudf-polars pylibcudf Issues specific to the pylibcudf package labels Jul 17, 2026

github-project-automation Bot added this to cuDF Python Jul 17, 2026

mroeschke mentioned this pull request Jul 17, 2026

[FEAT][TRACKER] Unsupported polars.Expressions in cudf-polars #23151

Open

GPUtester moved this to In Progress in cuDF Python Jul 17, 2026

coderabbitai Bot reviewed Jul 17, 2026

View reviewed changes

Comment thread python/cudf_polars/cudf_polars/dsl/expressions/unary.py

Comment thread python/cudf_polars/tests/expressions/test_numeric_unaryops.py

Comment thread python/pylibcudf/pylibcudf/unary.pyx

TomAugspurger approved these changes Jul 17, 2026

View reviewed changes

mroeschke added 3 commits July 17, 2026 17:25

Merge remote-tracking branch 'upstream/main' into cudf_polars/enh/to_…

28be8dc

…physical

Test hash consistency. Ensure all seeds are hashed

3fb62e8

Add numpydoc docstring to bit_cast

65ea3e2

coderabbitai Bot reviewed Jul 17, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Support `pl.Expr.to_physical/hash`#23313

Support `pl.Expr.to_physical/hash`#23313
mroeschke wants to merge 7 commits into
rapidsai:mainfrom
mroeschke:cudf_polars/enh/to_physical

mroeschke commented Jul 17, 2026

Uh oh!

coderabbitai Bot commented Jul 17, 2026 •

edited

Loading

Summary by CodeRabbit

Walkthrough

Changes

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

TomAugspurger Jul 17, 2026

Uh oh!

TomAugspurger Jul 17, 2026

Uh oh!

mroeschke Jul 17, 2026

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

mroeschke commented Jul 17, 2026

Description

Checklist

Uh oh!

coderabbitai Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Walkthrough

Changes

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

TomAugspurger Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

TomAugspurger Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

mroeschke Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

coderabbitai Bot commented Jul 17, 2026 •

edited

Loading