Skip to content

[Tests] Migrate tvm.testing.parameters() to pytest.mark.parametrize#19803

Merged
tqchen merged 1 commit into
apache:mainfrom
tlopex:migrate-parameters-to-parametrize
Jun 17, 2026
Merged

[Tests] Migrate tvm.testing.parameters() to pytest.mark.parametrize#19803
tqchen merged 1 commit into
apache:mainfrom
tlopex:migrate-parameters-to-parametrize

Conversation

@tlopex

@tlopex tlopex commented Jun 17, 2026

Copy link
Copy Markdown
Member

This pr phases out the custom tvm.testing.parameters() helper in favor of native pytest.mark.parametrize. parameters() itself is left in place for now and removed in a follow-up, together with updating the framework self-test (tests/python/testing/test_tvm_testing_features.py) that exercises it.

Migration rules

  • A group consumed only by test functions becomes pytest.mark.parametrize.
  • Single-name groups are unwrapped from 1-tuples to bare values.
  • A group shared by multiple tests uses a module-level named list; a test that uses only a subset of a group's names is parametrized only on the names in its signature.
  • pytest.mark.parametrize is stacked above the existing, unrelated tvm.testing.parametrize_targets(...), which is kept as-is.

Per-file pytest collection case counts are unchanged, except the two intentional changes below.

Behavior changes (intentional)

  • tests/python/relax/test_training_optimizer_numeric.py: the names lr and weight_decay were rebound across three parameters() groups, so test_sgd and test_momentum_sgd silently used the adam group's lr/weight_decay (and test_momentum_sgd cross-producted with it: 2/6/2 = 10 cases). Native parametrize gives each test its own co-located group: 2/3/2 = 7 cases. This fixes that latent rebinding bug; the case count drops 10 -> 7 and test_momentum_sgd now exercises its own weight_decay values.
  • tests/python/target/test_arm_target.py: its parameters() group was orphaned (no test consumed those names) — removed the dead definition.

Note: for tests that also use tvm.testing.parametrize_targets, the generated test ids reorder the target (e.g. test_unary[abs-True-llvm] -> test_unary[llvm-abs-True]); values and case counts are unchanged.

Phase out the custom `tvm.testing.parameters()` helper in favor of native
`@pytest.mark.parametrize`. `parameters()` itself is left in place for now and
removed in a follow-up, together with updating the framework self-test
(`tests/python/testing/test_tvm_testing_features.py`) that exercises it.

Migration rules
- A group consumed only by test functions becomes `@pytest.mark.parametrize`.
- Single-name groups are unwrapped from 1-tuples to bare values.
- A group shared by multiple tests uses a module-level named list; a test that
  uses only a subset of a group's names is parametrized only on the names in its
  signature.
- `@pytest.mark.parametrize` is stacked above the existing, unrelated
  `@tvm.testing.parametrize_targets(...)`, which is kept as-is.

Per-file pytest collection case counts are unchanged, except the two intentional
changes below.

Behavior changes (intentional)
- tests/python/relax/test_training_optimizer_numeric.py: the names `lr` and
  `weight_decay` were rebound across three `parameters()` groups, so `test_sgd`
  and `test_momentum_sgd` silently used the *adam* group's `lr`/`weight_decay`
  (and `test_momentum_sgd` cross-producted with it: 2/6/2 = 10 cases). Native
  parametrize gives each test its own co-located group: 2/3/2 = 7 cases. This
  fixes that latent rebinding bug; the case count drops 10 -> 7 and
  `test_momentum_sgd` now exercises its own `weight_decay` values.
- tests/python/target/test_arm_target.py: its `parameters()` group was orphaned
  (no test consumed those names) — removed the dead definition.

Note: for tests that also use `@tvm.testing.parametrize_targets`, the generated
test ids reorder the target (e.g. `test_unary[abs-True-llvm]` ->
`test_unary[llvm-abs-True]`); values and case counts are unchanged.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors several test files by replacing the custom tvm.testing.parameters with standard @pytest.mark.parametrize decorators. A review comment points out a potential issue in tests/python/s_tir/base/test_tir_te_extern_primfunc.py where ("A") is used instead of a proper 1-tuple ("A",), which could lead to unexpected behavior if the parameter name is changed to a multi-character string.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[func_4, ("C", "A", "D", "E"), verify_func_4],
)
_primfunc_cases = [
[func_1, ("A"), verify_func_1],

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

In Python, ("A") is evaluated as a simple string "A" rather than a 1-tuple. Since params is expected to be an iterable of parameter names (strings), passing a plain string will cause the test to iterate over its individual characters (e.g., 'A'). While this happens to work for a single-character name like "A", it is extremely fragile and will fail with a KeyError if the parameter name is changed to a multi-character string (e.g., "A_buf").\n\nPlease change ("A") to a proper 1-tuple ("A",) or a list ["A"] to ensure it is treated as a sequence of strings.

Suggested change
[func_1, ("A"), verify_func_1],
[func_1, ("A",), verify_func_1],

@tqchen tqchen merged commit 71466bb into apache:main Jun 17, 2026
12 checks passed
tlopex added a commit to tlopex/tvm that referenced this pull request Jun 17, 2026
All in-tree uses of tvm.testing.parameters() were migrated to native
@pytest.mark.parametrize in apache#19803, so remove the helper itself along with the
plugin machinery that only served it:
- python/tvm/testing/utils.py: delete the parameters() function and the
  _parametrize_group counter.
- python/tvm/testing/plugin.py: delete _parametrize_correlated_parameters and
  its call in pytest_generate_tests.
- tests/python/testing/test_tvm_testing_features.py: drop the joint-parameter
  tests that exercised parameters() (the parameter() and fixture() tests stay).

This removes the public tvm.testing.parameters symbol; tvm.testing.parameter
(singular) and tvm.testing.fixture are unchanged. Use @pytest.mark.parametrize
instead.
tlopex added a commit that referenced this pull request Jun 17, 2026
All in-tree uses of tvm.testing.parameters() were migrated to native
pytest.mark.parametrize in #19803, so remove the helper itself along
with the plugin machinery that only served it:
- python/tvm/testing/utils.py: delete the parameters() function and the
_parametrize_group counter.
- python/tvm/testing/plugin.py: delete
_parametrize_correlated_parameters and its call in
pytest_generate_tests.
- tests/python/testing/test_tvm_testing_features.py: drop the
joint-parameter tests that exercised parameters() (the parameter() and
fixture() tests stay).

This removes the public tvm.testing.parameters symbol;
tvm.testing.parameter (singular) and tvm.testing.fixture are unchanged.
Use pytest.mark.parametrize instead.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants