[Tests] Migrate tvm.testing.parameters() to pytest.mark.parametrize#19803
Conversation
Phase out the custom `tvm.testing.parameters()` helper in favor of native `@pytest.mark.parametrize`. `parameters()` itself is left in place for now and removed in a follow-up, together with updating the framework self-test (`tests/python/testing/test_tvm_testing_features.py`) that exercises it. Migration rules - A group consumed only by test functions becomes `@pytest.mark.parametrize`. - Single-name groups are unwrapped from 1-tuples to bare values. - A group shared by multiple tests uses a module-level named list; a test that uses only a subset of a group's names is parametrized only on the names in its signature. - `@pytest.mark.parametrize` is stacked above the existing, unrelated `@tvm.testing.parametrize_targets(...)`, which is kept as-is. Per-file pytest collection case counts are unchanged, except the two intentional changes below. Behavior changes (intentional) - tests/python/relax/test_training_optimizer_numeric.py: the names `lr` and `weight_decay` were rebound across three `parameters()` groups, so `test_sgd` and `test_momentum_sgd` silently used the *adam* group's `lr`/`weight_decay` (and `test_momentum_sgd` cross-producted with it: 2/6/2 = 10 cases). Native parametrize gives each test its own co-located group: 2/3/2 = 7 cases. This fixes that latent rebinding bug; the case count drops 10 -> 7 and `test_momentum_sgd` now exercises its own `weight_decay` values. - tests/python/target/test_arm_target.py: its `parameters()` group was orphaned (no test consumed those names) — removed the dead definition. Note: for tests that also use `@tvm.testing.parametrize_targets`, the generated test ids reorder the target (e.g. `test_unary[abs-True-llvm]` -> `test_unary[llvm-abs-True]`); values and case counts are unchanged.
There was a problem hiding this comment.
Code Review
This pull request refactors several test files by replacing the custom tvm.testing.parameters with standard @pytest.mark.parametrize decorators. A review comment points out a potential issue in tests/python/s_tir/base/test_tir_te_extern_primfunc.py where ("A") is used instead of a proper 1-tuple ("A",), which could lead to unexpected behavior if the parameter name is changed to a multi-character string.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| [func_4, ("C", "A", "D", "E"), verify_func_4], | ||
| ) | ||
| _primfunc_cases = [ | ||
| [func_1, ("A"), verify_func_1], |
There was a problem hiding this comment.
In Python, ("A") is evaluated as a simple string "A" rather than a 1-tuple. Since params is expected to be an iterable of parameter names (strings), passing a plain string will cause the test to iterate over its individual characters (e.g., 'A'). While this happens to work for a single-character name like "A", it is extremely fragile and will fail with a KeyError if the parameter name is changed to a multi-character string (e.g., "A_buf").\n\nPlease change ("A") to a proper 1-tuple ("A",) or a list ["A"] to ensure it is treated as a sequence of strings.
| [func_1, ("A"), verify_func_1], | |
| [func_1, ("A",), verify_func_1], |
All in-tree uses of tvm.testing.parameters() were migrated to native @pytest.mark.parametrize in apache#19803, so remove the helper itself along with the plugin machinery that only served it: - python/tvm/testing/utils.py: delete the parameters() function and the _parametrize_group counter. - python/tvm/testing/plugin.py: delete _parametrize_correlated_parameters and its call in pytest_generate_tests. - tests/python/testing/test_tvm_testing_features.py: drop the joint-parameter tests that exercised parameters() (the parameter() and fixture() tests stay). This removes the public tvm.testing.parameters symbol; tvm.testing.parameter (singular) and tvm.testing.fixture are unchanged. Use @pytest.mark.parametrize instead.
All in-tree uses of tvm.testing.parameters() were migrated to native pytest.mark.parametrize in #19803, so remove the helper itself along with the plugin machinery that only served it: - python/tvm/testing/utils.py: delete the parameters() function and the _parametrize_group counter. - python/tvm/testing/plugin.py: delete _parametrize_correlated_parameters and its call in pytest_generate_tests. - tests/python/testing/test_tvm_testing_features.py: drop the joint-parameter tests that exercised parameters() (the parameter() and fixture() tests stay). This removes the public tvm.testing.parameters symbol; tvm.testing.parameter (singular) and tvm.testing.fixture are unchanged. Use pytest.mark.parametrize instead.
This pr phases out the custom
tvm.testing.parameters()helper in favor of nativepytest.mark.parametrize.parameters()itself is left in place for now and removed in a follow-up, together with updating the framework self-test (tests/python/testing/test_tvm_testing_features.py) that exercises it.Migration rules
pytest.mark.parametrize.pytest.mark.parametrizeis stacked above the existing, unrelatedtvm.testing.parametrize_targets(...), which is kept as-is.Per-file pytest collection case counts are unchanged, except the two intentional changes below.
Behavior changes (intentional)
lrandweight_decaywere rebound across threeparameters()groups, sotest_sgdandtest_momentum_sgdsilently used the adam group'slr/weight_decay(andtest_momentum_sgdcross-producted with it: 2/6/2 = 10 cases). Native parametrize gives each test its own co-located group: 2/3/2 = 7 cases. This fixes that latent rebinding bug; the case count drops 10 -> 7 andtest_momentum_sgdnow exercises its ownweight_decayvalues.parameters()group was orphaned (no test consumed those names) — removed the dead definition.Note: for tests that also use
tvm.testing.parametrize_targets, the generated test ids reorder the target (e.g.test_unary[abs-True-llvm]->test_unary[llvm-abs-True]); values and case counts are unchanged.