Extend parameter file loader to parse TOML in addition to YAML#874
Extend parameter file loader to parse TOML in addition to YAML#874huyhoang171106 wants to merge 2 commits intonteract:mainfrom
Conversation
Signed-off-by: Nguyen Huy Hoang <181364121+huyhoang171106@users.noreply.github.com>
Signed-off-by: Nguyen Huy Hoang <181364121+huyhoang171106@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR aims to extend Papermill’s --parameters_file loader to accept TOML files (issue #719) while preserving existing YAML behavior.
Changes:
- Added a conditional import for
tomllib(withtomlifallback) inpapermill/iorw.py. - Updated CLI help text for
--parameters_fileto mention TOML support.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
papermill/iorw.py |
Introduces tomllib/tomli import plumbing, but no parsing logic is wired in yet. |
papermill/cli.py |
Updates --parameters_file help text to say “YAML or TOML”, without corresponding behavior changes. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| try: | ||
| import tomllib | ||
| except ModuleNotFoundError: | ||
| try: | ||
| import tomli as tomllib | ||
| except ModuleNotFoundError: | ||
| tomllib = None | ||
|
|
There was a problem hiding this comment.
The newly added tomllib/tomli import is currently unused anywhere in this module, which will fail Ruff/Pyflakes (unused import/unused variable). Either wire this into the parameters-file loader (e.g., TOML parsing) or remove the import until it’s used. If TOML support is intended for Python 3.10, also ensure the tomli fallback is an actual dependency (or raise a clear error when tomllib is None).
| try: | |
| import tomllib | |
| except ModuleNotFoundError: | |
| try: | |
| import tomli as tomllib | |
| except ModuleNotFoundError: | |
| tomllib = None |
| @click.option('--parameters', '-p', nargs=2, multiple=True, help='Parameters to pass to the parameters cell.') | ||
| @click.option('--parameters_raw', '-r', nargs=2, multiple=True, help='Parameters to be read as raw string.') | ||
| @click.option('--parameters_file', '-f', multiple=True, help='Path to YAML file containing parameters.') | ||
| @click.option('--parameters_file', '-f', multiple=True, help='Path to YAML or TOML file containing parameters.') |
There was a problem hiding this comment.
This help text now claims --parameters_file accepts TOML, but the implementation still routes to read_yaml_file(...), which only loads YAML/JSON and enforces extensions ['.json', '.yaml', '.yml']. Either implement TOML support end-to-end (loader + CLI wiring) or keep the help text YAML-only to avoid misleading users.
| @click.option('--parameters_file', '-f', multiple=True, help='Path to YAML or TOML file containing parameters.') | |
| @click.option('--parameters_file', '-f', multiple=True, help='Path to YAML file containing parameters.') |
Summary
The current
read_yaml_filepath is YAML-specific and is used by CLI--parameters_file. To support issue #719 cleanly, this loader should accept TOML files (primarily.toml) while preserving existing YAML behavior and backward compatibility for current callers.Files changed
papermill/iorw.py(modified)papermill/cli.py(modified)Testing
What does this PR do?
Fixes #<issue_number>