Skip to content

Validate license files are valid UTF-8 text in bdist_wheel - #5310

Open
CAOShurong wants to merge 1 commit into
pypa:mainfrom
CAOShurong:codex/4936-license-files-utf8
Open

Validate license files are valid UTF-8 text in bdist_wheel#5310
CAOShurong wants to merge 1 commit into
pypa:mainfrom
CAOShurong:codex/4936-license-files-utf8

Conversation

@CAOShurong

Copy link
Copy Markdown

Fixes #4936

Problem

PEP 639 requires tools to assume license file content is valid UTF-8 encoded text, and states that tools SHOULD validate this and raise an error if it is not. setuptools currently copies whatever files match the license-files patterns — including binary files like BadLicense.doc, LICENSE.pdf or even a stray license.py matched by the default globs (LICEN[CS]E*) — straight into .dist-info/licenses/ with no warning or error (issue #4936).

Fix

Add _validate_license_file_encoding() in setuptools/command/bdist_wheel.py. It runs for every license file just before it is copied into the wheel's licenses/ folder (in egg2dist), and raises InvalidConfigError when UTF-8 decoding fails:

error: license file 'license.pdf' is not valid UTF-8 encoded text.
Per PEP 639, license files must be UTF-8 text; remove it from
`license-files` or exclude it via more specific patterns.

Because the validation lives in bdist_wheel's copy loop, it covers both explicit patterns and the default globs, on every platform, regardless of how the file list was resolved.

Verification (all on real Windows 11)

  • New regression test test_binary_license_file_raises: fails on main (binary file silently bundled), passes with this change — RED/GREEN proven.
  • Full setuptools/tests/test_bdist_wheel.py: 29 passed, 2 skipped (unchanged from baseline).
  • test_wheel.py + config/ suites: 204 passed, 1 xfailed (unchanged).
  • ruff check + ruff format --check clean; mypy setuptools/command/bdist_wheel.py: no issues.
  • End-to-end repro of the issue scenario via setuptools.build_meta.build_wheel():
    • license.pdf containing invalid UTF-8 bytes + default globs → build now fails with the error above (previously succeeded and shipped the binary).
    • Valid UTF-8 LICENSE.txt → wheel builds normally, license entries present in .dist-info/licenses/.
  • POSIX behavior unchanged: the check only triggers on genuine decode failures; all previously-valid builds keep passing (UTF-8 text with BOM-free multi-byte content decodes identically on any platform).

Notes for reviewers

  • The PEP says SHOULD (not MUST), so if maintainers prefer a warning first (e.g. SetuptoolsWarning with a deprecation-style due date) I'm happy to convert this — the mechanism would stay identical, only the raise becomes an emit.
  • Scope deliberately limited to the wheel-copy path; sdist handling is untouched.

DCO sign-off included in the commit.

@mergify

mergify Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

Per PEP 639, tools MUST assume license file content is UTF-8 and SHOULD
validate this. Previously binary files matched by license-files glob
patterns (e.g. LICENSE.pdf via the default LICEN[CS]E* pattern) were
silently copied into the wheel's dist-info/licenses directory.

Raise InvalidConfigError from bdist_wheel's egg2dist when any bundled
license file fails UTF-8 decoding.

Fixes pypa#4936

Signed-off-by: Shurong Cao <170531907+CAOShurong@users.noreply.github.com>
@CAOShurong
CAOShurong force-pushed the codex/4936-license-files-utf8 branch from 8a84894 to df1ee4a Compare August 24, 2026 09:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] License-Files entries are not checked for UTF-8 compliance, they just get copied directly into the licenses folder

1 participant