Fix executable bit lost on package data files that double as modules - #5318
Closed
afonsojanu wants to merge 1 commit into
Closed
Fix executable bit lost on package data files that double as modules#5318afonsojanu wants to merge 1 commit into
afonsojanu wants to merge 1 commit into
Conversation
A data file that also ends in .py gets copied twice on its way into build/lib: once by build_module(), which treats it as an ordinary module and copies it without preserving mode, and again by build_package_data(), which does ask for the mode to be preserved. Both copies land on the exact same target. copy_file() skips its own work once source and target already agree on mtime, and preserve_times on the first copy already brought them into agreement, since it ran first and used the same source file. So by the time the second, mode-preserving copy runs, it sees nothing to do and returns early, before it ever reaches the code that would fix the mode up. The wrong permissions from the first copy are what ends up in the wheel. build_package_data() now forces its copy the same way self.force already does elsewhere, since it has to be authoritative about the final state of a package data file regardless of what an unrelated earlier copy left behind on the same path. Added a test mirroring the existing executable-data test, but with a .py-suffixed data file, which is what actually exercises the double copy. Fixes pypa#5296
Contributor
|
Tick the box to add this pull request to the merge queue (same as
|
Author
|
Closing this in favor of #5319, which I opened separately for the same issue. Ended up investigating this from two directions at once and only caught the overlap after both were already up. #5319's fix is more targeted: it copies the mode directly instead of forcing the whole file through copy_file's update check, which avoids re-copying package data that's genuinely already up to date. Sorry for the duplicate noise. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of changes
Since setuptools 84.0.0, a package data file whose name ends in
.pycan lose its executable permission bit when it ends up in a wheel. It's a double-copy issue: such a file gets copied intobuild/libtwice on its way there, once throughbuild_module()(which treats it as an ordinary module and doesn't preserve mode) and once throughbuild_package_data()(which does ask for mode to be preserved). Both target the exact same path.copy_file()skips its own copy once source and target already agree on mtime. The first copy already brought them into agreement (viapreserve_times), so by the time the second, mode-preserving copy runs, it sees nothing to do and returns before ever reaching the code that fixes the mode. Whatever mode the first copy left behind (the OS default for a newly created file) is what survives into the wheel.build_package_data()now forces its own copy the same wayself.forcealready does for other commands, since it needs to be authoritative about a package data file's final state no matter what an unrelated earlier copy left on the same path.I bisected this to the "Sync with distutils @ e8eb878" merge (#5292), specifically the
pypa/distutils#379mtime fix: preserving mtime with full nanosecond precision (rather than the previous, less exact copy) is what made the two copies' timestamps compare as exactly equal, which is what makescopy_file()'s update-skip trigger on the second one. The underlying double-copy-with-different-preserve_mode-settings issue looks like it predates that, just masked by imprecise timestamp comparisons before.Closes #5296
Pull Request Checklist
newsfragments/.(See documentation for details)