-
Notifications
You must be signed in to change notification settings - Fork 626
Avoid double-caching when ccache is installed in PATH #2524
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2524 +/- ##
==========================================
- Coverage 71.24% 71.14% -0.10%
==========================================
Files 64 65 +1
Lines 35593 35833 +240
==========================================
+ Hits 25357 25495 +138
- Misses 10236 10338 +102 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
ec91704 to
608d5c6
Compare
|
update: fix rustfmt error |
|
@sylvestre please review |
CodSpeed Performance ReportMerging this PR will degrade performance by 5.29%Comparing Summary
Performance Changes
Footnotes
|
1fe7bb9 to
4bb5579
Compare
|
@sylvestre can I help in any way to push this along? |
src/util.rs
Outdated
| use std::env; | ||
| use std::path::Path; | ||
|
|
||
| const CCACHE_DIRS: &[&str] = &["/usr/lib/ccache", "/usr/lib64/ccache"]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The CCACHE_DIRS list only contains Linux paths. Consider adding macOS Homebrew paths (/usr/local/opt/ccache/libexec, /opt/homebrew/opt/ccache/libexec) or adding a comment explaining why they're omitted
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also, it would be nice to be able to configure things somewhere
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can generalize it to any path that contains a component ccache.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
About configuration, is it still needed if we just filter out directories with ccache components?
| /// This prevents double-caching when ccache is installed and has | ||
| /// /usr/lib/ccache or /usr/lib64/ccache in PATH, since those directories | ||
| /// contain wrapper scripts that would call ccache. | ||
| pub fn filter_ccache_from_path(env_vars: Vec<(OsString, OsString)>) -> Vec<(OsString, OsString)> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Function allocates a new Vec even when no ccache dirs are present in PATH. Consider short-circuiting to return the input Vec unchanged when filtering is not needed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure
| } = self; | ||
| // Filter out ccache directories from PATH to avoid double-caching | ||
| // when ccache is also installed on the system. | ||
| let env_vars = filter_ccache_from_path(env_vars.to_vec()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Filter is only applied to SingleCompileCommand but NvccCompileCommand (in nvcc.rs) also implements CompileCommandImpl and may need the same treatment it is intentional ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not intentional, I'll take a look.
|
sorry for the latency |
4bb5579 to
2a7f25c
Compare
|
v2:
|
2a7f25c to
ecce16b
Compare
|
v3:
|
On Linux, ccache is typically installed in $PATH as /usr/lib64/ccache/g++ or similar. If we keep it there, all compilations will be cached by both ccache and sccache. While the user could easily disable ccache with CCACHE_DISABLE, it's reasonable to assume many will forget and will have their disk space doubly consumed by both caches. Better to recognize this and disable ccache under sccache. This patch does this by removing the ccache binary paths from $PATH. The check is fuzzy - any directory that has a `ccache` component is removed. It's assumed that people won't have directories with such names in $PATH that are not ccache masquarade directories. Fixes mozilla#2519
ecce16b to
ce9f95b
Compare
|
v4:
|
On Linux, ccache is typically installed in $PATH as /usr/lib64/ccache/g++ or similar. If we keep it there, all compilations will be cached by both ccache and sccache.
While the user could easily disable ccache with CCACHE_DISABLE, it's reasonable to assume many will forget and will have their disk space doubly consumed by both caches. Better to recognize this and disable ccache under sccache.
This patch does this by removing the ccache binary paths from $PATH.
Fixes #2519