Skip to content

downloader: CacheKey: include decompression flag#4067

Open
AkihiroSuda wants to merge 1 commit intolima-vm:masterfrom
AkihiroSuda:fix-cachekey-decomp
Open

downloader: CacheKey: include decompression flag#4067
AkihiroSuda wants to merge 1 commit intolima-vm:masterfrom
AkihiroSuda:fix-cachekey-decomp

Conversation

@AkihiroSuda
Copy link
Member

The caches are now created separately for compressed and decompressed contents. When a decompressed content is cached and a compressed content is requested, the downloader now correctly returns the compressed content.

  • URLSHA/data: unmodified original data
  • URLSHA/sha256.digest: digest of the original data
  • URLSHA+decomp/data: decompressed data
  • URLSHA+decomp/sha256.digest: digest of the original (i.e., compressed) data

Caching a decompressed content does not automatically cache the original compressed content.

@AkihiroSuda AkihiroSuda added this to the v2.0.0 milestone Sep 22, 2025
@AkihiroSuda AkihiroSuda marked this pull request as draft September 22, 2025 16:32
k += "+decomp"
}
return k
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(On second thought this might be rather confusing when the original data is not compressed)

@AkihiroSuda AkihiroSuda requested review from a team and jandubois September 24, 2025 00:12
@AkihiroSuda AkihiroSuda marked this pull request as ready for review October 1, 2025 09:14
@AkihiroSuda AkihiroSuda requested a review from a team October 9, 2025 07:30
@AkihiroSuda AkihiroSuda modified the milestones: v2.0.0, v2.1.0 (?) Oct 16, 2025
The caches are now created separately for compressed and decompressed contents.
When a decompressed content is cached and a compressed content is requested,
the downloader now correctly returns the compressed content.

- URLSHA/data:                 unmodified original data
- URLSHA/sha256.digest:        digest of the original data
- URLSHA+decomp/data:          decompressed data
- URLSHA+decomp/sha256.digest: digest of the *original* (i.e., compressed) data

Caching a decompressed content does not automatically cache the original compressed content.

Signed-off-by: Akihiro Suda <[email protected]>
@AkihiroSuda
Copy link
Member Author

ping @lima-vm/maintainers This PR has been open for more than 4 months

return fmt.Sprintf("%x", sha256.Sum256([]byte(remote)))
func CacheKey(remote string, decompress bool) string {
k := fmt.Sprintf("%x", sha256.Sum256([]byte(remote)))
if decompress {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To address your concern, can we?

Suggested change
if decompress {
if decompress && decompressor(remote) != "" {

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants