Skip to content

mutable: zero out tail slots dropped by Filter and FilterI#890

Open
c-tonneslan wants to merge 2 commits into
samber:masterfrom
c-tonneslan:fix/mutable-filter-clear-tail
Open

mutable: zero out tail slots dropped by Filter and FilterI#890
c-tonneslan wants to merge 2 commits into
samber:masterfrom
c-tonneslan:fix/mutable-filter-clear-tail

Conversation

@c-tonneslan
Copy link
Copy Markdown
Contributor

Fixes #842.

A caller who didn't reassign the original slice would see the dropped slots holding leftover values from the in-place compaction. The issue's example is the clearest demo: removing `bar` from `[foo bar baz]` left the original slice looking like `[foo baz baz]` with `baz` duplicated. The new shorter slice returned by `Filter` was correct, the backing array wasn't.

Zero out the unused tail after compaction, the same approach the standard library uses in `slices.Delete`. Written as an explicit loop rather than `clear` so this still compiles on the module's `go 1.18` minimum.

While in there:

  • The Filter doc said the predicate takes the element and its index. It doesn't (only `FilterI` does).
  • The Map and MapI docs said "the function returns the modified slice." Neither has a return value.
  • Updated the example output to match the new tail-zeroing behavior.

Reduced the noise in the doc comments at the same time.

A caller that didn't reassign the original slice would see the removed
slots holding leftover values from the in-place compaction. The example
in the issue shows it best: removing "bar" from [foo bar baz] produced
[foo baz baz] in the original slice, with baz duplicated.

Zero out the unused tail after compaction, the same trick the standard
library uses in slices.Delete. Written as an explicit loop instead of
clear() so this still compiles on the module's go 1.18 minimum.

Also fixes the doc comments while I'm here: Filter's predicate doesn't
take an index, Map doesn't return a slice. Updated the example output
to match the new tail-zeroing behavior.

Fixes samber#842.

Signed-off-by: Charlie Tonneslan <[email protected]>
@samber
Copy link
Copy Markdown
Owner

samber commented May 23, 2026

By default, this is faster to skip zeroing.

Can you tell me more about why it would be useful for you?

@codecov
Copy link
Copy Markdown

codecov Bot commented May 23, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.65%. Comparing base (7de2b0d) to head (682c5fb).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #890   +/-   ##
=======================================
  Coverage   92.64%   92.65%           
=======================================
  Files          32       32           
  Lines        5110     5116    +6     
=======================================
+ Hits         4734     4740    +6     
  Misses        273      273           
  Partials      103      103           
Flag Coverage Δ
unittests 92.65% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@samber samber added the breaking change Introduces changes that break backward compatibility or alter the public API. label May 23, 2026
@c-tonneslan
Copy link
Copy Markdown
Contributor Author

Two main reasons:

  1. It matches what the stdlib does. slices.Delete zeros the tail after compaction for the same reason. The Go team chose correctness over the small write cost back in 1.21, and the package comment is explicit about it.

  2. Memory retention for pointer types. If T is *Foo or a struct holding pointers, the leftover values in the tail keep those objects alive after the caller "removed" them. Long-running programs (servers, caches, anything in a filter-rebuild loop) accumulate this silently. With zeroing the GC can actually reclaim.

The visible bug from #842 (duplicate baz after removing bar) is the small case. The GC-retention one is the bigger foot-gun, but it's silent until prod.

If you'd rather skip zeroing for value-type slices specifically, I can branch on that. Happy to follow up. But matching stdlib feels like the right default.

@samber
Copy link
Copy Markdown
Owner

samber commented May 24, 2026

Ok, the argument (2) legitimate the extra cost.

Let's merge ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking change Introduces changes that break backward compatibility or alter the public API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

mutable.Filter duplicates an element

2 participants