Skip to content

Build Filter Optimization:: Fused derive#31

Merged
DhruvilK7 merged 1 commit intoRibbonFilter:mainfrom
DhruvilK7:query
Mar 14, 2026
Merged

Build Filter Optimization:: Fused derive#31
DhruvilK7 merged 1 commit intoRibbonFilter:mainfrom
DhruvilK7:query

Conversation

@DhruvilK7
Copy link
Copy Markdown
Contributor

Fuse Phase 2 (derive) into addRange so that each key hash is derived and banded in a single pass, eliminating the intermediate []hashResult allocation that accounted for ~53% of total build memory. The one-ahead pipeline (nextHr = derive(hashes[i+1]) before add(currHr)) preserves the existing prefetch-friendly access pattern while removing one full O(n) allocation and traversal.

benchmark                                        old ns/op     new ns/op     delta
BenchmarkBuildFilter/w=64/n=1000-11              107618        105230        -2.22%
BenchmarkBuildFilter/w=64/n=10000-11             332135        288198        -13.23%
BenchmarkBuildFilter/w=64/n=100000-11            3820237       3633888       -4.88%
BenchmarkBuildFilter/w=128/n=1000-11             157540        154127        -2.17%
BenchmarkBuildFilter/w=128/n=10000-11            698448        640460        -8.30%
BenchmarkBuildFilter/w=128/n=100000-11           6744833       6238794       -7.50%

benchmark                                        old allocs    new allocs    delta
BenchmarkBuildFilter/w=64/n=1000-11              8             7             -12.50%
BenchmarkBuildFilter/w=64/n=10000-11             8             7             -12.50%
BenchmarkBuildFilter/w=64/n=100000-11            8             7             -12.50%
BenchmarkBuildFilter/w=128/n=1000-11             9             8             -11.11%
BenchmarkBuildFilter/w=128/n=10000-11            9             8             -11.11%
BenchmarkBuildFilter/w=128/n=100000-11           9             8             -11.11%

benchmark                                        old bytes     new bytes     delta
BenchmarkBuildFilter/w=64/n=1000-11              54160         21392         -60.50%
BenchmarkBuildFilter/w=64/n=10000-11             532496        204816        -61.54%
BenchmarkBuildFilter/w=64/n=100000-11            5226640       2023568       -61.28%
BenchmarkBuildFilter/w=128/n=1000-11             60816         28048         -53.88%
BenchmarkBuildFilter/w=128/n=10000-11            622608        294928        -52.63%
BenchmarkBuildFilter/w=128/n=100000-11           5996688       2793616       -53.41%

@DhruvilK7 DhruvilK7 merged commit 910b402 into RibbonFilter:main Mar 14, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant