Open
Conversation
Implements a more efficient way to check if any key with a given prefix exists, avoiding the overhead of setting up a full merge iterator. The implementation: - Checks memtables using range iteration on the skip list - Checks SST tables using existing range iterators - Returns early as soon as a matching key is found - Respects MVCC semantics with sequence number filtering This is significantly more efficient than the current pattern of using `tree.prefix().next()?.is_some()` because it avoids initializing merge iterators across all levels and stops at the first match. Closes fjall-rs#138
Contributor
|
There is kind of an edge case where a prefix is fully tombstoned, in that case, the function could return a false positive (should be easy to test). And actually looking at it, @zaidoon1 already implemented this in https://github.com/fjall-rs/lsm-tree/pull/186/files#diff-ee87832434edd670d369ff1e6d5c0b08c959ef0cd85e865a7b726451f623fbf2R329-R361, but not for |
Contributor
I can add support for that, although since the prefix filter pr is not going to be merged soon, i'll hold off until you are ready to merge it before i fix merge conflicts/address this feedback |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements an optimized
contains_prefixmethod to efficiently check if any key with a given prefix exists in the tree.Motivation
Currently, users check for prefix existence using
.prefix().next()?.is_some(), which has significant overhead:This PR provides a more efficient alternative that returns as soon as any matching key is found.
Implementation
The new
contains_prefixmethod:trueimmediately upon finding the first matching keyAPI
Example usage:
Performance Benefits
Testing
get)Closes #138