Skip to content

Clear index state information of FormerIndexes based on the index name rather than subspace key#4024

Open
alecgrieser wants to merge 2 commits intoFoundationDB:mainfrom
alecgrieser:00514-clear-former-index-state-on-name
Open

Clear index state information of FormerIndexes based on the index name rather than subspace key#4024
alecgrieser wants to merge 2 commits intoFoundationDB:mainfrom
alecgrieser:00514-clear-former-index-state-on-name

Conversation

@alecgrieser
Copy link
Copy Markdown
Collaborator

There was a bug in the removeFormerIndex method which is called when a store is opened. It previously was clearing out the index state information (which states whether the index is built or not) based on whether the FormerIndex's subspace key. However, this is incorrect, as the index state space is keyed by index name (see #514 about switching this to be by subspace key). This means that the old index state will persist in the database, and we won't properly clear it.

This fixes things by using the former index name instead. Note that the former name was added after other fields in the FormerIndex, and so it may not be reliably set. Newer creations of the FormerIndex (including RecordMetaDataBuilder::removeIndex) should preserve that information, but that's not guaranteed. The change therefore needs to handle the fact that the subspace key may not be set. I've elected to update the logic so that we just give up on clearing out the index state if the name is not set, though that is technically a behavior change. In particular, if the user uses the default subspace key (which is the index name), we could attempt to guess at what the former name was by looking at the key. I decided this was probably a bad idea, though, as you could have a pair of indexes that were named "index_1" and "index_2" but had the index subpsace keys of "index_2" and "index_1". It seems a little unlikely, but the cost of incorrectly leaving index state information is relatively low (the main cost being that we may not be able to mark an index on new types that re-uses the name as READABLE), whereas the danger of getting that wrong is higher (we could end up incorrectly marking an unbuilt index as READABLE).

This fixes #515. It also probably suggests that we should look at #4019 with some skepticism, as we are now actively using the former index name data for something.

…ame rather than subspace key

There was a bug in the `removeFormerIndex` method which is called when a store is opened. It previously was clearing out the index state information (which states whether the index is built or not) based on whether the `FormerIndex`'s subspace key. However, this is incorrect, as the index state space is keyed by index name (see FoundationDB#514 about switching this to be by subspace key). This means that the old index state will persist in the database, and we won't properly clear it.

This fixes things by using the former index name instead. Note that the former name was added after other fields in the `FormerIndex`, and so it may not be reliably set. Newer creations of the `FormerIndex` (including `RecordMetaDataBuilder::removeIndex`) should preserve that information, but that's not guaranteed. The change therefore needs to handle the fact that the subspace key may not be set. I've elected to update the logic so that we just give up on clearing out the index state if the name is not set, though that is technically a behavior change. In particular, if the user uses the default subspace key (which is the index name), we could attempt to guess at what the former name was by looking at the key. I decided this was probably a bad idea, though, as you could have a pair of indexes that were named `"index_1"` and `"index_2"` but had the index subpsace keys of `"index_2"` and `"index_1"`. It seems a little unlikely, but the cost of incorrectly leaving index state information is relatively low (the main cost being that we may not be able to mark an index on new types that re-uses the name as `READABLE`), whereas the danger of getting that wrong is higher (we could end up incorrectly marking an unbuilt index as `READABLE`).

This fixes FoundationDB#515. It also probably suggests that we should look at FoundationDB#4019 with some skepticism, as we are now actively using the former index name data for something.
@alecgrieser alecgrieser added the bug fix Change that fixes a bug label Mar 25, 2026
@MMcM
Copy link
Copy Markdown
Collaborator

MMcM commented Mar 25, 2026

I somewhat prefer #514, though I am willing to accept arguments about that being a more significant change than is called for to fix all this.

@alecgrieser
Copy link
Copy Markdown
Collaborator Author

The way I view it, we would probably want both. The current situation is dicey, in that we end up clearing out the wrong location in the index state space, and that seems like something we need to address. If we addressed #514, we'd only be able to do that on a new format version, and so I think we'd need this (or something like it) on at old format versions.

We did try and address #514 with #662, and we got bogged down a bit in the details. I do think it would be a good change, though, if we could make it work.

@alecgrieser alecgrieser marked this pull request as ready for review March 27, 2026 13:50
@alecgrieser alecgrieser requested a review from MMcM March 27, 2026 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug fix Change that fixes a bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Deleting an index does not remove it from the disabled list

2 participants