perf(Propagator): Always commit in batches#58406
Closed
provokateurin wants to merge 1 commit intomasterfrom
Closed
perf(Propagator): Always commit in batches#58406provokateurin wants to merge 1 commit intomasterfrom
provokateurin wants to merge 1 commit intomasterfrom
Conversation
CarlSchwan
approved these changes
Feb 18, 2026
d6b5320 to
15d2b5b
Compare
Member
Author
|
Tests are red, but I think they just need some manual calls to |
Member
Author
|
I tried to fix a few of these issues, but it looks like it's not easily possible this way. Some sort of global propagator that can commit all batches in all propagators of all storages at once would be necessary. Depending on how that is done it might also be possible to gain even more performance by having a single query for propagation instead of one query per propagator. |
Signed-off-by: provokateurin <kate@provokateurin.de>
15d2b5b to
cccb7b0
Compare
Member
Author
|
A global batch propagator makes this a little better, but it doesn't solve all problems. Some code relies on the assumption that changes are propagated right away and I don't feel comfortable with breaking that assumption, as it could result in really bad behavior changes. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I'm not sure this is a good idea or not, but I've seen that parents are updated multiple times within the same request.
This should help, as they will only get updated once, but it might also result in more memory consumption if many changes need to be propagated within a single request.
We could also go half way and automatically commit if there are too many parents, like the scanner logic already does. This would at least limit the memory consumption and should still perform better.