Skip to content

Comments

perf(Propagator): Always commit in batches#58406

Closed
provokateurin wants to merge 1 commit intomasterfrom
perf/propagator/always-commit-in-batches
Closed

perf(Propagator): Always commit in batches#58406
provokateurin wants to merge 1 commit intomasterfrom
perf/propagator/always-commit-in-batches

Conversation

@provokateurin
Copy link
Member

@provokateurin provokateurin commented Feb 18, 2026

I'm not sure this is a good idea or not, but I've seen that parents are updated multiple times within the same request.
This should help, as they will only get updated once, but it might also result in more memory consumption if many changes need to be propagated within a single request.
We could also go half way and automatically commit if there are too many parents, like the scanner logic already does. This would at least limit the memory consumption and should still perform better.

@provokateurin provokateurin added this to the Nextcloud 34 milestone Feb 18, 2026
@provokateurin provokateurin requested a review from a team as a code owner February 18, 2026 15:33
@provokateurin provokateurin requested review from Altahrim and ArtificialOwl and removed request for a team February 18, 2026 15:33
@provokateurin provokateurin force-pushed the perf/propagator/always-commit-in-batches branch from d6b5320 to 15d2b5b Compare February 18, 2026 15:40
@provokateurin
Copy link
Member Author

Tests are red, but I think they just need some manual calls to commitBatch().

@provokateurin
Copy link
Member Author

I tried to fix a few of these issues, but it looks like it's not easily possible this way. Some sort of global propagator that can commit all batches in all propagators of all storages at once would be necessary. Depending on how that is done it might also be possible to gain even more performance by having a single query for propagation instead of one query per propagator.

@provokateurin provokateurin added 2. developing Work in progress and removed 3. to review Waiting for reviews labels Feb 23, 2026
@provokateurin provokateurin marked this pull request as draft February 23, 2026 15:13
Signed-off-by: provokateurin <kate@provokateurin.de>
@provokateurin provokateurin force-pushed the perf/propagator/always-commit-in-batches branch from 15d2b5b to cccb7b0 Compare February 24, 2026 08:01
@provokateurin
Copy link
Member Author

A global batch propagator makes this a little better, but it doesn't solve all problems.

Some code relies on the assumption that changes are propagated right away and I don't feel comfortable with breaking that assumption, as it could result in really bad behavior changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants