Skip to content

feat: catch-up for read-only ProviderFactorys#23357

Merged
klkvr merged 7 commits intomainfrom
klkvr/catch-up
Apr 6, 2026
Merged

feat: catch-up for read-only ProviderFactorys#23357
klkvr merged 7 commits intomainfrom
klkvr/catch-up

Conversation

@klkvr
Copy link
Copy Markdown
Member

@klkvr klkvr commented Apr 3, 2026

Implements catch-up logic for read-only ProviderFactory.

It is implemented like the following:

/// For read-only factories, checks whether the MDBX committed txn ID has advanced since the
/// last sync and, if so, catches up the RocksDB secondary instance and re-initializes the
/// static file index.
///
/// The fast path (txnid unchanged) is lock-free. Only the actual sync I/O is serialized.
///
/// No-op for read-write factories.
pub fn sync_providers_if_needed(&self) -> ProviderResult<()> {
let Some(sync_state) = &self.read_only_sync else { return Ok(()) };
let current_txnid = self.db.last_txnid().unwrap_or(0);
// Fast path: no contention when nothing changed.
if current_txnid == sync_state.last_synced_txnid.load(Ordering::Relaxed) {
return Ok(());
}
// Slow path: serialize the actual catch-up I/O.
let _guard = sync_state.sync_lock.lock().unwrap_or_else(|e| e.into_inner());
// Double-check after acquiring the lock — another thread may have already synced.
if current_txnid == sync_state.last_synced_txnid.load(Ordering::Relaxed) {
return Ok(());
}
self.rocksdb_provider.try_catch_up_with_primary()?;
self.static_file_provider.initialize_index()?;
sync_state.last_synced_txnid.store(current_txnid, Ordering::Relaxed);
Ok(())
}

This function is invoked in 2 cases:

  1. For every created DatabaseProvider or before every read directly from static files (those bypass mdbx entirely)
  2. Every time when mdbx dir changes — this part is identical to static files watcher we have right now and is kept to make sure that read-only factories don't fall too far behind when unused, causing potentially long catch up to happen on next access

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 3, 2026

⚠️ Changelog not found.

A changelog entry is required before merging. We've generated a suggested changelog based on your changes:

Preview
---
reth-provider: minor
reth-chain-state: minor
reth-db: minor
reth-engine-tree: patch
reth-cli-commands: patch
reth-e2e-test-utils: patch
---

Added on-demand MDBX syncing for read-only `ProviderFactory` instances via a new `with_read_only_sync` method and `ReadOnlySyncState` tracker. Refactored `ReadOnlyConfig` to watch the MDBX directory instead of static files (renamed `watch_static_files` to `watch`, simplified `StaticFileProvider::read_only` signature). Enhanced `TestBlockBuilder` with a `with_state` mode that generates proper `BundleState` with account/storage changes and reverts, and added a comprehensive reorg consistency test for read-only providers.

Add changelog to commit this to your branch.

/// This uses `MDBX_DBG_LEGACY_MULTIOPEN` which recovers POSIX file locks on close.
/// It may cause unexpected pauses and does not perfectly mirror multi-process behavior.
/// Use only in tests.
pub fn enable_legacy_multiopen() {
Copy link
Copy Markdown
Collaborator

@joshieDo joshieDo Apr 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we might be able to use this in tempo test suite as well cc @SuperFluffy

Comment on lines +97 to +100
/// State for on-demand syncing of `RocksDB` secondary and static file indexes.
///
/// Only set for read-only factories.
read_only_sync: Option<Arc<ReadOnlySyncState>>,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we be more specific that this should be used for out of process concurrent access?

///
/// If `watch` is true, also watches the static files directory for `.conf` changes to
/// eagerly catch up in the background.
pub fn with_read_only_sync(mut self, watch: bool) -> Self
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we be more specific that this should be used for out of process concurrent access?

@github-project-automation github-project-automation bot moved this from Backlog to In Progress in Reth Tracker Apr 6, 2026
@klkvr klkvr added this pull request to the merge queue Apr 6, 2026
Merged via the queue into main with commit 0e4f143 Apr 6, 2026
35 checks passed
@klkvr klkvr deleted the klkvr/catch-up branch April 6, 2026 19:59
@github-project-automation github-project-automation bot moved this from In Progress to Done in Reth Tracker Apr 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants