Enable gossip during optimistic sync for data columns#10304
Enable gossip during optimistic sync for data columns#10304gfukushima wants to merge 39 commits intoConsensys:masterfrom
Conversation
…GossipManager Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
...rc/main/java/tech/pegasys/teku/statetransition/datacolumns/DataColumnSidecarManagerImpl.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
| final boolean subscribeAllCustodySubnetsEnabled) { | ||
| subnetSubscriptions = dataColumnSidecarSubnetSubscriptions; | ||
| this.dasGossipLogger = dasGossipLogger; | ||
| this.subscribedToAllCustodySubnetsEnabled = subscribeAllCustodySubnetsEnabled; |
There was a problem hiding this comment.
It's not a good way to get this information.
Node could be subscribed to all custody subnets because:
- flag subscribe-custody..
- flag custody-override
- node is supernode because of the number of the validators served
Because number of the validators served could change dynamically on any epoch, this case should be handled too. For example check DataColumnSidecarRecoveringCustodyImpl how the flag isSuperNode is set. You need something similar but the check should be recurrent (because number of the validators could change), we do it usually onSlot, but there is no onSlot here, I'm not sure what will be the best option. We should do the check at least once per epoch.
There was a problem hiding this comment.
I've changed the PR to contemplate the scenarios where you mentioned but the check isn't recurrent, I think this is better than the current state we can work on make the check every epoch on another PR, this might be useful in case the node stays in optimistic sync for too long, correct?
There was a problem hiding this comment.
It's a bad idea trying to duplicate logic of CustodyGroupCountManager. Moreover, you don't cover one of the most common customer's case: validator node with 128+ validators without any command line flags. It should be subscribed to all custody subnets. And it's a reason we need updates of the custody count status: (not all) validators are loaded on initialization stage, when constructor is called, keys decryption takes time, and real numbers breaks in a bit later, that's why we always try to follow status in other components.
There was a problem hiding this comment.
since this gossip subscription is checked (for possible new subscriptions start) every epoch I believe we don't need a mechanism to keep checking constantly, I'm relying on the DB variable to tell if the node has changed status and is supernode when the change happens, if so before the epoch transition we should start the subscription for the data column gossip. If this isn't good enough than maybe we can think about a event based logic.
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
…en-sync-optimistic' into enable-gossip-for-data-column-when-sync-optimistic
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
…ange this to a supplier Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
| @Override | ||
| public SafeFuture<InternalValidationResult> onDataColumnSidecarGossip( | ||
| final DataColumnSidecar dataColumnSidecar, final Optional<UInt64> arrivalTimestamp) { | ||
| LOG.info("Received data column sidecar from GOSSIP: {}", dataColumnSidecar); |
| dataColumnSidecarGossipManager.unsubscribeFromSubnetId(subnetId); | ||
| } | ||
|
|
||
| private Supplier<Boolean> isSuperNode() { |
There was a problem hiding this comment.
we shouldn't do this
how about something like this?
zilm13@24c4595
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
networking/eth2/src/main/java/tech/pegasys/teku/networking/eth2/Eth2P2PNetworkBuilder.java
Show resolved
Hide resolved
…en-sync-optimistic
…en-sync-optimistic' into enable-gossip-for-data-column-when-sync-optimistic
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
networking/eth2/src/main/java/tech/pegasys/teku/networking/eth2/P2PConfig.java
Show resolved
Hide resolved
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
…en-sync-optimistic' into enable-gossip-for-data-column-when-sync-optimistic
…stodyGroupCountManagerImpl Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
...tetransition/src/test/java/tech/pegasys/teku/statetransition/util/SuperNodeSupplierTest.java
Show resolved
Hide resolved
.../beaconchain/src/main/java/tech/pegasys/teku/services/beaconchain/BeaconChainController.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
…en-sync-optimistic' into enable-gossip-for-data-column-when-sync-optimistic
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
…en-sync-optimistic' into enable-gossip-for-data-column-when-sync-optimistic
PR Description
This PR uses the flag
subscribedToAllCustodySubnetsEnabledto reflect whether gossip should be enabled during optimistic sync.Fixed Issue(s)
Fixes #10275
Documentation
doc-change-requiredlabel to this PR if updates are required.Changelog
Note
Medium Risk
Changes gossip subscription behavior during optimistic sync and adds new config-driven gating, which could affect network load and data propagation if misclassified or miswired.
Overview
Enables data column sidecar gossip during optimistic sync when the node is considered a super node (either explicitly via new
P2PConfig.isSubscribedToAllCustodySubnetsEnabled()or by having the maximum Fulu custody group count).This introduces a reusable
SuperNodeSupplierand threads it throughBeaconChainController→Eth2P2PNetworkBuilder→GossipForkSubscriptions*→DataColumnSidecarGossipManager, which now gates optimistic-sync enablement viaisEnabledDuringOptimisticSync(). It also centralizes “super node” detection viaMiscHelpersFulu.isSuperNode, adds trace logging on sidecar gossip receipt, and updates tests/build deps accordingly.Written by Cursor Bugbot for commit 19732b5. This will update automatically on new commits. Configure here.