Transactions network with reused host#7067
Open
sstanculeanu wants to merge 33 commits into
Open
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## rc/supernova #7067 +/- ##
================================================
- Coverage 76.81% 76.78% -0.03%
================================================
Files 846 847 +1
Lines 113181 113481 +300
================================================
+ Hits 86936 87141 +205
- Misses 20404 20468 +64
- Partials 5841 5872 +31 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
✅ Integration Tests passed successfully! 📊 MultiversX Automated Test Report: View Report 🔄 Build Details:
🚀 Environment Variables:
|
|
✅ Integration Tests passed successfully! 📊 MultiversX Automated Test Report: View Report 🔄 Build Details:
🚀 Environment Variables:
|
|
✅ Integration Tests passed successfully! 📊 MultiversX Automated Test Report: View Report 🔄 Build Details:
🚀 Environment Variables:
|
… into transactions-network-reuse-host
|
✅ Integration Tests passed successfully! 📊 MultiversX Automated Test Report: View Report 🔄 Build Details:
🚀 Environment Variables:
|
|
❌ Integration Tests completed with failures or errors. 📊 MultiversX Automated Test Report: View Report 🔄 Build Details:
🚀 Environment Variables:
|
into transactions-network-reuse-host # Conflicts: # common/common.go # common/interface.go # go.mod # go.sum # process/errors.go # process/factory/interceptorscontainer/baseInterceptorsContainerFactory.go # process/factory/interceptorscontainer/metaInterceptorsContainerFactory.go # process/factory/interceptorscontainer/shardInterceptorsContainerFactory.go # update/factory/fullSyncInterceptors.go
|
❌ Integration Tests completed with failures or errors. 📊 MultiversX Automated Test Report: View Report 🔄 Build Details:
🚀 Environment Variables:
|
…ersx/mx-chain-go into transactions-network-reuse-host # Conflicts: # common/common.go # dataRetriever/requestHandlers/requestHandler.go # epochStart/bootstrap/storageProcess.go # factory/processing/processComponents.go # go.mod # go.sum # integrationTests/singleShard/transaction/interceptedResolvedTx/interceptedResolvedTx_test.go # node/nodeTesting.go # process/errors.go # process/interface.go
sstanculeanu
changed the base branch from
rc/supernova
to
feat/supernova-async-exec
…ersx/mx-chain-go into transactions-network-reuse-host # Conflicts: # common/interface.go # dataRetriever/factory/resolverscontainer/args.go # dataRetriever/factory/resolverscontainer/metaResolversContainerFactory.go # dataRetriever/factory/resolverscontainer/metaResolversContainerFactory_test.go # dataRetriever/factory/resolverscontainer/shardResolversContainerFactory.go # dataRetriever/factory/resolverscontainer/shardResolversContainerFactory_test.go # epochStart/bootstrap/process.go # factory/processing/processComponents.go # integrationTests/testHeartbeatNode.go # integrationTests/testProcessorNode.go
Contributor
There was a problem hiding this comment.
Pull request overview
Introduces a dedicated transactions subnetwork (reusing the same libp2p host) to apply distinct GossipSub settings for transaction-related topics, and refactors topic wiring to be network-aware and to use centralized topic constants.
Changes:
- Add a TransactionsNetwork subnetwork and configuration plumbing for subnetworks with custom PubSub peer counts.
- Route transaction-related topics (transactions / unsignedTransactions / rewardsTransactions) to the transactions subnetwork when SupernovaFlag is enabled, including epoch-change migration logic.
- Refactor topic handling to use common topic constants and update topic creation/processor registration APIs to include a network type.
Reviewed changes
Copilot reviewed 82 out of 83 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| update/factory/fullSyncResolversContainerFactory.go | Updates resolver registration to pass explicit network type. |
| update/factory/fullSyncInterceptors.go | Routes interceptor topics via network selection (main/full-archive/transactions). |
| testscommon/p2pmocks/messengerStub.go | Updates mocks for network-aware CreateTopic/RegisterMessageProcessor; adds UnJoinTopic stub. |
| testscommon/components/components.go | Adds transactions subnetwork config in test component defaults. |
| process/txsSender/txsSender_test.go | Switches tx topic constants to common.* in tests. |
| process/txsSender/txsSender.go | Switches tx topic constant to common.TransactionTopic. |
| process/p2p/mock/topicResolverMock.go | Adds mock resolver supporting RequestTopic for epoch topic-move tests. |
| process/p2p/export_test.go | Exposes test helper to toggle internal topicsMoved flag. |
| process/p2p/epochChangeTopicsHandler_test.go | Adds tests covering epoch-driven migration of tx topics to transactions network. |
| process/p2p/epochChangeTopicsHandler.go | Adds epoch-based handler to move tx interceptors/resolvers onto transactions network. |
| process/mock/topicHandlerStub.go | Updates topic handler stub to network-aware CreateTopic/RegisterMessageProcessor. |
| process/interface.go | Updates TopicHandler API to accept network type; adds EpochChangeTopicsHandler interface. |
| process/factory/interceptorscontainer/shardInterceptorsContainerFactory_test.go | Updates tests for network-aware topic creation/registration and common topic constants. |
| process/factory/interceptorscontainer/shardInterceptorsContainerFactory.go | Switches rewards tx topic constant to common.RewardsTransactionTopic. |
| process/factory/interceptorscontainer/metaInterceptorsContainerFactory_test.go | Updates tests for network-aware topic creation/registration and common topic constants. |
| process/factory/interceptorscontainer/metaInterceptorsContainerFactory.go | Switches rewards tx topic constant to common.RewardsTransactionTopic. |
| process/factory/interceptorscontainer/baseInterceptorsContainerFactory.go | Routes topic creation/registration through selected network type; uses common.* tx topics. |
| process/factory/factory.go | Removes tx topic constants (moved to common). |
| process/errors.go | Adds errors used by epoch-change topic migration logic. |
| process/coordinator/process_test.go | Updates unsigned tx topic usage to common.UnsignedTransactionTopic. |
| process/coordinator/process.go | Routes broadcast topic selection via common.* tx topic constants. |
| p2p/disabled/networkMessenger.go | Adapts disabled messenger to new network-aware topic APIs; adds UnJoinTopic. |
| p2p/constants.go | Adds TransactionsNetwork network type constant. |
| p2p/config/config.go | Re-exports subnetwork-related config types. |
| node/nodeTesting_test.go | Updates tx topic constant usage to common.TransactionTopic. |
| node/nodeTesting.go | Updates tx topic constant usage to common.TransactionTopic. |
| node/nodeHelper.go | Updates tx topic constant usage to common.TransactionTopic. |
| node/chainSimulator/components/syncedTxsSender.go | Updates tx topic constant usage to common.TransactionTopic. |
| node/chainSimulator/components/syncedMessenger_test.go | Updates tests for network-aware CreateTopic/RegisterMessageProcessor. |
| node/chainSimulator/components/syncedMessenger.go | Adapts synced messenger APIs to accept network type; adds UnJoinTopic. |
| node/chainSimulator/components/syncedBroadcastNetwork_test.go | Updates tests for network-aware CreateTopic/RegisterMessageProcessor. |
| integrationTests/testProcessorNode.go | Uses network type when creating topics; passes EnableEpochsHandler to resolvers args. |
| integrationTests/testInitializer.go | Adds subnetwork config; makes messenger creation network-type aware; updates topic constants usage. |
| integrationTests/testHeartbeatNode.go | Adapts messenger creation/topic creation/registration to network type; updates topic base fields. |
| integrationTests/state/stateTrieSync/stateTrieSync_test.go | Uses network type when creating consensus topic. |
| integrationTests/singleShard/transaction/interceptedResolvedUnsignedTx/interceptedResolvedUnsignedTx_test.go | Uses common.UnsignedTransactionTopic; comment formatting updates. |
| integrationTests/singleShard/transaction/interceptedResolvedTx/interceptedResolvedTx_test.go | Uses common.RewardsTransactionTopic. |
| integrationTests/singleShard/transaction/interceptedBulkUnsignedTx/interceptedBulkUnsignedTx_test.go | Uses common.UnsignedTransactionTopic; comment formatting updates. |
| integrationTests/resolvers/smartContractsResults/scrs_test.go | Uses common.UnsignedTransactionTopic. |
| integrationTests/resolvers/rewards/rewards_test.go | Uses common.RewardsTransactionTopic; comment formatting updates. |
| integrationTests/p2p/networkSharding-hbv2/networkSharding_test.go | Adds transactions subnetwork config to integration test P2P config. |
| integrationTests/p2p/antiflood/common.go | Uses network type for topic creation/processor registration in antiflood integration tests. |
| integrationTests/multiShard/transaction/interceptedResolvedBulkTx/interceptedResolvedBulkTx_test.go | Uses common.TransactionTopic; comment formatting updates. |
| integrationTests/longTests/antiflooding/antiflooding_test.go | Uses network type for topic creation/processor registration; comment formatting updates. |
| go.sum | Updates mx-chain-communication-go dependency checksum to new pseudo-version. |
| go.mod | Bumps mx-chain-communication-go to pseudo-version that supports subnetwork APIs. |
| factory/processing/processComponents.go | Instantiates and wires EpochChangeTopicsHandler; passes EnableEpochsHandler into resolver factories. |
| factory/heartbeat/heartbeatV2Components_test.go | Updates mocks for network-aware CreateTopic. |
| factory/heartbeat/heartbeatV2Components.go | Creates heartbeat topics using explicit network type. |
| factory/consensus/consensusComponents_test.go | Updates mocks for network-aware CreateTopic. |
| factory/consensus/consensusComponents.go | Creates consensus topic / registers processor on main network explicitly. |
| epochStart/metachain/baseRewards.go | Uses common.RewardsTransactionTopic constant for rewards broadcast topic. |
| epochStart/bootstrap/syncEpochStartMeta_test.go | Updates messenger stub signature for network-aware RegisterMessageProcessor. |
| epochStart/bootstrap/syncEpochStartMeta.go | Creates/registers topics with explicit network type during epoch start meta sync. |
| epochStart/bootstrap/storageProcess.go | Creates consensus topic with explicit network type. |
| epochStart/bootstrap/process_test.go | Updates CreateTopic mock signature; sets epochStartMeta in tests. |
| epochStart/bootstrap/process.go | Instantiates epoch-change topic handler during bootstrap and forces epoch handling; creates topics with explicit network type; passes EnableEpochsHandler to resolvers factory args. |
| epochStart/bootstrap/fromLocalStorage.go | Creates consensus topic with explicit network type. |
| dataRetriever/requestHandlers/requestHandler.go | Switches requester topics to common.* tx topic constants. |
| dataRetriever/interface.go | Updates TopicHandler API to accept network type. |
| dataRetriever/factory/storageRequestersContainer/shardRequestersContainerFactory_test.go | Updates messenger stub signatures for network-aware CreateTopic/RegisterMessageProcessor. |
| dataRetriever/factory/storageRequestersContainer/shardRequestersContainerFactory.go | Switches rewards requester base topic to common.RewardsTransactionTopic. |
| dataRetriever/factory/storageRequestersContainer/metaRequestersContainerFactory_test.go | Updates messenger stub signatures for network-aware CreateTopic/RegisterMessageProcessor. |
| dataRetriever/factory/storageRequestersContainer/metaRequestersContainerFactory.go | Switches rewards requester base topic to common.RewardsTransactionTopic. |
| dataRetriever/factory/storageRequestersContainer/baseRequestersContainerFactory.go | Switches tx/unsigned base topics to common.* constants. |
| dataRetriever/factory/resolverscontainer/shardResolversContainerFactory_test.go | Updates stubs for network-aware APIs; adds EnableEpochsHandler to args. |
| dataRetriever/factory/resolverscontainer/shardResolversContainerFactory.go | Routes tx resolver registration by network; switches to common.* topics; adds EnableEpochsHandler plumbing. |
| dataRetriever/factory/resolverscontainer/metaResolversContainerFactory_test.go | Updates stubs for network-aware APIs; adds EnableEpochsHandler to args. |
| dataRetriever/factory/resolverscontainer/metaResolversContainerFactory.go | Routes meta resolver registration by network; switches to common.* topics; adds EnableEpochsHandler plumbing. |
| dataRetriever/factory/resolverscontainer/baseResolversContainerFactory.go | Adds EnableEpochsHandler validation; registers tx resolvers on transactions network when enabled; makes other registrations network-explicit. |
| dataRetriever/factory/resolverscontainer/args.go | Adds EnableEpochsHandler to resolver factory args. |
| dataRetriever/factory/requestersContainer/shardRequestersContainerFactory_test.go | Updates CreateTopic stub signature for network-aware API. |
| dataRetriever/factory/requestersContainer/shardRequestersContainerFactory.go | Switches rewards requester base topic to common.RewardsTransactionTopic; updates excluded topic to common.TransactionTopic. |
| dataRetriever/factory/requestersContainer/metaRequestersContainerFactory.go | Switches rewards requester base topic to common.RewardsTransactionTopic. |
| dataRetriever/factory/requestersContainer/baseRequestersContainerFactory.go | Switches tx/unsigned requester base topics to common.* constants. |
| consensus/broadcast/shardChainMessenger_test.go | Switches transaction map key to common.TransactionTopic. |
| config/tomlConfig_test.go | Extends TOML parsing test to include SubNetworks configuration. |
| common/proofs.go | Removes consensus dependency by using new common.ProofHandler interface. |
| common/interface.go | Adds ProofHandler interface used by common proofs helpers. |
| common/constants.go | Moves transaction topic constants into common. |
| common/common.go | Adds ShouldUseTransactionsNetwork helper for topic routing decisions. |
| cmd/node/config/p2p.toml | Adds SubNetworks configuration section for transactions network. |
| cmd/node/config/fullArchiveP2P.toml | Adds SubNetworks configuration section for transactions network in full-archive config. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| # SubNetworks holds the configuration for subnetworks | ||
| # currently only transactions live on a separate subnetwork with different gossip configuration | ||
| # default gossip configuration: https://github.com/libp2p/go-libp2p-pubsub/blob/v0.9.3/gossipsub.go#L33 |
There was a problem hiding this comment.
The comment references the default GossipSub configuration at go-libp2p-pubsub v0.9.3, but this repository depends on a newer go-libp2p-pubsub version. This link is likely outdated/misleading; consider updating it to the current dependency version (or linking to master/a commit) to keep the documentation accurate.
Suggested change
| # default gossip configuration: https://github.com/libp2p/go-libp2p-pubsub/blob/v0.9.3/gossipsub.go#L33 | |
| # default gossip configuration: https://github.com/libp2p/go-libp2p-pubsub/blob/master/gossipsub.go#L33 |
|
|
||
| # SubNetworks holds the configuration for subnetworks | ||
| # currently only transactions live on a separate subnetwork with different gossip configuration | ||
| # default gossip configuration: https://github.com/libp2p/go-libp2p-pubsub/blob/v0.9.3/gossipsub.go#L33 |
There was a problem hiding this comment.
The embedded TOML sample references the default GossipSub configuration at go-libp2p-pubsub v0.9.3, but this repo uses a newer go-libp2p-pubsub version. Updating this link (or linking to master/a commit) would avoid the test fixture documenting an outdated config reference.
Suggested change
| # default gossip configuration: https://github.com/libp2p/go-libp2p-pubsub/blob/v0.9.3/gossipsub.go#L33 | |
| # default gossip configuration: https://github.com/libp2p/go-libp2p-pubsub/blob/master/gossipsub.go#L33 |
Comment on lines
+63
to
+66
| isFullArchive: args.IsFullArchive, | ||
| } | ||
| args.EpochNotifier.RegisterNotifyHandler(handler) | ||
|
|
There was a problem hiding this comment.
NewEpochChangeTopicsHandler calls args.EpochNotifier.RegisterNotifyHandler(handler) but checkArgEpochChangeTopicsHandler never validates that args.EpochNotifier is non-nil. If a nil notifier is provided this will panic at runtime. Add a nil-check in checkArgEpochChangeTopicsHandler (e.g. return process.ErrNilEpochNotifier).
Comment on lines
+602
to
+604
| return strings.Contains(topic, TransactionTopic) || | ||
| strings.Contains(topic, UnsignedTransactionTopic) || | ||
| strings.Contains(topic, RewardsTransactionTopic) |
There was a problem hiding this comment.
ShouldUseTransactionsNetwork uses strings.Contains to classify transaction topics. This can produce false positives if any other topic happens to include these substrings. Since the code constructs topics by prefixing with the base topic constant, strings.HasPrefix (or an explicit equality/prefix check for the exact base + shard identifier patterns) would be more precise and safer.
Suggested change
| return strings.Contains(topic, TransactionTopic) || | |
| strings.Contains(topic, UnsignedTransactionTopic) || | |
| strings.Contains(topic, RewardsTransactionTopic) | |
| return strings.HasPrefix(topic, TransactionTopic) || | |
| strings.HasPrefix(topic, UnsignedTransactionTopic) || | |
| strings.HasPrefix(topic, RewardsTransactionTopic) |
|
|
||
| # SubNetworks holds the configuration for subnetworks | ||
| # currently only transactions live on a separate subnetwork with different gossip configuration | ||
| # default gossip configuration: https://github.com/libp2p/go-libp2p-pubsub/blob/v0.9.3/gossipsub.go#L33 |
There was a problem hiding this comment.
The comment references the default GossipSub configuration at go-libp2p-pubsub v0.9.3, but this repository depends on a newer go-libp2p-pubsub version. This link is likely outdated/misleading; consider updating it to the current dependency version (or linking to master/a commit) to keep the documentation accurate.
Suggested change
| # default gossip configuration: https://github.com/libp2p/go-libp2p-pubsub/blob/v0.9.3/gossipsub.go#L33 | |
| # default gossip configuration: https://github.com/libp2p/go-libp2p-pubsub/blob/master/gossipsub.go#L33 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reasoning behind the pull request
Proposed changes
Testing procedure
Pre-requisites
Based on the Contributing Guidelines the PR author and the reviewers must check the following requirements are met:
featbranch created?featbranch merging, do all satellite projects have a proper tag insidego.mod?