New header body pair cache by miiu96 · Pull Request #7620 · multiversx/mx-chain-go

miiu96 · 2026-01-16T10:16:52Z

Reasoning behind the pull request

New header body pair cache

Pre-requisites

Based on the Contributing Guidelines the PR author and the reviewers must check the following requirements are met:

was the PR targeted to the correct branch?
if this is a larger feature that probably needs more than one PR, is there a feat branch created?
if this is a feat branch merging, do all satellite projects have a proper tag inside go.mod?

# Conflicts: # process/asyncExecution/headersExecutor.go # process/asyncExecution/headersExecutor_test.go # testscommon/processMocks/executionManagerMock.go

sstanculeanu · 2026-01-16T14:11:10Z

+
+	headerNonce := pair.Header.GetNonce()
+	c.cacheByNonce[headerNonce] = pair
+	c.lastAddedNonce = headerNonce


what if this is called with a lower nonce? in this case, I think we should remove all headers with higher nonces from cache

refactored this
last added nonce is no longer needed, also is no need to remove the pairs with higher nonce
because with this implementation we will fetch pairs by providing the correct nonce.

ok, indeed we have RemoveAtNonceAndHigher that does that

sstanculeanu · 2026-01-16T14:12:43Z

+	c.mutex.Lock()
+	defer c.mutex.Unlock()
+
+	delete(c.cacheByNonce, nonce)


update lastAddedNonce nonce as well?

sstanculeanu · 2026-01-16T14:14:12Z

+	require.False(t, found)
+}
+
+func TestHeaderBodyCache_ConcurrentAccess(t *testing.T) {


maybe add one more test that calls all methods concurrently

sstanculeanu · 2026-01-16T14:17:23Z


 			// blocking operation
-			headerBodyPair, ok := he.blocksQueue.Pop()
+			/// get pair by nonce from blockchain


Suggested change

/// get pair by nonce from blockchain

// get pair by nonce from blockchain

sstanculeanu · 2026-01-16T14:17:47Z

-			headerBodyPair, ok := he.blocksQueue.Pop()
+			/// get pair by nonce from blockchain
+			lastExecutedHeader := he.blockChain.GetLastExecutedBlockHeader()
+			if lastExecutedHeader == nil {


Suggested change

if lastExecutedHeader == nil {

if check.IfNil(lastExecutedHeader) {

sstanculeanu · 2026-01-16T14:18:47Z

+				continue
 			}

 			if check.IfNil(headerBodyPair.Header) || check.IfNil(headerBodyPair.Body) {


I think this can be removed now

sstanculeanu · 2026-01-16T14:19:26Z

 	}

+	lastExecutionResult := he.blockChain.GetLastExecutionResult()
+	if lastExecutionResult != nil {


Suggested change

if lastExecutionResult != nil {

if check.IfNil(lastExecutionResult) {

sstanculeanu · 2026-01-16T14:20:53Z

 	args := createMockArgs()
-	blocksQueue := queue.NewBlocksQueue()
-	args.BlocksQueue = blocksQueue
+	blocksQueue := cache.NewHeaderBodyCache()


Suggested change

blocksQueue := cache.NewHeaderBodyCache()

blocksCache := cache.NewHeaderBodyCache()

sstanculeanu · 2026-01-16T14:21:33Z

-	RemoveAtNonceAndHigher(nonce uint64) []uint64
-	ValidateQueueIntegrity() error
-	Clean(lastAddedNonce uint64)
+	GetByNonce(nonce uint64) (cache.HeaderBodyPair, bool)


rename interface as well

sstanculeanu · 2026-01-16T14:22:03Z


 	executionResultsTracker := executionTrack.NewExecutionResultsTracker()
-	tpn.BlocksQueue = queue.NewBlocksQueue()
+	tpn.BlocksQueue = headersCache.NewHeaderBodyCache()


rename to BlocksCache

mariusmihaic · 2026-01-20T11:35:16Z

+	c.mutex.Lock()
+	defer c.mutex.Unlock()
+
+	nonces := make([]uint64, 0)
+	for nonce := range c.cacheByNonce {
+		if nonce >= providedNonce {
+			delete(c.cacheByNonce, nonce)
+			nonces = append(nonces, nonce)
+		}
+	}
+


You can move the mutex to be used only for this for, up until sort.Slice

mariusmihaic · 2026-01-20T11:36:11Z

+		// Should be pair2 (note: they are different pointers for Header/Body so Equal check works if pointer comparison)
+		// Wait, slice/map/func in structs make Go equality tricky unless pointers.
+		// These are pointers to structs, so it should be fine.


Perhaps remove these AI generated comments

mariusmihaic · 2026-01-20T11:36:56Z

+		require.True(t, found)
+		// Should be pair2 (note: they are different pointers for Header/Body so Equal check works if pointer comparison)
+		// Wait, slice/map/func in structs make Go equality tricky unless pointers.
+		// These are pointers to structs, so it should be fine.


This doesn't actually tests pointer comparison.
It should require that retrievedPair == pair

comment removed.

mariusmihaic · 2026-01-20T11:38:52Z

 		log.Warn("executionManager.Close - failed to close headers executor", "error", err)
 	}

-	em.blocksQueue.Close()


Should we clean the cache here? Does it make sense?

it is not needed, because at the restart the cache will be initialized again.

The base branch was changed.

codecov · 2026-01-22T05:45:28Z

Codecov Report

❌ Patch coverage is 72.82609% with 75 lines in your changes missing coverage. Please review.
✅ Project coverage is 77.60%. Comparing base (8c69291) to head (629fc30).
⚠️ Report is 47 commits behind head on feat/supernova-async-exec.

Files with missing lines	Patch %	Lines
process/asyncExecution/headersExecutor.go	52.83%	19 Missing and 6 partials ⚠️
...xecution/executionTrack/executionResultsTracker.go	48.48%	16 Missing and 1 partial ⚠️
process/sync/baseSync.go	75.00%	6 Missing and 2 partials ⚠️
node/chainSimulator/process/processor.go	30.00%	7 Missing ⚠️
...syncExecution/executionManager/executionManager.go	55.55%	3 Missing and 1 partial ⚠️
process/common.go	57.14%	3 Missing ⚠️
process/coordinator/process.go	40.00%	1 Missing and 2 partials ⚠️
process/asyncExecution/cache/headerBodyCache.go	96.55%	1 Missing and 1 partial ⚠️
process/block/preprocess/basePreProcess.go	66.66%	1 Missing and 1 partial ⚠️
process/block/preprocess/transactionsV2.go	90.00%	1 Missing and 1 partial ⚠️
... and 2 more

Additional details and impacted files

@@                      Coverage Diff                      @@
##           feat/supernova-async-exec    #7620      +/-   ##
=============================================================
- Coverage                      77.62%   77.60%   -0.03%     
=============================================================
  Files                            877      877              
  Lines                         121763   121752      -11     
=============================================================
- Hits                           94515    94480      -35     
- Misses                         20987    21004      +17     
- Partials                        6261     6268       +7

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Fix tests

# Conflicts: # process/asyncExecution/queue/blocksQueue.go # process/asyncExecution/queue/blocksQueue_test.go

…ameter

# Conflicts: # node/chainSimulator/process/processor.go

…balance fix unexecutable tx due to initial balance

fix on concurrent header execution and revert

# Conflicts: # process/sync/baseSync.go

# Conflicts: # factory/processing/processComponents.go # integrationTests/testFullNode.go # integrationTests/testProcessorNode.go # integrationTests/testSyncNode.go # integrationTests/vm/staking/metaBlockProcessorCreator.go # process/asyncExecution/headersExecutor_test.go # process/asyncExecution/queue/blocksQueue.go # process/asyncExecution/queue/blocksQueue_test.go # process/block/baseProcess_test.go # process/block/export_test.go

Header body queue capacity in config.toml

refactor blocks queue

f782261

miiu96 self-assigned this Jan 16, 2026

miiu96 added 3 commits January 16, 2026 12:21

fix linter

199c147

fixes

cecc4cf

fix race

a19303b

miiu96 changed the base branch from feat/supernova-async-exec to fix-referenced-gaps January 16, 2026 13:30

Merge branch 'fix-referenced-gaps' into header-cache

33fe1ec

# Conflicts: # process/asyncExecution/headersExecutor.go # process/asyncExecution/headersExecutor_test.go # testscommon/processMocks/executionManagerMock.go

sstanculeanu reviewed Jan 16, 2026

View reviewed changes

fixes after review

2447a6b

sstanculeanu previously approved these changes Jan 20, 2026

View reviewed changes

Merge branch 'fix-referenced-gaps' into header-cache

5a57b67

mariusmihaic self-requested a review January 20, 2026 11:19

mariusmihaic reviewed Jan 20, 2026

View reviewed changes

extra log and fixes after review

3481307

Base automatically changed from fix-referenced-gaps to feat/supernova-async-exec January 20, 2026 14:59

Merge branch 'feat/supernova-async-exec' into header-cache

d07bac4

miiu96 changed the base branch from feat/supernova-async-exec to fix-edge-on-bootstrap January 20, 2026 15:01

Merge branch 'fix-edge-on-bootstrap' into header-cache

8fcf73c

Base automatically changed from fix-edge-on-bootstrap to feat/supernova-async-exec January 20, 2026 15:54

An error occurred while trying to automatically change base from fix-edge-on-bootstrap to feat/supernova-async-exec January 20, 2026 15:54

Merge branch 'feat/supernova-async-exec' into header-cache

9e974a3

miiu96 changed the base branch from feat/supernova-async-exec to proper-fix-on-consensus-revert January 21, 2026 08:20

Merge branch 'proper-fix-on-consensus-revert' into header-cache

3698e96

Base automatically changed from proper-fix-on-consensus-revert to dynamic-time-for-selection January 21, 2026 10:54

Base automatically changed from dynamic-time-for-selection to feat/supernova-async-exec January 21, 2026 12:30

miiu96 and others added 2 commits January 21, 2026 17:19

fixes

5283363

Merge branch 'feat/supernova-async-exec' into header-cache

d4d3535

sstanculeanu previously approved these changes Jan 22, 2026

View reviewed changes

AdoAdoAdo and others added 25 commits January 23, 2026 18:13

fix tests

8fd4b14

Merge pull request #7638 from multiversx/fix-replacement-header

a8c35ef

Fix tests

Merge branch 'feat/supernova-async-exec' into header-cache

318ee67

# Conflicts: # process/asyncExecution/queue/blocksQueue.go # process/asyncExecution/queue/blocksQueue_test.go

fixes after merge

59fbb07

fix on concurrent header execution and revert

86858c2

fix locking

38f749f

prevent adding stale execution results

8ae51bb

simplify

d33f35f

skip updating last executed header on replaced block

fb9d849

update checkAndUpdateContextBeforeExecution to include headerHash par…

6286183

…ameter

fix header hash in chain simulator and update tests

12c0788

Merge branch 'feat/supernova-async-exec' into header-cache

ff5147d

# Conflicts: # node/chainSimulator/process/processor.go

Merge branch 'header-cache' into fix-cleanup-on-consensus

9107e94

fix after merge

aaa6aa7

cleanup logging in tests

31bf957

fix unexecutable tx due to initial balance

8201e37

fix: update transaction cost calculation for async execution

ec76bf7

Merge pull request #7649 from multiversx/fix-unexecutable-tx-initial-…

d671a0b

…balance fix unexecutable tx due to initial balance

Merge pull request #7640 from multiversx/fix-cleanup-on-consensus

8ff38cd

fix on concurrent header execution and revert

Merge branch 'feat/supernova-async-exec' into header-cache

74a6acb

# Conflicts: # process/sync/baseSync.go

header body cache capacity

6df2424

Merge branch 'feat/supernova-async-exec' into header-cache

1481fa7

fixes after merge

df0de95

Merge pull request #7656 from multiversx/header-body-capacity

e6e1807

Header body queue capacity in config.toml

sstanculeanu approved these changes Feb 2, 2026

View reviewed changes

Merge branch 'feat/supernova-async-exec' into header-cache

629fc30

AdoAdoAdo approved these changes Feb 2, 2026

View reviewed changes

AdoAdoAdo merged commit c635e8b into feat/supernova-async-exec Feb 2, 2026
9 of 11 checks passed

AdoAdoAdo deleted the header-cache branch February 2, 2026 13:04

	/// get pair by nonce from blockchain
	// get pair by nonce from blockchain

	if lastExecutedHeader == nil {
	if check.IfNil(lastExecutedHeader) {

	if lastExecutionResult != nil {
	if check.IfNil(lastExecutionResult) {

	blocksQueue := cache.NewHeaderBodyCache()
	blocksCache := cache.NewHeaderBodyCache()

Conversation

miiu96 commented Jan 16, 2026

Reasoning behind the pull request

Pre-requisites

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov Bot commented Jan 22, 2026 •

edited

Loading