eth/consensus : implement eccpow consensus engine#10
Open
mmingyeomm wants to merge 3417 commits into
Open
Conversation
In `buildPayload()`, the background goroutine uses a `select` to wait on the recommit timer, the stop channel, and the end timer. When both `timer.C` and `payload.stop` are ready simultaneously, Go's `select` picks a case non-deterministically. This means the loop can enter the `timer.C` case and perform an unnecessary `generateWork` call even after the payload has been resolved. Add a non-blocking check of `payload.stop` at the top of the `timer.C` case to exit immediately when the payload has already been delivered.
Return the Amsterdam instruction set from `LookupInstructionSet` when `IsAmsterdam` is true, so Amsterdam rules no longer fall through to the Osaka jump table. --------- Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
For bal-devnet-3 we need to update the EIP-8024 implementation to the latest spec changes: ethereum/EIPs#11306 > Note: I deleted tests not specified in the EIP bc maintaining them through EIP changes is too error prone.
Pebble maintains a batch pool to recycle the batch object. Unfortunately batch object must be explicitly returned via `batch.Close` function. This PR extends the batch interface by adding the close function and also invoke batch.Close in some critical code paths. Memory allocation must be measured before merging this change. What's more, it's an open question that whether we should apply batch.Close as much as possible in every invocation.
Implements https://eips.ethereum.org/EIPS/eip-7778 --------- Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Mainnet was already overriding --cache to 4096. This PR just makes this the default.
…33927) The BatchSpanProcessor queue size was incorrectly set to DefaultMaxExportBatchSize (512) instead of DefaultMaxQueueSize (2048). I noticed the issue on bloatnet when analyzing the block building traces. During a particular run, the miner was including 1000 transactions in a single block. When telemetry is enabled, the miner creates a span for each transaction added to the block. With the queue capped at 512, spans were silently dropped when production outpaced the span export, resulting in incomplete traces with orphaned spans. While this doesn't eliminate the possibility of drops under extreme load, using the correct default restores the 4x buffer between queue capacity and export batch size that the SDK was designed around.
Fixes a regression in #33593 where a block gas limit > gasCap resulted in more execution than the gas cap.
Eth currently has a flaky test, related to the tx fetcher. The issue seems to happen when Unsubscribe is called while sub is nil. It seems that chain.Stop() may be invoked before the loop starts in some tests, but the exact cause is still under investigation through repeated runs. I think this change will at least prevent the error.
The computation of `MAIN_STORAGE_OFFSET` was incorrect, causing the last byte of the stem to be dropped. This means that there would be a collision in the hash computation (at the preimage level, not a hash collision of course) if two keys were only differing at byte 31.
…le tree (#33961) This is an optimization that existed for verkle and the MPT, but that got dropped during the rebase. Mark the nodes that were modified as needing recomputation, and skip the hash computation if this is not needed. Otherwise, the whole tree is hashed, which kills performance.
`GenerateChain` commits trie nodes asynchronously, and it can happen that some nodes aren't making it to the db in time for `GenerateChain` to open it and find the data it is looking for.
Add nil checks to prevent potential panics when keystore backend is unavailable in the Clef signer API.
Reduce allocations in calculation of tx cost. --------- Co-authored-by: weixie.cui <weixie.cui@okg.com> Co-authored-by: Sina M <1591639+s1na@users.noreply.github.com>
Updates go-eth-kzg to https://github.com/crate-crypto/go-eth-kzg/releases/tag/v1.5.0 Significantly reduces the allocations in VerifyCellProofBatch which is around ~5% of all allocations on my node --------- Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>
I observed failing tests in Hive `engine-withdrawals`: - https://hive.ethpandaops.io/#/test/generic/1772351960-ad3e3e460605c670efe1b4f4178eb422?testnumber=146 - https://hive.ethpandaops.io/#/test/generic/1772351960-ad3e3e460605c670efe1b4f4178eb422?testnumber=147 ```shell DEBUG (Withdrawals Fork on Block 2): NextPayloadID before getPayloadV2: id=0x01487547e54e8abe version=1 >> engine_getPayloadV2("0x01487547e54e8abe") << error: {"code":-38005,"message":"Unsupported fork"} FAIL: Expected no error on EngineGetPayloadV2: error=Unsupported fork ``` The same failure pattern occurred for Block 3. Per Shanghai engine_getPayloadV2 spec, pre-Shanghai payloads should be accepted via V2 and returned as ExecutionPayloadV1: - executionPayload: ExecutionPayloadV1 | ExecutionPayloadV2 - ExecutionPayloadV1 MUST be returned if payload timestamp < Shanghai timestamp - ExecutionPayloadV2 MUST be returned if payload timestamp >= Shanghai timestamp Reference: - https://github.com/ethereum/execution-apis/blob/main/src/engine/shanghai.md#engine_getpayloadv2 Current implementation only allows GetPayloadV2 on the Shanghai fork window (`[]forks.Fork{forks.Shanghai}`), so pre-Shanghai payloads are rejected with Unsupported fork. If my interpretation of the spec is incorrect, please let me know and I can adjust accordingly. --------- Co-authored-by: muzry.li <muzry.li1@ambergroup.io>
This PR fixes a regression introduced in https://github.com/ethereum/go-ethereum/pull/33836/changes Before PR 33836, running mainnet would automatically bump the cache size to 4GB and trigger a cache re-calculation, specifically setting the key-value database cache to 2GB. After PR 33836, this logic was removed, and the cache value is no longer recomputed if no command line flags are specified. The default key-value database cache is 512MB. This PR bumps the default key-value database cache size alongside the default cache size for other components (such as snapshot) accordingly.
We got a report for a bug in the tracing journal which has the responsibility to emit events for all state that must be reverted. The edge case is as follows: on CREATE operations the nonce is incremented. When a create frame reverts, the nonce increment associated with it does **not** revert. This works fine on master. Now one step further: if the parent frame reverts tho, the nonce **should** revert and there is the bug.
This PR contains two changes: Firstly, the finalized header will be resolved from local chain if it's not recently announced via the `engine_newPayload`. What's more importantly is, in the downloader, originally there are two code paths to push forward the pivot point block, one in the beacon header fetcher (`fetchHeaders`), and another one is in the snap content processer (`processSnapSyncContent`). Usually if there are new blocks and local pivot block becomes stale, it will firstly be detected by the `fetchHeaders`. `processSnapSyncContent` is fully driven by the beacon headers and will only detect the stale pivot block after synchronizing the corresponding chain segment. I think the detection here is redundant and useless.
…3984) The `--remove.chain` flag incorrectly described itself as selecting "state data" for removal, which could mislead operators into removing the wrong data category. This corrects the description to accurately reflect that the flag targets chain data (block bodies and receipts).
This PR allows users to prune their nodes up to the Prague fork. It indirectly depends on #32157 and can't really be merged before eraE files are widely available for download. The `--history.chain` flag becomes mandatory for `prune-history` command. Here I've listed all the edge cases that can happen and how we behave: ## prune-history Behavior | From | To | Result | |-------------|--------------|--------------------------| | full | postmerge | ✅ prunes | | full | postprague | ✅ prunes | | postmerge | postprague | ✅ prunes further | | postprague | postmerge | ❌ can't unprune | | any | all | ❌ use import-history | ## Node Startup Behavior | DB State | Flag | Result | |-------------|--------------|----------------------------------------------------------------| | fresh | postprague | ✅ syncs from Prague | | full | postprague | ❌ "run prune-history first" | | postmerge | postprague | ❌ "run prune-history first" | | postprague | postmerge | ❌ "can't unprune, use import-history or fix flag" | | pruned | all | ✅ accepts known prune points |
This PR adds a cmd tool fetchpayload which connects to a node and gets all the information in order to create a serialized payload that can then be passed to the zkvm.
`TestSubscribePendingTxHashes` hangs indefinitely because pending tx events are permanently missed due to a race condition in `NewPendingTransactions` (and `NewHeads`). Both handlers called their event subscription functions (`SubscribePendingTxs`, `SubscribeNewHeads`) inside goroutines, so the RPC handler returned the subscription ID to the client before the filter was installed in the event loop. When the client then sent a transaction, the event fired but no filter existed to catch it — the event was silently lost. - Move `SubscribePendingTxs` and `SubscribeNewHeads` calls out of goroutines so filters are installed synchronously before the RPC response is sent, matching the pattern already used by `Logs` and `TransactionReceipts` <!-- START COPILOT CODING AGENT TIPS --> --- 💬 We'd love your input! Share your thoughts on Copilot coding agent in our [2 minute survey](https://gh.io/copilot-coding-agent-survey). --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: s1na <1591639+s1na@users.noreply.github.com>
This PR introduces a separate transaction pool type for sparse blobpool. In sparse blobpool, PooledTransactions message delivers transactions without blobs, partial or full cells are downloaded by Cells message. Blobpool no longer stores transactions with complete sidecars, and it stores transactions without blobs, along with the corresponding cells. Because of this, a dedicated type distinct from types.Transaction is required. This PR introduces a type called `BlobTxForPool` and stores each sidecar field independently, in order to bypass the assumption that a sidecar always exists as a complete unit. Reintroducing the conversion queue was considered, but was ultimately omitted because type conversion should be sufficiently fast. With sparse blobpool, blob -> cell computation would take about ~13ms per blob. Not sure whether this is fast enough, but otherwise we can add the conversion queue later on the sparse blobpool branch.
1. should use !reflect.DeepEqual.
2. big.NewInt(0).SetBits([]big.Word{}) work around for DeepEqual when
big.Int is zero, unpack return a []big.Word{}.
Passing `--v2=false` currently still selects the v2 binding generator because the command checks whether the flag was set. This switches generation to use the boolean flag value, so explicit false continues to generate legacy bindings while `--v2` keeps selecting v2.
This PR introduces OnGasChangeV2 tracing hook, as the pre-requisite for landing EIP-8037. --------- Co-authored-by: Sina M <1591639+s1na@users.noreply.github.com>
This PR extends the journal to track the pre-transaction values of mutated balances, nonces, and code. At the end of the transaction, these values are used to filter out no-op changes, such as balance transitions from a-> b->a. These changes are excluded from the block-level access list. Additionally, there is a dedicated `bal.ConstructionBlockAccessList` objects for gathering the state reads and writes within the current transaction. These state writes will be keyed by the block accessList index. --------- Co-authored-by: jwasinger <j-wasinger@hotmail.com>
## Summary The `--rpc.telemetry.sample-ratio` flag declares `Value: 1.0` and `geth --help` advertises `(default: 1)`. In practice, however, omitting the flag produces a sample ratio of `0`, causing `sdktrace.TraceIDRatioBased(0)` to drop 100% of spans. Users who enable `--rpc.telemetry` see the `OpenTelemetry trace export enabled` log line and a clean startup, but no traces ever leave the process. The root cause is the interaction between two pieces of code: 1. `cmd/utils/flags.go:setOpenTelemetry` (added in #34062) only copies the flag value when `ctx.IsSet(...)` returns true: ```go if ctx.IsSet(RPCTelemetrySampleRatioFlag.Name) { tcfg.SampleRatio = ctx.Float64(RPCTelemetrySampleRatioFlag.Name) } ``` That is the right pattern for "don't clobber a config-file value with the CLI default," but it implies that something else must initialise the field when neither source sets it. 2. `node/defaults.go:DefaultConfig` never initialises `OpenTelemetry.SampleRatio`, leaving it at the float64 zero value. The result for the common CLI-only user (no TOML config) is `SampleRatio = 0` → every span is silently dropped, despite the documented default of 1. ## Change Seed `OpenTelemetry: OpenTelemetryConfig{SampleRatio: 1.0}` in `node.DefaultConfig` so the documented default matches runtime behavior and the `ctx.IsSet` guard in `setOpenTelemetry` continues to do what it was designed to do.
Avoids every legacy tx hash query hitting the blob pool on the path of BlobPool.GetRLP.
This PR fixes a bug in the current blobpool `Reset` function where it used the Transaction type instead of blobTxForPool. Decoding transactions fetched from the pool as Transaction type caused an error because the blobpool stores blobTxForPool types.
This method is similar to `eth_blobBaseFee` but returns the next base fee.
Return blockchain rewind failures from debug_setHead instead of ignoring them.
…4957) This PR finally lands EIP-7928, collecting the block accessList during the block execution and verifying against the block header. --------- Co-authored-by: jwasinger <j-wasinger@hotmail.com> Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
This PR implements the serving side of the eth71 BAL exchange messages. Until commit 4cd7092 also contained the requesting side, but since that part still needs more work, I'm splitting it out into a separate PR. The test injects BALs directly into rawdb. This can be removed once BAL generation is integrated into the chain maker. --------- Co-authored-by: Felix Lange <fjl@twurst.com>
…4967) Updates the static validation logic to cover additional edge cases (reflecting the state of the latest devnet branch, except cleaned up slightly). --------- Co-authored-by: Gary Rong <garyrong0905@gmail.com>
It's a change to BAL json marshalling and t8n tooling to adhere the EELS definition.
…33969) Adds a fast path for ExecutionPayloadEnvelope and BlobAndProofListV* that bypasses encoding/json's reflection and re-validation, which are expensive for large payloads with many blobs. Also hand-rolls the jsonrpcMessage wire encoding in the RPC codec to avoid a second re-validation pass when writing responses to the connection. Resolves #33814 --------- Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de> Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: healthykim <bsbs8645@snu.ac.kr> Co-authored-by: Felix Lange <fjl@twurst.com>
Fixes a regression where nil results from getBlobs were encoded as an empty array instead of null. --------- Co-authored-by: Felix Lange <fjl@twurst.com>
…3797) Adds Ledger signing support for BlobTxType (EIP-4844) and SetCodeTxType (EIP-7702) transactions. --------- Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>
…s field spec ordering (#35009) This PR: - Adds `engine_newPayloadWithWitnessV5`. The codebase already supports the previous `VX`, so only `V5` was missing. - Make the consensus witness format use the field [ordering defined in the spec](https://github.com/ethereum/execution-specs/blob/8d7e68f4b7fa58107647164d1749853a6ed00eb3/src/ethereum/forks/amsterdam/stateless_host_exec_witness.py#L175-L176) to make it canonical. cc @gballet --------- Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>
…#35026) - Adds tracing to the `GetBlobsV1/V2/V3` - Adds `blobs.requested` and `blobs.filled` attributes to `GetBlobsV1/V2/V3` spans. - Adds tracing to `BlobTxPool().GetBlobs()`
…#35025) The changes here enable us to fill tests with Amsterdam using geth EVM bin. This will be useful for block builder tests using `testing_buildBlockV1` endpoint and for filling benchmarking compute and stateful tests as Python is too slow for benchmark tests. Tested in [ethereum/execution-specs](https://github.com/ethereum/execution-specs) with: ``` uv run fill --clean --fork=Amsterdam tests/amsterdam/eip7928_block_level_access_lists/test_block_access_lists.py --evm-bin=$GETH_EVM_PATH ```
Return empty raw bytes when the GraphQL `Block.raw` resolver cannot load the block body. This matches the nil handling used by the other block-body-backed resolvers and avoids exposing RLP empty-list bytes as raw block data.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
implements eccpow consensus engine for Worldland Network