This PR builds on #29040 and updates it to the new version of the spec.
I filled the EEST tests and they pass.
Link to spec: https://eips.ethereum.org/EIPS/eip-7623
---------
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: lightclient <14004106+lightclient@users.noreply.github.com>
Co-authored-by: lightclient <lightclient@protonmail.com>
The total difficulty is the sum of all block difficulties from genesis
to a certain block. This value was used in PoW for deciding which chain
is heavier, and thus which chain to select. Since PoS has a different
fork selection algorithm, all blocks since the merge have a difficulty
of 0, and all total difficulties are the same for the past 2 years.
Whilst the TDs are mostly useless nowadays, there was never really a
reason to mess around removing them since they are so tiny. This
reasoning changes when we go down the path of pruned chain history. In
order to reconstruct any TD, we **must** retrieve all the headers from
chain head to genesis and then iterate all the difficulties to compute
the TD.
In a world where we completely prune past chain segments (bodies,
receipts, headers), it is not possible to reconstruct the TD at all. In
a world where we still keep chain headers and prune only the rest,
reconstructing it possible as long as we process (or download) the chain
forward from genesis, but trying to snap sync the head first and
backfill later hits the same issue, the TD becomes impossible to
calculate until genesis is backfilled.
All in all, the TD is a messy out-of-state, out-of-consensus computed
field that is overall useless nowadays, but code relying on it forces
the client into certain modes of operation and prevents other modes or
other optimizations. This PR completely nukes out the TD from the node.
It doesn't compute it, it doesn't operate on it, it's as if it didn't
even exist.
Caveats:
- Whenever we have APIs that return TD (devp2p handshake, tracer, etc.)
we return a TD of 0.
- For era files, we recompute the TD during export time (fairly quick)
to retain the format content.
- It is not possible to "verify" the merge point (i.e. with TD gone, TTD
is useless). Since we're not verifying PoW any more, just blindly trust
it, not verifying but blindly trusting the many year old merge point
seems just the same trust model.
- Our tests still need to be able to generate pre and post merge blocks,
so they need a new way to split the merge without TTD. The PR introduces
a settable ttdBlock field on the consensus object which is used by tests
as the block where originally the TTD happened. This is not needed for
live nodes, we never want to generate old blocks.
- One merge transition consensus test was disabled. With a
non-operational TD, testing how the client reacts to TTD is useless, it
cannot react.
Questions:
- Should we also drop total terminal difficulty from the genesis json?
It's a number we cannot react on any more, so maybe it would be cleaner
to get rid of even more concepts.
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This implements a basic mechanism to query the node's external IP using
a STUN server. There is a built-in list of public STUN servers for convenience.
The new detection mechanism must be selected explicitly using `--nat=stun`
and is not enabled by default in Geth.
Fixes#30881
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
Refactoring of the `evm` command moved where some commands were valid.
One command, `--bench`, used to work in `evm statetest`. The pluming is
still in place. This PR puts the `--bench` flag in the place the trace
flags were moved, and adds tests to validate the bench flag operates in
`run` and `statetest`
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
As part of trying to make the inputs and outputs of the evm subcommands
more streamlined and aligned, this PR modifies how `evm t8n` manages
output-files.
Previously, we do a kind of wonky thing where between each transaction,
we invoke a `getTracer` closure. In that closure, we create a new
output-file, a tracer, and then make the tracer stream output to the
file. We also fiddle a bit to ensure that the file becomes properly
closed.
It is a kind of hacky solution we have in place. This PR changes it, so
that from the execution-pipeline point of view, we have just a regular
tracer. No fiddling with re-setting it or closing files.
That particular tracer, however, is a bit special: it takes care of
creating new files per transaction (in the tx-start-hook) and closing
(on tx-end-hook). Also instantiating the right type of underlying
tracer, which can be a json-logger or a custom tracer.
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This PR addresses issue #30768 , which highlights that running
cmd/abigen/abigen --pkg my_package example.json (erroneously omitting
the --abi flag) generates an empty binding, when it should fail
explicitly.
---------
Co-authored-by: jwasinger <j-wasinger@hotmail.com>
This pull request delivers the new version of the state history, where
the raw storage key is used instead of the hash.
Before the cancun fork, it's supported by protocol to destruct a
specific account and therefore, all the storage slot owned by it should
be wiped in the same transition.
Technically, storage wiping should be performed through storage
iteration, and only the storage key hash will be available for traversal
if the state snapshot is not available. Therefore, the storage key hash
is chosen as the identifier in the old version state history.
Fortunately, account self-destruction has been deprecated by the
protocol since the Cancun fork, and there are no empty accounts eligible
for deletion under EIP-158. Therefore, we can conclude that no storage
wiping should occur after the Cancun fork. In this case, it makes no
sense to keep using hash.
Besides, another big reason for making this change is the current format
state history is unusable if verkle is activated. Verkle tree has a
different key derivation scheme (merkle uses keccak256), the preimage of
key hash must be provided in order to make verkle rollback functional.
This pull request is a prerequisite for landing verkle.
Additionally, the raw storage key is more human-friendly for those who
want to manually check the history, even though Solidity already
performs some hashing to derive the storage location.
---
This pull request doesn't bump the database version, as I believe the
database should still be compatible if users degrade from the new geth
version to old one, the only side effect is the persistent new version
state history will be unusable.
---------
Co-authored-by: Zsolt Felfoldi <zsfelfoldi@gmail.com>
This pull request refactors the genesis setup function, the major
changes are highlighted here:
**(a) Triedb is opened in verkle mode if `EnableVerkleAtGenesis` is
configured in chainConfig or the database has been initialized previously with
`EnableVerkleAtGenesis` configured**.
A new config field `EnableVerkleAtGenesis` has been added in the
chainConfig. This field must be configured with True if Geth wants to initialize
the genesis in Verkle mode.
In the verkle devnet-7, the verkle transition is activated at genesis.
Therefore, the verkle rules should be used since the genesis. In production
networks (mainnet and public testnets), verkle activation always occurs after
the genesis block. Therefore, this flag is only made for devnet and should be
deprecated later. Besides, verkle transition at non-genesis block hasn't been
implemented yet, it should be done in the following PRs.
**(b) The genesis initialization condition has been simplified**
There is a special mode supported by the Geth is that: Geth can be
initialized with an existing chain segment, which can fasten the node sync
process by retaining the chain freezer folder.
Originally, if the triedb is regarded as uninitialized and the genesis block can
be found in the chain freezer, the genesis block along with genesis state will be
committed. This condition has been simplified to checking the presence of chain
config in key-value store. The existence of chain config can represent the genesis
has been committed.
This fixes a regression introduced recently. Without this fix, it's not
possible to use statetests without `.json` suffix. This is problematic for
goevmlab `minimizer`, which appends the suffix `.min` during processing.
This PR implements EIP-7702: "Set EOA account code".
Specification: https://eips.ethereum.org/EIPS/eip-7702
> Add a new transaction type that adds a list of `[chain_id, address,
nonce, y_parity, r, s]` authorization tuples. For each tuple, write a
delegation designator `(0xef0100 ++ address)` to the signing account’s
code. All code reading operations must load the code pointed to by the
designator.
---------
Co-authored-by: Mario Vega <marioevz@gmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
This PR modifies how the metrics library handles `Enabled`: previously,
the package `init` decided whether to serve real metrics or just
dummy-types.
This has several drawbacks:
- During pkg init, we need to determine whether metrics are enabled or
not. So we first hacked in a check if certain geth-specific
commandline-flags were enabled. Then we added a similar check for
geth-env-vars. Then we almost added a very elaborate check for
toml-config-file, plus toml parsing.
- Using "real" types and dummy types interchangeably means that
everything is hidden behind interfaces. This has a performance penalty,
and also it just adds a lot of code.
This PR removes the interface stuff, uses concrete types, and allows for
the setting of Enabled to happen later. It is still assumed that
`metrics.Enable()` is invoked early on.
The somewhat 'heavy' operations, such as ticking meters and exp-decay,
now checks the enable-flag to prevent resource leak.
The change may be large, but it's mostly pretty trivial, and from the
last time I gutted the metrics, I ensured that we have fairly good test
coverage.
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
This PR attempts to clean up some ambiguities and quirks from recent
changes to evm flag handling.
This PR mainly focuses on `evm run` subcommand, to use the same flags
for configuring tracing/output options as `statetest/blocktest` does.
Additionally, it adds quite a lot of tests for expected outputs of the
various subcommands, to avoid accidental changes.
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
This PR refactors the structlog a bit, making it so that it can be used
in a streaming mode.
-------------
OBS: this PR makes a change in the input `config` config, the third
input-parem field to `debug.traceCall`. Previously, seteting it to e.g.
` {"enableMemory": true, "limit": 1024}` would mean that the response
was limited to `1024` items. Since an 'item' may include both memory and
storage, the actual size of the response was undertermined.
After this change, the response will be limited to `1024` __`bytes`__
(or thereabouts).
-----------
The commandline usage of structlog now uses the streaming mode, leaving
the non-streaming mode of operation for the eth_Call.
There are two benefits of streaming mode
1. Not have to maintain a long list of operations,
2. Not have to duplicate / n-plicate data, e.g. memory / stack /
returndata so that each entry has their own private slice.
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Lots of packages depend on eth/downloader just for the SyncMode type.
Since we have a dedicated package for eth protocol configuration, it
makes more sense to define SyncMode there, turning eth/downloader into
more of a leaf package.
* unify `staterunner` and `blockrunner` CLI flags, especially around
tracing
* added support for struct logger or json logging (although having issue
#30658)
* new --cross-check flag to validate the stateless witness collection
/ execution matches stateful
* adds support for tracing the stateless execution when a tracer is set
(to more easily debug differences)
* --human for more readable test summary
* directory or file input, so if you pass tests/spec-tests/fixtures/blockchain_tests it will execute all
blockchain tests
This change relocates the EVM tx context switching to the ApplyMessage function.
With this change, we can remove a lot of EVM.SetTxContext calls before
message execution.
### Tracing API changes
- This PR replaces the `GasPrice` field of the `VMContext` struct with
`BaseFee`. Users may instead take the effective gas price from
`tx.EffectiveGasTipValue(env.BaseFee)`.
---------
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
Since we don't really support custom networks anymore, we don't need the
bootnode utility. In case a discovery-only node is wanted, it can still be run using cmd/devp2p.
Reusing state between benchmark iterations resulted in inconsistent
results across runs, which surfaced in https://github.com/ethereum/go-ethereum/issues/30778 .
If these errors are triggered again, they will now trigger panic.
---------
Co-authored-by: Martin HS <martin@swende.se>
This pull request refactors the EVM constructor by removing the
TxContext parameter.
The EVM object is frequently overused. Ideally, only a single EVM
instance should be created and reused throughout the entire state
transition of a block, with the transaction context switched as needed
by calling evm.SetTxContext.
Unfortunately, in some parts of the code, the EVM object is repeatedly
created, resulting in unnecessary complexity. This pull request is the
first step towards gradually improving and simplifying this setup.
---------
Co-authored-by: Martin Holst Swende <martin@swende.se>
This is one further step towards removing account management from
`geth`. This PR deprecates the flag `unlock`, and makes the flag moot:
unlock via geth is no longer possible.
This PR moves chain config related code (config file processing, fork
logic, network defaults) from `beacon/types` and `beacon/blsync` into
`beacon/params` while the command line flag logic of the chain config is
moved into `cmd/utils`, thereby removing the cli dependencies from
package `beacon` and its sub-packages.
When `evm statetest --bench` is specified, benchmark the execution
similarly to `evm run`.
Also adds the ability to filter tests by name, index and fork.
---------
Co-authored-by: Martin Holst Swende <martin@swende.se>
This PR is a first step towards removing account management from geth,
and contains a lot of the user-facing changes.
With this PR, the `personal` namespace disappears. **Note**: `personal`
namespace has been deprecated for quite some time (since
https://github.com/ethereum/go-ethereum/pull/26390 1 year and 8 months
ago), and users who have wanted to use it has been forced to used the
flag `--rpc.enabledeprecatedpersonal`. So I think it's fairly
non-controversial to drop it at this point.
Specifically, this means:
- Account/wallet listing
-`personal.getListAccounts`
-`personal.listAccounts`
-`personal.getListWallets`
-`personal.listWallets`
- Lock/unlock
-`personal.lockAccount`
-`personal.openWallet`
-`personal.unlockAccount`
- Sign ops
-`personal.sign`
-`personal.sendTransaction`
-`personal.signTransaction`
- Imports / inits
-`personal.deriveAccount`
-`personal.importRawKey`
-`personal.initializeWallet`
-`personal.newAccount`
-`personal.unpair`
- Other:
-`personal.ecRecover`
The underlying keystores and account managent code is still in place,
which means that `geth --dev` still works as expected, so that e.g. the
example below still works:
```
> eth.sendTransaction({data:"0x6060", value: 1, from:eth.accounts[0]})
```
Also, `ethkey` and `clef` are untouched.
With the removal of `personal`, as far as I know we have no more API
methods which contain credentials, and if we want to implement
logging-capabilities of RPC ingress payload, it would be possible after
this.
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
Currently we have a custom TextMarshalerFlag. It's a nice idea, allowing
anything implementing text marshaller to be used as a flag. That said,
we only ever used it in one place because it's not that obvious how to
use and it needs some boilerplate on the type itself too, apart of the
heavy boilerplate got the custom flag.
All in all there's no *need* to drop this feature just now, but while
porting the cmds over to cli @v3, all other custom flags worker
perfectly, whereas this one started crashing deep inside the cli
package. The flag handling in v3 got rebuild on generics and there are a
number of new methods needed; and my guess is that maybe one of them
doesn't work like this flag currently is designed too.
We could definitely try and redesign this flag for cli v3... but all
that effort and boilerplate just to use it for 1 flag in 1 location,
seems not worth it. So for now I'm suggesting removing it and maybe
reconsider a similar feature in cli v3 with however it will work.
rebased https://github.com/ethereum/go-ethereum/pull/29766 . The
downstream branch appears to have been deleted and I don't have perms to
push to that fork.
`TerminalTotalDifficultyPassed` is removed. `TerminalTotalDifficulty`
must now be non-nil, and it is expected that networks are already
merged: we can only import PoW/Clique chains, not produce blocks on
them.
---------
Co-authored-by: stevemilk <wangpeculiar@gmail.com>
Breaking changes:
- The ChainConfig was exposed to tracers via VMContext passed in
`OnTxStart`. This is unnecessary specially looking through the lens of
live tracers as chain config remains the same throughout the lifetime of
the program. It was there so that native API-invoked tracers could
access it. So instead we moved it to the constructor of API tracers.
Non-breaking:
- Change the default config of the tracers to be `{}` instead of nil.
This way an extra nil check can be avoided.
Refactoring:
- Rename `supply` struct to `supplyTracer`.
- Un-export some hook definitions.
Changelog: https://golangci-lint.run/product/changelog/#1610
Removes `exportloopref` (no longer needed), replaces it with
`copyloopvar` which is basically the opposite.
Also adds:
- `durationcheck`
- `gocheckcompilerdirectives`
- `reassign`
- `mirror`
- `tenv`
---------
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
This fixes a few issues missed in #29052:
* `requests` must be hex encoded, so added a helper to marshal.
* The statedb was committed too early and so the result of the system
calls was lost.
* For devnet-4 we need to pull off the type byte prefix from the request
data.
This is a redo of #29052 based on newer specs. Here we implement EIPs
scheduled for the Prague fork:
- EIP-7002: Execution layer triggerable withdrawals
- EIP-7251: Increase the MAX_EFFECTIVE_BALANCE
Co-authored-by: lightclient <lightclient@protonmail.com>
This implements recent changes to EIP-7685, EIP-6110, and
execution-apis.
---------
Co-authored-by: lightclient <lightclient@protonmail.com>
Co-authored-by: Shude Li <islishude@gmail.com>
The bulk of this PR is authored by @lightclient , in the original
EOF-work. More recently, the code has been picked up and reworked for the new EOF
specification, by @MariusVanDerWijden , in https://github.com/ethereum/go-ethereum/pull/29518, and also @shemnon has contributed with fixes.
This PR is an attempt to start eating the elephant one small bite at a
time, by selecting only the eof-validation as a standalone piece which
can be merged without interfering too much in the core stuff.
In this PR:
- [x] Validation of eof containers, lifted from #29518, along with
test-vectors from consensus-tests and fuzzing, to ensure that the move
did not lose any functionality.
- [x] Definition of eof opcodes, which is a prerequisite for validation
- [x] Addition of `undefined` to a jumptable entry item. I'm not
super-happy with this, but for the moment it seems the least invasive
way to do it. A better way might be to go back and allowing nil-items or
nil execute-functions to denote "undefined".
- [x] benchmarks of eof validation speed
---------
Co-authored-by: lightclient <lightclient@protonmail.com>
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Danno Ferrin <danno.ferrin@shemnon.com>
Sina M
Ryan Tinianov
Péter Szilágyi
zhen peng
Danno Ferrin
Martin HS
Cedrick
rjl493456442
georgehao
dashangcun
Savely
gitglorythegreat
Felix Lange
lightclient
Marius van der Wijden
jwasinger
Felföldi Zsolt
Ng Wei Han
zhiqiangxu