In many cases, there is a need to create somewhat nontrivial bytecode. A
recent example is the verkle statetests, where we want a `CREATE2`- op
to create a contract, which can then be invoked, and when invoked does a
selfdestruct-to-self.
It is overkill to go full solidity, but it is also a bit tricky do
assemble this by concatenating bytes. This PR takes an approach that
has been used in in goevmlab for several years.
Using this utility, the case can be expressed as:
```golang
// Some runtime code
runtime := program.New().Ops(vm.ADDRESS, vm.SELFDESTRUCT).Bytecode()
// A constructor returning the runtime code
initcode := program.New().ReturnData(runtime).Bytecode()
// A factory invoking the constructor
outer := program.New().Create2AndCall(initcode, nil).Bytecode()
```
We have a lot of places in the codebase where we concatenate bytes, cast
from `vm.OpCode` . By taking tihs approach instead, thos places can be made a
bit more maintainable/robust.
Looking at the cpu profile of a burntpix benchmark, I noticed that a lot
of time was spent in gas-used, in the interpreter loop. It's an actual
call (not inlined), which explicitly wants to be ignored by tracing
("tracing.GasChangeIgnored"), so it can be safely and simply inlined.
The other change is in `pushX`. These also do a call to
`common.RightPadBytes`. I replaced that by a doing a corresponding `Lsh`
on the `u256` if needed. Note: it's needed only to make the stack output
look right, for fuzzers. It technically doesn't matter what we put
there: if code ends on a pushdata immediate, nothing will consume the
stack element. We could just as well just ignore it, if we didn't care
about fuzzers (which I do).
Seems quite a lot faster on burntpix, according to my runs.
This PR:
```
EVM gas used: 5642735088
execution time: 34.84609475s
allocations: 915683
allocated bytes: 175334088
```
```
EVM gas used: 5642735088
execution time: 36.671958278s
allocations: 915701
allocated bytes: 175340528
```
Master
```
EVM gas used: 5642735088
execution time: 49.349209526s
allocations: 915684
allocated bytes: 175333368
```
```
EVM gas used: 5642735088
execution time: 46.581006598s
allocations: 915681
allocated bytes: 175330728
```
---------
Co-authored-by: Sina M <1591639+s1na@users.noreply.github.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
Breaking changes:
- The ChainConfig was exposed to tracers via VMContext passed in
`OnTxStart`. This is unnecessary specially looking through the lens of
live tracers as chain config remains the same throughout the lifetime of
the program. It was there so that native API-invoked tracers could
access it. So instead we moved it to the constructor of API tracers.
Non-breaking:
- Change the default config of the tracers to be `{}` instead of nil.
This way an extra nil check can be avoided.
Refactoring:
- Rename `supply` struct to `supplyTracer`.
- Un-export some hook definitions.
This pull request introduces a state.Reader interface for state
accessing.
The interface could be implemented in various ways. It can be pure trie
only reader, or the combination of trie and state snapshot. What's more,
this interface allows us to have more flexibility in the future, e.g.
the
archive reader (for accessing archive state).
Additionally, this pull request removes the following metrics
- `chain/snapshot/account/reads`
- `chain/snapshot/storage/reads`
The struct-based tracing added in #29189 seems to have caused an issue
with the benchmark `BenchmarkTracerStepVsCallFrame`. On master we see
the following panic:
```console
BenchmarkTracerStepVsCallFrame
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x40 pc=0x1019782f0]
goroutine 37 [running]:
github.com/ethereum/go-ethereum/eth/tracers/js.(*jsTracer).OnOpcode(0x140004c4000, 0x0, 0x10?, 0x989680, 0x1, {0x101ea2298, 0x1400000e258}, {0x1400000e258?, 0x14000155928?, 0x10173020c?}, ...)
/Users/matt/dev/go-ethereum/eth/tracers/js/goja.go:328 +0x140
github.com/ethereum/go-ethereum/core/vm.(*EVMInterpreter).Run(0x14000307da0, 0x140003cc0d0, {0x0, 0x0, 0x0}, 0x0)
...
FAIL github.com/ethereum/go-ethereum/core/vm/runtime 0.420s
FAIL
```
The issue seems to be that `OnOpcode` expects that `OnTxStart` has
already been called to initialize the `env` value in the tracer. The JS
tracer uses it in `OnOpcode` for the `GetRefund()` method.
This patch resolves the issue by reusing the `Call` method already
defined in `runtime_test.go` which correctly calls `OnTxStart`.
Here we add a Go API for running tracing plugins within the main block import process.
As an advanced user of geth, you can now create a Go file in eth/tracers/live/, and within
that file register your custom tracer implementation. Then recompile geth and select your tracer
on the command line. Hooks defined in the tracer will run whenever a block is processed.
The hook system is defined in package core/tracing. It uses a struct with callbacks, instead of
requiring an interface, for several reasons:
- We plan to keep this API stable long-term. The core/tracing hook API does not depend on
on deep geth internals.
- There are a lot of hooks, and tracers will only need some of them. Using a struct allows you
to implement only the hooks you want to actually use.
All existing tracers in eth/tracers/native have been rewritten to use the new hook system.
This change breaks compatibility with the vm.EVMLogger interface that we used to have.
If you are a user of vm.EVMLogger, please migrate to core/tracing, and sorry for breaking
your stuff. But we just couldn't have both the old and new tracing APIs coexist in the EVM.
---------
Co-authored-by: Matthieu Vachon <matthieu.o.vachon@gmail.com>
Co-authored-by: Delweng <delweng@gmail.com>
Co-authored-by: Martin HS <martin@swende.se>
This change makes use of uin256 to represent balance in state. It touches primarily upon statedb, stateobject and state processing, trying to avoid changes in transaction pools, core types, rpc and tracers.
This PR removes the Debug field from vmconfig, making it so that if a tracer is set, debug=true is implied.
---------
Co-authored-by: 0xTylerHolmes <tyler@ethereum.org>
Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>
This PR changes the API so that uint64 is used for fork timestamps.
It's a good choice because types.Header also uses uint64 for time.
Co-authored-by: Felix Lange <fjl@twurst.com>
This makes non-JS tracers execute all block txs on a single goroutine.
In the previous implementation, we used to prepare every tx pre-state
on one goroutine, and then run the transactions again with tracing enabled.
Native tracers are usually faster, so it is faster overall to use their output as
the pre-state for tracing the next transaction.
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
This PR allows users to pass in a config object directly to the tracers. Previously only the struct logger was configurable.
It also adds an option to the call tracer which if enabled makes it ignore any subcall and collect only information about the top-level call. See #25419 for discussion.
The tracers will silently ignore if they are passed a config they don't care about.
This enables the following linters
- typecheck
- unused
- staticcheck
- bidichk
- durationcheck
- exportloopref
- gosec
WIth a few exceptions.
- We use a deprecated protobuf in trezor. I didn't want to mess with that, since I cannot meaningfully test any changes there.
- The deprecated TypeMux is used in a few places still, so the warning for it is silenced for now.
- Using string type in context.WithValue is apparently wrong, one should use a custom type, to prevent collisions between different places in the hierarchy of callers. That should be fixed at some point, but may require some attention.
- The warnings for using weak random generator are squashed, since we use a lot of random without need for cryptographic guarantees.
This adds a JS tracer runtime environment based on the Goja VM. The new
runtime replaces the duktape runtime, which will be removed soon.
Goja is implemented in Go and is faster for cases where the Go <-> JS
transition overhead dominates overall performance. It is faster because
duktape is written in C, and the transition cost includes the cost of using
cgo. Another reason for using Goja is that go-duktape is not maintained
anymore.
We expect the performace of JS tracing to be at least as good or better with
this change.
* all: mv loggers to eth/tracers
* core/vm: minor
* eth/tracers: tmp comment out testStoreCapture
* eth/tracers: uncomment and fix logger test
* eth/tracers: simplify test
* core/vm: re-add license
* core/vm: minor
* rename LogConfig to Config
This change introduces 2 new optional methods; `enter()` and `exit()` for js tracers, and makes `step()` optiona. The two new methods are invoked when entering and exiting a call frame (but not invoked for the outermost scope, which has it's own methods). Currently these are the data fields passed to each of them:
enter: type (opcode), from, to, input, gas, value
exit: output, gasUsed, error
The PR also comes with a re-write of the callTracer. As a backup we keep the previous tracing script under the name `callTracerLegacy`. Behaviour of both tracers are equivalent for the most part, although there are some small differences (improvements), where the new tracer is more correct / has more information.
Previously, the makeCallVariantGasCallEIP2929 charged the cold account access cost directly, leading to an incorrect gas cost passed to the tracer from the main execution loop.
This change still temporarily charges the cost (to allow for an accurate calculation of the available gas for the call), but then afterwards refunds it and instead returns the correct total gas cost to be then properly charged in the main loop.
Fixes the CaptureStart api to include the EVM, thus being able to set the statedb early on. This pr also exposes the struct we used internally in the interpreter to encapsulate the contract, mem, stack, rstack, so we pass it as a single struct to the tracer, and removes the error returns on the capture methods.
* remove uneeded convertion type
* remove redundant type in composite literal
* omit explicit type where implicit
* remove unused redundant parenthesis
* remove redundant import alias duktape
* core/vm/runtime: add test for blockhash
* core/evm: less iteration in blockhash
* core/vm/runtime: nitpickfix
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
With this commit, core/state's access to the underlying key/value database is
mediated through an interface. Database errors are tracked in StateDB and
returned by CommitTo or the new Error method.
Motivation for this change: We can remove the light client's duplicated copy of
core/state. The light client now supports node iteration, so tracing and storage
enumeration can work with the light client (not implemented in this commit).
* common: remove CurrencyToString
Move denomination values to params instead.
* common: delete dead code
* common: move big integer operations to common/math
This commit consolidates all big integer operations into common/math and
adds tests and documentation.
There should be no change in semantics for BigPow, BigMin, BigMax, S256,
U256, Exp and their behaviour is now locked in by tests.
The BigD, BytesToBig and Bytes2Big functions don't provide additional
value, all uses are replaced by new(big.Int).SetBytes().
BigToBytes is now called PaddedBigBytes, its minimum output size
parameter is now specified as the number of bytes instead of bits. The
single use of this function is in the EVM's MSTORE instruction.
Big and String2Big are replaced by ParseBig, which is slightly stricter.
It previously accepted leading zeros for hexadecimal inputs but treated
decimal inputs as octal if a leading zero digit was present.
ParseUint64 is used in places where String2Big was used to decode a
uint64.
The new functions MustParseBig and MustParseUint64 are now used in many
places where parsing errors were previously ignored.
* common: delete unused big integer variables
* accounts/abi: replace uses of BytesToBig with use of encoding/binary
* common: remove BytesToBig
* common: remove Bytes2Big
* common: remove BigTrue
* cmd/utils: add BigFlag and use it for error-checked integer flags
While here, remove environment variable processing for DirectoryFlag
because we don't use it.
* core: add missing error checks in genesis block parser
* common: remove String2Big
* cmd/evm: use utils.BigFlag
* common/math: check for 256 bit overflow in ParseBig
This is supposed to prevent silent overflow/truncation of values in the
genesis block JSON. Without this check, a genesis block that set a
balance larger than 256 bits would lead to weird behaviour in the VM.
* cmd/utils: fixup import
Reworked the EVM gas instructions to use 64bit integers rather than
arbitrary size big ints. All gas operations, be it additions,
multiplications or divisions, are checked and guarded against 64 bit
integer overflows.
In additon, most of the protocol paramaters in the params package have
been converted to uint64 and are now constants rather than variables.
* common/math: added overflow check ops
* core: vmenv, env renamed to evm
* eth, internal/ethapi, les: unmetered eth_call and cancel methods
* core/vm: implemented big.Int pool for evm instructions
* core/vm: unexported intPool methods & verification methods
* core/vm: added memoryGasCost overflow check and test
Reworked the EVM gas instructions to use 64bit integers rather than
arbitrary size big ints. All gas operations, be it additions,
multiplications or divisions, are checked and guarded against 64 bit
integer overflows.
In additon, most of the protocol paramaters in the params package have
been converted to uint64 and are now constants rather than variables.
* common/math: added overflow check ops
* core: vmenv, env renamed to evm
* eth, internal/ethapi, les: unmetered eth_call and cancel methods
* core/vm: implemented big.Int pool for evm instructions
* core/vm: unexported intPool methods & verification methods
* core/vm: added memoryGasCost overflow check and test
The run loop, which previously contained custom opcode executes have been
removed and has been simplified to a few checks.
Each operation consists of 4 elements: execution function, gas cost function,
stack validation function and memory size function. The execution function
implements the operation's runtime behaviour, the gas cost function implements
the operation gas costs function and greatly depends on the memory and stack,
the stack validation function validates the stack and makes sure that enough
items can be popped off and pushed on and the memory size function calculates
the memory required for the operation and returns it.
This commit also allows the EVM to go unmetered. This is helpful for offline
operations such as contract calls.
Implemented `runtime.Call` which uses - unlike Execute - the given state
for the execution and the address of the contract you wish to execute.
Unlike `Execute`, `Call` requires a config.
The runtime environment can be used for simple basic execution of
contract code without the requirement of setting up a full stack and
operates fully in memory.
Martin HS
Sina M
rjl493456442
lmittmann
lightclient
alex
Darioush Jalali
Justin Traglia
Delweng
Seungbae.yu
Péter Szilágyi
Alex Prut
Greg Colvin
Felix Lange
Jeffrey Wilcke
Jeffrey Wilcke