60e30a940b
core/rawdb: refactor db inspector for extending multiple ancient store ( #25896 )
...
This PR ports a few changes from PBSS:
- Fix the snapshot generator waiter in case the generation is not even initialized
- Refactor db inspector for ancient store
2 years ago
5a02b2d6d0
all: fix spelling mistakes ( #25961 )
2 years ago
bff84a99fe
cmd, core, eth, les, light: track deleted nodes ( #25757 )
...
* cmd, core, eth, les, light: track deleted nodes
* trie: add docs
* trie: address comments
* cmd, core, eth, les, light, trie: trie id
* trie: add tests
* trie, core: updates
* trie: fix imports
* trie: add utility print-method for nodeset
* trie: import err
* trie: fix go vet warnings
Co-authored-by: Martin Holst Swende <martin@swende.se>
2 years ago
b628d72766
build: upgrade to go 1.19 ( #25726 )
...
This changes the CI / release builds to use the latest Go version. It also
upgrades golangci-lint to a newer version compatible with Go 1.19.
In Go 1.19, godoc has gained official support for links and lists. The
syntax for code blocks in doc comments has changed and now requires a
leading tab character. gofmt adapts comments to the new syntax
automatically, so there are a lot of comment re-formatting changes in this
PR. We need to apply the new format in order to pass the CI lint stage with
Go 1.19.
With the linter upgrade, I have decided to disable 'gosec' - it produces
too many false-positive warnings. The 'deadcode' and 'varcheck' linters
have also been removed because golangci-lint warns about them being
unmaintained. 'unused' provides similar coverage and we already have it
enabled, so we don't lose much with this change.
2 years ago
5ddedd2f83
core, light, trie: remove DiskDB function from trie database ( #25690 )
2 years ago
2c5648d891
all: fix some typos ( #25551 )
...
* Fix some typos
* Fix some mistakes
* Revert 4byte.json
* Fix an incorrect fix
* Change files to fails
2 years ago
6da5c1644d
core/state, trie, light: add a TryDeleteAccount method ( #25531 )
...
* core/state, trie, light: Add a DeleteAccount method
* review feedback
* Update database.go
* pr triage feedback
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2 years ago
f67e54c92f
core: use TryGetAccount to read what TryUpdateAccount has written ( #25458 )
...
* core: use TryGetAccount to read where TryUpdateAccount has been used to write
* Gary's review feedback
* implement Gary's suggestion
* fix bug + rename NewSecure into NewStateTrie
* trie: add backwards-compatibility aliases for SecureTrie
* Update database.go
* make the linter happy
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2 years ago
8b53b92eb4
core, trie: rework trie committer ( #25320 )
...
* all: rework trie and trie committer
* all: get rid of internal cache in trie
* all: fixes
* trie: polish
* core, trie: address comments
* trie: fix imports
* core/state: address comments
* core/state/snapshot: polish
* trie: remove unused code
* trie: update tests
* trie: don't set db as nil
* trie: address comments
* trie: unskip test
2 years ago
1657e43931
core, les, eth: port snap sync changes ( #24898 )
...
core, eth, les, trie: rework snap sync
2 years ago
22defa5af7
all: introduce trie owner notion ( #24750 )
...
* cmd, core/state, light, trie, eth: add trie owner notion
* all: refactor
* tests: fix goimports
* core/state/snapshot: fix ineffasigns
Co-authored-by: Martin Holst Swende <martin@swende.se>
2 years ago
48605b5f61
core/statedb: always clear out access list when setting a new one
3 years ago
51e7968b8b
core/state: fix read-meters + simplify code ( #24304 )
3 years ago
443afc975c
core/state: move state account to core/types + abstracted "write account to trie" ( #23567 )
...
* core/state: abstracted "write account to trie" method
* fix appveyor build
* Apply suggestions from code review
Co-authored-by: Martin Holst Swende <martin@swende.se>
* review feedback
* core/state/accounts: move Account to core/types
* core/types: rename Account -> StateAccount
* core/state: restore EncodeRLP for stateObject
* core/types: add the missing file
* more review feedback
* more review feedback
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
3 years ago
a5a5237178
core, light, tests, trie: add state metrics ( #23433 )
3 years ago
82c5085399
cre/state: fix outdated statedb Prepare comment ( #23320 )
3 years ago
686b2884ee
all: removed blockhash from statedb ( #23126 )
...
This PR removes the blockhash from the statedb
3 years ago
bca8c03e57
core/state: remove unused methods ReturnGas, GetStorageProofByHash ( #23092 )
...
Co-authored-by: lidongwei <lidongwei@huobi.com>
3 years ago
cc606be74c
all: define London+baikal, undefine yolov3, add london override flag ( #22822 )
...
* all: define London+baikal, undefine yolov3, add london override flag
* cmd, core, params: add baikal genesis definition
4 years ago
7088f1e814
core, eth: faster snapshot generation ( #22504 )
...
* eth/protocols: persist received state segments
* core: initial implementation
* core/state/snapshot: add tests
* core, eth: updates
* eth/protocols/snapshot: count flat state size
* core/state: add metrics
* core/state/snapshot: skip unnecessary deletion
* core/state/snapshot: rename
* core/state/snapshot: use the global batch
* core/state/snapshot: add logs and fix wiping
* core/state/snapshot: fix
* core/state/snapshot: save generation progress even if the batch is empty
* core/state/snapshot: fixes
* core/state/snapshot: fix initial account range length
* core/state/snapshot: fix initial account range
* eth/protocols/snap: store flat states during the healing
* eth/protocols/snap: print logs
* core/state/snapshot: refactor (#4 )
* core/state/snapshot: refactor
* core/state/snapshot: tiny fix and polish
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
* core, eth: fixes
* core, eth: fix healing writer
* core, trie, eth: fix paths
* eth/protocols/snap: fix encoding
* eth, core: add debug log
* core/state/generate: release iterator asap (#5 )
core/state/snapshot: less copy
core/state/snapshot: revert split loop
core/state/snapshot: handle storage becoming empty, improve test robustness
core/state: test modified codehash
core/state/snapshot: polish
* core/state/snapshot: optimize stats counter
* core, eth: add metric
* core/state/snapshot: update comments
* core/state/snapshot: improve tests
* core/state/snapshot: replace secure trie with standard trie
* core/state/snapshot: wrap return as the struct
* core/state/snapshot: skip wiping correct states
* core/state/snapshot: updates
* core/state/snapshot: fixes
* core/state/snapshot: fix panic due to reference flaw in closure
* core/state/snapshot: fix errors in state generation logic + fix log output
* core/state/snapshot: remove an error case
* core/state/snapshot: fix condition-check for exhausted snap state
* core/state/snapshot: use stackTrie for small tries
* core/state/snapshot: don't resolve small storage tries in vain
* core/state/snapshot: properly clean up storage of deleted accounts
* core/state/snapshot: avoid RLP-encoding in some cases + minor nitpicks
* core/state/snapshot: fix error (+testcase)
* core/state/snapshot: clean up tests a bit
* core/state/snapshot: work in progress on better tests
* core/state/snapshot: polish code
* core/state/snapshot: fix trie iteration abortion trigger
* core/state/snapshot: fixes flaws
* core/state/snapshot: remove panic
* core/state/snapshot: fix abort
* core/state/snapshot: more tests (plus failing testcase)
* core/state/snapshot: more testcases + fix for failing test
* core/state/snapshot: testcase for malformed data
* core/state/snapshot: some test nitpicks
* core/state/snapshot: improvements to logging
* core/state/snapshot: testcase to demo error in abortion
* core/state/snapshot: fix abortion
* cmd/geth: make verify-state report the root
* trie: fix failing test
* core/state/snapshot: add timer metrics
* core/state/snapshot: fix metrics
* core/state/snapshot: udpate tests
* eth/protocols/snap: write snapshot account even if code or state is needed
* core/state/snapshot: fix diskmore check
* core/state/snapshot: review fixes
* core/state/snapshot: improve error message
* cmd/geth: rename 'error' to 'err' in logs
* core/state/snapshot: fix some review concerns
* core/state/snapshot, eth/protocols/snap: clear snapshot marker when starting/resuming snap sync
* core: add error log
* core/state/snapshot: use proper timers for metrics collection
* core/state/snapshot: address some review concerns
* eth/protocols/snap: improved log message
* eth/protocols/snap: fix heal logs to condense infos
* core/state/snapshot: wait for generator termination before restarting
* core/state/snapshot: revert timers to counters to track total time
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
4 years ago
bbfb1e4008
all: add support for EIP-2718, EIP-2930 transactions ( #21502 )
...
This adds support for EIP-2718 typed transactions as well as EIP-2930
access list transactions (tx type 1). These EIPs are scheduled for the
Berlin fork.
There very few changes to existing APIs in core/types, and several new APIs
to deal with access list transactions. In particular, there are two new
constructor functions for transactions: types.NewTx and types.SignNewTx.
Since the canonical encoding of typed transactions is not RLP-compatible,
Transaction now has new methods for encoding and decoding: MarshalBinary
and UnmarshalBinary.
The existing EIP-155 signer does not support the new transaction types.
All code dealing with transaction signatures should be updated to use the
newer EIP-2930 signer. To make this easier for future updates, we have
added new constructor functions for types.Signer: types.LatestSigner and
types.LatestSignerForChainID.
This change also adds support for the YoloV3 testnet.
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Ryan Schneider <ryanleeschneider@gmail.com>
4 years ago
52e5c38aa5
core/state: copy the snap when copying the state ( #22340 )
...
* core/state: copy the snap when copying the state
* core/state: deep-copy snap stuff during state Copy
4 years ago
42f9f1f073
core/state: convert prefetcher to concurrent per-trie loader
4 years ago
1e1865b73f
core: implement background trie prefetcher
...
Squashed from the following commits:
core/state: lazily init snapshot storage map
core/state: fix flawed meter on storage reads
core/state: make statedb/stateobjects reuse a hasher
core/blockchain, core/state: implement new trie prefetcher
core: make trie prefetcher deliver tries to statedb
core/state: refactor trie_prefetcher, export storage tries
blockchain: re-enable the next-block-prefetcher
state: remove panics in trie prefetcher
core/state/trie_prefetcher: address some review concerns
sq
4 years ago
017831dd5b
core, eth: split eth package, implement snap protocol ( #21482 )
...
This commit splits the eth package, separating the handling of eth and snap protocols. It also includes the capability to run snap sync (https://github.com/ethereum/devp2p/blob/master/caps/snap.md ) , but does not enable it by default.
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Martin Holst Swende <martin@swende.se>
4 years ago
810f9e057d
all: remove redundant conversions and import names ( #21903 )
4 years ago
18145adf08
core/state: maintain one more diff layer ( #21730 )
...
* core/state: maintain one more diff layer
* core/state: address comment
4 years ago
6487c002f6
all: implement EIP-2929 (gas cost increases for state access opcodes) + yolo-v2 ( #21509 )
...
* core/vm, core/state: implement EIP-2929 + YOLOv2
* core/state, core/vm: fix some review concerns
* core/state, core/vm: address review concerns
* core/vm: address review concerns
* core/vm: better documentation
* core/vm: unify sload cost as fully dynamic
* core/vm: fix typo
* core/vm/runtime: fix compilation flaw
* core/vm/runtime: fix renaming-err leftovers
* core/vm: renaming
* params/config: use correct yolov2 chainid for config
* core, params: use a proper new genesis for yolov2
* core/state/tests: golinter nitpicks
4 years ago
d8da0b3d81
core/state, eth, trie: stabilize memory use, fix memory leak
4 years ago
87c0ba9213
core, eth, les, trie: add a prefix to contract code ( #21080 )
4 years ago
2ff464b29d
core/state: fixed some comments ( #21450 )
4 years ago
295693759e
core/vm: less allocations for various call variants ( #21222 )
...
* core/vm/runtime/tests: add more benchmarks
* core/vm: initial work on improving alloc count for calls to precompiles
name old time/op new time/op delta
SimpleLoop/identity-precompile-10M-6 117ms ±75% 43ms ± 1% -63.09% (p=0.008 n=5+5)
SimpleLoop/loop-10M-6 79.6ms ± 4% 70.5ms ± 1% -11.42% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
SimpleLoop/identity-precompile-10M-6 24.4MB ± 0% 4.9MB ± 0% -79.94% (p=0.008 n=5+5)
SimpleLoop/loop-10M-6 13.2kB ± 0% 13.2kB ± 0% ~ (p=0.357 n=5+5)
name old allocs/op new allocs/op delta
SimpleLoop/identity-precompile-10M-6 382k ± 0% 153k ± 0% -59.99% (p=0.000 n=5+4)
SimpleLoop/loop-10M-6 40.0 ± 0% 40.0 ± 0% ~ (all equal)
* core/vm: don't allocate big.int for touch
name old time/op new time/op delta
SimpleLoop/identity-precompile-10M-6 43.3ms ± 1% 42.4ms ± 7% ~ (p=0.151 n=5+5)
SimpleLoop/loop-10M-6 70.5ms ± 1% 76.7ms ± 1% +8.67% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
SimpleLoop/identity-precompile-10M-6 4.90MB ± 0% 2.46MB ± 0% -49.83% (p=0.008 n=5+5)
SimpleLoop/loop-10M-6 13.2kB ± 0% 13.2kB ± 1% ~ (p=0.571 n=5+5)
name old allocs/op new allocs/op delta
SimpleLoop/identity-precompile-10M-6 153k ± 0% 76k ± 0% -49.98% (p=0.029 n=4+4)
SimpleLoop/loop-10M-6 40.0 ± 0% 40.0 ± 0% ~ (all equal)
* core/vm: reduce allocs in staticcall
name old time/op new time/op delta
SimpleLoop/identity-precompile-10M-6 42.4ms ± 7% 37.5ms ± 6% -11.68% (p=0.008 n=5+5)
SimpleLoop/loop-10M-6 76.7ms ± 1% 69.1ms ± 1% -9.82% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
SimpleLoop/identity-precompile-10M-6 2.46MB ± 0% 0.02MB ± 0% -99.35% (p=0.008 n=5+5)
SimpleLoop/loop-10M-6 13.2kB ± 1% 13.2kB ± 0% ~ (p=0.143 n=5+5)
name old allocs/op new allocs/op delta
SimpleLoop/identity-precompile-10M-6 76.4k ± 0% 0.1k ± 0% ~ (p=0.079 n=4+5)
SimpleLoop/loop-10M-6 40.0 ± 0% 40.0 ± 0% ~ (all equal)
* trie: better use of hasher keccakState
* core/state/statedb: reduce allocations in getDeletedStateObject
* core/vm: reduce allocations in all call derivates
* core/vm: reduce allocations in call variants
- Make returnstack `uint32`
- Use a `sync.Pool` of `stack`s
* core/vm: fix tests
* core/vm: goimports
* core/vm: tracer fix + staticcall gas fix
* core/vm: add back snapshot to staticcall
* core/vm: review concerns + make returnstack pooled + enable returndata in traces
* core/vm: fix some test tracer method signatures
* core/vm: run gencodec, minor comment polish
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
4 years ago
80e887d7bf
core/state: avoid escape analysis fault when accessing cached state
4 years ago
0b2edf05bb
core/state: make GetCodeSize mirror GetCode implementation wise
5 years ago
82f9ed49fa
core/state: avoid statedb.dbErr due to emptyCode ( #21051 )
...
* core/state: more verbose stateb errors
* core/state: fix flaw
* core/state: fixed lint
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
5 years ago
b0b65d017f
core/state: abort commit if read errors have occurred ( #21039 )
...
This finally adds the error check that the documentation of StateDB.dbErr
promises to do. dbErr was added in 9e5f03b6c
5 years ago
26d271dfbb
core/state/snapshot: implement storage iterator ( #20971 )
...
* core/state/snapshot: implement storage iterator
* core/state/snapshot, tests: implement helper function
* core/state/snapshot: fix storage issue
If an account is deleted in the tx_1 but recreated in the tx_2,
the it can happen that in this diff layer, both destructedSet
and storageData records this account. In this case, the storage
iterator should be able to iterate the slots belong to new account
but disable further iteration in deeper layers(belong to old account)
* core/state/snapshot: address peter and martin's comment
* core/state: address comments
* core/state/snapshot: fix test
5 years ago
074efe6c8d
core: fix two snapshot iterator flaws, decollide snap storage prefix
...
* core/state/snapshot/iterator: fix two disk iterator flaws
* core/rawdb: change SnapshotStoragePrefix to avoid prefix collision with preimagePrefix
5 years ago
328de180a7
core/state: fix resurrection state clearing and access
5 years ago
dcb22a9f99
core/state: fix account root hash update point
5 years ago
a4cf279494
core/state: extend snapshotter to handle account resurrections
5 years ago
92ec07d63b
core/state: fix an account resurrection issue
5 years ago
fd39f722a3
core: journal the snapshot inside leveldb, not a flat file
5 years ago
351a5903b0
core/rawdb, core/state/snapshot: runtime snapshot generation
5 years ago
d754091a87
core/state/snapshot: unlink snapshots from blocks, quad->linear cleanup
5 years ago
e146fbe4e7
core/state: lazy sorting, snapshot invalidation
5 years ago
542df8898e
core: initial version of state snapshots
5 years ago
5a9c96454e
trie: separate hashes and committer, collapse on commit
...
* trie: make db insert use size instead of full data
* core/state: minor optimization in state onleaf allocation
* trie: implement dedicated committer and hasher
* trie: use dedicated committer/hasher
* trie: linter nitpicks
* core/state, trie: avoid unnecessary storage trie load+commit
* trie: review feedback, mainly docs + minor changes
* trie: start deprecating old hasher
* trie: fix misspell+lint
* trie: deprecate hasher.go, make proof framework use new hasher
* trie: rename pure_committer/hasher to committer/hasher
* trie, core/state: fix review concerns
* trie: more review concerns
* trie: make commit collapse into hashnode, don't touch dirtyness
* trie: goimports fixes
* trie: remove panics
5 years ago
fc392395fb
core/state: add more verbosity to panic
5 years ago
dd21f079e8
core/state: fix staticcheck warnings ( #20357 )
...
Also remove dependency on gopkg.in/check.v1 in tests.
5 years ago
rjl493456442
Martin Holst Swende
Felix Lange
Justin Traglia
Guillaume Ballet
Péter Szilágyi
aaronbuchwald
Marius van der Wijden
Li Dongwei
lightclient
Alex Prut
Giuseppe Bertone