8cbdc8638f
core: define and test chain rewind corner cases ( #21409 )
...
* core: define and test chain reparation cornercases
* core: write up a variety of set-head tests
* core, eth: unify chain rollbacks, handle all the cases
* core: make linter smile
* core: remove commented out legacy code
* core, eth/downloader: fix review comments
* core: revert a removed recovery mechanism
4 years ago
93da0cf8a1
cmd, core, eth, light, trie: dump clean cache periodically ( #20391 )
...
* cmd, core, eth, light, trie: dump clean cache periodically
* eth: update config
* trie: minor fix
* core, trie: address comments
* eth: remove useless
* trie: print clean cache dump start too
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
4 years ago
295693759e
core/vm: less allocations for various call variants ( #21222 )
...
* core/vm/runtime/tests: add more benchmarks
* core/vm: initial work on improving alloc count for calls to precompiles
name old time/op new time/op delta
SimpleLoop/identity-precompile-10M-6 117ms ±75% 43ms ± 1% -63.09% (p=0.008 n=5+5)
SimpleLoop/loop-10M-6 79.6ms ± 4% 70.5ms ± 1% -11.42% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
SimpleLoop/identity-precompile-10M-6 24.4MB ± 0% 4.9MB ± 0% -79.94% (p=0.008 n=5+5)
SimpleLoop/loop-10M-6 13.2kB ± 0% 13.2kB ± 0% ~ (p=0.357 n=5+5)
name old allocs/op new allocs/op delta
SimpleLoop/identity-precompile-10M-6 382k ± 0% 153k ± 0% -59.99% (p=0.000 n=5+4)
SimpleLoop/loop-10M-6 40.0 ± 0% 40.0 ± 0% ~ (all equal)
* core/vm: don't allocate big.int for touch
name old time/op new time/op delta
SimpleLoop/identity-precompile-10M-6 43.3ms ± 1% 42.4ms ± 7% ~ (p=0.151 n=5+5)
SimpleLoop/loop-10M-6 70.5ms ± 1% 76.7ms ± 1% +8.67% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
SimpleLoop/identity-precompile-10M-6 4.90MB ± 0% 2.46MB ± 0% -49.83% (p=0.008 n=5+5)
SimpleLoop/loop-10M-6 13.2kB ± 0% 13.2kB ± 1% ~ (p=0.571 n=5+5)
name old allocs/op new allocs/op delta
SimpleLoop/identity-precompile-10M-6 153k ± 0% 76k ± 0% -49.98% (p=0.029 n=4+4)
SimpleLoop/loop-10M-6 40.0 ± 0% 40.0 ± 0% ~ (all equal)
* core/vm: reduce allocs in staticcall
name old time/op new time/op delta
SimpleLoop/identity-precompile-10M-6 42.4ms ± 7% 37.5ms ± 6% -11.68% (p=0.008 n=5+5)
SimpleLoop/loop-10M-6 76.7ms ± 1% 69.1ms ± 1% -9.82% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
SimpleLoop/identity-precompile-10M-6 2.46MB ± 0% 0.02MB ± 0% -99.35% (p=0.008 n=5+5)
SimpleLoop/loop-10M-6 13.2kB ± 1% 13.2kB ± 0% ~ (p=0.143 n=5+5)
name old allocs/op new allocs/op delta
SimpleLoop/identity-precompile-10M-6 76.4k ± 0% 0.1k ± 0% ~ (p=0.079 n=4+5)
SimpleLoop/loop-10M-6 40.0 ± 0% 40.0 ± 0% ~ (all equal)
* trie: better use of hasher keccakState
* core/state/statedb: reduce allocations in getDeletedStateObject
* core/vm: reduce allocations in all call derivates
* core/vm: reduce allocations in call variants
- Make returnstack `uint32`
- Use a `sync.Pool` of `stack`s
* core/vm: fix tests
* core/vm: goimports
* core/vm: tracer fix + staticcall gas fix
* core/vm: add back snapshot to staticcall
* core/vm: review concerns + make returnstack pooled + enable returndata in traces
* core/vm: fix some test tracer method signatures
* core/vm: run gencodec, minor comment polish
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
4 years ago
240d1851db
trie: quell linter in commiter.go ( #21329 )
4 years ago
6eef141aef
les: historical data garbage collection ( #19570 )
...
This change introduces garbage collection for the light client. Historical
chain data is deleted periodically. If you want to disable the GC, use
the --light.nopruning flag.
4 years ago
ddeea1e0c6
core: types: less allocations when hashing and tx handling ( #21265 )
...
* core, crypto: various allocation savings regarding tx handling
* core: reduce allocs for gas price comparison
This change reduces the allocations needed for comparing different transactions to each other.
A call to `tx.GasPrice()` copies the gas price as it has to be safe against modifications and
also needs to be threadsafe. For comparing and ordering different transactions we don't need
these guarantees
* core: added tx.GasPriceIntCmp for comparison without allocation
adds a method to remove unneeded allocation in comparison to tx.gasPrice
* core/types: pool legacykeccak256 objects in rlpHash
rlpHash is by far the most used function in core that allocates a legacyKeccak256 object on each call.
Since it is so widely used it makes sense to add pooling here so we relieve the GC.
On my machine these changes result in > 100 MILLION less allocations and > 30 GB less allocated memory.
* reverted some changes
* reverted some changes
* trie: use crypto.KeccakState instead of replicating code
Co-authored-by: Martin Holst Swende <martin@swende.se>
4 years ago
b482423e61
trie: reduce allocs in insertPreimage ( #21261 )
4 years ago
389da6aa48
trie: enforce monotonic range in prover and return end marker ( #21130 )
...
* trie: add hasRightElement indicator
* trie: ensure the range is monotonic increasing
* trie: address comment and fix lint
* trie: address comment
* trie: make linter happy
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
5 years ago
070a5e1252
trie: fix for range proof ( #21107 )
...
* trie: fix for range proof
* trie: fix typo
5 years ago
4f2784b38f
all: fix typos in comments ( #21118 )
5 years ago
65ce550b37
trie: extend range proofs with non-existence ( #21000 )
...
* trie: implement range proof with non-existent edge proof
* trie: fix cornercase
* trie: consider empty range
* trie: add singleSide test
* trie: support all-elements range proof
* trie: fix typo
* trie: tiny typos and formulations
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
5 years ago
234cc8e77f
eth/downloader: minor typo fixes in comments ( #21035 )
5 years ago
5cdc2dffda
trie: fix TestBadRangeProof unit test ( #21034 )
5 years ago
44ff3f3dc9
trie: initial implementation for range proof ( #20908 )
...
* trie: initial implementation for range proof
* trie: add benchmark
* trie: fix lint
* trie: fix minor issue
* trie: unset the edge valuenode as well
* trie: unset the edge valuenode as nilValuenode
5 years ago
6402c42b67
all: simplify and fix database iteration with prefix/start ( #20808 )
...
* core/state/snapshot: start fixing disk iterator seek
* ethdb, rawdb, leveldb, memorydb: implement iterators with prefix and start
* les, core/state/snapshot: iterator fixes
* all: remove two iterator methods
* all: rename Iteratee.NewIteratorWith -> NewIterator
* ethdb: fix review concerns
5 years ago
af4080b4b7
trie: fix concurrent usage of secKeyBuf, ref #20920
5 years ago
351a5903b0
core/rawdb, core/state/snapshot: runtime snapshot generation
5 years ago
542df8898e
core: initial version of state snapshots
5 years ago
a1313b5b1e
trie: make hasher parallel when number of changes are large ( #20488 )
...
* trie: make hasher parallel when number of changes are large
* trie: remove unused field dirtyCount
* trie: rename unhashedCount/unhashed
5 years ago
5a9c96454e
trie: separate hashes and committer, collapse on commit
...
* trie: make db insert use size instead of full data
* core/state: minor optimization in state onleaf allocation
* trie: implement dedicated committer and hasher
* trie: use dedicated committer/hasher
* trie: linter nitpicks
* core/state, trie: avoid unnecessary storage trie load+commit
* trie: review feedback, mainly docs + minor changes
* trie: start deprecating old hasher
* trie: fix misspell+lint
* trie: deprecate hasher.go, make proof framework use new hasher
* trie: rename pure_committer/hasher to committer/hasher
* trie, core/state: fix review concerns
* trie: more review concerns
* trie: make commit collapse into hashnode, don't touch dirtyness
* trie: goimports fixes
* trie: remove panics
5 years ago
9b09c0fc83
* trie: utilize callbacks instead of amassing lists in ref/unref ( #20529 )
...
* trie/tests: add benchmarks and update trie tests
* trie: update benchmark tests
* trie: utilize callbacks instead of amassing lists of hashes in database ref/unref
* trie: replace remaining non-callback based accesses
5 years ago
4f7b7f84ae
add node.go unit test file node_test.go ( #20028 )
...
* add node.go unit test file node_test.go
* add node_test.go file license and rollback trie_test.go
* fix unused variable v
* trie: fix license year
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
5 years ago
13ccf6016e
trie: track dirty cache metrics, track clean writes on commit
5 years ago
08611cfd75
trie: remove dead code ( #20405 )
5 years ago
9d67222f4e
trie: replace bigcache with fastcache ( #19971 )
5 years ago
0abcf03fde
trie: remove unused code ( #20366 )
5 years ago
ecdbb402ee
trie: remove node ordering slice in sync batch ( #19929 )
...
When we flush a batch of trie nodes into database during the state
sync, we should guarantee that all children should be flushed before
parent.
Actually the trie nodes commit order is strict by: children -> parent.
But when we flush all ready nodes into db, we don't need the order
anymore since
(1) they are all ready nodes (no more dependency)
(2) underlying database provides write atomicity
5 years ago
df6c08a485
core, trie: decode the value for storage dump ( #19943 )
...
* core, trie: decode the value for storage dump
* core/state: address comment
5 years ago
5183483c53
core/state, p2p/discover, trie, whisper: avoid unnecessary conversions ( #19870 )
...
No need to convert these types.
5 years ago
dcc4adfcd7
cmd/geth: wrong memory size sanitizing on OpenBSD ( #19793 )
5 years ago
536b3b416c
cosensus, core, eth, params, trie: fixes + clique history cap
6 years ago
80469bea0c
all: integrate the freezer with fast sync
...
* all: freezer style syncing
core, eth, les, light: clean up freezer relative APIs
core, eth, les, trie, ethdb, light: clean a bit
core, eth, les, light: add unit tests
core, light: rewrite setHead function
core, eth: fix downloader unit tests
core: add receipt chain insertion test
core: use constant instead of hardcoding table name
core: fix rollback
core: fix setHead
core/rawdb: remove canonical block first and then iterate side chain
core/rawdb, ethdb: add hasAncient interface
eth/downloader: calculate ancient limit via cht first
core, eth, ethdb: lots of fixes
* eth/downloader: print ancient disable log only for fast sync
6 years ago
9effd64290
core, eth, trie: bloom filter for trie node dedup during fast sync ( #19489 )
...
* core, eth, trie: bloom filter for trie node dedup during fast sync
* eth/downloader, trie: address review comments
* core, ethdb, trie: restart fast-sync bloom construction now and again
* eth/downloader: initialize fast sync bloom on startup
* eth: reenable eth/62 until we properly remove it
6 years ago
14868a37fb
trie: clarify why verifyProof doesn't check hashes ( #19530 )
...
* trie: fix merkle proof
* trie: use hasher instead of allocate keccack256 every time
* trie: add comments
6 years ago
4a4abc41d4
trie: approximate the wasted cache metaspace closer
6 years ago
4bf0d11e7c
trie: there's no point in retrieving the metaroot
6 years ago
59e1953246
core, ethdb, trie: mode dirty data to clean cache on flush ( #19307 )
...
This PR is a more advanced form of the dirty-to-clean cacher (#18995 ),
where we reuse previous database write batches as datasets to uncache,
saving a dirty-trie-iteration and a dirty-trie-rlp-reencoding per block.
6 years ago
876f357364
trie: disable fnv64a hashing of hashes for bigcache ( #19314 )
...
* trie: disable fnv64a hashing of hashes for bigcache
* trie/database: add very important period
6 years ago
91eec1251c
cmd, core, eth, trie: get rid of trie cache generations ( #19262 )
...
* cmd, core, eth, trie: get rid of trie cache generations
* core, trie: get rid of remainder of cache gen boilerplate
6 years ago
8111b9dda5
ethdb, trie: tiny API tidy-up from the database rework pr
6 years ago
054412e335
all: clean up and proerly abstract database access
6 years ago
514a9472ad
trie: prefer nil slices over zero-length slices ( #19084 )
6 years ago
4f85c2b88b
trie: fix error in node decoding ( #19111 )
6 years ago
bb7c786b09
trie: add missing unlock call in error case ( #18985 )
6 years ago
33d233d3e1
vendor, crypto, swarm: switch over to upstream sha3 package
6 years ago
2843001ac2
trie: fix overflow in write cache parent tracking ( #18165 )
...
trie/database: fix overflow in parent tracking
6 years ago
d136e985e8
trie: go fmt package
6 years ago
68be45e5f8
trie: return hasher to pool ( #18116 )
...
* trie: return hasher to pool
* trie: minor code formatting fix
6 years ago
434dd5bc00
cmd, core, eth, light, trie: add trie read caching layer
6 years ago
53634f1e04
trie: remove unused originalRoot field ( #17862 )
6 years ago
Péter Szilágyi
gary rong
Martin Holst Swende
Guillaume Ballet
Marius van der Wijden
Martin Michlmayr
me020523
Felix Lange
Christian Muehlhaeuser
Matthew Halpern
HackyMiner
Dave McGregor
Łukasz Kurowski