17 Commits (4f3ba6742fbf1cc585048fcc45072523c6de173d)

Author	SHA1	Message	Date
gary rong	7088f1e814	core, eth: faster snapshot generation (#22504) ... * eth/protocols: persist received state segments * core: initial implementation * core/state/snapshot: add tests * core, eth: updates * eth/protocols/snapshot: count flat state size * core/state: add metrics * core/state/snapshot: skip unnecessary deletion * core/state/snapshot: rename * core/state/snapshot: use the global batch * core/state/snapshot: add logs and fix wiping * core/state/snapshot: fix * core/state/snapshot: save generation progress even if the batch is empty * core/state/snapshot: fixes * core/state/snapshot: fix initial account range length * core/state/snapshot: fix initial account range * eth/protocols/snap: store flat states during the healing * eth/protocols/snap: print logs * core/state/snapshot: refactor (#4) * core/state/snapshot: refactor * core/state/snapshot: tiny fix and polish Co-authored-by: rjl493456442 <garyrong0905@gmail.com> * core, eth: fixes * core, eth: fix healing writer * core, trie, eth: fix paths * eth/protocols/snap: fix encoding * eth, core: add debug log * core/state/generate: release iterator asap (#5) core/state/snapshot: less copy core/state/snapshot: revert split loop core/state/snapshot: handle storage becoming empty, improve test robustness core/state: test modified codehash core/state/snapshot: polish * core/state/snapshot: optimize stats counter * core, eth: add metric * core/state/snapshot: update comments * core/state/snapshot: improve tests * core/state/snapshot: replace secure trie with standard trie * core/state/snapshot: wrap return as the struct * core/state/snapshot: skip wiping correct states * core/state/snapshot: updates * core/state/snapshot: fixes * core/state/snapshot: fix panic due to reference flaw in closure * core/state/snapshot: fix errors in state generation logic + fix log output * core/state/snapshot: remove an error case * core/state/snapshot: fix condition-check for exhausted snap state * core/state/snapshot: use stackTrie for small tries * core/state/snapshot: don't resolve small storage tries in vain * core/state/snapshot: properly clean up storage of deleted accounts * core/state/snapshot: avoid RLP-encoding in some cases + minor nitpicks * core/state/snapshot: fix error (+testcase) * core/state/snapshot: clean up tests a bit * core/state/snapshot: work in progress on better tests * core/state/snapshot: polish code * core/state/snapshot: fix trie iteration abortion trigger * core/state/snapshot: fixes flaws * core/state/snapshot: remove panic * core/state/snapshot: fix abort * core/state/snapshot: more tests (plus failing testcase) * core/state/snapshot: more testcases + fix for failing test * core/state/snapshot: testcase for malformed data * core/state/snapshot: some test nitpicks * core/state/snapshot: improvements to logging * core/state/snapshot: testcase to demo error in abortion * core/state/snapshot: fix abortion * cmd/geth: make verify-state report the root * trie: fix failing test * core/state/snapshot: add timer metrics * core/state/snapshot: fix metrics * core/state/snapshot: udpate tests * eth/protocols/snap: write snapshot account even if code or state is needed * core/state/snapshot: fix diskmore check * core/state/snapshot: review fixes * core/state/snapshot: improve error message * cmd/geth: rename 'error' to 'err' in logs * core/state/snapshot: fix some review concerns * core/state/snapshot, eth/protocols/snap: clear snapshot marker when starting/resuming snap sync * core: add error log * core/state/snapshot: use proper timers for metrics collection * core/state/snapshot: address some review concerns * eth/protocols/snap: improved log message * eth/protocols/snap: fix heal logs to condense infos * core/state/snapshot: wait for generator termination before restarting * core/state/snapshot: revert timers to counters to track total time Co-authored-by: Martin Holst Swende <martin@swende.se> Co-authored-by: Péter Szilágyi <peterke@gmail.com>	4 years ago
gary rong	5e9f5ca5d3	core/state/snapshot: write snapshot generator in batch (#22163) ... * core/state/snapshot: write snapshot generator in batch * core: refactor the tests * core: update tests * core: update tests	4 years ago
Péter Szilágyi	c4deebbf1e	core/state/snapshot: add generation logs to storage too	4 years ago
Melvin Junhee Woo	167ff563d1	core/state/snapshot: gethring -> gathering typo (#22104)	4 years ago
Péter Szilágyi	017831dd5b	core, eth: split eth package, implement snap protocol (#21482) ... This commit splits the eth package, separating the handling of eth and snap protocols. It also includes the capability to run snap sync (https://github.com/ethereum/devp2p/blob/master/caps/snap.md) , but does not enable it by default. Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de> Co-authored-by: Martin Holst Swende <martin@swende.se>	4 years ago
Péter Szilágyi	7b7b327ff2	core/state/snapshot: update generator marker in sync with flushes	4 years ago
gary rong	b63e3c37a6	core: improve snapshot journal recovery (#21594) ... * core/state/snapshot: introduce snapshot journal version * core: update the disk layer in an atomic way * core: persist the disk layer generator periodically * core/state/snapshot: improve logging * core/state/snapshot: forcibly ensure the legacy snapshot is matched * core/state/snapshot: add debug logs * core, tests: fix tests and special recovery case * core: polish * core: add more blockchain tests for snapshot recovery * core/state: fix comment * core: add recovery flag for snapshot * core: add restart after start-after-crash tests * core/rawdb: fix imports * core: fix tests * core: remove log * core/state/snapshot: fix snapshot * core: avoid callbacks in SetHead * core: fix setHead cornercase where the threshold root has state * core: small docs for the test cases Co-authored-by: Péter Szilágyi <peterke@gmail.com>	4 years ago
Péter Szilágyi	a15d71a255	core/state/snapshot: stop generator if it hits missing trie nodes (#21649) ... * core/state/snapshot: exit Geth if generator hits missing trie nodes * core/state/snapshot: error instead of hard die on generator fault * core/state/snapshot: don't enable logging on the tests	4 years ago
Péter Szilágyi	3ee91b9f2e	core/state/snapshot: reduce disk layer depth during generation	4 years ago
gary rong	26d271dfbb	core/state/snapshot: implement storage iterator (#20971) ... * core/state/snapshot: implement storage iterator * core/state/snapshot, tests: implement helper function * core/state/snapshot: fix storage issue If an account is deleted in the tx_1 but recreated in the tx_2, the it can happen that in this diff layer, both destructedSet and storageData records this account. In this case, the storage iterator should be able to iterate the slots belong to new account but disable further iteration in deeper layers(belong to old account) * core/state/snapshot: address peter and martin's comment * core/state: address comments * core/state/snapshot: fix test	5 years ago
Péter Szilágyi	6e05ccd845	core/state/snapshot, tests: sync snap gen + snaps in consensus tests	5 years ago
Péter Szilágyi	06d4470b41	core: fix broken tests due to API changes + linter	5 years ago
Péter Szilágyi	351a5903b0	core/rawdb, core/state/snapshot: runtime snapshot generation	5 years ago
Martin Holst Swende	f300c0df01	core/state/snapshot: replace bigcache with fastcache	5 years ago
Péter Szilágyi	d754091a87	core/state/snapshot: unlink snapshots from blocks, quad->linear cleanup	5 years ago
Martin Holst Swende	e146fbe4e7	core/state: lazy sorting, snapshot invalidation	5 years ago
Péter Szilágyi	542df8898e	core: initial version of state snapshots	5 years ago