7 Commits (de23cf910b814867d5c5d1ad6164835d79069638)

Author	SHA1	Message	Date
rjl493456442	1941c5e6c9	core/rawdb: untie freezer and ancient chain data (#24684) ... Previously freezer has only been used for storing ancient chain data, while obviously it can be used more. This PR unties the chain data and freezer, keep the minimal freezer structure and move all other logic (like incrementally freezing block data) into a separate structure called ChainFreezer. This PR also extends the database interface by adding a new ancient store function AncientDatadir which can return the root directory of ancient store. The ancient root directory can be used when we want to open some other ancient-stores (e.g. reverse diff freezer).	3 years ago
Eng Zer Jun	8d066f1f42	all: use T.TempDir to create temporary test directories (#24633) ... This commit replaces ioutil.TempDir with t.TempDir in tests. The directory created by t.TempDir is automatically removed when the test and all its subtests complete. Prior to this commit, temporary directory created using ioutil.TempDir had to be removed manually by calling os.RemoveAll, which is omitted in some tests. The error handling boilerplate e.g. defer func() { if err := os.RemoveAll(dir); err != nil { t.Fatal(err) } } is also tedious, but t.TempDir handles this for us nicely. Reference: https://pkg.go.dev/testing#T.TempDir Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	3 years ago
Sina Mahmoodi	fd4f60f49b	core/rawdb: simple legacy receipt converter (#24028) ... * cmd,core: add simple legacy receipt converter core/rawdb: use forEach in migrate core/rawdb: batch reads in forEach core/rawdb: make forEach anonymous fn cmd/geth: check for legacy receipts on node startup fix err msg Co-authored-by: rjl493456442 <garyrong0905@gmail.com> fix log Co-authored-by: rjl493456442 <garyrong0905@gmail.com> fix some review comments add warning to cmd drop isLegacy fn from migrateTable params add test for windows rename test replacing in windows case * minor fix * sanity check for tail-deletion * add log before moving files around * speed-up hack for mainnet * fix mainnet check, use networkid instead * check mainnet genesis * review fixes * resume previous migration attempt * core/rawdb: lint fix Co-authored-by: Martin Holst Swende <martin@swende.se>	3 years ago
rjl493456442	538a868384	core/rawdb, cmd, ethdb, eth: implement freezer tail deletion (#23954) ... * core/rawdb, cmd, ethdb, eth: implement freezer tail deletion * core/rawdb: address comments from martin and sina * core/rawdb: fixes cornercase in tail deletion * core/rawdb: separate metadata into a standalone file * core/rawdb: remove unused code * core/rawdb: add random test * core/rawdb: polish code * core/rawdb: fsync meta file before manipulating the index * core/rawdb: fix typo * core/rawdb: address comments	3 years ago
Sina Mahmoodi	4aab440ee2	core/rawdb: enforce readonly in freezer instantiation (#24119) ... * freezer: add readonly flag to table * freezer: enforce readonly in table repair * freezer: enforce readonly in newFreezer * minor fix * minor * core/rawdb: test that writing during readonly fails * rm unused log * check readonly on batch append * minor * Revert "check readonly on batch append" This reverts commit 2ddb5ec4ba. * review fixes * minor test refactor * attempt at fixing windows issue * add comment re windows sync issue * k->kind * open readonly db for genesis check Co-authored-by: Martin Holst Swende <martin@swende.se>	3 years ago
Martin Holst Swende	b7a6409cc1	core/rawdb: better error message in freezer (#23901) ... * core/rawdb: better error message in freezer * Apply suggestions from code review	3 years ago
Martin Holst Swende	794c6133ef	core/rawdb: freezer batch write (#23462) ... This change is a rewrite of the freezer code. When writing ancient chain data to the freezer, the previous version first encoded each individual item to a temporary buffer, then wrote the buffer. For small item sizes (for example, in the block hash freezer table), this strategy causes a lot of system calls for writing tiny chunks of data. It also allocated a lot of temporary []byte buffers. In the new version, we instead encode multiple items into a re-useable batch buffer, which is then written to the file all at once. This avoids performing a system call for every inserted item. To make the internal batching work, the ancient database API had to be changed. While integrating this new API in BlockChain.InsertReceiptChain, additional optimizations were also added there. Co-authored-by: Felix Lange <fjl@twurst.com>	3 years ago