core/rawdb: fix high memory usage in freezer (#21243)

The ancients variable in the freezer is a list of hashes, which
identifies all of the hashes to be frozen. The slice is being allocated
with a capacity of `limit`, which is the number of the last block
this batch will attempt to add to the freezer. That means we are
allocating memory for all of the blocks in the freezer, not just
the ones to be added.

If instead we allocate `limit - f.frozen`, we will only allocate
enough space for the blocks we're about to add to the freezer. On
mainnet this reduces usage by about 320 MB.
pull/21251/head
AusIV 4 years ago committed by GitHub
parent 5435e0d1a1
commit 3ebfeb09fe
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
  1. 2
      core/rawdb/freezer.go

@ -311,7 +311,7 @@ func (f *freezer) freeze(db ethdb.KeyValueStore) {
var (
start = time.Now()
first = f.frozen
ancients = make([]common.Hash, 0, limit)
ancients = make([]common.Hash, 0, limit-f.frozen)
)
for f.frozen < limit {
// Retrieves all the components of the canonical block

Loading…
Cancel
Save