@ -5,13 +5,12 @@ description: Overview of Geth's database architecture
Since v1.9.0, Geth has divided its database into two parts. Recent blocks and state data are kept in quick-access storage, but older blocks and receipts ("ancients") are stored in a "freezer" database. The point of this separation is to minimize the dependency on expensive, sensitive SSDs, and instead push the less frequently-accessed data into a database that can be supported by cheaper and more durable drives. Storing less data in the faster LevelDB database also enables faster compactions and improves the database performance by allowing more state trie nodes to be held in active memory for a given cache-size.
# Recent blocks
Geth stores recent blocks in a LevelDB database. This is a persistent key-value store that can be queried very quickly. The LevelDB database is supposed to be run on top of a fast SSD hard disk so that the disk IO is not bottlenecked by the underlying hardware. In addition to basic storage, the LevelDB database supports batch writes and iterations over the keyspace in binary-alphabetical order.
The database is periodically compacted to reduce the operational cost of accessing indivdual items. This is achieved by flattening the underlying data store for a given range of keys. Any deleted or overwritte items in that key range are removed and the surviving data is reorganized for efficiency.
Geth also tracks several performance metrics for the LevelDB database that can be monitored via the metrics subsystem. These are:
Geth also tracks several performance metrics for the LevelDB database that can be monitored via the metrics subsystem. These are:
@ -28,10 +27,9 @@ Geth also tracks several performance metrics for the LevelDB database that can b
| `nonlevel0CompGauge` | Gauge for tracking the number of table compaction in non0 level |
| `seekCompGauge` | Gauge for tracking the number of table compaction caused by read opt |
## Freezer/ancients
Older segments of the chain are moved out of the fast LevelDB database and into a freezer database the is less performant. Nodes rarely need to access these files so IO speed is less important and the bulk of the chain data can be stored on a cheaper HDD. Once blocks pass some threshold age (90,000 blocks behind the head by default) the block and receipt data is flattened and saved as a raw binary blob of data along with an index entry file used for identification.
Older segments of the chain are moved out of the fast LevelDB database and into a freezer database the is less performant. Nodes rarely need to access these files so IO speed is less important and the bulk of the chain data can be stored on a cheaper HDD. Once blocks pass some threshold age (90,000 blocks behind the head by default) the block and receipt data is flattened and saved as a raw binary blob of data along with an index entry file used for identification.
Geth also tracks some basic metrics relating to the ancients database that can be monitored:
@ -41,9 +39,7 @@ Geth also tracks some basic metrics relating to the ancients database that can b
| `writeMeter` | Meter for measuring the effective amount of data written |
| `sizeGauge` | Gauge for tracking the combined size of all freezer tables |
The ancients data is saved entirely separately from the fast-access recent data, meaning it can be stored in a different location. The default location for the ancient chain segments is inside the `chaindata` directory, which is inside `datadir`, but it can be defined by passing `--datadir.ancient <path>` to Geth on startup. The freezer is designed to have a read operation complexity of O(1), involving only a read for index items (6 bytes) and a read for the data. This design makes the freezer performant enough to run on a slow HDD disk, permitting people to run Ethereum nodes without requiring a huge SSD. The ancient data can also be moved later by manually copying the directory to a new location and then starting Geth passing the new path to `--datadir.ancient`.
The ancients data is saved entirely separately from the fast-access recent data, meaning it can be stored in a different location. The default location for the ancient chain segments is inside the `chaindata` directory, which is inside `datadir`, but it can be defined by passing `--datadir.ancient <path>` to Geth on startup. The freezer is designed to have a read operation complexity of O(1), involving only a read for index items (6 bytes) and a read for the data. This design makes the freezer performant enough to run on a slow HDD disk, permitting people to run Ethereum nodes without requiring a huge SSD. The ancient data can also be moved later by manually copying the directory to a new location and then starting Geth passing the new path to `--datadir.ancient`.
## Using the freezer
@ -55,4 +51,4 @@ This can be used to deliberately clean up a node. Passing `--datadir --removedb`
If Geth stops unexpectedly the database can be corrupted. This is known as an "unclean shutdown" and it can lead to a variety of problems for the node when it is restarted. It is always best to shut down Geth gracefully, i.e. using a shutdown command such as `ctrl-c`, `docker stop -t 300 <container ID>` or `systemctl stop` (although please note that `systemctl stop` has a default timeout of 90s - if Geth takes longer than this to gracefully shut down it will quit forcefully. Update the `TimeoutSecs` variable in `systemd.service` to override this value to something larger, at least 300s). This way, Geth knows to write all relevant information into the database to allow the node to restart properly later. This can involve >1GB of information being written to the LevelDB database which can take several minutes.
If an unexpected shutdown does occur, the `removedb` subcommand can be used to delete the state database and resync it from the ancient database. This should get the database back up and running.
If an unexpected shutdown does occur, the `removedb` subcommand can be used to delete the state database and resync it from the ancient database. This should get the database back up and running.
@ -273,8 +273,6 @@ To start the node, the followijng command can be run:
build/bin/geth
```
Additionally all the developer tools provided with Geth (`clef`, `devp2p`, `abigen`, `bootnode`, `evm` and `rlpdump`) can be compiled by running `make all`. More information about these tools can be found [here](https://github.com/ethereum/go-ethereum#executables).
To build a stable release, e.g. v1.9.21, the command `git checkout v1.9.21` retrieves that specific version. Executing that command before running `make geth` switches Geth to a stable branch.
Configures how often in-memory state tries are persisted to disk. The interval needs to be in a format parsable by a [time.Duration](https://pkg.go.dev/time#ParseDuration). Note that the interval is not wall-clock time. Rather it is accumulated block processing time after which the state should be flushed.
For example the value `0s` will essentially turn on archive mode. If set to `1h`, it means that after one hour of effective block processing time, the trie would be flushed. If one block takes 200ms, a flush would occur every `5*3600=18000`blocks. The default interval for mainnet is `1h`.
For example the value `0s` will essentially turn on archive mode. If set to `1h`, it means that after one hour of effective block processing time, the trie would be flushed. If one block takes 200ms, a flush would occur every `5*3600=18000` blocks. The default interval for mainnet is `1h`.
**Note:** this configuration will not be presisted through restarts.
@ -360,7 +359,6 @@ For example the value `0s` will essentially turn on archive mode. If set to `1h`
Returns a printed representation of the stacks of all goroutines. Note that the web3 wrapper for this method takes care of the printing and does not return the string.
@ -5,6 +5,6 @@ description: introduction to the private-network boot-strapping tool, Puppeth
<Note>Puppeth was [removed from Geth](https://github.com/ethereum/go-ethereum/pull/26581) in January 2023.</Note>
Puppeth was a tool for quickly spinning up and managing private development networks. The user was guided through the process by a command line wizard instead of having to configure the network manually. However, this tool has been discontinued and removed from the Geth repository.
Puppeth was a tool for quickly spinning up and managing private development networks. The user was guided through the process by a command line wizard instead of having to configure the network manually. However, this tool has been discontinued and removed from the Geth repository.
Instructions for setting up a private network using Ethash or Clique are available on our [private networks page](/docs/fundamentals/private-network.md).
Instructions for setting up a private network using Ethash or Clique are available on our [private networks page](/docs/fundamentals/private-network.md).
Nicolás Quiroz
2 years ago