@ -5,7 +5,6 @@ description: Overview of Geth's database architecture
Since v1.9.0, Geth has divided its database into two parts. Recent blocks and state data are kept in quick-access storage, but older blocks and receipts ("ancients") are stored in a "freezer" database. The point of this separation is to minimize the dependency on expensive, sensitive SSDs, and instead push the less frequently-accessed data into a database that can be supported by cheaper and more durable drives. Storing less data in the faster LevelDB database also enables faster compactions and improves the database performance by allowing more state trie nodes to be held in active memory for a given cache-size.
Since v1.9.0, Geth has divided its database into two parts. Recent blocks and state data are kept in quick-access storage, but older blocks and receipts ("ancients") are stored in a "freezer" database. The point of this separation is to minimize the dependency on expensive, sensitive SSDs, and instead push the less frequently-accessed data into a database that can be supported by cheaper and more durable drives. Storing less data in the faster LevelDB database also enables faster compactions and improves the database performance by allowing more state trie nodes to be held in active memory for a given cache-size.
# Recent blocks
# Recent blocks
Geth stores recent blocks in a LevelDB database. This is a persistent key-value store that can be queried very quickly. The LevelDB database is supposed to be run on top of a fast SSD hard disk so that the disk IO is not bottlenecked by the underlying hardware. In addition to basic storage, the LevelDB database supports batch writes and iterations over the keyspace in binary-alphabetical order.
Geth stores recent blocks in a LevelDB database. This is a persistent key-value store that can be queried very quickly. The LevelDB database is supposed to be run on top of a fast SSD hard disk so that the disk IO is not bottlenecked by the underlying hardware. In addition to basic storage, the LevelDB database supports batch writes and iterations over the keyspace in binary-alphabetical order.
@ -28,7 +27,6 @@ Geth also tracks several performance metrics for the LevelDB database that can b
| `nonlevel0CompGauge` | Gauge for tracking the number of table compaction in non0 level |
| `nonlevel0CompGauge` | Gauge for tracking the number of table compaction in non0 level |
| `seekCompGauge` | Gauge for tracking the number of table compaction caused by read opt |
| `seekCompGauge` | Gauge for tracking the number of table compaction caused by read opt |
## Freezer/ancients
## Freezer/ancients
Older segments of the chain are moved out of the fast LevelDB database and into a freezer database the is less performant. Nodes rarely need to access these files so IO speed is less important and the bulk of the chain data can be stored on a cheaper HDD. Once blocks pass some threshold age (90,000 blocks behind the head by default) the block and receipt data is flattened and saved as a raw binary blob of data along with an index entry file used for identification.
Older segments of the chain are moved out of the fast LevelDB database and into a freezer database the is less performant. Nodes rarely need to access these files so IO speed is less important and the bulk of the chain data can be stored on a cheaper HDD. Once blocks pass some threshold age (90,000 blocks behind the head by default) the block and receipt data is flattened and saved as a raw binary blob of data along with an index entry file used for identification.
@ -41,10 +39,8 @@ Geth also tracks some basic metrics relating to the ancients database that can b
| `writeMeter` | Meter for measuring the effective amount of data written |
| `writeMeter` | Meter for measuring the effective amount of data written |
| `sizeGauge` | Gauge for tracking the combined size of all freezer tables |
| `sizeGauge` | Gauge for tracking the combined size of all freezer tables |
The ancients data is saved entirely separately from the fast-access recent data, meaning it can be stored in a different location. The default location for the ancient chain segments is inside the `chaindata` directory, which is inside `datadir`, but it can be defined by passing `--datadir.ancient <path>` to Geth on startup. The freezer is designed to have a read operation complexity of O(1), involving only a read for index items (6 bytes) and a read for the data. This design makes the freezer performant enough to run on a slow HDD disk, permitting people to run Ethereum nodes without requiring a huge SSD. The ancient data can also be moved later by manually copying the directory to a new location and then starting Geth passing the new path to `--datadir.ancient`.
The ancients data is saved entirely separately from the fast-access recent data, meaning it can be stored in a different location. The default location for the ancient chain segments is inside the `chaindata` directory, which is inside `datadir`, but it can be defined by passing `--datadir.ancient <path>` to Geth on startup. The freezer is designed to have a read operation complexity of O(1), involving only a read for index items (6 bytes) and a read for the data. This design makes the freezer performant enough to run on a slow HDD disk, permitting people to run Ethereum nodes without requiring a huge SSD. The ancient data can also be moved later by manually copying the directory to a new location and then starting Geth passing the new path to `--datadir.ancient`.
## Using the freezer
## Using the freezer
If Geth cannot find the freezer, either because the database was deleted or because Geth received an incorrect path, then the node becomes unstable. It is explicitly forbidden to start Geth with an invalid path to the freezer. However, if the state database goes missing Geth can rebuild all its indices based upon data from the freezer and then do a snap-sync on top of it to fill in the missing state data.
If Geth cannot find the freezer, either because the database was deleted or because Geth received an incorrect path, then the node becomes unstable. It is explicitly forbidden to start Geth with an invalid path to the freezer. However, if the state database goes missing Geth can rebuild all its indices based upon data from the freezer and then do a snap-sync on top of it to fill in the missing state data.
@ -273,8 +273,6 @@ To start the node, the followijng command can be run:
build/bin/geth
build/bin/geth
```
```
Additionally all the developer tools provided with Geth (`clef`, `devp2p`, `abigen`, `bootnode`, `evm` and `rlpdump`) can be compiled by running `make all`. More information about these tools can be found [here](https://github.com/ethereum/go-ethereum#executables).
Additionally all the developer tools provided with Geth (`clef`, `devp2p`, `abigen`, `bootnode`, `evm` and `rlpdump`) can be compiled by running `make all`. More information about these tools can be found [here](https://github.com/ethereum/go-ethereum#executables).
To build a stable release, e.g. v1.9.21, the command `git checkout v1.9.21` retrieves that specific version. Executing that command before running `make geth` switches Geth to a stable branch.
To build a stable release, e.g. v1.9.21, the command `git checkout v1.9.21` retrieves that specific version. Executing that command before running `make geth` switches Geth to a stable branch.
Configures how often in-memory state tries are persisted to disk. The interval needs to be in a format parsable by a [time.Duration](https://pkg.go.dev/time#ParseDuration). Note that the interval is not wall-clock time. Rather it is accumulated block processing time after which the state should be flushed.
Configures how often in-memory state tries are persisted to disk. The interval needs to be in a format parsable by a [time.Duration](https://pkg.go.dev/time#ParseDuration). Note that the interval is not wall-clock time. Rather it is accumulated block processing time after which the state should be flushed.
@ -360,7 +359,6 @@ For example the value `0s` will essentially turn on archive mode. If set to `1h`
Returns a printed representation of the stacks of all goroutines. Note that the web3 wrapper for this method takes care of the printing and does not return the string.
Returns a printed representation of the stacks of all goroutines. Note that the web3 wrapper for this method takes care of the printing and does not return the string.
Nicolás Quiroz
2 years ago