description: Introduction to writing custom tracers for Geth
---
In addition to the default opcode tracer and the built-in tracers, Geth offers the possibility to write custom code
that hook to events in the EVM to process and return the data in a consumable format. Custom tracers can be
written either in Javascript or Go. JS tracers are good for quick prototyping and experimentation as well as for
less intensive applications. Go tracers are performant but require the tracer to be compiled together with the Geth source code.
* TOC
{:toc}
In addition to the default opcode tracer and the built-in tracers, Geth offers the possibility to write custom code that hook to events in the EVM to process and return the data in a consumable format. Custom tracers can be written either in Javascript or Go. JS tracers are good for quick prototyping and experimentation as well as for less intensive applications. Go tracers are performant but require the tracer to be compiled together with the Geth source code.
## Custom Javascript tracing
Transaction traces include the complete status of the EVM at every point during the transaction execution, which
can be a very large amount of data. Often, users are only interested in a small subset of that data. Javascript trace
filters are available to isolate the useful information. Detailed information about `debug_traceTransaction` and its
component parts is available in the [reference documentation](/docs/rpc/ns-debug#debug_tracetransaction).
Transaction traces include the complete status of the EVM at every point during the transaction execution, which can be a very large amount of data. Often, users are only interested in a small subset of that data. Javascript trace filters are available to isolate the useful information. Detailed information about `debug_traceTransaction` and its component parts is available in the [reference documentation](/content/docs/developers/interacting-with-geth/rpc/ns-debug#debug_tracetransaction).
### A simple filter
Filters are Javascript functions that select information from the trace to persist and discard based on some
conditions. The following Javascript function returns only the sequence of opcodes executed by the transaction as a
comma-separated list. The function could be written directly in the Javascript console, but it is cleaner to
write it in a separate re-usable file and load it into the console.
Filters are Javascript functions that select information from the trace to persist and discard based on some conditions. The following Javascript function returns only the sequence of opcodes executed by the transaction as a comma-separated list. The function could be written directly in the Javascript console, but it is cleaner to write it in a separate re-usable file and load it into the console.
1. Create a file, `filterTrace_1.js`, with this content:
@ -71,12 +59,9 @@ write it in a separate re-usable file and load it into the console.
More information about the `JSON.stringify` function is available [here](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify).
The commands above worked by calling the same `debug.traceTransaction` function that was previously
explained in [basic traces](https://geth.ethereum.org/docs/dapp/tracing), but with a new parameter, `tracer`.
This parameter takes the JavaScript object formated as a string. In the case of the trace above, it is:
The commands above worked by calling the same `debug.traceTransaction` function that was previously explained in [basic traces](https://geth.ethereum.org/docs/dapp/tracing), but with a new parameter, `tracer`. This parameter takes the JavaScript object formated as a string. In the case of the trace above, it is:
```javascript
{
@ -94,14 +79,11 @@ This object has three member functions:
In this case, `retVal` is used to store the list of strings to return in `result`.
The `step` function adds to `retVal` the program counter and the name of the opcode there. Then, in `result`, this
list is returned to be sent to the caller.
The `step` function adds to `retVal` the program counter and the name of the opcode there. Then, in `result`, this list is returned to be sent to the caller.
### Filtering with conditions
For actual filtered tracing we need an `if` statement to only log relevant information. For example, to isolate
the transaction's interaction with storage, the following tracer could be used:
For actual filtered tracing we need an `if` statement to only log relevant information. For example, to isolate the transaction's interaction with storage, the following tracer could be used:
```javascript
tracer = function(tx) {
@ -121,9 +103,7 @@ tracer = function(tx) {
} // tracer = function ...
```
The `step` function here looks at the opcode number of the op, and only pushes an entry if the opcode is
`SLOAD` or `SSTORE` ([here is a list of EVM opcodes and their numbers](https://github.com/wolflo/evm-opcodes)).
We could have used `log.op.toString()` instead, but it is faster to compare numbers rather than strings.
The `step` function here looks at the opcode number of the op, and only pushes an entry if the opcode is `SLOAD` or `SSTORE` ([here is a list of EVM opcodes and their numbers](https://github.com/wolflo/evm-opcodes)). We could have used `log.op.toString()` instead, but it is faster to compare numbers rather than strings.
The output looks similar to this:
@ -143,18 +123,13 @@ The output looks similar to this:
### Stack Information
The trace above reports the program counter (PC) and whether the program read from storage or wrote to it.
That alone isn't particularly useful. To know more, the `log.stack.peek` function can be used to peek
into the stack. `log.stack.peek(0)` is the stack top, `log.stack.peek(1)` the entry below it, etc.
The trace above reports the program counter (PC) and whether the program read from storage or wrote to it. That alone isn't particularly useful. To know more, the `log.stack.peek` function can be used to peek into the stack. `log.stack.peek(0)` is the stack top, `log.stack.peek(1)` the entry below it, etc.
The values returned by `log.stack.peek` are Go `big.Int` objects. By default they are converted to JavaScript
floating point numbers, so you need `toString(16)` to get them as hexadecimals, which is how 256-bit values such as
storage cells and their content are normally represented.
The values returned by `log.stack.peek` are Go `big.Int` objects. By default they are converted to JavaScript floating point numbers, so you need `toString(16)` to get them as hexadecimals, which is how 256-bit values such as storage cells and their content are normally represented.
#### Storage Information
The function below provides a trace of all the storage operations and their parameters. This gives
a more complete picture of the program's interaction with storage.
The function below provides a trace of all the storage operations and their parameters. This gives a more complete picture of the program's interaction with storage.
```javascript
tracer = function(tx) {
@ -195,13 +170,10 @@ The output is similar to:
#### Operation Results
One piece of information missing from the function above is the result on an `SLOAD` operation. The
state we get inside `log` is the state prior to the execution of the opcode, so that value is not
known yet. For more operations we can figure it out for ourselves, but we don't have access to the
One piece of information missing from the function above is the result on an `SLOAD` operation. The state we get inside `log` is the state prior to the execution of the opcode, so that value is not known yet. For more operations we can figure it out for ourselves, but we don't have access to the
storage, so here we can't.
The solution is to have a flag, `afterSload`, which is only true in the opcode right after an
`SLOAD`, when we can see the result at the top of the stack.
The solution is to have a flag, `afterSload`, which is only true in the opcode right after an `SLOAD`, when we can see the result at the top of the stack.
```javascript
tracer = function(tx) {
@ -251,14 +223,10 @@ The output now contains the result in the line that follows the `SLOAD`.
### Dealing With Calls Between Contracts
So the storage has been treated as if there are only 2<sup>256</sup> cells. However, that is not true.
Contracts can call other contracts, and then the storage involved is the storage of the other contract.
We can see the address of the current contract in `log.contract.getAddress()`. This value is the execution
context - the contract whose storage we are using - even when code from another contract is executed (by using
[`CALLCODE` or `DELEGATECALL`][solidity-delcall]).
So the storage has been treated as if there are only 2<sup>256</sup> cells. However, that is not true. Contracts can call other contracts, and then the storage involved is the storage of the other contract. We can see the address of the current contract in `log.contract.getAddress()`. This value is the execution context - the contract whose storage we are using - even when code from another contract is executed (by using
[`CALLCODE` or `DELEGATECALL`](https://docs.soliditylang.org/en/v0.8.14/introduction-to-smart-contracts.html#delegatecall-callcode-and-libraries)).
However, `log.contract.getAddress()` returns an array of bytes. To convert this to the familiar hexadecimal
representation of Ethereum addresses, `this.byteHex()` and `array2Hex()` can be used.
However, `log.contract.getAddress()` returns an array of bytes. To convert this to the familiar hexadecimal representation of Ethereum addresses, `this.byteHex()` and `array2Hex()` can be used.
```javascript
tracer = function(tx) {
@ -345,10 +313,7 @@ The output is similar to:
## Other traces
This tutorial has focused on `debug_traceTransaction()` which reports information about individual transactions. There are
also RPC endpoints that provide different information, including tracing the EVM execution within a block, between two blocks,
for specific `eth_call`s or rejected blocks. The fill list of trace functions can be explored in the
[reference documentation][debug-docs].
This tutorial has focused on `debug_traceTransaction()` which reports information about individual transactions. There are also RPC endpoints that provide different information, including tracing the EVM execution within a block, between two blocks, for specific `eth_call`s or rejected blocks. The full list of trace functions can be explored in the [reference documentation](/content/docs/interacting_with_geth/RPC/ns-debug.md).
## Custom Go tracing
@ -356,7 +321,7 @@ for specific `eth_call`s or rejected blocks. The fill list of trace functions ca
Custom tracers can also be made more performant by writing them in Go. The gain in performance mostly comes from the fact that Geth doesn't need
to interpret JS code and can execute native functions. Geth comes with several built-in [native tracers](https://github.com/ethereum/go-ethereum/tree/master/eth/tracers/native) which can serve as examples. Please note that unlike JS tracers, Go tracing scripts cannot be simply passed as an argument to the API. They will need to be added to and compiled with the rest of the Geth source code.
In this section a simple native tracer that counts the number of opcodes will be covered. First follow the instructions to [clone and build](install-and-build/installing-geth#build-from-source-code) Geth from source code. Next save the following snippet as a `.go` file and add it to `eth/tracers/native`:
In this section a simple native tracer that counts the number of opcodes will be covered. First follow the instructions to [clone and build](/content/docs/getting_started/Installing-Geth.md) Geth from source code. Next save the following snippet as a `.go` file and add it to `eth/tracers/native`:
```go
package native
@ -454,5 +419,4 @@ As can be seen every method of the [EVMLogger interface](https://pkg.go.dev/gith
description: Introduction to mobile development with Geth
---
Embedding clients into mobile devices is an important part of Ethereum's decentralization vision.
This is because being able to verify data, follow the chain and submit transactions without
relying on centralized intermediaries is critical for censorship resistant access
to the network. Doing so on a mobile device is the most convenient route for many users.
This relies on Geth running a [light client](/docs/interface/les) on the mobile
device and exposing an API that developers can use to build mobile apps on top of Geth. This
page outlines how to download Geth for mobile and how to get started with managing Ethereum
accounts in mobile applications. Ethereum mobile development is relatively nascent, but there is
an active developer community. For further information on Geth mobile development visit the
#mobile channel in the [Geth discord](https://discord.gg/wQdpS5aA).
Embedding clients into mobile devices is an important part of Ethereum's decentralization vision. This is because being able to verify data, follow the chain and submit transactions without relying on centralized intermediaries is critical for censorship resistant access to the network. Doing so on a mobile device is the most convenient route for many users. This relies on Geth running a [light client](/docs/interface/les) on the mobile
device and exposing an API that developers can use to build mobile apps on top of Geth. This page outlines how to download Geth for mobile and how to get started with managing Ethereum accounts in mobile applications. Ethereum mobile development is relatively nascent, but there is an active developer community. For further information on Geth mobile development visit the #mobile channel in the [Geth discord](https://discord.gg/wQdpS5aA).
## Download and install
@ -20,12 +12,9 @@ an active developer community. For further information on Geth mobile developmen
#### Android Studio
Geth for Mobile bundles can be downloaded directly from [the download page](https://geth.ethereum.org/downloads/)
and inserted into a project in Android Studio via `File -> New -> New module... -> Import .JAR/.AAR Package`.
Geth for Mobile bundles can be downloaded directly from [the download page](https://geth.ethereum.org/downloads/) and inserted into a project in Android Studio via `File -> New -> New module... -> Import .JAR/.AAR Package`.
It is also necessary to configure `gradle` to link the mobile library bundle to the
application. This can be done by adding a new entry to the `dependencies` section of the
`build.gradle` script, pointing it to the module that was just added (named `geth` by default).
It is also necessary to configure `gradle` to link the mobile library bundle to the application. This can be done by adding a new entry to the `dependencies` section of the `build.gradle` script, pointing it to the module that was just added (named `geth` by default).
```gradle
dependencies {
@ -36,9 +25,7 @@ dependencies {
#### Manual build
Geth can also be built it locally using a `make` command. This will create an Android
archive called `geth.aar` in the `build/bin` folder that can be imported into Android
Studio as described above.
Geth can also be built it locally using a `make` command. This will create an Android archive called `geth.aar` in the `build/bin` folder that can be imported into Android Studio as described above.
```shell
$ make android
@ -49,9 +36,7 @@ Import "build/bin/geth.aar" to use the library.
### iOS
Geth must be downloaded and built locally for IoS. Building locally is achieved using the
`make` command. This will create an iOS XCode framework called `Geth.framework` in the
`build/bin` folder that can be imported into XCode as described above.
Geth must be downloaded and built locally for IoS. Building locally is achieved using the `make` command. This will create an iOS XCode framework called `Geth.framework` in the `build/bin` folder that can be imported into XCode as described above.
```bash
$ make ios
@ -62,54 +47,37 @@ Import "build/bin/Geth.framework" to use the library.
## Mobile API
Similarly to the reusable [Go libraries](/docs/dapp/native), the mobile wrappers focus on
three main usage areas:
Similarly to the reusable [Go libraries](content/docs/developers/dapp-developer/native-accounts.md), the mobile wrappers focus on three main usage areas:
- Simplified client side account management
- Remote node interfacing via different transports
- Contract interactions through auto-generated bindings
The Geth mobile API is broadly equivalent to the [Go API](/docs/dapp/native).
The source code can be found in the `mobile` section of Geth's
The Geth mobile API is broadly equivalent to the [Go API](/content/docs/developers/dapp-developer/native-accounts.md). The source code can be found in the `mobile` section of Geth's
Best practise for account management is to do it client-side, with all sensitive information
self-contained inside the local application. This ensures the developer/user retains
fine-grained control over the access permissions for user-data instead of outsourcing security
Best practise for account management is to do it client-side, with all sensitive information self-contained inside the local application. This ensures the developer/user retains fine-grained control over the access permissions for user-data instead of outsourcing security
to a third party.
To support this, Geth provides an accounts library that includes the tools required
for secure account management via encrypted keystores and passphrase protected accounts,
similarly to running a full Geth node.
To support this, Geth provides an accounts library that includes the tools required for secure account management via encrypted keystores and passphrase protected accounts, similarly to running a full Geth node.
### Encrypted keystores
Access keys to Ethereum accounts should never be stored in plain-text. Instead, they should
be stored encrypted so that even if the mobile device is accessed by a malicious third party
the keys are still hidden under an additional layer of security. Geth provides a keystore
that enables developers to store keys securely using the [`secp256k1` elliptic curve](sec2),
implemented using [`libsecp256k`][secp256k1] and wrapped by [Geth accounts][accounts-go].
Accounts are stored on disk in the [Web3 Secret Storage][secstore] format. Developers should be
aware of these implementation details but are not required to deeply understand the cryptographic
primitives in order to use the keystore.
One thing that should be understood, though, is that the cryptographic primitives underpinning
the keystore can operate in *light* or *standard* mode. Light mode is computationally cheaper, while
standard mode has extra security. Light mode is appropriate for mobile devices, but developers
Access keys to Ethereum accounts should never be stored in plain-text. Instead, they should be stored encrypted so that even if the mobile device is accessed by a malicious third party the keys are still hidden under an additional layer of security. Geth provides a keystore that enables developers to store keys securely using the [`secp256k1` elliptic curve](https://www.secg.org/sec2-v2.pdf), implemented using [`libsecp256k`](https://github.com/bitcoin-core/secp256k1) and wrapped by [Geth accounts](https://godoc.org/github.com/ethereum/go-ethereum/accounts).
Accounts are stored on disk in the [Web3 Secret Storage](https://github.com/ethereum/wiki/wiki/Web3-Secret-Storage-Definition) format. Developers should be aware of these implementation details but are not required to deeply understand the cryptographic primitives in order to use the keystore.
One thing that should be understood, though, is that the cryptographic primitives underpinning the keystore can operate in *light* or *standard* mode. Light mode is computationally cheaper, while standard mode has extra security. Light mode is appropriate for mobile devices, but developers
should be aware that there is a security trade-off.
* *standard* needs 256MB memory and 1 second processing on a modern CPU to access a key
* *standard* needs 256MB memory and 1 second processing on a modern CPU to access a key
* *light* needs 4MB memory and 100 millisecond processing on a modern CPU to access a key
### Keystores on Android (Java)
The encrypted keystore on Android is implemented by the `KeyStore` class from the
`org.ethereum.geth` package. The configuration constants are located in the `Geth`
abstract class, similarly from the `org.ethereum.geth` package.
Hence to do client side account management on Android, two classes should be
imported into the Java code:
The encrypted keystore on Android is implemented by the `KeyStore` class from the `org.ethereum.geth` package. The configuration constants are located in the `Geth` abstract class, similarly from the `org.ethereum.geth` package. Hence to do client side account management on Android, two classes should be imported into the Java code:
```java
import org.ethereum.geth.Geth;
@ -122,23 +90,13 @@ Then new encrypted keystore can be created via:
KeyStore ks = new KeyStore("/path/to/keystore", Geth.LightScryptN, Geth.LightScryptP);
```
The keystore should be in a location writable by the local mobile application but
on-readable for other installed applications such as inside the app's data directory.
If the `KeyStore` is created from within a class extending an Android object, access
to the `Context.getFilesDir()` method is probably provided via `this.getFilesDir()`,
so the keystore path could be set to `this.getFilesDir() + "/keystore"`.
The last two arguments of the `KeyStore` constructor are the crypto parameters defining
how resource-intensive the keystore encryption should be. The choices are
`Geth.StandardScryptN, Geth.StandardScryptP`, `Geth.LightScryptN, Geth.LightScryptP` or
custom numbers. The *light* version is recommended.
The keystore should be in a location writable by the local mobile application but on-readable for other installed applications such as inside the app's data directory. If the `KeyStore` is created from within a class extending an Android object, access to the `Context.getFilesDir()` method is probably provided via `this.getFilesDir()`, so the keystore path could be set to `this.getFilesDir() + "/keystore"`.
The last two arguments of the `KeyStore` constructor are the crypto parameters defining how resource-intensive the keystore encryption should be. The choices are `Geth.StandardScryptN, Geth.StandardScryptP`, `Geth.LightScryptN, Geth.LightScryptP` or custom numbers. The *light* version is recommended.
### Keystores on iOS (Swift 3)
The encrypted keystore on iOS is implemented by the `GethKeyStore` class from the `Geth`
framework. The configuration constants are located in the same namespace as global
variables. Hence to do client side account management on iOS, `Geth` framework should be
The encrypted keystore on iOS is implemented by the `GethKeyStore` class from the `Geth` framework. The configuration constants are located in the same namespace as global variables. Hence to do client side account management on iOS, `Geth` framework should be
imported into the Swift code:
```swift
@ -151,60 +109,31 @@ Then a new encrypted account manager can be created using:
let ks = GethNewKeyStore("/path/to/keystore", GethLightScryptN, GethLightScryptP);
```
The keystore folder needs to be in a location writable by the local mobile application
but non-readable for other installed applications such as inside the app's document
directory. The document directory shopuld be retrievable using
so the keystore path could be `datadir + "/keystore"`.
The keystore folder needs to be in a location writable by the local mobile application but non-readable for other installed applications such as inside the app's document directory. The document directory shopuld be retrievable using
`let datadir = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true)[0]`, so the keystore path could be `datadir + "/keystore"`.
The last two arguments of the `GethNewKeyStore` factory method are the crypto parameters
defining how resource-intensive the keystore encryption should be. The choices are
`GethStandardScryptN, GethStandardScryptP`, `GethLightScryptN, GethLightScryptP` or
custom numbers. The *light* version is recommended.
The last two arguments of the `GethNewKeyStore` factory method are the crypto parameters defining how resource-intensive the keystore encryption should be. The choices are `GethStandardScryptN, GethStandardScryptP`, `GethLightScryptN, GethLightScryptP` or custom numbers. The *light* version is recommended.
### Account lifecycle
The encyrpted keystore can be used for the entire account lifecycle requirements of a mobile
application. This includes the basic functionality of creating new accounts and deleting
existing ones as well as more advanced functions like updating access credentials and account
The encyrpted keystore can be used for the entire account lifecycle requirements of a mobile application. This includes the basic functionality of creating new accounts and deleting existing ones as well as more advanced functions like updating access credentials and account
import/export.
Although the keystore defines the encryption strength it uses to store accounts,
there is no global master password that can grant access to all of them. Rather each
account is maintained individually, and stored on disk in its [encrypted format][secstore]
individually, ensuring a much cleaner and stricter separation of credentials.
This individuality means that any operation requiring access to an account will
need to provide the necessary authentication credentials for that particular account in
the form of a passphrase:
* When creating a new account, the caller must supply a passphrase to encrypt the account
with. This passphrase will be required for any subsequent access.
* When deleting an existing account, the caller must supply a passphrase to verify
ownership of the account. This isn't cryptographically necessary, rather a protective
measure against accidental loss of accounts.
* When updating an existing account, the caller must supply both current and new
passphrases. After completing the operation, the account will not be accessible via the
old passphrase.
* When exporting an existing account, the caller must supply both the current passphrase
to decrypt the account, as well as an export passphrase to re-encrypt it with before
returning the key-file to the user. This is required to allow moving accounts between
devices without sharing original credentials.
* When importing a new account, the caller must supply both the encryption passphrase of
the key-file being imported, as well as a new passphrase with which to store the
account. This is required to allow storing account with different credentials than used
for moving them around.
*Please note, there is no recovery mechanisms for losing the passphrases. The
cryptographic properties of the encrypted keystore (if using the provided parameters)
guarantee that account credentials cannot be brute forced in any meaningful time.*
Although the keystore defines the encryption strength it uses to store accounts, there is no global master password that can grant access to all of them. Rather each account is maintained individually, and stored on disk in its [encrypted format][secstore] individually, ensuring a much cleaner and stricter separation of credentials.
This individuality means that any operation requiring access to an account will need to provide the necessary authentication credentials for that particular account in the form of a passphrase:
* When creating a new account, the caller must supply a passphrase to encrypt the account with. This passphrase will be required for any subsequent access.
* When deleting an existing account, the caller must supply a passphrase to verify ownership of the account. This isn't cryptographically necessary, rather a protective measure against accidental loss of accounts.
* When updating an existing account, the caller must supply both current and new passphrases. After completing the operation, the account will not be accessible via the old passphrase.
* When exporting an existing account, the caller must supply both the current passphrase to decrypt the account, as well as an export passphrase to re-encrypt it with before returning the key-file to the user. This is required to allow moving accounts between devices without sharing original credentials.
* When importing a new account, the caller must supply both the encryption passphrase of the key-file being imported, as well as a new passphrase with which to store the account. This is required to allow storing account with different credentials than used for moving them around.
*Please note, there is no recovery mechanisms for losing the passphrases. The cryptographic properties of the encrypted keystore (if using the provided parameters) guarantee that account credentials cannot be brute forced in any meaningful time.*
### Accounts on Android (Java)
An Ethereum account on Android is implemented by the `Account` class from the
`org.ethereum.geth` package. Assuming an instance of a `KeyStore` called
`ks` exists, all of the described lifecycle operations can be executed with
a handful of function calls:
An Ethereum account on Android is implemented by the `Account` class from the `org.ethereum.geth` package. Assuming an instance of a `KeyStore` called `ks` exists, all of the described lifecycle operations can be executed with a handful of function calls:
```java
// Create a new account with the specified encryption passphrase.
Although instances of `Account` can be used to access various information about specific
Ethereum accounts, they do not contain any sensitive data (such as passphrases or private
keys), rather they act solely as identifiers for client code and the keystore.
Although instances of `Account` can be used to access various information about specific Ethereum accounts, they do not contain any sensitive data (such as passphrases or private keys), rather they act solely as identifiers for client code and the keystore.
### Accounts on iOS (Swift 3)
An Ethereum account on iOS is implemented by the `GethAccount` class from the `Geth`
framework. Assuming an instance of a `GethKeyStore` called `ks` exists, all of the described
lifecycle operations can be executed with a handful of function calls:
An Ethereum account on iOS is implemented by the `GethAccount` class from the `Geth` framework. Assuming an instance of a `GethKeyStore` called `ks` exists, all of the described lifecycle operations can be executed with a handful of function calls:
```swift
// Create a new account with the specified encryption passphrase.
Although instances of `GethAccount` can be used to access various information about
specific Ethereum accounts, they do not contain any sensitive data (such as passphrases or
private keys), rather they act solely as identifiers for client code and the keystore.
Although instances of `GethAccount` can be used to access various information about specific Ethereum accounts, they do not contain any sensitive data (such as passphrases or private keys), rather they act solely as identifiers for client code and the keystore.
## Signing authorization
As mentioned above, account objects do not hold the sensitive private keys of the
associated Ethereum accounts - they are merely placeholders to identify the cryptographic
keys with. All operations that require authorization (e.g. transaction signing) are
performed by the account manager after granting it access to the private keys.
There are a few different ways one can authorize the account manager to execute signing
operations. Since the different methods have very different security guarantees,
it is essential to be clear on how each works:
* **Single authorization**: The simplest way to sign a transaction via the keystore is to
provide the passphrase of the account every time something needs to be signed, which
will ephemerally decrypt the private key, execute the signing operation and immediately
throw away the decrypted key. The drawbacks are that the passphrase needs to be queried
from the user every time, which can become annoying if done frequently; or the
application needs to keep the passphrase in memory, which can have security
consequences if not done properly; and depending on the keystore's configured strength,
constantly decrypting keys can result in non-negligible resource requirements.
* **Multiple authorizations**: A more complex way of signing transactions via the
keystore is to unlock the account via its passphrase once, and allow the account
manager to cache the decrypted private key, enabling all subsequent signing requests to
complete without the passphrase. The lifetime of the cached private key may be managed
manually (by explicitly locking the account back up) or automatically (by providing a
timeout during unlock). This mechanism is useful for scenarios where the user may need
to sign many transactions or the application would need to do so without requiring user
input. The crucial aspect to remember is that **anyone with access to the account
manager can sign transactions while a particular account is unlocked** (e.g. device
left unattended; application running untrusted code).
As mentioned above, account objects do not hold the sensitive private keys of the associated Ethereum accounts - they are merely placeholders to identify the cryptographic keys with. All operations that require authorization (e.g. transaction signing) are performed by the account manager after granting it access to the private keys.
There are a few different ways one can authorize the account manager to execute signing operations. Since the different methods have very different security guarantees, it is essential to be clear on how each works:
* **Single authorization**: The simplest way to sign a transaction via the keystore is to provide the passphrase of the account every time something needs to be signed, which will ephemerally decrypt the private key, execute the signing operation and immediately throw away the decrypted key. The drawbacks are that the passphrase needs to be queried from the user every time, which can become annoying if done frequently; or the application needs to keep the passphrase in memory, which can have security consequences if not done properly; and depending on the keystore's configured strength, constantly decrypting keys can result in non-negligible resource requirements.
* **Multiple authorizations**: A more complex way of signing transactions via the keystore is to unlock the account via its passphrase once, and allow the account manager to cache the decrypted private key, enabling all subsequent signing requests tocomplete without the passphrase. The lifetime of the cached private key may be managed manually (by explicitly locking the account back up) or automatically (by providing a timeout during unlock). This mechanism is useful for scenarios where the user may need to sign many transactions or the application would need to do so without requiring user input. The crucial aspect to remember is that **anyone with access to the account manager can sign transactions while a particular account is unlocked** (e.g. device left unattended; application running untrusted code).
### Signing on Android (Java)
Assuming an instance of a `KeyStore` called `ks` exists, a new account to sign transactions
can be created using its `newAccount` method. For this demonstation a hard-coded
example transaction is created to sign:
Assuming an instance of a `KeyStore` called `ks` exists, a new account to sign transactions can be created using its `newAccount` method. For this demonstation a hard-coded example transaction is created to sign:
```java
// Create a new account to sign transactions with
@ -322,8 +223,7 @@ signed = ks.signTx(signer, tx, chain);
### Signing on iOS (Swift 3)
Assuming an instance of a `GethKeyStore` called `ks` exists, a new account
can be created to sign transactions with its `newAccount` method. For
Assuming an instance of a `GethKeyStore` called `ks` exists, a new account can be created to sign transactions with its `newAccount` method. For
this demonstation a hard-coded example transaction is created to sign:
This page introduced Geth for mobile. In addition to download and installation instructions, basic
account management was demonstrated for mobile applications on iOS and Android.
This page introduced Geth for mobile. In addition to download and installation instructions, basic account management was demonstrated for mobile applications on iOS and Android.
description: Introduction to account management in Go native applications.
---
Geth provides a simple, yet thorough accounts package that includes all the tools developers
need to leverage all the security of Geth's crypto implementation in a Go native application.
The account management is done client side with all sensitive data held inside the application.
This gives the user control over access permissions without relying on any third party.
Geth provides a simple, yet thorough accounts package that includes all the tools developers need to leverage all the security of Geth's crypto implementation in a Go native application. The account management is done client side with all sensitive data held inside the application. This gives the user control over access permissions without relying on any third party.
**Note Geth's built-in account management is convenient and straightforward to use, but
best practise is to use the external tool *Clef* for key management.**
{:toc}
- this will be removed by the toc
**Note Geth's built-in account management is convenient and straightforward to use, but best practise is to use the external tool *Clef* for key management.**
## Encrypted keystores
Access keys to Ethereum accounts should never be stored in plain-text. Instead, they should be
stored encrypted so that even if the mobile device is accessed by a malicious third party the
keys are still hidden under an additional layer of security. Geth provides a keystore that enables
developers to store keys securely. The Geth keystore uses [Scrypt][scrypt-docs] to store keys that are encoded
using the [`secp256k1`][secp256k1] elliptic curve. Accounts are stored on disk in the
[Web3 Secret Storage][wss] format. Developers should be aware of these implementation details
Access keys to Ethereum accounts should never be stored in plain-text. Instead, they should be stored encrypted so that even if the mobile device is accessed by a malicious third party the keys are still hidden under an additional layer of security. Geth provides a keystore that enables developers to store keys securely. The Geth keystore uses [Scrypt](https://pkg.go.dev/golang.org/x/crypto/scrypt) to store keys that are encoded using the [`secp256k1`](https://www.secg.org/sec2-v2.pdf) elliptic curve. Accounts are stored on disk in the [Web3 Secret Storage](https://github.com/ethereum/wiki/wiki/Web3-Secret-Storage-Definition) format. Developers should be aware of these implementation details
but are not required to deeply understand the cryptographic primitives in order to use the keystore.
One thing that should be understood, though, is that the cryptographic primitives underpinning the
keystore can operate in light or standard mode. Light mode is computationally cheaper, while standard
mode has extra security. Light mode is appropriate for mobile devices, but developers should be
aware that there is a security trade-off.
One thing that should be understood, though, is that the cryptographic primitives underpinning the keystore can operate in light or standard mode.Light mode is computationally cheaper, while standard mode has extra security. Light mode is appropriate for mobile devices, but developers should be aware that there is a security trade-off.
* standard needs 256MB memory and 1 second processing on a modern CPU to access a key
* light needs 4MB memory and 100 millisecond processing on a modern CPU to access a key
The encrypted keystore is implemented by the [`accounts.Manager`][accounts-manager] struct
from the [`accounts`][accounts-pkg] package, which also contains the configuration constants for the
*standard* or *light* security modes described above. Hence client side account management
The encrypted keystore is implemented by the [`accounts.Manager`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager) struct from the [`accounts`](https://godoc.org/github.com/ethereum/go-ethereum/accounts) package, which also contains the configuration constants for the *standard* or *light* security modes described above. Hence client side account management
simply requires importing the `accounts` package into the application code.
accounts.LightScryptP`][pkg-constants] or custom values (requiring understanding of the underlying
cryptography). The *standard* version is recommended.
The path to the keystore folder needs to be a location that is writable by the local user but non-readable for other system users, such as inside the user's home directory.
The last two arguments of [`keystore.NewKeyStore`](https://godoc.org/github.com/ethereum/go-ethereum/accounts/keystore#NewKeyStore) are the crypto parameters defining how resource-intensive the keystore encryption should be. The options are [`accounts.StandardScryptN, accounts.StandardScryptP`, `accounts.LightScryptN, accounts.LightScryptP`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#pkg-constants) or custom values (requiring understanding of the underlying cryptography). The *standard* version is recommended.
## Account lifecycle
Once an encrypted keystore for Ethereum accounts exists it, it can be used to manage accounts for the
entire account lifecycle requirements of a Go native application. This includes the basic functionality
of creating new accounts and deleting existing ones as well as updating access credentials,
exporting existing accounts, and importing them on other devices.
Once an encrypted keystore for Ethereum accounts exists it, it can be used to manage accounts for the entire account lifecycle requirements of a Go native application. This includes the basic functionality of creating new accounts and deleting existing ones as well as updating access credentials, exporting existing accounts, and importing them on other devices.
Although the keystore defines the encryption strength it uses to store accounts, there is no global master
password that can grant access to all of them. Rather each account is maintained individually, and stored on
disk in its [encrypted format][wss] individually, ensuring a much cleaner and stricter separation of
credentials.
Although the keystore defines the encryption strength it uses to store accounts, there is no global master password that can grant access to all of them. Rather each account is maintained individually, and stored on disk in its [encrypted format](https://github.com/ethereum/wiki/wiki/Web3-Secret-Storage-Definition) individually, ensuring a much cleaner and stricter separation of credentials.
This individuality means that any operation requiring access to an account will need to provide the
necessary authentication credentials for that particular account in the form of a passphrase:
This individuality means that any operation requiring access to an account will need to provide the necessary authentication credentials for that particular account in the form of a passphrase:
* When creating a new account, the caller must supply a passphrase to encrypt the account
with. This passphrase will be required for any subsequent access, the lack of which
will forever forfeit using the newly created account.
* When creating a new account, the caller must supply a passphrase to encrypt the account with. This passphrase will be required for any subsequent access, the lack of which will forever forfeit using the newly created account.
* When deleting an existing account, the caller must supply a passphrase to verify
ownership of the account. This isn't cryptographically necessary, rather a protective
measure against accidental loss of accounts.
* When deleting an existing account, the caller must supply a passphrase to verify ownership of the account. This isn't cryptographically necessary, rather a protective measure against accidental loss of accounts.
* When updating an existing account, the caller must supply both current and new
passphrases. After completing the operation, the account will not be accessible via the
old passphrase any more.
* When updating an existing account, the caller must supply both current and new passphrases. After completing the operation, the account will not be accessible via the old passphrase any more.
* When exporting an existing account, the caller must supply both the current passphrase
to decrypt the account, as well as an export passphrase to re-encrypt it with before
returning the key-file to the user. This is required to allow moving accounts between
machines and applications without sharing original credentials.
* When exporting an existing account, the caller must supply both the current passphrase to decrypt the account, as well as an export passphrase to re-encrypt it with before returning the key-file to the user. This is required to allow moving accounts between machines and applications without sharing original credentials.
* When importing a new account, the caller must supply both the encryption passphrase of
the key-file being imported, as well as a new passhprase with which to store the
account. This is required to allow storing account with different credentials than used
for moving them around.
* When importing a new account, the caller must supply both the encryption passphrase of the key-file being imported, as well as a new passhprase with which to store the account. This is required to allow storing account with different credentials than used for moving them around.
***Please note, there are no recovery mechanisms for lost passphrases. The
cryptographic properties of the encrypted keystore (using the provided parameters)
guarantee that account credentials cannot be brute forced in any meaningful time.***
***Please note, there are no recovery mechanisms for lost passphrases. The cryptographic properties of the encrypted keystore (using the provided parameters) guarantee that account credentials cannot be brute forced in any meaningful time.***
An Ethereum account is implemented by the [`accounts.Account`][accounts-account] struct from
the Geth [accounts][accounts-pkg] package. Assuming an instance of an
[`accounts.Manager`][accounts-manager] called `am` exists, all of the described lifecycle
operations can be executed with a handful of function calls (error handling omitted).
An Ethereum account is implemented by the [`accounts.Account`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Account) struct from the Geth [accounts](https://godoc.org/github.com/ethereum/go-ethereum/accounts) package. Assuming an instance of an [`accounts.Manager`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager) called `am` exists, all of the described lifecycle operations can be executed with a handful of function calls (error handling omitted).
```go
// Create a new account with the specified encryption passphrase.
*Although instances of [`accounts.Account`][accounts-account] can be used to access various information about
specific Ethereum accounts, they do not contain any sensitive data (such as passphrases or private keys),
rather they act solely as identifiers for client code and the keystore.*
*Although instances of [`accounts.Account`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Account) can be used to access various information about specific Ethereum accounts, they do not contain any sensitive data (such as passphrases or private keys), rather they act solely as identifiers for client code and the keystore.*
## Signing authorization
Account objects do not hold the sensitive private keys of the associated Ethereum accounts.
Account objects are placeholders that identify the cryptographic keys. All operations that
require authorization (e.g. transaction signing) are performed by the account manager after
granting it access to the private keys.
There are a few different ways to authorize the account manager to execute signing
operations, each having its advantages and drawbacks. Since the different methods have
wildly different security guarantees, it is essential to be clear on how each works:
* **Single authorization**: The simplest way to sign a transaction via the account
manager is to provide the passphrase of the account every time something needs to be
signed, which will ephemerally decrypt the private key, execute the signing operation
and immediately throw away the decrypted key. The drawbacks are that the passphrase
needs to be queried from the user every time, which can become annoying if done
frequently or the application needs to keep the passphrase in memory, which can have
security consequences if not done properly. Depending on the keystore's configured
strength, constantly decrypting keys can result in non-negligible resource
requirements.
* **Multiple authorizations**: A more complex way of signing transactions via the account
manager is to unlock the account via its passphrase once, and allow the account manager
to cache the decrypted private key, enabling all subsequent signing requests to
complete without the passphrase. The lifetime of the cached private key may be managed
manually (by explicitly locking the account back up) or automatically (by providing a
timeout during unlock). This mechanism is useful for scenarios where the user may need
to sign many transactions or the application would need to do so without requiring user
input. The crucial aspect to remember is that **anyone with access to the account
manager can sign transactions while a particular account is unlocked** (e.g.
application running untrusted code).
Assuming an instance of an [`accounts.Manager`][accounts-manager] called `am` exists, a new
account can be created to sign transactions using [`NewAccount`][new-account]. Creating transactions
is out of scope for this page so instead a random [`common.Hash`][common-hash] will be signed instead.
Account objects do not hold the sensitive private keys of the associated Ethereum accounts. Account objects are placeholders that identify the cryptographic keys. All operations that require authorization (e.g. transaction signing) are performed by the account manager after granting it access to the private keys.
There are a few different ways to authorize the account manager to execute signing operations, each having its advantages and drawbacks. Since the different methods have wildly different security guarantees, it is essential to be clear on how each works:
* **Single authorization**: The simplest way to sign a transaction via the account manager is to provide the passphrase of the account every time something needs to be signed, which will ephemerally decrypt the private key, execute the signing operation and immediately throw away the decrypted key. The drawbacks are that the passphrase needs to be queried from the user every time, which can become annoying if done frequently or the application needs to keep the passphrase in memory, which can have security consequences if not done properly. Depending on the keystore's configured strength, constantly decrypting keys can result in non-negligible resource requirements.
* **Multiple authorizations**: A more complex way of signing transactions via the account manager is to unlock the account via its passphrase once, and allow the account manager to cache the decrypted private key, enabling all subsequent signing requests to complete without the passphrase. The lifetime of the cached private key may be managed manually (by explicitly locking the account back up) or automatically (by providing a timeout during unlock). This mechanism is useful for scenarios where the user may need to sign many transactions or the application would need to do so without requiring user input. The crucial aspect to remember is that **anyone with access to the account manager can sign transactions while a particular account is unlocked** (e.g. application running untrusted code).
Assuming an instance of an [`accounts.Manager`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager) called `am` exists, a new account can be created to sign transactions using [`NewAccount`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager.NewAccount). Creating transactions is out of scope for this page so instead a random [`common.Hash`](https://godoc.org/github.com/ethereum/go-ethereum/common#Hash) will be signed instead.
For information on creating transactions in Go native applications see the [Go API page](/docs/dapp/native).
Note that [`SignWithPassphrase`][sign-w-phrase] takes an [`accounts.Account`][accounts-account] as the
signer, whereas [`Sign`][accounts-sign] takes only a [`common.Address`][common-address]. The reason
for this is that an [`accounts.Account`][accounts-account] object may also contain a custom key-path, allowing
[`SignWithPassphrase`][sign-w-phrase] to sign using accounts outside of the keystore; however
[`Sign`][accounts-sign] relies on accounts already unlocked within the keystore, so it cannot specify custom paths.
Note that [`SignWithPassphrase`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager.SignWithPassphrase) takes an [`accounts.Account`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Account) as the signer, whereas [`Sign`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager.Sign) takes only a [`common.Address`](https://godoc.org/github.com/ethereum/go-ethereum/common#Address). The reason for this is that an [`accounts.Account`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Account) object may also contain a custom key-path, allowing [`SignWithPassphrase`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager.SignWithPassphrase) to sign using accounts outside of the keystore; however [`Sign`](https://godoc.org/github.com/ethereum/go-ethereum/accounts#Manager.Sign) relies on accounts already unlocked within the keystore, so it cannot specify custom paths.
## Summary
Account management is a fundamental pillar of Ethereum development. Geth's Go API provides the tools required
to integrate best-practise account security into Go native applications using a simple set of Go functions.
Account management is a fundamental pillar of Ethereum development. Geth's Go API provides the tools required to integrate best-practise account security into Go native applications using a simple set of Go functions.
description: Intriduction to generating bindings for using Geth features in Go native applications
---
This page introduces the concept of server-side native dapps. Geth provides the tools required
to generate [Go][go-link] language bindings to any Ethereum contract that is compile-time type safe,
highly performant and can be generated completely automatically from a compiled contract.
This page introduces the concept of server-side native dapps. Geth provides the tools required to generate [Go](https://github.com/golang/go/wiki#getting-started-with-go) language bindings to any Ethereum contract that is compile-time type safe, highly performant and can be generated completely automatically from a compiled contract.
Interacting with a contract on the Ethereum blockchain from Go is already possible via the
RPC interfaces exposed by Ethereum clients. However, writing the boilerplate code that
translates Go language constructs into RPC calls and back is time consuming and brittle -
implementation bugs can only be detected during runtime and it's almost impossible to evolve
a contract as even a tiny change in Solidity is awkward to port over to Go. Therefore,
Geth provides tools for easily converting contract code into Go code that can be used directly
in Go applications.
Interacting with a contract on the Ethereum blockchain from Go is already possible via the RPC interfaces exposed by Ethereum clients. However, writing the boilerplate code that translates Go language constructs into RPC calls and back is time consuming and brittle - implementation bugs can only be detected during runtime and it's almost impossible to evolve a contract as even a tiny change in Solidity is awkward to port over to Go. Therefore, Geth provides tools for easily converting contract code into Go code that can be used directly in Go applications.
This page provides an introduction to generating Go contract bindings and using them in a simple
Go application.
{:toc}
- this will be removed by the toc
This page provides an introduction to generating Go contract bindings and using them in a simple Go application.
## Prerequisites
This page is fairly beginner-friendly and designed for people starting out with
writing Go native dapps. The core concepts will be introduced gradually as a developer
would encounter them. However, some basic familiarity with [Ethereum](https://ethereum.org),
[Solidity](https://docs.soliditylang.org/en/v0.8.15/) and [Go](https://go.dev/) is
assumed.
This page is fairly beginner-friendly and designed for people starting out with writing Go native dapps. The core concepts will be introduced gradually as a developer would encounter them. However, some basic familiarity with [Ethereum](https://ethereum.org), [Solidity](https://docs.soliditylang.org/en/v0.8.15/) and [Go](https://go.dev/) is assumed.
## What is an ABI?
Ethereum smart contracts have a schema that defines its functions and return types in the form
of a JSON file. This JSON file is known as an *Application Binary Interface*, or ABI. The ABI
acts as a specification for precisely how to encode data sent to a contract and how to
decode the data the contract sends back. The ABI is the only essential piece of information required to
generate Go bindings. Go developers can then use the bindings to interact with the contract
from their Go application without having to deal directly with data encoding and decoding.
An ABI is generated when a contract is compiled.
Ethereum smart contracts have a schema that defines its functions and return types in the form of a JSON file. This JSON file is known as an *Application Binary Interface*, or ABI. The ABI acts as a specification for precisely how to encode data sent to a contract and how to decode the data the contract sends back. The ABI is the only essential piece of information required to generate Go bindings. Go developers can then use the bindings to interact with the contract from their Go application without having to deal directly with data encoding and decoding. An ABI is generated when a contract is compiled.
## Abigen: Go binding generator
Geth includes a source code generator called `abigen` that can convert Ethereum ABI definitions
into easy to use, type-safe Go packages. With a valid Go development environment
set up and the go-ethereum repository checked out correctly, `abigen` can be built as follows:
Geth includes a source code generator called `abigen` that can convert Ethereum ABI definitions into easy to use, type-safe Go packages. With a valid Go development environment set up and the go-ethereum repository checked out correctly, `abigen` can be built as follows:
```
$ cd $GOPATH/src/github.com/ethereum/go-ethereum
@ -54,9 +29,7 @@ $ go build ./cmd/abigen
### Generating the bindings
To demonstrate the binding generator a contract is required. The contract `Storage.sol` implements two
very simple functions: `store` updates a user-defined `uint256` to the contract's storage, and `retrieve`
displays the value stored in the contract to the user. The Solidity code is as follows:
To demonstrate the binding generator a contract is required. The contract `Storage.sol` implements two very simple functions: `store` updates a user-defined `uint256` to the contract's storage, and `retrieve` displays the value stored in the contract to the user. The Solidity code is as follows:
```solidity
// SPDX-License-Identifier: GPL-3.0
@ -83,18 +56,13 @@ contract Storage {
This contract can be pasted into a text file and saved as `Storage.sol`.
The following code snippet shows how an ABI can be generated for `Storage.sol`
using the Solidity compiler `solc`.
The following code snippet shows how an ABI can be generated for `Storage.sol` using the Solidity compiler `solc`.
```shell
solc --abi Storage.sol -o build
```
The ABI can also be generated in other ways such as using the `compile` commands in development
frameworks such as [Truffle][truffle-link], [Hardhat][hardhat-link] and [Brownie][brownie-link]
or in the online IDE [Remix][remix-link]. ABIs for existing
verified contracts can be downloaded from [Etherscan](etherscan.io).
The ABI can also be generated in other ways such as using the `compile` commands in development frameworks such as [Truffle](https://trufflesuite.com/docs/truffle/), [Hardhat](https://hardhat.org/) and [Brownie](https://eth-brownie.readthedocs.io/en/stable/) or in the online IDE [Remix](https://remix.ethereum.org/). ABIs for existing verified contracts can be downloaded from [Etherscan](etherscan.io).
The ABI for `Storage.sol` (`Storage.abi`) looks as follows:
@ -115,9 +83,7 @@ Where the flags are:
* `--type`: Optional Go type name to assign to the binding struct
* `--out`: Optional output path for the generated Go source file (not set = stdout)
This will generate a type-safe Go binding for the Storage contract. The generated code will
look something like the snippet below, the full version of which can be viewed
This will generate a type-safe Go binding for the Storage contract. The generated code will look something like the snippet below, the full version of which can be viewed [here](https://gist.github.com/jmcook1186/a78e59d203bb54b06e1b81f2cda79d93).
```go
// Code generated - DO NOT EDIT.
@ -169,16 +135,12 @@ type Storage struct {
```
`Storage.go` contains all the bindings required to interact with `Storage.sol` from a Go application.
However, this isn't very useful unless the contract is actually deployed on Ethereum or one of
Ethereum's testnets. The following sections will demonstrate how to deploy the contract to
`Storage.go` contains all the bindings required to interact with `Storage.sol` from a Go application. However, this isn't very useful unless the contract is actually deployed on Ethereum or one of Ethereum's testnets. The following sections will demonstrate how to deploy the contract to
an Ethereum testnet and interact with it using the Go bindings.
### Deploying contracts to Ethereum
In the previous section, the contract ABI was sufficient for generating the contract bindings from its ABI.
However, deploying the contract requires some additional information in the form of the compiled
bytecode.
In the previous section, the contract ABI was sufficient for generating the contract bindings from its ABI. However, deploying the contract requires some additional information in the form of the compiled bytecode.
The bytecode is obtained by running the compiler again but this passing the `--bin` flag, e.g.
@ -193,8 +155,7 @@ Then `abigen` can be run again, this time passing `Storage.bin`:
This will generate something similar to the bindings generated in the previous section. However,
an additional `DeployStorage` function has been injected:
This will generate something similar to the bindings generated in the previous section. However, an additional `DeployStorage` function has been injected:
```go
// DeployStorage deploys a new Ethereum contract, binding an instance of Storage to it.
View the full file [here](https://gist.github.com/jmcook1186/91124cfcbc7f22dcd3bb4f148d2868a8).
The new `DeployStorage()` function can be used to deploy the contract to an Ethereum testnet from a Go application. To do this
requires incorporating the bindings into a Go application that also handles account management, authorization and Ethereum backend
to deploy the contract through. Specifically, this requires:
The new `DeployStorage()` function can be used to deploy the contract to an Ethereum testnet from a Go application. To do this requires incorporating the bindings into a Go application that also handles account management, authorization and Ethereum backend to deploy the contract through. Specifically, this requires:
1. A running Geth node connected to an Ethereum testnet (recommended Goerli)
2. An account in the keystore prefunded with enough ETH to cover gas costs for deploying and interacting with the contract
Assuming these prerequisites exist, a new `ethclient` can be instantiated with the local Geth node's ipc file, providing
access to the testnet from the Go application. The key can be instantiated as a variable in the application by copying the
JSON object from the keyfile in the keystore.
Assuming these prerequisites exist, a new `ethclient` can be instantiated with the local Geth node's ipc file, providing access to the testnet from the Go application. The key can be instantiated as a variable in the application by copying the JSON object from the keyfile in the keystore.
Putting it all together would result in:
@ -275,8 +232,7 @@ func main() {
}
```
Running this code requests the creation of a brand new `Storage` contract on the Goerli blockchain.
The contract functions can be called while the contract is waiting to be mined.
Running this code requests the creation of a brand new `Storage` contract on the Goerli blockchain. The contract functions can be called while the contract is waiting to be included in a block.
@ -285,8 +241,7 @@ Transaction waiting to be mined: 0x6a81231874edd2461879b7280ddde1a857162a744e365
Pending name: Storage contract in Go!
```
Once mined, the contract exists permanently at its deployment address and can now be interacted with
from other applications without ever needing to be redeployed.
Once the contract deployment has been included in a validated block, the contract exists permanently at its deployment address and can now be interacted with from other applications without ever needing to be redeployed.
Note that `DeployStorage` returns four variables:
@ -301,15 +256,11 @@ Note that `DeployStorage` returns four variables:
### Accessing an Ethereum contract
To interact with a contract already deployed on the blockchain, the deployment `address` is required and
a `backend` through which to access Ethereum must be defined. The binding generator provides an RPC
backend out-of-the-box that can be used to attach to an existing Ethereum node via IPC, HTTP or WebSockets.
To interact with a contract already deployed on the blockchain, the deployment `address` is required and a `backend` through which to access Ethereum must be defined. The binding generator provides an RPC backend out-of-the-box that can be used to attach to an existing Ethereum node via IPC, HTTP or WebSockets.
As in the previous section, a Geth node running on an Ethereum testnet (recommend Goerli) and an account
with some test ETH to cover gas is required. The `Storage.sol` deployment address is also needed.
As in the previous section, a Geth node running on an Ethereum testnet (recommend Goerli) and an account with some test ETH to cover gas is required. The `Storage.sol` deployment address is also needed.
Again, an instance of `ethclient` can be created, passing the path to Geth's ipc file. In the example
below this backend is assigned to the variable `conn`.
Again, an instance of `ethclient` can be created, passing the path to Geth's ipc file. In the example below this backend is assigned to the variable `conn`.
```go
// Create an IPC based RPC connection to a remote node
@ -320,9 +271,7 @@ if err != nil {
}
```
The functions available for interacting with the `Storage` contract are defined in `Storage.go`. To create
a new instance of the contract in a Go application, the `NewStorage()` function can be used. The function
is defined in `Storage.go` as follows:
The functions available for interacting with the `Storage` contract are defined in `Storage.go`. To create a new instance of the contract in a Go application, the `NewStorage()` function can be used. The function is defined in `Storage.go` as follows:
```go
// NewStorage creates a new instance of Storage, bound to a specific deployed contract.
`NewStorage()` takes two arguments: the deployment address and a backend (`conn`) and returns
an instance of the deployed contract. In the example below, the instance is assigned to `store`.
`NewStorage()` takes two arguments: the deployment address and a backend (`conn`) and returns an instance of the deployed contract. In the example below, the instance is assigned to `store`.
```go
@ -366,9 +314,7 @@ func main() {
```
The contract instance is then available to interact with in the Go application. To read a value from
the blockchain, for example the `value` stored in the contract, the contract's `Retrieve()` function
can be called. Again, the function is defined in `Storage.go` as follows:
The contract instance is then available to interact with in the Go application. To read a value from the blockchain, for example the `value` stored in the contract, the contract's `Retrieve()` function can be called. Again, the function is defined in `Storage.go` as follows:
```go
// Retrieve is a free data retrieval call binding the contract method 0x2e64cec1.
Note that the `Retrieve()` function requires a parameter to be passed, even though the
original Solidity contract didn't require any at all none. The parameter required is
a `*bind.CallOpts` type, which can be used to fine tune the call. If no adjustments to the
call are required, pass `nil`. Adjustments to the call include:
Note that the `Retrieve()` function requires a parameter to be passed, even though the original Solidity contract didn't require any at all none. The parameter required is a `*bind.CallOpts` type, which can be used to fine tune the call. If no adjustments to the call are required, pass `nil`. Adjustments to the call include:
* `Pending`: Whether to access pending contract state or the current stable one
* `GasLimit`: Place a limit on the computing resources the call might consume
@ -416,17 +359,9 @@ Value: 56
### Transacting with an Ethereum contract
Invoking a method that changes contract state (i.e. transacting) is a bit more involved,
as a live transaction needs to be authorized and broadcast into the network. **Go bindings
require local signing of transactions and do not delegate this to a remote node.** This is
to keep accounts private within dapps, and not shared (by default) between them.
Invoking a method that changes contract state (i.e. transacting) is a bit more involved, as a live transaction needs to be authorized and broadcast into the network. **Go bindings require local signing of transactions and do not delegate this to a remote node.** This is to keep accounts private within dapps, and not shared (by default) between them.
Thus to allow transacting with a contract, your code needs to implement a method that
given an input transaction, signs it and returns an authorized output transaction. Since
most users have their keys in the [Web3 Secret Storage][web3-ss-link] format, the `bind`
package contains a small utility method (`bind.NewTransactor(keyjson, passphrase)`) that can
create an authorized transactor from a key file and associated password, without the user
needing to implement key signing themselves.
Thus to allow transacting with a contract, your code needs to implement a method that given an input transaction, signs it and returns an authorized output transaction. Since most users have their keys in the [Web3 Secret Storage](https://github.com/ethereum/wiki/wiki/Web3-Secret-Storage-Definition) format, the `bind` package contains a small utility method (`bind.NewTransactor(keyjson, passphrase)`) that can create an authorized transactor from a key file and associated password, without the user needing to implement key signing themselves.
Changing the previous code snippet to update the value stored in the contract:
Similar to the method invocations in the previous section which only read contract state,
transacting methods also require a mandatory first parameter, a `*bind.TransactOpts` type,
which authorizes the transaction and potentially fine tunes it:
Similar to the method invocations in the previous section which only read contract state, transacting methods also require a mandatory first parameter, a `*bind.TransactOpts` type, which authorizes the transaction and potentially fine tunes it:
* `From`: Address of the account to invoke the method with (mandatory)
* `Signer`: Method to sign a transaction locally before broadcasting it (mandatory)
@ -487,21 +420,14 @@ which authorizes the transaction and potentially fine tunes it:
* `GasPrice`: Explicitly set the gas price to run the transaction with (optional)
* `Value`: Any funds to transfer along with the method call (optional)
The two mandatory fields are automatically set by the `bind` package if the auth options are
constructed using `bind.NewTransactor`. The nonce and gas related fields are automatically
derived by the binding if they are not set. Unset values are assumed to be zero.
The two mandatory fields are automatically set by the `bind` package if the auth options are constructed using `bind.NewTransactor`. The nonce and gas related fields are automatically derived by the binding if they are not set. Unset values are assumed to be zero.
### Pre-configured contract sessions
Reading and state modifying contract-calls require a mandatory first parameter which can
authorize and fine tune some of the internal parameters. However, most of the time the
same accounts and parameters will be used to issue many transactions, so constructing
the call/transact options individually quickly becomes unwieldy.
Reading and state modifying contract-calls require a mandatory first parameter which can authorize and fine tune some of the internal parameters. However, most of the time the same accounts and parameters will be used to issue many transactions, so constructing the call/transact options individually quickly becomes unwieldy.
To avoid this, the generator also creates specialized wrappers that can be pre-configured with
tuning and authorization parameters, allowing all the Solidity defined methods to be invoked
without needing an extra parameter.
To avoid this, the generator also creates specialized wrappers that can be pre-configured with tuning and authorization parameters, allowing all the Solidity defined methods to be invoked without needing an extra parameter.
These are named similarly to the original contract type name but suffixed with `Sessions`:
In the past, abigen allowed compilation and binding of a Solidity source file directly to a Go package in a single step.
This feature has been discontinued from [v1.10.18](https://github.com/ethereum/go-ethereum/releases/tag/v1.10.18)
onwards due to maintenance synchronization challenges with the compiler in Geth.
In the past, abigen allowed compilation and binding of a Solidity source file directly to a Go package in a single step. This feature has been discontinued from [v1.10.18](https://github.com/ethereum/go-ethereum/releases/tag/v1.10.18) onwards due to maintenance synchronization challenges with the compiler in Geth.
The compilation and binding steps can be joined together into a pipeline, for example:
The `abigen` command was made in such a way as to integrate easily into existing
Go toolchains: instead of having to remember the exact command needed to bind an Ethereum
contract into a Go project, `go generate` can handle all the fine details.
The `abigen` command was made in such a way as to integrate easily into existing Go toolchains: instead of having to remember the exact command needed to bind an Ethereum contract into a Go project, `go generate` can handle all the fine details.
Place the binding generation command into a Go source file before the package definition:
@ -545,19 +468,12 @@ Place the binding generation command into a Go source file before the package de
//go:generate abigen --sol Storage.sol --pkg main --out Storage.go
```
After which whenever the Solidity contract is modified, instead of needing to remember and
run the above command, we can simply call `go generate` on the package (or even the entire
source tree via `go generate ./...`), and it will correctly generate the new bindings for us.
After which whenever the Solidity contract is modified, instead of needing to remember and run the above command, we can simply call `go generate` on the package (or even the entire source tree via `go generate ./...`), and it will correctly generate the new bindings for us.
## Blockchain simulator
Being able to deploy and access deployed Ethereum contracts from native Go code is a powerful
feature. However, using public testnets as a backend does not lend itself well to
*automated unit testing*. Therefore, Geth also implements a *simulated blockchain*
that can be set as a backend to native contracts the same way as a live RPC backend, using the
command `backends.NewSimulatedBackend(genesisAccounts)`. The code snippet below shows how this
can be used as a backend in a Go applicatioon.
Being able to deploy and access deployed Ethereum contracts from native Go code is a powerful feature. However, using public testnets as a backend does not lend itself well to *automated unit testing*. Therefore, Geth also implements a *simulated blockchain* that can be set as a backend to native contracts the same way as a live RPC backend, using the command `backends.NewSimulatedBackend(genesisAccounts)`. The code snippet below shows how this can be used as a backend in a Go application.
```go
package main
@ -599,19 +515,7 @@ func main() {
}
```
Note, that it is not necessary to wait for a local private chain miner, or testnet miner to
integrate the currently pending transactions. To mine the next block, simply `Commit()` the simulator.
## Summary
To make interacting with Ethereum contracts easier for Go developers, Geth provides tools that generate
contract bindings automatically. This makes contract functions available in Go native applications.
To make interacting with Ethereum contracts easier for Go developers, Geth provides tools that generate contract bindings automatically. This makes contract functions available in Go native applications.
description: Introduction to reporting metrics from Geth
---
Geth includes a variety of optional metrics that can be reported to the user. However, metrics are disabled by default to save on the computational overhead for the average user. Users that choose to see more detailed metrics can enable them using the `--metrics` flag when starting Geth. Some metrics are classed as especially expensive and are only enabled when the `--metrics.expensive` flag is supplied. For example, per-packet network traffic data is considered expensive.
The goal of the Geth metrics system is that - similar to logs - arbitrary metric collections can be added to any part of the code without requiring fancy constructs to analyze them (counter variables, public interfaces, crossing over the APIs, console hooks, etc). Instead, metrics should be "updated" whenever and wherever needed and be automatically collected, surfaced through the APIs, queryable and visualizable for analysis.
Joe
2 years ago