System Implementation with D-QBFT

Author: AmpereChain Team Published: Dec 08, 2023 Updated: Dec 08, 2023

Building a Robust Public Network with D-QBFT Technology

A public network provides a configurable environment for testing, utilizing the D-QBFT (proof of authority) consensus protocol.

The D-QBFT network, as demonstrated in this tutorial, employs the block header validator selection method to manage validators. For a tutorial on how to implement the contract validator selection method, follow the steps in the example smart contract repository.

INFO: The steps within this tutorial lead to the creation of an isolated Ethereum private network. However, it's important to note that this network may not be fully protected or secure. We strongly recommend running the public network behind a properly configured firewall.

This tutorial is designed to configure a public network using D-QBFT, primarily for educational and enterprise purposes. D-QBFT necessitates the presence of four validators to achieve Byzantine fault tolerance.

Prerequisites​

Steps​

On the right-hand side of this page, you'll find a list of steps to create a public network using D-QBFT with four nodes, each of which serves as a validator.

1. Create Directories: To begin, it's essential to create a data directory for each node, as well as directories for your private network and all four nodes.

QBFT-Network/
├── Node-1
│   ├── data
├── Node-2
│   ├── data
├── Node-3
│   ├── data
└── Node-4
    ├── data

2. Create a configuration file​

The configuration file defines the QBFT genesis file and the number of node key pairs to generate.

The configuration file consists of two nested JSON nodes. The first node is the "genesis" property, which defines the QBFT genesis file. It's worth noting that the "extraData" string is generated automatically by Besu in the resulting genesis file. The second node is the "blockchain" property, which sets the number of key pairs to be generated.

Copy the following configuration file definition to a file called qbftConfigFile.json and save it in the QBFT-Network directory:

{
  "genesis": {
    "config": {
      "chainId": 1337,
      "berlinBlock": 0,
      "qbft": {
        "blockperiodseconds": 2,
        "epochlength": 30000,
        "requesttimeoutseconds": 4
      }
    },
    "nonce": "0x0",
    "timestamp": "0x58ee40ba",
    "gasLimit": "0x47b760",
    "difficulty": "0x1",
    "mixHash": "0x63746963616c2062797a616e74696e65206661756c7420746f6c6572616e6365",
    "coinbase": "0x0000000000000000000000000000000000000000",
    "alloc": {
      "fe3b557e8fb62b89f4916b721be55ceb828dbd73": {
        "privateKey": "8f2a55949038a9610f50fb23b5883af3b4ecb3c3bb792cbcefbd1542c692be63",
        "comment": "private key and this comment are ignored.  In a real chain, the private key should NOT be stored",
        "balance": "0xad78ebc5ac6200000"
      },
      "627306090abaB3A6e1400e9345bC60c78a8BEf57": {
        "privateKey": "c87509a1c067bbde78beb793e6fa76530b6382a4c0241e5e4a9ec0a0f44dc0d3",
        "comment": "private key and this comment are ignored.  In a real chain, the private key should NOT be stored",
        "balance": "90000000000000000000000"
      },
      "f17f52151EbEF6C7334FAD080c5704D77216b732": {
        "privateKey": "ae6ae8e5ccbfb04590405997ee2d52d2b330726137b875053c36d94e974d162f",
        "comment": "private key and this comment are ignored.  In a real chain, the private key should NOT be stored",
        "balance": "90000000000000000000000"
      }
    }
  },
  "blockchain": {
    "nodes": {
      "generate": true,
      "count": 4
    }
  }
}

NOTE

We recommend specifying the latest milestone when creating the genesis file for a private network. This ensures you are using the most up-to-date protocol and have access to the most recent opcodes.

DANGER

Do not use the accounts in alloc in the genesis file on Mainnet or any public network except for testing. The private keys display, which means the accounts are not secure.

3. Generate node keys and a genesis file​

In the QBFT-Network directory, generate the node key and genesis file:

besu operator generate-blockchain-config --config-file=qbftConfigFile.json --to=networkFiles --private-key-file-name=key

Besu creates the following in the networkFiles directory:

  • genesis.json - The genesis file including the extraData property specifying the four nodes are validators.

  • A directory for each node named using the node address and containing the public and private key for each node.

networkFiles/
├── genesis.json
└── keys
    ├── 0x438821c42b812fecdcea7fe8235806a412712fc0
    │   ├── key
    │   └── key.pub
    ├── 0xca9c2dfa62f4589827c0dd7dcf48259aa29f22f5
    │   ├── key
    │   └── key.pub
    ├── 0xcd5629bd37155608a0c9b28c4fd19310d53b3184
    │   ├── key
    │   └── key.pub
    └── 0xe96825c5ab8d145b9eeca1aba7ea3695e034911a
        ├── key
        └── key.pub

4. Copy the genesis file to the QBFT-Network directory​

Copy the genesis.json file to the QBFT-Network directory.

5. Copy the node private keys to the node directories​

For each node, copy the key files to the data directory for that node

QBFT-Network/
├── genesis.json
├── Node-1
│   ├── data
│   │    ├── key
│   │    ├── key.pub
├── Node-2
│   ├── data
│   │    ├── key
│   │    ├── key.pub
├── Node-3
│   ├── data
│   │    ├── key
│   │    ├── key.pub
├── Node-4
│   ├── data
│   │    ├── key
│   │    ├── key.pub

6. Start the first node as the bootnode​

In the Node-1 directory, start Node-1:

  • MacOS

  • Windows

besu --data-path=data --genesis-file=../genesis.json --rpc-http-enabled --rpc-http-api=ETH,NET,QBFT --host-allowlist="*" --rpc-http-cors-origins="all"

The command line:

When the node starts, the enode URL displays. Copy the enode URL to specify Node-1 as the bootnode in the following steps.

7. Start Node-2​

Start another terminal, change to the Node-2 directory and start Node-2 specifying the Node-1 enode URL copied when starting Node-1 as the bootnode:

  • MacOS

  • Windows

besu --data-path=data --genesis-file=../genesis.json --bootnodes=<Node-1 Enode URL> --p2p-port=30304 --rpc-http-enabled --rpc-http-api=ETH,NET,QBFT --host-allowlist="*" --rpc-http-cors-origins="all" --rpc-http-port=8546

The command line specifies:

  • The data directory for Node-2 using the --data-path option.

  • A different port to Node-1 for P2P discovery using the --p2p-port option.

  • A different port to Node-1 for HTTP JSON-RPC using the --rpc-http-port option.

  • The enode URL of Node-1 using the --bootnodes option.

  • Other options as for Node-1.

8. Start Node-3​

Start another terminal, change to the Node-3 directory and start Node-3 specifying the Node-1 enode URL copied when starting Node-1 as the bootnode:

  • MacOS

  • Windows

besu --data-path=data --genesis-file=../genesis.json --bootnodes=<Node-1 Enode URL> --p2p-port=30305 --rpc-http-enabled --rpc-http-api=ETH,NET,QBFT --host-allowlist="*" --rpc-http-cors-origins="all" --rpc-http-port=8547

The command line specifies:

  • The data directory for Node-3 using the --data-path option.

  • A different port to Node-1 and Node-2 for P2P discovery using the --p2p-port option.

  • A different port to Node-1 and Node-2 for HTTP JSON-RPC using the --rpc-http-port option.

  • The bootnode as for Node-2.

  • Other options as for Node-1.

9. Start Node-4​

Start another terminal, change to the Node-4 directory and start Node-4 specifying the Node-1 enode URL copied when starting Node-1 as the bootnode:

  • MacOS

  • Windows

besu --data-path=data --genesis-file=../genesis.json --bootnodes=<Node-1 Enode URL> --p2p-port=30306 --rpc-http-enabled --rpc-http-api=ETH,NET,QBFT --host-allowlist="*" --rpc-http-cors-origins="all" --rpc-http-port=8548

The command line specifies:

  • The data directory for Node-4 using the --data-path option.

  • A different port to Node-1, Node-2, and Node-3 for P2P discovery using the --p2p-port option.

  • A different port to Node-1, Node-2, and Node-3 for HTTP JSON-RPC using the --rpc-http-port option.

  • The bootnode as for Node-2.

  • Other options as for Node-1.

10. Confirm the private network is working​

Start another terminal, use curl to call the JSON-RPC API qbft_getvalidatorsbyblocknumber method and confirm the network has four validators:

curl -X POST --data '{"jsonrpc":"2.0","method":"qbft_getValidatorsByBlockNumber","params":["latest"], "id":1}' localhost:8545

The result displays the four validators:

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": [
    "0x73ced0bd3def2e2d9859e3bd0882683a2e6835fb",
    "0x7a175f3542ceb60bf80fb536b3f42e7a30c0a6d7",
    "0x7f6efa6e34f8c9b591a9ad4763e21b3fca31bcd6",
    "0xc64140f1c9d5bb82e54976e568ad39958c3e94be"
  ]
}

Look at the logs to confirm Besu is producing blocks:

2021-05-26 08:47:00.221+10:00 | EthScheduler-Workers-0 | INFO  | PersistBlockTask | Imported #1 / 0 tx / 0 om / 0 (0.0%) gas / (0x4ee4456536e2793523df87288fae76518089eec91c3f7e05e220f1f4d3f6f95b) in 0.016s. Peers: 4
2021-05-26 08:47:02.071+10:00 | pool-8-thread-1 | INFO  | QbftBesuControllerBuilder | Imported #2 / 0 tx / 0 pending / 0 (0.0%) gas / (0x6fc47ada7146d75f6a46911d8d4038795b0c99970bbd4ce0c6d6aa60955f66fe)
2021-05-26 08:47:04.051+10:00 | pool-8-thread-1 | INFO  | QbftBesuControllerBuilder | Imported #3 / 0 tx / 0 pending / 0 (0.0%) gas / (0x3cb663880a65103266b11a8d8631beca5c482d515ac287125aa077b2e31b80b0)
2021-05-26 08:47:06.058+10:00 | pool-8-thread-1 | INFO  | QbftBesuControllerBuilder | Produced #4 / 0 tx / 0 pending / 0 (0.0%) gas / (0xc2927915ac0c94bab5fc9acea6608455f1c857d69e97191dc2c39e4ac411817b)
2021-05-26 08:47:08.058+10:00 | pool-8-thread-1 | INFO  | QbftBesuControllerBuilder | Imported #5 / 0 tx / 0 pending / 0 (0.0%) gas / (0xba63471d62c936733add9b884f5213c3842af9f52460268e39e0666ab82f02a5)

INFO: If the key files were not copied to the correct directory in step 5, the network will not start producing blocks.

The logs for each node should indicate the public key was loaded from the data/key directory:

2021-05-26 08:43:16.592+10:00 | main | INFO  | KeyPairUtil | Loaded public key 0x931d32f1aec4e45b150ee38f3c74157a750fc53f523e63fe2b07bf3fce43a3de64587fc9aaf3736444f2e3eef0eea90be3b67d18be7b5b2b7cb2fcd670416a7e from <path-to-network>/QBFT-Network/Node-1/data/key

If the keys were not copied to the correct directory, Besu creates a key when starting up:

2021-05-26 08:43:16.592+10:00 | main | INFO  | KeyPairUtil | Generated new public key 0x1a4a2ade5ebc0a85572e2492e0cdf3e96b8928c75fa55b4425de8849850cf9b3a8cad1e27d98a3d3afac326a5e8788dbe6cc40249715c92825aebb28abe3e346 and stored it to <path-to-network>/QBFT-Network/Node-1/data/key

If a new key was created, the validator key specified in the configuration does not match the created key and the node cannot participate in creating blocks.

Next steps​

Use the D-QBFT API to remove or add validators, or import accounts to MetaMask and send transactions as described in the Quickstart tutorial.

NOTE: To add or remove nodes as validators you need the node address. The directory created for each node has the node address as the name.

Besu doesn't support private key management.

You can switch from the block header validator selection method configured here, to the contract validator selection method by updating the genesis file and configuring a transition.

Stop the nodes​

When finished using the private network, stop all nodes using ++ctrl+c++ in each terminal window.

TIP: To restart the D-QBFT network in the future, start from step 6.

*[Byzantine fault tolerant]: Ability to function correctly and reach consensus despite nodes failing or propagating incorrect information to peers

PreviousTechnical ArchitectureNextPractical Applications

Last updated