Introduction

Harmony is a blockchain designed to provide fast, secure, and scalable infrastructure. This article will cover the current state and plans for Harmony’s architecture.

RPC Full Node

RPC full nodes serve as the entry point to the Harmony network. They handle requests for querying information in the blockchain or processing transactions. In 2020, Harmony transitioned from original RPC nodes to elastic RPC clusters to handle the increasing volume of RPC requests. The transition was made to improve the performance and scalability of the network. However, now that the traffic amount has decreased, we have transitioned back to the legacy RPC nodes.

The motive for this transition is to prevent resources from being wasted and extra costs from occurring. We have fully terminated our ERPC clusters and have set up the required nodes to properly handle RPC requests. Harmony currently has three full nodes fully utilized in serving requests.

Harmony will constantly monitor the health and traffic of the RPC nodes. If more or less traffic occurs, the team will scale the nodes for the robustness of the network. Harmony’s ultimate goal is to scale the network using light clients alongside full nodes. For detailed information regarding light clients, please check out the following article:

Internal Validator Node

Leader node is responsible for constructing new blocks and broadcasting them to validators. Validator nodes verify the validity of blocks to ensure that the ones from malicious leaders do not get appended to the chain. Currently, leaders are chosen from the twenty internal validator nodes hosted by Harmony (five validator nodes per shard). Each validator node contains five BLS keys each, which totals twenty-five BLS keys per shard. These keys are the actual identity that gets selected as the leader.

As of now, only the internal validators are considered when choosing leaders for each shard. To ensure the continuity of the network, if a leader is not fully functioning, a view change will kick in and another leader will be chosen. Fully terminating all the internal validators is not viable at the moment because there will be no possible leader candidates to produce blocks. Introducing external validators as possible leader candidates is a feature Harmony is currently developing and planning to complete by Q1 of 2023. Once completed, external validators will be part of the leader rotation process, allowing the community to play a more active role in block validation.

Harmony’s plan is to fully externalize the voting slots as our end goal is a fully decentralized network. The plan is as follows:

The governance for the stages of termination is happening currently: HIP-29. If the governance is approved, two nodes will be terminated per shard, making it three nodes per shard. Each node will have one less slot making it four slots per node and twelve slots per shard. Once the change is implemented, the network will have increased external voting power.

Harmony has implemented various measures to ensure the stability and availability of the network, including the use of multiple consensus mechanisms (FBFT and EPoS), automatic failover mechanisms, and real-time monitoring and alerting systems. These already guarantee the robustness of our network, however as the network becomes more decentralized, dependency on the external validator community is bound to grow. Harmony will continue to coordinate with the community to further assure the resilience of the network.

The multiple stages will allow Harmony to cooperate with the community on finding a robust solution for a decentralized, secure network.

Concern for Centralization

Though not specific to the HIP-29 proposal, there is one concern we need to address as external voting power greatens: centralization. Increasing the external voting power may also increase the centralization of the network as larger stakeholders can possibly absorb the voting slots. A smaller number of participants with larger stakes may have more influence over the network’s decision.

Previously, a hard fork was implemented in order to place a limit of BLS keys per shard. The motivation was to prevent the likelihood of a shard being taken over by a small group of misbehaving validators. Each validator would have a limit on the number of keys they possess per shard.

Regardless, there are more measures Harmony needs to take in order to further push towards decentralization. There are several proposals, such as limiting the total number of keys per validator, limiting the number of rewards per validator, and limiting the max a “whale” can delegate per validator. Note that these are just proposals that may or may not be implemented. Regardless, Harmony will work with the community in search of the most appropriate approach to achieving decentralization. Harmony will provide further updates on addressing the issue of centralization.

Bootnode

Bootnodes are responsible for bootstrapping nodes in the network. When a new node is introduced to the network, bootnode provides a set of nodes for the new nodes to sync with. The sync would be done in a peer-to-peer manner. This will allow the new node to have the necessary data to participate in the network.

Harmony is currently hosting four bootnodes. Bootnodes are relatively cheaper than other services as lightweight instances can handle the functionality. Harmony plans on fully externalizing all services but as of now, bootnodes are a critical service that needs to be part of our network.

DNS (Sync) Node

DNS nodes are another solution for syncing nodes. They are full nodes that Harmony designates to handle syncing. If nodes do not want to sync from peers bootstrapped by bootnodes, they can sync from the DNS nodes run by Harmony. The interaction between DNS nodes and nodes requiring sync is similar to the server and client model. They do not communicate with each other. Rather, new nodes call out to DNS nodes and they simply respond with the appropriate data.

Maintaining DNS nodes is expensive due to the requirement for high bandwidth and large storage. In order to optimize our infrastructure and cost, Harmony plans to retire the DNS nodes and operate the syncing functionality in a different form: stream sync.

Stream Sync

Nodes must be in sync with the latest state of the blockchain network. For this reason, stream sync is implemented as a new method for Harmony nodes to sync their states between nodes.

Stream sync involves a set of steps that each node follows to ensure its state is up to date. First, the node finds a set of neighboring nodes during the bootstrapping process. The node uses a connection manager to manage all of these connections. Then, the node will create stream channels for each neighbor through a stream manager. Once the stream channels have been established, the node can begin to sync its state.

The node starts by asking each neighbor for their current block height through a request manager. Once the requests have been fulfilled, the node can calculate the maximum block height. With the maximum block height, the node then asks each neighbor to share a piece of their state. The sharing process continues until the node fetches all states up to the current block.

Harmony stream sync uses a unique ID to identify nodes in the same shards. Nodes in the non-beacon chain use a downloader to fetch the last blocks of each epoch from the beacon chain. These blocks are essential for the nodes in the non-beacon chain as they enable retrieval of shard states and committee lists.

The benefit of stream sync is that it provides a faster state sync time with minimal network bandwidth. The process makes the entire network more decentralized as well because nodes will not be relying on designated sets of nodes. Rather, nodes are able to sync from anyone within the same shard. Stream sync ultimately can be seen as an upgrade to not only the traditional peer-to-peer sync but also the DNS sync. It removes the dependency on DNS nodes and eliminates the high overload on the few designated syncing nodes. Stream sync is planned to release during Q1 of 2023 and once the feature is fully functioning, Harmony will retire DNS nodes.

Snapshot Node

Snapshot nodes are used to store backups of the database. Snapshot nodes repeat the process of syncing and capturing snapshots in decentralized storage, Storj. The snapshots will allow new nodes to efficiently catch up to the current state of the network.

Without snapshot nodes, a new node would have to sync all the data from other peer nodes or DNS nodes. However, due to the immense amount of data, the syncing process would take an unreasonable amount of time and put a burden on both nodes. In order to avoid this complication, new nodes would first obtain data from snapshot nodes. Once sync from the snapshot node is completed, new nodes can then sync the rest of the data from other full nodes, whether peer or DNS nodes. This would offload the amount of data for other full nodes and make the network efficient overall.

Harmony currently operates one snapshot node per shard, making it four services within our Mainnet. Snapshot nodes can only be retired if Harmony’s sync is reasonably fast. Until then, the service will be operated to ensure the convenience of the network.

Archival Node

Archival nodes contain all the data of the Harmony blockchain. Currently, a single archival node in shard 0 contains 22TB of data. Spinning up a new archival node would take approximately three weeks or more depending on the node’s connectivity. Due to its immense size, it is crucial for us to maintain a clean copy.

Harmony is currently maintaining three archival nodes. The purpose of redundancy is to prevent a single point of failure. For example, if one archival node is dysfunctional and requires a new setup, another archival will be occupied with syncing the dysfunctional one. At that stage, only one archival node will be available to handle external requests. In order to prevent network stoppage, Harmony is maintaining more than two archival nodes.

At the current state of Harmony, three archival nodes are sufficient to handle traffic and provide network robustness. If more demands are required, Harmony will scale the archival nodes to fulfill the network needs.

Conclusion

Harmony’s focus is to make substantial advancements and the appropriate changes in our infrastructure. We look forward to witnessing the impact they will bring to our network. Kindly stay tuned for more infrastructure and protocol updates!