Summary

This report discloses a discovered vulnerability within the Harmony blockchain protocol, specifically in the staking precompile implementation and handling state reverts. The vulnerability was initially observed during the synchronization of a testnet node from scratch. This process was halted due to an error named “invalid merkle root”, indicating a discrepancy between the state of the local node and the state reported by a remote node. The problematic block was identified as block 2058022.

Details

The block contained a transaction to a Yield Warrior NFT (a runner-up in the Encode Club Hackathon). Inference from the block data revealed that the Yield Warrior NFT used the staking precompile from a smart contract to participate in staking. Transactions from this block showed one transaction related to the Yield Warrior NFT that failed and reverted, while the following transaction, also associated with staking, was successful. This observation raised a hypothesis of a potential problem in the code handling state reverts.

Detailed examination pointed towards PR #4374, a significant part of the Ethereum `statedb` code to Harmony. The specific problem stemmed from a change that unintentionally modified the handling of reverts, leading to a vulnerability where state reverts were not processed correctly.

Vulnerability Proof of Concept

If the revert of the validator wrapper failed, but the state database revert succeeded, it would be possible for an actor to create delegated ONEs without an actual transfer of tokens. This could be achieved by delegating through a smart contract and then causing it to revert, costing only the transaction gas. This scenario could be exploited indefinitely if the actor had a minimum of 100 ONE (+ gas).

An example of this is illustrated below:

function exploitTest(address validatorAddress) public payable {
  delegate(validatorAddress, msg.value);
  revert();
}

This allowed the contract to retain its original balance of ONE tokens while having additional staked tokens that could be undelegated.

Fix

A patch was designed and implemented to correct this vulnerability by restoring the removed line of code from PR #4374. The patch ensured that the correct state was restored in case of reverts.

diff --git a/core/state/journal.go b/core/state/journal.go
index 210f3880..c5e3f743 100644
--- a/core/state/journal.go
+++ b/core/state/journal.go
@@ -156,7 +156,8 @@ func (v validatorWrapperChange) dirtied() *common.Address {
 }
 
  // revert undoes the changes introduced by this journal entry.
 -func (v validatorWrapperChange) revert(*DB) {
 +func (v validatorWrapperChange) revert(s *DB) {
    + s.stateValidators[*(v.address)] = v.prev
 }
 
 func (ch createObjectChange) revert(s *DB) {

Impact and Exploitation Analysis

The vulnerability was introduced with the release of version 2023.2.1. However, due to the inability to achieve consensus with this version, it was not widely deployed across the validator network. The vulnerability became more exposed when the consensus issue was resolved with the release of version 2023.2.4. There existed a potential period during which an actor, aware of the issue and possessing the technical expertise, could have garnered the necessary quorum to exploit this vulnerability.

A thorough review of transactions and the explorer revealed no evidence of exploitation. However, this conclusion is not definitive due to the lack of internal transactions in the explorer. To confirm that no exploitation occurred, a process of syncing an older version of the database using the patched binary was initiated. This process should identify instances where the vulnerability was exploited, as the patched binary will reject such blocks.

Preventive Measures

A corrective patch was developed promptly upon the discovery of the vulnerability. This patch was merged into a subsequent update (v8090-v2023.2.1–430-gc7a63bab-dirty) and deployed to all internal nodes. While planning to upgrade external nodes, the vulnerability was not disclosed publicly until all validators had completed the upgrade. An analysis of the network’s transactions showed that the network was not exploited.

Prevention of Similar Issues

Future prevention of similar issues includes submitting smaller pull requests to ensure that changes are easier to review and less prone to error. Greater reviewer involvement is also suggested to increase the likelihood of identifying potential problems. Including test cases for these specific scenarios is also recommended for any PR testing to ensure the expected behavior occurs.

Conclusion

While this incident was unfortunate, it has presented a valuable learning opportunity, contributing to the robustness and security of the Harmony protocol. Through our collective efforts, we identified and corrected this vulnerability, safeguarding our community’s and users’ interests. We continue to strive for and uphold the highest standards in all aspects of our work and sincerely appreciate the community’s continued support and trust.

Technical Overview Report — Security Incident on July 19, 2023 was originally published in Harmony on Medium, where people are continuing the conversation by highlighting and responding to this story.

Abstract

As blockchain technology advances, there is a growing demand for enhanced transaction speeds and efficiency. The Harmony Protocol, a sharded blockchain built to promote the creation and use of decentralized applications (dApps), is pioneering in this area by implementing a 1-second finality. This study provides a comprehensive analysis of this monumental leap, signaling a significant progression in the blockchain realm.

The paper explores the prevalent challenges in the blockchain environment, presenting Harmony’s unique strategies to address these as part of Harmony Improvement Proposal 30 (HIP-30). With HIP-30’s substantial amendments, Harmony is on track to achieving 1-second finality. Our primary findings suggest that these enhancements will dramatically improve transaction times, addressing longstanding scalability concerns in the blockchain sector.

We further discuss the introduction of responsiveness into blockchain protocols, examining its importance and reviewing two significant protocols: PaLa and Simplex. We delve into how their techniques can be adapted for Harmony and the potential benefits and drawbacks this could entail.

Lastly, we offer an in-depth comparison of Harmony’s performance against traditional blockchains, focusing on dApp development, user adoption, and overall blockchain scalability. The paper concludes by emphasizing the importance of these innovations in shaping the future of blockchain technology, with implications for the Harmony Protocol and the broader blockchain community seeking to improve transaction efficiency and overall network performance.

Introduction

Blockchain technology has emerged as a pivotal innovation, paving the way for decentralized solutions in various industries. At the heart of this technology lies the principle of transaction finality — the guarantee that once a transaction has been included in the blockchain, it cannot be reversed or altered. This concept is paramount for any blockchain-based system, ensuring immutability and trust within the network. Nevertheless, achieving transaction finality has been challenging for many blockchain networks due to inherent constraints in decentralized system design and the trade-off between decentralization, security, and scalability. This paper aims to delve into the technical details and the innovative strides the Harmony protocol takes to enhance transaction finality time to a groundbreaking 1-second.

Since its inception, the Harmony protocol has been a forerunner in addressing the most pressing challenges in the blockchain ecosystem. Employing a sharded blockchain design and leveraging its unique Effective Proof-of-Stake (EPoS) consensus algorithm, Harmony has already achieved a remarkable 2-second transaction finality. This impressive feat was realized by minimizing the number of communication rounds during the consensus process, reducing the time needed to confirm each transaction.

In mathematical terms, we can express the transaction finality as a function of the number of communication rounds, n, and the time required for a round-trip message, t (assuming a fully connected network). This relationship can be expressed as:

F = n ⋅ t

Where F is the transaction finality time.

In Harmony’s current state with 2-second finality, we can see that n communication rounds and t time per round work together to achieve this. However, to further reduce this finality time to 1-second, Harmony is taking a significant leap by strategically reducing the n value (communication rounds), thus decreasing the F value (finality time) while maintaining the network’s security and decentralization.

As we delve further into the technical details, it will become evident how Harmony’s innovative approach solves one of blockchain technology’s key challenges and sets a new precedent for transaction speeds in the industry. This in-depth exploration will unravel the potential implications of 1-second transaction finality for user experience, network scalability, and overall performance in the Harmony network, setting the stage for the upcoming sections of this paper.

Background and Overview of Harmony Protocol

Harmony Protocol is a cutting-edge blockchain network implementing deep sharding and novel consensus mechanisms. Utilizing sharding technology at the state, network, and transaction level, Harmony aims to overcome one of the most significant challenges in the blockchain space: scalability with security. Coupled with a consensus mechanism known as Effective Proof-of-Stake (EPoS), Harmony provides a solution for the trilemma in the blockchain space, efficiently balancing decentralization, security, and scalability.

Sharding in Harmony Protocol

Sharding technology forms the backbone of Harmony’s scalable architecture. “Sharding” refers to dividing a larger set of nodes into smaller sets, or “shards”, each capable of parallel processing transactions and smart contracts. This is analogous to splitting a larger database into smaller, more manageable parts, each operating independently in harmony.

Mathematically, we denote the number of shards as S, where each shard comprises n nodes. Therefore, the total number of nodes in the network, N, can be expressed as N = S ⋅ n. The critical advantage of sharding is its potential for linear scaling; as the network grows (i.e., N increases), the number of shards (S) can also increase, leading to more parallel processing and higher throughput.

Effective Proof-of-Stake (EPoS)

The bedrock of Harmony’s consensus mechanism, the Effective Proof-of-Stake (EPoS), actively counters the typical centralizing propensities ordinary in orthodox Proof-of-Stake systems. By implementing a staking threshold, EPoS limits the sway of wealthier nodes, thereby capping a node’s voting shares at a dynamic limit dictated by the median stake.

Within this context, the individual node’s stake is expressed as s_i and the median stake as s_median. The calculation for the effective stake, s_e(i), unfolds as follows:

The introduction of the logarithmic component within the latter equation guarantees that the effective stake does not escalate in direct proportion to the actual stake. This formula fosters a more equitable and balanced staking mechanism.

Fast Byzantine Fault Tolerance (FBFT)

The Fast Byzantine Fault Tolerance (FBFT) consensus algorithm is pivotal in Harmony’s 2-second transaction finality. FBFT is an optimized version of the Practical Byzantine Fault Tolerance (PBFT) algorithm, designed to deliver faster consensus by reducing the number of communication rounds.

In PBFT, three phases of communication are required (Pre-Prepare, Prepare, and Commit), each requiring n round-trip messages, where n is the number of validator nodes. However, FBFT optimizes this process by piggybacking the Prepare and Commit phases, reducing the communication complexity from O(n²) to O(n), achieving faster transaction finality.

The time for transaction finality F can be given by the equation mentioned in the introduction, F = n ⋅ t. By reducing n through the FBFT, Harmony effectively achieved 2-second transaction finality.

As Harmony continues to optimize its infrastructure, it is crucial to understand these fundamental building blocks to appreciate the engineering feat of a 1-second transaction finality. The upcoming sections will discuss how these elements interplay with Harmony’s transition to this unprecedented fast finality.

Proposed Shift to 1-second Finality

Harmony Protocol, leveraging its innovative technology stack, proposes an ambitious transition from the current 2-second finality to 1-second finality. This significant stride promises to optimize transaction speeds further, improving user experience and elevating Harmony’s competitive edge in the high-stake race of blockchain scalability.

Description of the Proposal

In pursuit of 1-second finality, Harmony presented the Harmony Improvement Proposal 30 (HIP-30). The HIP-30 aims to reduce the number of nodes in Shard 1 from 250 to 200 and accelerate development towards this goal. The key idea is to optimize consensus latency by minimizing the number of communication rounds needed between nodes. This strategy plays a critical role in achieving 1-second finality.

The original proposal suggested distinguishing between Shard 0 and Shard 1, with 250 nodes maintained on Shard 0 and 200 nodes hosted on Shard 1. This approach aimed to facilitate dynamic, live experiments without jeopardizing network stability. However, on careful reflection, it becomes clear that imposing a difference in the number of nodes between shards might complicate the sharding design unduly. To uphold the integrity and simplicity of the sharding architecture, it is advisable to maintain uniformity among shards regarding node count. Diverging from this could introduce unforeseen complications and problems; hence, uniformity across shards is strongly recommended.

Technological and Mathematical Changes

Achieving a 1-second block finality in Harmony involves a series of critical technological and mathematical adaptations. Chief among these is the need to curtail consensus latency significantly. We can achieve this by fine-tuning the Fast Byzantine Fault Tolerance (FBFT) algorithm to decrease the communication rounds required for transaction validation.

Traditionally, FBFT’s time complexity is encapsulated by the equation T = k ⋅ n ⋅ t. In this expression, T signifies the total time taken for consensus, k denotes the number of communication rounds, n represents the number of nodes, and t is the average latency per node. However, this model assumes sequential communication between the leader and validators, which is not always the practice case.

In a scenario where communication happens in parallel and block write times to local databases are a factor, a more representative model could be expressed as T = k ⋅ max(t, db), where db denotes the time to write a block to the local database.

To halve the total consensus time, T, for Harmony — the essence of 1-second finality — we need T_new = T/2. Consequently, this necessitates strategic optimization of k, t, and db. For instance, under the HIP-30 proposal, Harmony plans to reduce n from 250 to 200. A 20% reduction in the number of nodes could potentially lead to a comparable decrease in consensus time, edging Harmony closer to its target 1-second finality.

Next, reducing nodes per shard also means a decrease in BLS keys per validator. This necessitates a well-crafted strategy to distribute the BLS keys equitably, maintaining the unique balance of Harmony’s Effective Proof-of-Stake (EPoS) model.

Moreover, as we shift towards 1-second finality, upholding the integrity of cross-shard communications becomes paramount. With shards processing transactions in parallel, stringent coordination and synchronization between shards is essential to prevent double-spending and ensure overall consistency.

Harmony’s ambitious stride towards 1-second finality underpins its dedication to pushing the envelope of blockchain technology, heralding a new epoch of highly efficient, scalable networks. The following sections will probe deeper into the potential ramifications of this transition for Harmony Protocol and the broader blockchain ecosystem.

Comparative Analysis

A comprehensive comparative analysis offers valuable insights into the implications and potential outcomes of transitioning to a 1-second finality on the Harmony Protocol. The comparison focuses on qualitative and quantitative aspects, providing a holistic perspective on this significant shift.

Detailed Comparison between Current and Proposed Finality Times

Harmony currently operates with a 2-second finality time, already an exceptional achievement in the blockchain ecosystem. This speed is made possible by the protocol’s innovative sharding and EPoS design and utilizing FBFT for consensus. However, the move to 1-second finality promises further to amplify the network’s efficiency and user experience.

The current 2-second finality is characterized by the equation T = k ⋅ n ⋅ t with n = 250 nodes in Shard 1. Reducing n to 200 in the proposed scenario leads to a 20% reduction in the consensus time, bringing it closer to the 1-second target.

Potential Improvements and Drawbacks of 1-second Finality

The leap to 1-second finality bears significant advantages. Foremost among these is the drastic user experience enhancement facilitated by near-immediate transaction confirmations. This increase in speed paves the way for highly responsive and scalable decentralized applications (dApps), especially in the burgeoning DeFi sector, bolstering Harmony’s appeal to developers.

Further, the shorter finality time could spur a surge in transaction volume, thus amplifying network throughput. Theoretically, this change could double Harmony’s transaction processing capacity per time unit.

Nevertheless, transitioning to 1-second finality has potential challenges. For one, the reduction in node quantity may have implications for network resilience against Byzantine behavior. Accelerated consensus could also demand more from node hardware, raising the entry bar for some validators.

Moreover, the increase in block generation associated with a faster finality time is anticipated to accelerate the growth of storage space requirements. This aspect necessitates careful consideration and planning to ensure it is a manageable factor in the network’s scalability.

Quantitative Comparison of Potential Transaction Volume, Security Implications, and Other Key Metrics

Regarding security, the decreased number of nodes per shard could marginally lower resilience against Byzantine nodes. The mathematical relation f < n/3 defines the fault tolerance threshold in a Byzantine fault-tolerant system, where f is the maximum number of Byzantine nodes the system can tolerate, and n is the total number of nodes. The theoretical resilience is marginally lowered with n reducing from 250 to 200. However, Harmony’s EPoS model can mitigate this to a great extent.

The management of finality time reduction needs to be approached with precision to circumvent a rise in the rate of orphan blocks, as this could detract from network efficiency. To elaborate, orphan blocks or “stale blocks” are valid blocks that do not make it into the main blockchain, often due to simultaneous block production by miners or lag in network propagation. A high rate of these orphan blocks suggests an inefficient usage of resources since computational power is allocated to blocks that do not lengthen the chain. In a quantitative context, if we let r represent the block production rate and d symbolize network delay, it is vital to ensure r < d to keep orphan rates at bay.

The transition to a 1-second finality offers impressive potential improvements but demands meticulous design and implementation to manage potential drawbacks. The succeeding sections will examine the practical considerations of implementing this transition and the potential future Harmony Protocol developments.

Mathematical Modeling of 1-second Finality

Mathematical modeling provides the logical structure that underpins the feasibility and implications of transitioning to a 1-second finality. By representing the elements of the Harmony system and their interactions through mathematical equations, we can predict system behavior under various conditions, offering valuable insights into the potential dynamics of the proposed changes.

Introduction to the Models and Equations Used in the Paper

This paper utilizes several mathematical models to analyze and predict the behavior of the Harmony system under the proposed changes.

Consensus Time Model: The consensus time (T) in Harmony is approximated as T = k ⋅ n ⋅ t, where k is a constant, n is the number of nodes, and t is the time taken for a single communication round. Reducing n from 250 to 200 would lead to a direct reduction in consensus time.

Fault Tolerance Model: The fault tolerance threshold f in a Byzantine fault-tolerant system is defined by f < n/3. Reducing the number of nodes (n) from 250 to 200 marginally lowers the theoretical resilience.

Orphan Block Rate Model: If r is the block production rate and d is the network delay, maintaining r < d is essential to prevent high orphan rates.

Detailed Discussion of the Mathematical Modeling of the Proposed System

Our mathematical modeling represents the interactions within the Harmony system, considering the parameters and variables that influence transaction finality time, throughput, security, and efficiency.

In the Consensus Time Model, by decreasing n from 250 to 200, there is an immediate reduction in the consensus time — a significant step towards achieving the 1-second finality.

The Transaction Throughput Model predicts that with the finality time halved, the transaction throughput should theoretically double, implying that Harmony could process twice the current volume of transactions per second.

The Fault Tolerance Model suggests a marginal decrease in the network’s tolerance to Byzantine nodes. However, this potential drawback is mitigated by Harmony’s EPoS model, which punishes malicious validators by slashing their stakes.

Lastly, the Orphan Block Rate Model alerts us to the potential risk of an increased orphan block rate if the transition to 1-second finality is not appropriately managed. Reducing the finality time without appropriate adjustments to the network delay could result in a higher orphan rate, reducing the network’s overall efficiency.

Analysis of the Model Results and Their Implications

The model results illuminate the potential for impressive improvements and possible challenges that the transition to 1-second finality may bring.

The consensus time and transaction throughput models indicate a notable performance enhancement in transaction speed and volume, promising a vastly improved user experience and a more competitive protocol for DeFi applications.

Meanwhile, the fault tolerance and orphan block rate models highlight the need for meticulous design and implementation in the transition to prevent potential drawbacks, particularly concerning security and efficiency. Despite the reduced resilience against Byzantine nodes, Harmony’s EPoS model safeguards against malicious activities, ensuring the network’s robustness.

Conclusively, these models present a comprehensive overview of the network’s potential performance under 1-second finality, guiding the design and implementation process. The following section will delve into the practical considerations of executing this transition.

Responsiveness in Blockchain Protocols

The concept of responsiveness within blockchain protocols significantly enhances such systems’ performance, scalability, and efficiency. It refers to a protocol’s capability to adapt to real-world network conditions and function optimally within the network latency rather than being constrained by a predetermined, hardcoded delay limit, denoted as Δ.

In essence, responsiveness manifests as the protocol’s capability to progress within the “real network delay,” denoted as δ. This actual network delay is typically less than or equal to Δ and represents the time it takes for information to propagate through the network.

In practice, the responsiveness of a blockchain protocol has profound implications for its speed, performance, and user experience. A responsive protocol adjusts its operation to real-time network conditions, allowing it to achieve faster transaction confirmations during periods of low network latency. It ensures that the protocol functions at its maximum potential and is not limited by a maximum delay set during its design phase.

Therefore, The principle of responsiveness becomes pivotal when designing protocols that aim for high-speed transaction confirmations, such as Harmony’s proposed 1-second finality. By implementing a responsive design, Harmony can make significant strides in its quest for rapid finality without jeopardizing the security and reliability of the network. In the following subsections, we will delve deeper into this concept by examining some recently proposed responsive consensus protocols and exploring how their techniques might be incorporated into Harmony’s design.

Review of PaLa and Simplex Consensus Protocols

The concept of responsiveness in blockchain protocols is an active area of research, with numerous contributions being made in the recent past. Two notable contributions include the PaLa: A Simple Partially Synchronous Blockchain and Simplex Consensus: A Simple and Fast Consensus Protocol papers.

PaLa: A Simple Partially Synchronous Blockchain

The PaLa paper proposes a new protocol named PaLa, which operates under a partially synchronous network model. It is a simple protocol that combines ideas from Tendermint and HotStuff while aiming for high responsiveness. The protocol assumes a partially synchronous model with a known upper bound Δ on message delay, but the actual delay δ is unknown. The protocol guarantees that if the network delay is less than Δ, then the protocol completes in roughly 2δ time.

PaLa is designed to achieve two key goals. The first is safety, where the protocol guarantees that no two honest validators commit different blocks at the same height. The second is liveness, where every honest validator eventually commits some block at every height. PaLa achieves these goals by carefully structuring the voting process involving two voting steps: prevote and precommit. In essence, a validator must witness more than 2/3 of validators prevoting a block to precommit it and more than 2/3 of validators precommitting a block to commit it.

Simplex Consensus: A Simple and Fast Consensus Protocol

The Simplex Consensus study presents an innovative consensus protocol that, while maintaining the integrity of safety and liveness, streamlines the consensus process with a unique approach to consensus rounds and voting. Named Simplex, this protocol ensures network robustness more effectively and efficiently than the predecessor, PaLa.

Each round in Simplex comprises three key phases: Proposal, Soft vote, and Certify. Initially, an honest validator suggests a block during the Proposal phase. This is followed by the Soft vote phase, where the validator waits to gather a requisite number of soft votes. Lastly, the Certify phase involves gathering sufficient certifications to endorse and confirm a block.

A key feature of Simplex lies in its progressive approach within the actual network delay δ, which trumps the traditional method within the worst-case upper-bound Δ. This allows the protocol to achieve responsiveness. Moreover, Simplex introduces a pioneering mechanism that authorizes validators to securely transition to the next round when the present round seems stagnant, thereby enhancing overall efficiency.

The strategies to actualize responsiveness in the Simplex protocol involve meticulously structured voting mechanisms and transitions between rounds. Validators make choices based on the input from a supermajority of other validators. This way, the protocol’s progress is intimately linked to the actual network delay instead of a pessimistic estimate. Such approaches are crucial when optimizing transaction finality times, especially in protocols like Harmony.

Potential Implementation in Harmony Protocol

The methodologies from both PaLa and Simplex protocols offer intriguing possibilities for Harmony Protocol in its quest for achieving a 1-second finality. Adapting these techniques, Harmony can redesign its consensus algorithm to be responsive, offering increased performance.

Implementation Considerations

First, Harmony’s current use of FBFT can be enhanced by incorporating some principles from PaLa and Simplex protocols. The voting mechanism in PaLa, which involves two steps — prevote and precommit — could provide a blueprint. Similarly, the Proposal, Soft vote, and Certify phases from Simplex could be implemented within the FBFT algorithm. However, this would require careful redesign to avoid creating new vulnerabilities or disrupting existing efficiencies.

The quest for improved performance in the Harmony protocol, particularly during periods of high transactional load or spam attacks, is not without its hurdles. One such challenge is the time it takes to insert a block into the local database — a process that has historically constituted a notable system bottleneck. However, the solution we once implemented, a local sharding database strategy, did not result in sufficient performance gains to justify its operational overhead, especially when faced with a full disk.

This approach, which involved data splitting across different disks to distribute the load and decrease insertion time, brought about performance improvements of only 10–20%. Although available, this feature currently needs to be used due to its inadequate efficiency-cost trade-off.

However, it is worth mentioning that obstacles like these can provide valuable insights into the network’s operational efficiency. This specific challenge and others guide our ongoing exploration of strategies like implementing a responsive consensus algorithm (such as PaLa or Simplex). As we persist in this endeavor, we inch closer to Harmony’s aspiration of realizing 1-second finality and amplifying our transaction processing capability twofold.

While both protocols operate under the assumption of a partially synchronous model, Harmony must identify the most suitable delay bounds for its network. It must also redesign its slot allocation and leader selection mechanisms to align with the updated consensus protocol.

Potential Benefits and Drawbacks

The primary benefit of adopting the responsive consensus approach is the potential to achieve a faster block finality time — possibly reaching the desired 1-second finality. This could drastically improve transaction throughput and overall network performance.

Additionally, in the context of Harmony’s Effective Proof-of-Stake (EPoS) and shard chains, responsiveness might enhance the robustness of the consensus algorithm. A protocol that can adjust to actual network delays could better handle fluctuations in network conditions, thereby maintaining high levels of performance and reliability.

Transitioning to a responsive protocol poses challenges. This would require an extensive redesign of Harmony’s consensus mechanism. Any changes to this critical aspect of the network should be cautious to avoid new security vulnerabilities or unintended consequences. Theoretical benefits must be empirically validated.

Incorporating the strategies from PaLa and Simplex into Harmony Protocol could catalyze significant enhancements in transaction finality speed. However, such an integration warrants rigorous exploration and a comprehensive understanding of its potential impacts before application. Despite the potential advantages, the necessity of these implementations comes into question considering empirical evidence from the Harmony network. Historical data indicates that shards 1, 2, and 3 successfully managed 1-second finality without any need for node reduction in the past. Thus, Harmony could attain its objective of 1-second finality without necessitating the full adoption of PaLa and Simplex, underscoring the importance of this research line in Harmony’s ongoing evolution.

Further Investigation into Responsive Protocols

The promising results of incorporating responsiveness in consensus protocols, as evidenced by PaLa and Simplex, warrant deeper investigation. Further exploration into the mechanisms underlying these protocols and a comprehensive understanding of their strengths and weaknesses will provide the foundation necessary for the proposed adaptation.

Modifying the Harmony Codebase for Reduced Block Time

As Harmony aims to optimize its network for faster transaction finality, it is crucial to delve into the codebase to understand where key parameters governing block times are set and how they can be adapted. This exploration is a fundamental step toward transforming the theoretical foundations of PaLa and Simplex into practical implementations on the Harmony network.

In Harmony’s codebase, specifically within the consensus_service.go file, one can locate the BlockPeriod field in the consensus object. This field indicates the time interval between consecutive block generations.

consensus.BlockPeriod = 5 * time.Second

This line sets the block period to five seconds. However, a conditional statement later adjusts this interval for certain epochs.

// Enable 2s block time at the twoSecondsEpoch
if consensus.Blockchain().Config().IsTwoSeconds(nextEpoch) {
  consensus.BlockPeriod = 2 * time.Second
}

This conditional checks whether the next epoch is configured for two-second blocks and, if so, changes the BlockPeriod to two seconds.

While this approach provides a straightforward way to manage the block time, it inherently restricts the system’s flexibility. Any adjustments to the block time currently necessitate code changes and subsequent deployment across the network.

To drive Harmony toward achieving a 1-second block time, the block time setting in the codebase must be carefully revised. This requires a comprehensive mathematical model and rigorous testing to validate the robustness of the network under the proposed changes. One potential modification could be to introduce a condition for one-second block periods.

// Enable 1s block time at the oneSecondEpoch
if consensus.Blockchain().Config().IsOneSecond(nextEpoch) {
  consensus.BlockPeriod = 1 * time.Second
}

This modification suggests a conditional epoch where one-second block periods become active. However, such a transition must be cautiously implemented, as shorter block times could introduce additional strain on the network. Challenges such as increased load, network congestion, and a potential rise in orphaned blocks must be addressed during this transition. Ensuring the network’s stability and performance under these circumstances will be critical in Harmony’s journey toward 1-second finality.

Implications for Blockchain Technology

The advent of 1-second finality in the Harmony protocol is set to reverberate throughout the blockchain ecosystem. While it is a significant milestone for Harmony itself, the ripple effects will likely inspire a wave of advancements in blockchain technology.

First and foremost, the possibility of 1-second finality would serve as a beacon for scalability in the blockchain ecosystem. With this achievement, Harmony would double its transaction throughput, thereby substantially increasing the potential transaction volume that can be handled per unit of time. This dramatic improvement in scalability would be a boon for decentralized applications, particularly in the DeFi sector, where transaction speed and volume are critical.

The impact on security, while nuanced, is also worth noting. Though fewer nodes might marginally decrease the network’s theoretical fault tolerance, implementing this transition with Harmony’s robust EPoS model safeguards against malicious activities. Therefore, this model can serve as a blueprint for other blockchain networks looking to enhance their performance while maintaining high-security standards.

Decentralization, a core tenet of blockchain technology, would not be compromised under this transition. The number of nodes would still be substantial to ensure a decentralized network and the democratic ethos of the Harmony ecosystem would continue to thrive.

Conclusion

The adoption of 1-second finality, as detailed in this paper, signifies a landmark moment for Harmony and the broader landscape of blockchain technology. Our mathematical models indicate a significant increase in transaction speed and capacity, alluding to a superior user experience and an enhanced protocol well-suited for high-volume applications, particularly in the sphere of DeFi.

Despite the potential benefits, the journey to 1-second finality requires careful management to address potential security and efficiency implications. Our mathematical analysis shows that reducing the number of nodes could slightly lower the network’s resilience to Byzantine behaviors. However, the protection offered by Harmony’s EPoS model against such adversarial actions emphasizes the network’s robustness.

This study paves the way for future research in various domains. For example, a deeper evaluation of the EPoS model under the newly proposed conditions is warranted. Further, a comprehensive exploration of 1-second finality’s impact on factors such as network latency, computational overhead, and resource utilization could yield significant insights. Lastly, empirical testing and evaluation of the proposed system will provide practical data to corroborate our theoretical conclusions.

As explored in this paper, the progression toward 1-second finality within the Harmony protocol holds immense potential. It exemplifies Harmony’s dedication to ongoing advancement and innovation, continually redefining the limits of what is feasible in the blockchain ecosystem.

Crafting Swift Consensus: The Journey to 1-Second Finality on Harmony was originally published in Harmony on Medium, where people are continuing the conversation by highlighting and responding to this story.

Harmony’s unique consensus mechanism, Effective Proof-of-Stake (EPoS), already provides an innovative approach to scalability and security. However, the nature of technology mandates continuous improvement to maintain security, efficiency, and competitiveness. This necessitates exploring new methods of consensus management, such as Secret Leader Rotation.

Leader-based consensus mechanisms are common in many blockchain networks where a specific node or participant is chosen to propose a block. However, these mechanisms are often publicly known, making them vulnerable to targeted attacks. Secret Leader Rotation is an emerging solution that aims to tackle this issue, which is already being researched in Ethereum’s ecosystem. Through this strategy, a leader is chosen secretly, ensuring that only the selected participant knows their role in advance. This adds a layer of obscurity and can prevent certain kinds of attacks, improving the security and stability of the network.

Harmony Protocol incorporates Secret Leader Rotation, adding complexity. External leaders can introduce new variables and potential vulnerabilities into the system. Understanding and navigating these new dynamics could improve Harmony Protocol’s consensus mechanism and blockchain field.

In the following sections, we will delve into the current state of Harmony Protocol’s consensus mechanism, understand Secret Leader Rotation in-depth, explore its potential challenges and benefits in Harmony Protocol, and propose a suitable strategy for its implementation. The goal is to offer an advanced and comprehensive exploration of Secret Leader Rotation within the context of the Harmony Protocol and contribute to the broader discussion on improving security and efficiency in blockchain networks.

Background

Blockchain technology is built on the principle of decentralization, where decisions are made collectively by the participants in the network, typically through a process known as consensus. One particular challenge in consensus mechanisms is the process of leader selection — choosing a specific node or participant to propose the next block. This process, however, can be susceptible to various security vulnerabilities.

In many blockchains, including Harmony Protocol, this process is public, meaning the chosen leader is known ahead of time. While this approach offers advantages regarding transparency and predictability, it also introduces potential vulnerabilities. A malicious actor could target the known leader with a Denial-of-Service (DoS) attack, preventing them from proposing the block and potentially disrupting the network’s operations.

With its Effective Proof-of-Stake (EPoS) consensus mechanism, Harmony Protocol has shown significant promise in delivering high scalability and security in the blockchain space. However, with the protocol’s plan to incorporate external leaders into the rotation, managing leader selection while minimizing vulnerabilities becomes even more critical.

This is where the concept of Secret Leader Rotation comes into play. Originating from the Ethereum network’s research, Secret Leader Rotation aims to add a layer of security by making the selection process private. In this setup, the chosen leader is unknown until they propose a block. This approach can deter targeted attacks.

Understanding the nuances of Secret Leader Rotation, its benefits, and potential challenges is an essential prerequisite for evaluating its implementation in Harmony Protocol’s existing consensus mechanism. Especially considering the incorporation of external leaders, it is crucial to comprehend how this could impact network dynamics and security.

In the following sections, we will explore the technicalities of Secret Leader Rotation, discuss its potential implications for Harmony Protocol, and offer insights into its incorporation in a network with external leader rotation.

Current State of Leader Rotation in Harmony Protocol

Harmony Protocol is renowned for its unique approach to consensus through its Effective Proof-of-Stake (EPoS) model, allowing it to balance decentralization, security, and scalability. The crux of this mechanism lies in its leader rotation method, which significantly contributes to the protocol’s robust security and speed.

In Harmony, validators are selected based on the number of tokens staked and sorted by their stake amount. These validators then participate in the consensus process; leaders are elected to propose the next block. This process is public and known beforehand, meaning all participants can identify the next leader in the network.

Harmony implements a Fast Byzantine Fault Tolerance (FBFT) consensus mechanism, a traditional Practical Byzantine Fault Tolerance (PBFT) variant. In FBFT, a leader is elected from the validator set and is responsible for proposing the next block in the blockchain. The other nodes in the network then validate the proposed block, and if it receives more than two-thirds of the vote, the block is added to the chain.

One aspect that sets Harmony apart is its use of Verifiable Random Functions (VRF) for leader selection, ensuring that the process is random and unbiased. However, as Harmony plans to allow external leaders into the rotation, the public nature of this process raises concerns about potential vulnerabilities, notably the risk of targeted DoS attacks.

Harmony employs a function f: V -> S, which, while secure, allows external observers to predict with some degree of certainty the validator-slot assignments. We represent this as a function g: S -> V, where g is a statistical model used to predict f.

The possibility of such attacks disrupting Harmony’s network operations underscores the importance of exploring new methods to enhance the security of the leader rotation process. This is where Secret Leader Rotation, a concept emerging from Ethereum’s research, holds potential. In the following sections, we will delve into the concept of Secret Leader Rotation, assess its potential in the context of Harmony Protocol, and provide a comprehensive exploration of its potential integration into Harmony’s consensus mechanism.

Introduction to Secret Leader Rotation

Secret Leader Rotation is a groundbreaking concept that aims to enhance the security of blockchain networks. This methodology primarily addresses the vulnerability of consensus mechanisms where the identity of block proposers is publicly known ahead of time. With Secret Leader Rotation, the leader’s identity (or block proposer) remains hidden until the block is proposed, significantly reducing the window for targeted attacks.

This ingenious approach was initially discussed in the Ethereum community as part of ongoing research into securing Proof-of-Stake (PoS) networks. However, Secret Leader Rotation’s fundamental principles broadly apply to other consensus mechanisms and blockchain networks.

Secret Leader Rotation utilizes cryptographic procedures to maintain secrecy. Each validator submits a commitment to a secret. These commitments are shuffled and re-randomized. Only validators who submitted the selected commitment can identify it.

For Ethereum, one particular realization of this idea is called Single Secret Leader Election (SSLE), implemented in a Whisk protocol. However, the flexibility of the fundamental idea allows for adaptations to various blockchain environments.

Let V be the set of validators where V = {v1, v2, v3, …, vn}. A function f: V -> S maps each validator to a slot in the upcoming block. The function is cryptographically secure, so only the selected validator can identify their assigned slot.

In the context of the Harmony Protocol, Secret Leader Rotation offers potential solutions to enhance network security, especially with the planned inclusion of external leaders into the rotation. In the following sections, we will delve deeper into the potential integration of Secret Leader Rotation in Harmony Protocol, evaluate its benefits, and analyze potential challenges that may arise.

Challenges in Implementing Secret Leader Rotation in Harmony Protocol

While Secret Leader Rotation offers significant advantages in enhancing network security, its implementation in the Harmony Protocol is challenging. This section discusses the potential obstacles that need to be considered.

1. Adaptation of Cryptographic Procedures: The core of Secret Leader Rotation is the cryptographic procedures that maintain the secrecy of the block proposer’s identity. Harmony currently uses Verifiable Random Functions (VRF) for leader selection. Integrating or adapting these cryptographic procedures to maintain compatibility with Harmony’s existing infrastructure may pose a challenge.
2. Consensus Mechanism Compatibility: Harmony’s consensus is based on Fast Byzantine Fault Tolerance (FBFT), a PBFT variant. Implementing Secret Leader Rotation requires thorough investigation to ensure it can work seamlessly with FBFT without compromising the efficiency and security of the network.
3. Performance Considerations: The introduction of Secret Leader Rotation may affect the performance of the Harmony network. This includes factors like network latency, speed of block validation, and overall transaction throughput. Extensive testing is required to ensure that performance is not negatively impacted.
4. Transitioning Complexity: If Secret Leader Rotation is to be implemented, Harmony would need to transition from the current state to a new one. This process might be complex and disruptive. Furthermore, testing this new system to ensure it functions as expected while maintaining Harmony’s operational performance could be a considerable challenge.
5. Potential for Increased Complexity in Staking: One of the impacts of Secret Leader Rotation might be the increased complexity for validators regarding staking and rewards. Given that the staking model is an integral part of Harmony’s design, changing the leader rotation could add layers of complexity that may impact the staking strategies of validators.

While substantial, these challenges do not necessarily pose insurmountable barriers to implementing Secret Leader Rotation in Harmony Protocol. Instead, they highlight areas needing careful research, planning, and testing in the implementation process. In the following section, we explore potential strategies for overcoming these challenges.

Adopting Secret Leader Rotation in Harmony Protocol

Given the outlined challenges, adopting Secret Leader Rotation within Harmony requires a strategic and carefully planned approach. The following steps outline how Harmony might implement this concept, incorporating solutions to potential challenges:

1. Theoretical Analysis and Compatibility Check: Conduct a thorough analysis of Secret Leader Rotation and its compatibility with Harmony’s consensus mechanism, FBFT. Assess how it can be integrated without disrupting the current performance and security of the network. This analysis should also account for Harmony’s unique sharding model.
2. Design and Development of Adapted Cryptographic Procedures: Develop a specific set of cryptographic procedures to support Secret Leader Rotation that aligns with the existing Harmony protocol. This step must account for Harmony’s current use of VRF in leader selection, finding a way to retain the randomness while incorporating secrecy.
3. Test Network Implementation and Performance Assessment: Implement the newly developed procedures within a test network to evaluate performance. The focus here should be on maintaining, if not improving, the existing network latency, speed of block validation, and overall transaction throughput.
4. Staking Model Assessment: Analyze how Secret Leader Rotation impacts Harmony’s staking model. Detailed studies on the impact of validator rewards and the overall economic model should be conducted to ensure the continued fairness and attractiveness of Harmony’s staking ecosystem.
5. Gradual Deployment and Community Engagement: Assuming successful testing and economic evaluation, plan a phased deployment of the Secret Leader Rotation mechanism. Keeping the Harmony community, particularly validators, informed and engaged throughout this process will be critical to a successful transition.
6. Continued Evaluation and Iteration: Even post-deployment, it is crucial to continue monitoring the performance and security of the network. Regular evaluation allows for the timely detection and resolution of potential issues and provides data to iterate and further improve the system.

By following this roadmap, Harmony Protocol can strategically incorporate Secret Leader Rotation into its consensus mechanism, enhancing its security, resilience, and overall network performance.

Simulation and Results

In this section of the research paper, we will simulate the implementation of the Secret Leader Rotation in Harmony Protocol’s consensus mechanism to ascertain its feasibility, efficiency, and security. The simulation will be categorized into three main areas:

1. Network Performance: Simulate the effects of the Secret Leader Rotation on Harmony’s network performance. Key metrics to be evaluated here include block proposal time, block confirmation time, overall transaction speed, and network latency.
2. Security Enhancement: The simulation must consider the impact on network security, especially regarding resistance to DoS attacks and potential manipulations of the leader selection process. This part will involve simulated attack scenarios to determine the robustness of the secret leader rotation mechanism.
3. Staking Model Impact: Analyze the effect of the secret leader rotation on the incentives for validators in Harmony’s staking model. This should simulate various scenarios of validator behavior to assess if the secret leader rotation mechanism affects their rewards, stake, and overall willingness to participate in the protocol.

The results from these simulations will provide an understanding of how the Secret Leader Rotation affects Harmony Protocol in practical terms, providing valuable insights that guide decision-making processes for potential implementation. Comparisons to the current state of Harmony Protocol and other blockchain platforms using similar or alternative mechanisms would be beneficial in illustrating the relative advantages and potential drawbacks of adopting Secret Leader Rotation.

Through a set of simulations, we will demonstrate the effectiveness of our modified function f’ in preventing predictive modeling. We can measure this using a function h: G -> R where G is the set of all possible predictive models g and R is a measure of the model’s accuracy. If our modified function f’ is practical, then for all g in G, h(g) should be close to 0.5 (i.e., equivalent to a random guess).

The results analysis will also highlight areas requiring further optimization, providing a roadmap for continued research and development. It will be crucial to interpret these results considering the potential variations in real-world conditions compared to simulation environments.

Potential Impact and Benefits of Secret Leader Rotation in Harmony Protocol

Integrating the Secret Leader Rotation into Harmony’s consensus mechanism could significantly enhance the protocol. Below are some potential impacts and benefits that could be realized:

1. Enhanced Network Security: By adopting Secret Leader Rotation, the Harmony Protocol could significantly decrease the vulnerability of block proposers to DoS attacks and enhance network security. The secret selection of block proposers obscures potential attack vectors, adding an extra layer of defense to the protocol.
2. Fair Validator Participation: With Secret Leader Rotation, every validator has a random, independent chance of being chosen to propose a block without revealing this information ahead of time. This could result in more balanced and fair participation among validators, as the selection process is random and confidential.
3. Scalability and Performance: While the impact on scalability and performance needs to be verified by thorough testing and simulation, it is plausible that Secret Leader Rotation could improve network performance. The network’s efficiency could be enhanced by ensuring that only one proposer is chosen for each slot and minimizing the likelihood of conflicting proposals.
4. Decentralization and Equality: Implementing Secret Leader Rotation could contribute to greater decentralization within the network. The mechanism supports a more equitable distribution of block proposals and rewards by preventing more extensive, sophisticated validators from gaining a disproportionate advantage (as they may in a scenario where upcoming validators are publicly known).
5. Attractiveness to Validators: Given the added security and fairness in validator selection, Harmony Protocol might become more attractive to potential validators. This could increase the network’s security further and stimulate greater community participation.

If successful, applying f’ would significantly increase the security of the Harmony protocol. We can represent this with a function j: F -> I where F is the set of all assignment functions (including f and f’), and I is a measure of the protocol’s immunity to DoS attacks. It is expected that j(f’) > j(f), indicating that the Harmony protocol with secret leader rotation is more secure.

While these potential benefits are promising, they must be weighed against the challenges and costs of implementing Secret Leader Rotation. Therefore, a balanced perspective considering the potential benefits and drawbacks is crucial in assessing this mechanism’s feasibility in the Harmony Protocol.

Conclusion

Integrating Secret Leader Rotation into Harmony Protocol presents a compelling approach to bolstering network security, enhancing decentralization, and fostering fairer validator participation. The cryptographic secrecy and randomness of the process shield validators from targeted attacks, thereby reinforcing the overall resilience of the network.

Moreover, by ensuring every validator has an equal opportunity to propose a block without advance public disclosure, Secret Leader Rotation could contribute to the democratization of the network. This could bolster the network’s appeal to potential validators, fostering a broader and more robust community of participants.

While the benefits are promising, it is crucial to address the potential challenges posed by the implementation. As discussed, the practicality of introducing Secret Leader Rotation will hinge on various factors, including the Harmony Protocol’s existing architecture, potential modifications to the staking model, and the capacity to manage potential security risks associated with secret validator selection.

Through extensive simulation and testing, we can gain empirical insights into the feasibility and implications of adopting Secret Leader Rotation in the Harmony Protocol through extensive simulation and testing. The results of these simulations, coupled with continuous research and development, will be vital in steering the future direction of Harmony’s consensus mechanism. In the evolving landscape of blockchain technology, such innovations will be crucial to maintaining Harmony’s competitive edge, ensuring the security of its network, and promoting an inclusive and equitable ecosystem.

Fortifying the Harmony Protocol: Exploring the Potential of Secret Leader Rotation was originally published in Harmony on Medium, where people are continuing the conversation by highlighting and responding to this story.

Overview

Consensus protocols in a blockchain network are an essential procedure for the participants to reach a common agreement about the state of the network. Blockchain is a distributed system, thus there might be times when the consensus mechanism does not operate in the desired manner due to faulty participants.

As a network decentralizes and externalizes to a greater degree, these unexpected problems become more prevalent. In order to prevent the network from halting, a mechanism called view change is a necessity. In this article, we will explore Harmony’s implementation of the view change mechanism.

Background

Harmony employs Fast Byzantine Fault Tolerance (FBFT), an enhanced version of Practical Byzantine Fault Tolerance (PBFT), as its consensus protocol. The protocol consists of a leader who proposes a block with a batch of transactions and validators who, as the name implies, validate the legitimacy of the block. The protocol is composed of two modes: normal and view change.

In the normal mode, there is a series of steps in which a block is produced, validated, and committed by each participant (for an explained detail of the process, refer to Harmony’s Technical Whitepaper). The other mode is the view change mode. This mode is triggered by the validators if a block is not received within a designated timeframe or the block received is an invalid one. Regardless of the underlying factor, view change is initiated whenever the next block is unable to be committed. This is to prevent consensus congestion within the network. In other words, the view change mechanism guarantees the “liveness” of the network.

In the context of blockchain, liveness is a fundamental property that ensures that the network operates and makes progress ceaselessly. It refers to the requirement that the blockchain continues to grow without getting stuck or becoming stagnant. This essentially means that new blocks must be continuously produced and committed to the chain, allowing the chain’s height to increase and ultimately, the network to reach consensus on the latest state of the ledger. These properties are all guaranteed by view change.

Harmony’s Implementation

Every time a new view is started, whether it be from the previous committed block or a view change, a timer for the respective mode is started. The timers are stored in the Consensus struct as the consensusTimeout.

type Consensus struct {
  ....
  // 2 types of timeouts: normal and viewchange
  consensusTimeout map[TimeoutType]*utils.Timeout
  ....
} 

Validators: Starting a View Change

Within consensus_v2.go, there exists a Start() method for the Consensus struct that runs the consensus logic and waits for the next block. Within 27 seconds (Harmony’s view timeout is set to 27 seconds), if the respective process is not completed and the corresponding timer times out, the startViewChange method is triggered as shown below.

if k != timeoutViewChange {
     consensus.getLogger().Warn().Msg("[ConsensusMainLoop] Ops Consensus Timeout!!!")
     consensus.startViewChange()
     break
 } else {
     consensus.getLogger().Warn().Msg("[ConsensusMainLoop] Ops View Change Timeout!!!")
     consensus.startViewChange()
     break
 }

The process for startViewChange looks as follows:

consensus.consensusTimeout[timeoutConsensus].Stop()
consensus.consensusTimeout[timeoutBootstrap].Stop()
consensus.current.SetMode(ViewChanging)
nextViewID, duration := consensus.getNextViewID()
consensus.setViewChangingID(nextViewID)
// ...
consensus.LeaderPubKey = consensus.getNextLeaderKey(nextViewID)
// ...
consensus.consensusTimeout[timeoutViewChange].SetDuration(duration)
defer consensus.consensusTimeout[timeoutViewChange].Start()

All timers nonrelevant to view change are stopped. Then, the required values, such as the mode, next view ID, and the next leader, are configured. The duration for the current view change is configured as well and we defer the timer from being started at the end of the logic (in Golang, the defer command defers the execution of a function until the surrounding function returns). This ensures that the view change timer starts only after everything has been prepared.

members := consensus.Decider.Participants()
consensus.vc.AddViewIDKeyIfNotExist(nextViewID, members)


// init my own payload
if err := consensus.vc.InitPayload(
   consensus.FBFTLog,
   nextViewID,
   consensus.getBlockNum(),
   consensus.priKey,
   members); err != nil {
   consensus.getLogger().Error().Err(err).Msg("[startViewChange] Init Payload Error")
}

Now the participants, or the members of the current validators, are added to the current viewChange struct. With all the configurations for the next view, along with the members, the payload to be included for starting the view change is initialized.

// for view change, send separate view change per public key
// do not do multi-sign of view change message
for _, key := range consensus.priKey {
   if !consensus.isValidatorInCommittee(key.Pub.Bytes) {
       continue
   }
   msgToSend := consensus.constructViewChangeMessage(&key)
   if err := consensus.msgSender.SendWithRetry(
       consensus.getBlockNum(),
       msg_pb.MessageType_VIEWCHANGE,
       []nodeconfig.GroupID{
           nodeconfig.NewGroupIDByShardID(nodeconfig.ShardID(consensus.ShardID))},
       p2p.ConstructMessage(msgToSend),
   ); err != nil {
       consensus.getLogger().Err(err).
           Msg("[startViewChange] could not send out the ViewChange message")
   }
}

Then, the node constructs the view change message for all the public keys it hosts and sends it to the relevant group within the shard where the node is located. Once these logics have been triggered, assuming no error has occurred, the deferred logic for starting the view change timer is triggered. At this point, the initiation of the view change process is completed.

Note that since the view change timer has been started, the new view message from the new leader must be received within 27 seconds. If the new leader fails to do so or sends an invalid message, another view change will be started following the abovementioned logic. However, this time, with an incremented view ID.

New Leader: Starting a New View

First, a list of sanity checks is performed to: ensure that only the valid leader performs the logic, prevent the logic from running again if the quorum has been achieved, and verify that the received message is a valid one. Once all sanity checks are passed, the following logic is executed:

// already checked the length of SenderPubkeys in onViewChangeSanityCheck
senderKey := recvMsg.SenderPubkeys[0]


// update the dictionary key if the viewID is first time received
members := consensus.Decider.Participants()
consensus.vc.AddViewIDKeyIfNotExist(recvMsg.ViewID, members)


// do it once only per viewID/Leader
if err := consensus.vc.InitPayload(consensus.FBFTLog,
   recvMsg.ViewID,
   recvMsg.BlockNum,
   consensus.priKey,
   members); err != nil {
   consensus.getLogger().Error().Err(err).Msg("[onViewChange] Init Payload Error")
   return
}

err = consensus.vc.ProcessViewChangeMsg(consensus.FBFTLog, consensus.Decider, recvMsg)
if err != nil {
   consensus.getLogger().Error().Err(err).
       Uint64("viewID", recvMsg.ViewID).
       Uint64("blockNum", recvMsg.BlockNum).
       Str("msgSender", senderKey.Bytes.Hex()).
       Msg("[onViewChange] process View Change message error")
   return
}

If the received message is the very first one, the new leader needs to update its viewChange struct with the update viewID and the members. Once the struct has been updated, the payload is initialized with the current configuration. Then, the received view change message is processed.

// received enough view change messages, change state to normal consensus
if consensus.Decider.IsQuorumAchievedByMask(consensus.vc.GetViewIDBitmap(recvMsg.ViewID)) && consensus.isViewChangingMode() {
   // no previous prepared message, go straight to normal mode
   // and start proposing new block
   if consensus.vc.IsM1PayloadEmpty() {
       if err := consensus.startNewView(recvMsg.ViewID, newLeaderPriKey, true); err != nil {
           consensus.getLogger().Error().Err(err).Msg("[onViewChange] startNewView failed")
           return
       }
       go consensus.ReadySignal(SyncProposal)
       return
   }


   payload := consensus.vc.GetM1Payload()
   if err := consensus.selfCommit(payload); err != nil {
       consensus.getLogger().Error().Err(err).Msg("[onViewChange] self commit failed")
       return
   }
   if err := consensus.startNewView(recvMsg.ViewID, newLeaderPriKey, false); err != nil {
       consensus.getLogger().Error().Err(err).Msg("[onViewChange] startNewView failed")
       return
   }
}

If the quorum has been achieved after processing the current message, the leader can transition from the ViewChange to the Normal mode. However, in doing so, the leader needs to notify the validators that it has completed the view change process with a new view change message. The message is constructed and broadcast only after ensuring that the payload has been initialized and committed. The transition and broadcasting logic are fulfilled in the startNewView method as shown below.

if !consensus.isViewChangingMode() {
     return errors.New("not in view changing mode anymore")
 }


 msgToSend := consensus.constructNewViewMessage(
     viewID, newLeaderPriKey,
 )
 if msgToSend == nil {
     return errors.New("failed to construct NewView message")
 }


 if err := consensus.msgSender.SendWithRetry(
     consensus.getBlockNum(),
     msg_pb.MessageType_NEWVIEW,
     []nodeconfig.GroupID{
         nodeconfig.NewGroupIDByShardID(nodeconfig.ShardID(consensus.ShardID))},
     p2p.ConstructMessage(msgToSend),
 ); err != nil {
     return errors.New("failed to send out the NewView message")
 }
 consensus.getLogger().Info().
     Str("myKey", newLeaderPriKey.Pub.Bytes.Hex()).
     Hex("M1Payload", consensus.vc.GetM1Payload()).
     Msg("[startNewView] Sent NewView Messge")


 consensus.msgSender.StopRetry(msg_pb.MessageType_VIEWCHANGE)

First, the mode is checked to ensure that we are still in the appropriate mode. Then the leader constructs a new view message that will eventually be broadcasted with retries. The retry guarantees a successful message relay by attempting to broadcast it multiple times in case of failure. Once the message has been successfully sent out, the retry is halted.

consensus.current.SetMode(Normal)
consensus.consensusTimeout[timeoutViewChange].Stop()
consensus.setViewIDs(viewID)
consensus.resetViewChangeState()
consensus.consensusTimeout[timeoutConsensus].Start()


consensus.getLogger().Info().
   Uint64("viewID", viewID).
   Str("myKey", newLeaderPriKey.Pub.Bytes.Hex()).
   Msg("[startNewView] viewChange stopped. I am the New Leader")

if reset {
   consensus.resetState()
}
consensus.setLeaderPubKey(newLeaderPriKey.Pub)

The mode is changed back to Normal and the view change timer is stopped. The viewID is updated and the view change state is reset as it has been completed. The normal timer is started and the node’s public key is set as the public key of the current view’s leader. With this, view change is completed and the normal mode is started with the new leader proposing blocks.

Proof of Test

View change has been part of the Harmony protocol since the very beginning. In the following section, we will initiate a view change within our Testnet to observe the real-time logic in action. Let’s dive into this process.

First, we will have to find out who the leader is. This can be done using the following command:

curl -d '{
    "jsonrpc": "2.0",
    "method": "hmy_getLeader",
    "params": [],
    "id": 1
}' -H 'Content-Type:application/json' -X POST 'https://api.s0.b.hmny.io'

The output looks as follows:

{
   "id" : 1,
   "jsonrpc" : "2.0",
   "result" : "one1yc06ghr2p8xnl2380kpfayweguuhxdtupkhqzw"
}

The public key of the current leader shows as one1yc06ghr2p8xnl2380kpfayweguuhxdtupkhqzw. We will need to connect to this instance to proceed with the view change test.

The protocol’s current leader rotation logic is hard coded amongst the internal validators’ public keys. The leader is chosen only from those candidates (the leader candidates can be checked out in the protocol repository).

The intention was to establish the network’s foundation initially through internal validators. Once the network attained a sufficient level of stability and resilience, the subsequent step involved the process of complete externalization without full reliance on the internal validators. The objective has never changed and still remains to gradually transition towards complete decentralization, wherein the network is operated and governed solely by community validators. As the leader rotation externalizes, view change will assume a pivotal role.

Coming back to the test scenario, we will now ssh into the instance hosting the leader and verify that the validator is indeed the leader.

$ ./hmy utility metadata | jq -r '.result | "is-leader: \(.["is-leader"])"'
is-leader: true

After confirming, we will connect to the next possible leader candidate. We see that the next leader candidate is one1wh4p0kuc7unxez2z8f82zfnhsg4ty6dupqyjt2. However, the order follows a pattern in which the succeeding key would be part of the latter shard. Harmony’s Testnet has 2 shards, thus one1wh4p0kuc7unxez2z8f82zfnhsg4ty6dupqyjt2 would be part of Shard 1, and the subsequent one one1puj38zamhlu89enzcdjw6rlhlqtyp2c675hjg5, with the BLS key of 99d0835797ca0683fb7b1d14a882879652ddcdcfe0d52385ffddf8012ee804d92e5c05a56c9d7fc663678e36a158a28cwill be the actual succeeding leader candidate for Shard 0.

var TNHarmonyAccountsV1 = []DeployAccount{
 {Index: "0", Address: "one1yc06ghr2p8xnl2380kpfayweguuhxdtupkhqzw", BLSPublicKey: "e7f54994bc5c02edeeb178ce2d34db276a893bab5c59ac3d7eb9f077c893f9e31171de6236ba0e21be415d8631e45b91"},
 {Index: "1", Address: "one1wh4p0kuc7unxez2z8f82zfnhsg4ty6dupqyjt2", BLSPublicKey: "4bf54264c1bfa68ca201f756e882f49e1e8aaa5ddf42deaf4690bc3977497e245af40f3ad4003d7a6121614f13033b0b"},
 {Index: "4", Address: "one1puj38zamhlu89enzcdjw6rlhlqtyp2c675hjg5", BLSPublicKey: "99d0835797ca0683fb7b1d14a882879652ddcdcfe0d52385ffddf8012ee804d92e5c05a56c9d7fc663678e36a158a28c"},
 {Index: "5", Address: "one1x6meu5tqzuz0dyseju80zd2c2ftumnm0l06h4t", BLSPublicKey: "f441b75470919983ba18a0525b1c101af42cae052c6d50f74d1553eebbe78ef226849c5e5a7fb2ba563eec6b20380c00"},
 {Index: "8", Address: "one1dpm37ppsgvepjfyvsamas25md280ctgmcfjlfx", BLSPublicKey: "8a211eb5e9334341fd2498fb5d6b922b4a0984d6a4ea0b5631c1904de5fe21fd6889c9c032d862546ca50a5c41294b0c"},
 {Index: "9", Address: "one1mrprl7pxuqpzp0a84d5kd2zenpt578rr4d68ru", BLSPublicKey: "2e9aa982036860eccb0880702c5d71665761f8d4e6ab5f3d8c3aee25b3e68a2c7eaa3cd85972c7f9a3c19d3fed3d5d01"},
 ...
}

We will now ssh into the instance and inspect that the instance does indeed have the desired key.

$ ./hmy utility metadata | jq -r '.result.blskey | "blskey: \(.[0])"'
blskey: 99d0835797ca0683fb7b1d14a882879652ddcdcfe0d52385ffddf8012ee804d92e5c05a56c9d7fc663678e36a158a28c

After successfully establishing a connection with the candidate instance, we will proceed to restart the Harmony binary on the leader instance. This restart will trigger the initiation of the view change process. As the binary goes through the stop-and-start sequence, the leader temporarily suspends processing incoming messages, thus halting the consensus. Consequently, validators will begin transmitting new view change messages to the candidate once a timeout has elapsed. If a quorum is achieved within a given timeframe, the candidate will become the new leader. Let us observe if this is indeed what happens.

Once the Harmony binary on the leader has been restarted, the view change will kick in. Let’s look at the real-time log of the new leader (previously leader candidate). The logs contain an abundance of information, for simplicity, we will check out only the relevant ones to understand the process.

{"phase":"Prepare","mode":"Normal","message":"[ConsensusMainLoop] Ops Consensus Timeout!!!"}
{"phase":"Prepare","mode":"ViewChanging","message":"[getNextViewID]"}
{"phase":"Prepare","mode":"ViewChanging","message":"[getNextLeaderKey] got leaderPubKey from coinbase"}
{"phase":"Prepare","mode":"ViewChanging","message":"[getNextLeaderKey] next Leader"}
{"phase":"Prepare","mode":"ViewChanging","message":"[startViewChange]"}
{"phase":"Prepare","mode":"ViewChanging","message":"[constructViewChangeMessage]"}

Once the consensus has timed out, the mode of the consensus will change to ViewChanging. The next view ID and next leader public key are fetched. With the received information, the view change message is constructed and broadcast. The above process is carried out by all the validators, thus the leader candidate will participate as well.

{"phase":"Prepare","mode":"ViewChanging","message":"[startNewView] Sent NewView Messge"}
{"phase":"Prepare","mode":"Normal","message":"[ResetViewChangeState] Resetting view change state"}
{"phase":"Prepare","mode":"Normal","message":"[startNewView] viewChange stopped. I am the New Leader"}
{"blockNum":12051992,"message":"PROPOSING NEW BLOCK ------------------------------------------------"}

Now, these above logs pertain exclusively to the new leader as they are logic triggered by the view change messages that only the new leader will receive. Once the leader has received enough of the view change message, the new view message is constructed and broadcast to validators. Then, the consensus will be reset to theNormal mode and the confirmation message, claiming that the node itself is indeed the leader, will be displayed. At this point, the view change process is completed and the new leader continues the consensus by proposing new blocks!

{"phase":"Prepare","mode":"ViewChanging","message":"[onNewView] Received NewView Message"}
{"phase":"Prepare","mode":"ViewChanging","message":"[ResetViewChangeState] Resetting view change state"}
{"phase":"Announce","mode":"Normal","message":"onNewView === announce"}
{"phase":"Announce","mode":"Normal","message":"new leader changed"}

For all other, non-leader validators, the above logs can be observed. After the new leader sends out the new view message, the message is received. Once the message is verified, the consensus is reset to theNormal mode and the Announce phase. Finally, a message claiming that the leader has changed is logged and the consensus is resumed. With this, the view change is completed on both sides of the leader and the validators.

The leader change can be examined in the Testnet Explorer as well. Looking at the block history, we can observe that the view change has been processed successfully! The leader has changed from one1yc06ghr2p8xnl2380kpfayweguuhxdtupkhqzw to one1puj38zamhlu89enzcdjw6rlhlqtyp2c675hjg5 at block 12051992.

For more data regarding view change, check out our newly added metrics page in our explorer!

Conclusion

The view change mechanism is an essential component in Harmony’s consensus protocol. By employing the view change mechanism, the protocol prevents consensus congestion and ensures the liveness of the network by facilitating the continuous production and commitment of new blocks. View change will play a pivotal role in further advancing the network to complete decentralization.

Path to Decentralization: View Change was originally published in Harmony on Medium, where people are continuing the conversation by highlighting and responding to this story.

Summary

On May 2, 2023, Shard 0 (S0) consensus was interrupted for approximately 3 hours, resulting in internal validators nodes being unable to add block 41256348. The issue was resolved by reconfiguring the DNS node to point to the explorer node, allowing synchronization with external nodes. Further investigation into the code hash issue is underway to prevent future incidents.

Impact

Shard 0 consensus was interrupted and no transactions were processed for approximately 3 hours. Shard 0 consensus was restored at 2023–05–02, 22:18:46 UTC.

Timeline

• May 2, 07:12 PM (UTC): PagerDuty alerts for consensus stuck on shard 0
• May 2, 08:54 PM (UTC): Revert from block 4125637 to 4125636
• May 2, 10:14 PM (UTC): DNS is updated
• May 2, 10:17 PM (UTC): Nodes are all restarted
• May 2, 10:26 PM (UTC): Issue is resolved - consensus resumes

Root Cause

Internal S0 validator nodes could not add block 41256348 due to an error loading code hash of a smart contract. Internal non-consensus nodes, explorer nodes, and external validators (not yet upgraded) were not impacted, and have successfully added block 41256348. Internal and external nodes were at different heights and could not reach a consensus.

Action Taken

To resolve the problem, the internal S0 validator nodes were restarted. In addition, the DNS node was reconfigured to point to the explorer node. This allowed the internal S0 validator nodes to synchronize with the latest block in sync with the external node.

May 10 Outage Details:

The Problem: The protocol team noticed that a number of internal validators were unable to sync to block 41613066. The issue was reported as a repeating error, where the validators were not signing blocks as expected. The corresponding non-signing BLS keys were identified, and their associated IP addresses were extracted.

Diagnostics: The team used a Python script ( non-signing-keys.py ) to identify the non-signing keys and associated validators. The output indicated that all non-signing keys were linked to internal Harmony validators. There were no external validators involved in this issue.

Comparison with a Previous Similar Issue: The team noted that this issue had a different behavior compared to a previous similar one. In the past, restarting the nodes had resolved the issue. This time, even though all explorer nodes (which were DNS nodes) had already reached block 41613066, the internal validators were still stuck.

The Error Messages: The main error encountered during this issue was related to a missing code hash. The error message read: “can’t load code hash 422c55ce5f5e1ece63fb9347a319ccffb476da0a4e422082ab6714fe1285b3c2: not found”. This message was appearing in the logs of the nodes that were unable to sync.

The Restart: The team decided to restart some nodes while keeping one node that couldn’t sync for further investigation. After the restart, some nodes were still logging errors related to the missing code hash. However, they started to write new blocks to the explorer db, while the block number stayed behind.

Explorer DB vs Chain DB: The team noticed that the explorer db and the chain db were behaving differently. The explorer db was being updated with new blocks, but the chain db seemed to lag behind.

Further Investigations: The team decided to not restart the remaining non-syncing nodes until the problem was isolated. They also noticed that the beacon-ness of the sync was wrong on one of the nodes, which could be unrelated to the main issue but was still worth investigating.

Logging Issues: The team found that the debug logging couldn’t be enabled on any of the three nodes without a restart, which would hinder their ability to diagnose the issue.

Flag Setting Issue: The team noticed that a flag was wrongly set as false, irrespective of the Shard ID. The code responsible for this was identified as part of the old syncing code.

Persistent Error: The missing code hash error was persisting on some nodes even after a restart. The error logs indicated that the problem was related to the addition of a new block (41613066) to the blockchain.

Incorrect Beacon Flag: A point of confusion was the isBeacon flag, which was set to false. The team speculated that it should be true and identified a related fix in a pull request (PR#4428).

Concurrency Issue: The team discussed a potential issue with the SetCurrentCommitSig function being called concurrently by two goroutines. This could lead to inconsistencies due to the indeterminate order of goroutine execution. A fix for this issue was proposed in a separate pull request (PR#4429).

Cache Issue: The team also considered the possibility of a cache-related problem, as the same error persisted across all stuck nodes.

To summarize, the team is dealing with an issue where internal validators are unable to sync to a certain block. The problem seems to be related to a missing code hash, and the usual fix of restarting the nodes has not resolved the issue completely. Further investigation is ongoing, with the focus on the different behaviors of the explorer db and chain db, and the incorrect beacon-ness of the sync.

What’s Next

• Fix the code hash issue (In Progress)
• Hardfork (v2023.2.0) postponed until the root cause is formally fixed

Follow Up

For more questions and discussion, refer to our talk forum.

[Post Mortem] Tuesday 2 May 2023 was originally published in Harmony on Medium, where people are continuing the conversation by highlighting and responding to this story.

Executive Summary

In Q1 2023, Harmony Protocol made significant progress in Stream Sync, State Sync, Cross-Shard Transactions, and Leader Rotation, along with numerous protocol updates and enhancements. As we move forward into Q2 2023, our roadmap focuses on State Pruning, Light Client development, and several new initiatives to optimize the Harmony network. However, it is essential to emphasize that these new incentives are recommendations and will not be implemented without community involvement and governance voting.

Key initiatives under consideration include downscaling from 4 shards to 1, reducing nodes from 250 to 100, reallocating annual token emissions with a 90% focus on ecosystem development, and improving transaction finality from 2 seconds to 0.8 seconds. Additionally, validator optimization and potential Boneh-Lynn-Shacham (BLS) signing key upgrades are planned to enhance efficiency, security, and cross-chain compatibility. These new incentives are designed to encourage new validators to enter the protocol, as they promote a more diverse and competitive environment that rewards innovation, efficiency, and dedication to the Harmony ecosystem. By attracting fresh talent and fostering a healthy ecosystem, Harmony aims to bring positive enhancements to the system.

Harmony Protocol’s primary goal is to bolster network performance, foster a thriving ecosystem, and drive further adoption within the blockchain space while ensuring that the community’s voice is heard and respected in the decision-making process.

Q1 Recap: Key Achievements and Initiatives

1. Stream Sync: We introduced an advanced synchronization process called Stream Sync, which builds upon the existing State Sync mechanism to improve network efficiency and performance further. Stream Sync leverages cutting-edge algorithms and data structures to minimize data exchange between nodes, reducing network overhead and accelerating new nodes’ integration. Using a pipelined approach, Stream Sync processes multiple state chunks concurrently, significantly reducing the time required for nodes to catch up with the current network state. This improvement ensures the availability of up-to-date and accurate data across the network, benefiting dApp developers and end-users. The development updates for Stream Sync include the implementation of versioned Merkle Trees for better data integrity verification, optimized caching mechanisms to expedite state data retrieval, and the introduction of advanced data compression techniques for efficient data transmission. The Harmony team has also focused on enhancing fault tolerance and reliability to ensure a seamless and robust synchronization experience.
2. State Sync: Harmony’s enhanced state synchronization, known as State Sync, is crucial for maintaining network performance and data consistency. It uses optimized algorithms and data structures, such as Merkle Trees, to minimize the data exchanged between nodes during the synchronization. The process involves dividing the state data into smaller chunks and synchronizing them in parallel, reducing the time required for nodes to catch up. Harmony accelerates new node onboarding and ensures data availability across the network by implementing efficient data compression techniques and prioritizing critical state data during synchronization. Development updates in State Sync include the introduction of Staged Stream Sync v1.0, which further improves performance and reliability, and implementation of caching mechanisms for archival nodes to reduce data access times.
3. Cross-Shard Transactions: Harmony’s development team has been working on multiple aspects of cross-shard transactions to improve performance and user experience. One key element is the implementation of composable commits, which allows transaction outputs from one shard to be used as inputs in another shard. This enables seamless interoperability between shards and maintains transaction atomicity. Harmony also employs efficient routing algorithms to minimize communication overhead between shards and reduce transaction latency. Shard coordination is improved through atomic cross-shard commit protocols and two-phase commit mechanisms, ensuring transaction consistency and finality across shards. Recent updates include planning a hard fork to implement chain ID fixes, cross-shard transaction optimizations, and imitating Ethereum’s syncing behavior in Harmony’s RPC calls for a more familiar developer experience.
4. Leader Rotation: The updated leader rotation mechanism in Harmony Protocol enhances network security and stability by ensuring that the next leader is responsive and available before the rotation process begins. This is achieved by introducing aliveness checks, verifying the next leader’s responsiveness and availability, and mitigating the risk of selecting an unresponsive or malicious leader. Harmony also employs a more secure and unbiased algorithm for leader selection, utilizing Verifiable Random Functions (VRFs) to generate unpredictable and unique leader candidates. This method strengthens the network’s resilience against attacks and collusion attempts. Development updates include the implementation of aliveness checks, improvements to the reshuffling process for shard leaders, and enhancements to the leader rotation mechanism by verifying the next leader’s availability before transitioning.

Protocol updates during Q1:

• Staged Stream Sync v1.0: Introduced a new version of the state synchronization mechanism to improve performance and reliability.
• Multiple SDK dependency updates: Various software development kit dependencies were updated to ensure compatibility and address security vulnerabilities.
• Feature registry: Implemented a system for registering and managing features, allowing better control over protocol enhancements.
• Enable caching on archival mode: Improved the performance of archival nodes by enabling caching, reducing the time required to access historical data.
• Fix epoch chain initialization issues: Resolved problems related to epoch chain initialization, ensuring smooth transitions between epochs.
• GoLang upgrades: Updated the Go programming language used within the protocol, benefiting from performance improvements and bug fixes.
• Fixes to config migration issues: Addressed issues with configuration migration, ensuring smooth updates and protocol changes.
• Add logging for NthNextHmy panic: Enhanced logging to help diagnose and address issues related to NthNextHmy panic.
• Resharding Leader updates: Improved the reshuffling process for shard leaders, increasing network security and stability.
• Schedule hard fork for chainid_fix and cx_one_transfer_precompile: Planned a hard fork to implement important protocol updates, including chain ID fixes and cross-shard transaction optimizations.
• Imitate eth_syncing behavior: Mirrored Ethereum’s syncing behavior in Harmony’s RPC calls, providing a more familiar experience for Ethereum developers.
• Fix self-query issues: Resolved issues related to self-querying, improving the reliability of node communication.
• Multiple Docker and GitHub action updates: Updated various Docker configurations and GitHub workflows to streamline development and deployment processes.
• Fix the max height issue: Addressed a problem calculating the maximum blockchain height, improving overall chain stability.
• Check the next leader’s aliveness: Enhanced the leader rotation mechanism by verifying the next leader’s availability before transitioning.
• Add TriesInMemory Flag: Introduced a flag to control the number of tries stored in memory, optimizing memory usage.
• Add Gas Price Oracle User Configuration: Enabled user configuration of the gas price oracle, allowing for more flexible and dynamic gas pricing.
• Fix p2p host discovery for legacy sync: Resolved issues with peer-to-peer host discovery in legacy sync mode, improving network connectivity.
• Fix state handling of self destruct: Corrected the handling of self-destruct operations, ensuring proper state updates and contract termination.
• Fixed Bitmap race error: Resolved a race condition in bitmap operations, enhancing network stability and performance.
• Removed unused methods: Eliminated deprecated and unused methods, streamlining the codebase and reducing potential security risks.
• Rotate external validators: Implemented rotation of external validators, promoting decentralization and network security.
• Add configurable HTTP and eth_call timeouts to RPC: Users can configure HTTP and eth_call timeouts, providing greater flexibility and performance tuning.
• Do not return extcode for validators: Improved privacy by preventing the exposure of external validator codes.
• Removed multiple engine dependencies: Simplified the dependency structure by removing unnecessary engine components, reducing potential conflicts and vulnerabilities.
• Fixes to context passing: Corrected issues with context passing, improving the overall reliability and efficiency of the protocol.

Q2 Objectives and Initiatives

1. State Pruning: Implement an efficient mechanism that utilizes data structures like Merkle Patricia Trees and reference counting to track and remove obsolete state data. This process will help reduce node storage requirements by eliminating unused or outdated information while maintaining consistency and data integrity. By periodically pruning the state tree, nodes benefit from lower storage and memory overhead, faster synchronization times, and improved overall performance.
2. Light Client Development: Continue the development of light clients by incorporating features such as succinct block headers, Merkle proofs, and Bloom filters, which enable lightweight clients to verify transactions and access blockchain data without downloading the entire chain. These clients, tailored for resource-constrained devices like smartphones and IoT devices, will improve performance by minimizing bandwidth and storage requirements. In addition, implementing protocols like the Light Ethereum Subprotocol (LES) and the Ultra-light Beam Synchronization (ULC) will ensure fast and efficient syncing of light clients, providing secure and accessible blockchain interaction for a broader range of users and devices.

New Proposed Initiatives:

• Downscale from 4 shards to 1 shard and 250 to 100 nodes.
• Restricting validators to use one BLS key per node.
• Split annual token emission of 441M as 10% to validators and 90% to the ecosystem.
• Improve transaction finality from 2 seconds to 0.8 seconds.
• Optional: Upgrade validator Boneh-Lynn-Shacham (BLS) signing keys, before the next Ethereum upgrade, for efficient bridges and light clients.

As we continue to refine and optimize the Harmony Protocol, we believe involving our dedicated community in critical decision-making is essential. To ensure that the interests of our community members are taken into account, we will provide an opportunity for you to vote and give feedback on the proposed tokenomic changes, including the 90% and 10% reward adjustments, as well as the reduction in the total number of nodes to less than 100. Your input and participation in these crucial decisions will help us shape the future of the Harmony Protocol, aligning it with the collective vision of our growing community.

The proposed tokenomics and node reduction changes are designed to foster a more sustainable and dedicated validator ecosystem within Harmony. By prioritizing long-term, committed validators, we aim to create an environment that encourages homebrew validators to run on bare metal infrastructure instead of relying on cloud server usage. These changes are intended to motivate and empower validators who actively contribute to the Harmony ecosystem by developing dApps, developer tools, exchanges, and other valuable use cases.

We recognize that these adjustments might discourage some current validators; however, our primary focus is empowering those who genuinely believe in Harmony’s long-term vision and success. By supporting and promoting validators that bring value to the network and invest in its future, we can create a stronger, more vibrant community that propels Harmony towards its ultimate goal of widespread adoption and innovation.

Downscaling and Token Emission

In Q2, one of our primary initiatives will be to restructure the Harmony network by consolidating from 4 shards to a single shard and simultaneously reducing the number of nodes from 250 to 100. This strategic move will have several consequences for the network regarding technical performance and operational costs.

Reducing the number of shards and nodes will lead to a 2.5x decrease in the total elected validators in the network, streamlining the consensus process and reducing the communication overhead. This consolidation will also lower the validator operations budget, as fewer nodes will need to be maintained and secured, allowing for a more efficient allocation of resources.

Reducing the number of nodes from 250 to 100 will improve message propagation and consensus speed, potentially enhancing the network’s transaction throughput and finality time. This is because fewer nodes will participate in the consensus process, reducing communication complexity and speeding up the overall process.

While a reduction in nodes may raise concerns about the potential impact on decentralization, it is essential to note that the Harmony Protocol employs a secure and efficient sharding mechanism that mitigates the risk of centralization. The network can maintain its decentralization and security even with fewer nodes by effectively distributing the transaction load and validator responsibilities across the single shard.

In conclusion, downsizing the number of nodes in Harmony Protocol can lead to performance improvements without inherently compromising decentralization, provided that the proper mechanisms are in place to maintain a secure and distributed network. This can be achieved through community engagement and transparent governance mechanisms that encourage a healthy balance of power within the network. Ensuring a broad and diverse distribution of delegations among the remaining validators will further support decentralization.

Regarding the annual token emission of 441M, we plan to allocate 10% of this amount (44.1M tokens) to validators and the remaining 90% (396.9M tokens) to ecosystem development. This distribution will ensure that validators remain fairly rewarded for their contributions to the network’s security and stability, even though their overall rewards will be reduced. The focus on ecosystem development will accelerate depegged asset recovery and drive developer growth, ultimately attracting more projects and users to the platform.

In the future scenario, with the reduced 10% allocation to validators, the average earnings per slot for validators will still be maintained:

44,100,000 tokens / 100 nodes * $0.02 per token = $8,820 average earnings per slot.

By readjusting the token distribution, we can maintain a healthy balance between incentivizing validators to maintain high-performance nodes and providing ample resources for the growth of the Harmony ecosystem. This approach aims to achieve sustainable network growth and adoption while keeping the network secure and decentralized.

The Q2 initiative to downscale the network will increase operational costs and technical performance efficiency. By carefully allocating token emissions, we aim to strike a balance between incentivizing validators and fostering a thriving ecosystem, ultimately contributing to the long-term success of the Harmony Protocol.

Transaction Finality Improvement

We aim to improve transaction finality from 2 seconds to 0.8 seconds by optimizing the consensus mechanism and network communication. Transaction finality speed is influenced by various factors, including the number of nodes participating in the consensus, the volume of network messages exchanged, and the shard leader’s efficiency in computing aggregate signatures.

In the existing setup with 250 nodes, network latency, and communication overhead contribute significantly to the 2-second finality time. However, it is important to note that with the current load, shards 1, 2, and 3 can achieve 1-second finality, but the system is intentionally set to wait for 2 seconds. By reducing the number of nodes to 100, we expect to decrease the finality time to 0.8 seconds, as calculated by the formula: 2 / (250/100) = 0.8 seconds. This reduction in nodes decreases the number of required message exchanges and limits the communication overhead, leading to a faster consensus agreement. The modified setup would allow us to capitalize on the existing performance capabilities of the individual shards, further optimizing the overall network performance and transaction processing times.

In addition to the node reduction, we will explore other optimizations to enhance the shard leader’s computation of aggregate signatures. These optimizations can further reduce the time required for the shard leader to compute and validate signatures, thereby accelerating transaction finality. For instance, we can investigate using more efficient cryptographic algorithms, parallel signature verification, and batching techniques.

We also plan to analyze and improve the underlying network infrastructure, such as optimizing peer-to-peer connections, implementing more efficient message dissemination protocols, and utilizing network compression techniques. These improvements will help reduce network latency and further enhance transaction finality.

In summary, by decreasing the number of nodes, optimizing shard leader computations, and enhancing the network infrastructure, we aim to significantly improve transaction finality from 2 seconds to 0.8 seconds, offering a more responsive and efficient blockchain experience for Harmony Protocol users.

Validator Optimization and BLS Key Upgrade

In Q2, we plan to re-align validators by adjusting the distribution of the foundation treasury’s tokens, which includes the current 305M tokens ($6.2M) staked from the foundation. The new delegation will allocate 1.20B tokens ($24M) among the validators. This optimization will consider validator performance, security measures, and uptime to ensure a fair and effective distribution. By rebalancing the delegated tokens, we aim to maintain a robust and decentralized network with evenly distributed voting power, reducing the likelihood of cooperation or centralization.

We are also exploring options for using the rewards generated from staking these tokens for initiatives such as depegged asset recovery, supporting DApps, and fostering developer growth within the Harmony ecosystem. By directing some rewards towards these areas, we intend to create a vibrant and sustainable environment for developers and users, driving further adoption and innovation in the blockchain space.

In addition to validator optimization, we are considering upgrading the validator Boneh-Lynn-Shacham (BLS) signing keys in addition to validator optimization before the next Ethereum upgrade. Upgrading to BLS signatures offers multiple benefits, primarily due to their unique aggregation property, which allows multiple signatures to be combined into a single, compact signature. This reduces the size of the blockchain’s signature data and decreases the computation required for verification, thereby enhancing the performance of cross-chain bridges and light clients.

Harmony Protocol can achieve more efficient communication between Ethereum and other blockchain ecosystems by adopting BLS signatures, resulting in faster and more secure cross-chain transactions. BLS signatures have a deterministic and constant-time verification process, which reduces the risk of timing attacks and improves the protocol’s security. This upgrade will ensure that Harmony stays compatible with future Ethereum upgrades, enhancing cross-chain compatibility and user experience.

The upgrade process will require thorough testing and benchmarking to ensure compatibility and seamless integration with the existing protocol. Moreover, this optional initiative will demand coordination with the validator community to facilitate the smooth transition to the upgraded BLS signing keys.

By re-aligning validators and potentially upgrading to BLS signing keys, Harmony Protocol aims to bolster network efficiency, security, and cross-chain compatibility, ultimately providing an enhanced user experience and fostering greater adoption in the blockchain space.

Conclusion

In Q1, Harmony Protocol made significant progress in State Sync, Cross-Shard Transactions, and Leader Rotation, alongside numerous protocol updates and improvements. As we move into Q2, our focus shifts to State Pruning, continued Light Client development, and new initiatives such as downscaling the network, improving transaction finality, and re-aligning validators. We aim to strengthen the Harmony Protocol ecosystem by achieving these objectives and driving further growth and adoption in the blockchain space.

Pros of Moving from 4 Shards to 1 Shard:

• Simplified consensus process: With a single shard, the consensus process becomes more straightforward to manage, improving overall efficiency.
• Reduced communication overhead: Less inter-shard communication will be required, minimizing network overhead and enhancing performance.
• Lower validator operational costs: Validators will only need to maintain nodes for a single shard, reducing operational and maintenance costs.
• Easier developer experience: Developers will have a more accessible time building and deploying dApps on a single shard without worrying about cross-shard transactions.

Cons of Moving from 4 Shards to 1 Shard:

• Potentially reduced scalability: One of the critical features of sharding is increased scalability. Moving from 4 shards to 1 may raise concerns about the protocol’s ability to scale efficiently in the future.
• Negative community perception: The community might perceive this change as a step back from the original vision of Harmony’s scalable infrastructure. They might question the decision to scale down, particularly after the significant work that has gone into developing cross-shard transactions.
• Reconfiguration of existing dApps: dApps built to work with multiple shards may require updates and reconfiguration to function correctly on a single shard.
• Possible centralization concerns: Reducing the number of shards may raise concerns about increased centralization, as fewer shards can lead to a higher concentration of validators in the single shard.

Harmony Protocol Technical Quarterly Report and Roadmap for Q2 2023 was originally published in Harmony on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction

Harmony strives to create a decentralized platform fully externalized and self-sustaining. Harmony will offer the required infrastructure and internal services until the desired state of the network is achieved. This article aims to provide open access to Harmony’s cost analysis, projections, and security practices.

Cost Overview

Below is a screenshot of Harmony’s monthly costs from November 2022 to the projected cost of June 2023. As captured below, Harmony has been working to save costs while ensuring the resilience of the network. We were able to reduce the total cost from $120,000 to $51,000, and the current projected cost for June 2023 is approximately $15,000 (note that these projections can change as more or fewer demands are required). Migration from AWS and termination of underutilized instances and plans have been and will be the major contributing factors to the decrease in cost.

The initiative of these activities is not to simply reduce costs but rather to pave a path for a self-sustaining, decentralized network. This has been Harmony’s driving mission from the start. In order to achieve the goal, adjustments to our services in accordance with the current need of the network are a must.

Migration from AWS

Before the major migrations and terminations, the majority of Harmony’s services were concentrated in our two AWS accounts: “development” and “main”. The purpose of the development account is for engineers to develop and host services before landing in the main account. The purpose of the main account is to host services that were fully developed and fully integrated into the network. Because new features and services were continuously developed and tested, the development account hosted a lot more services than the main one, explaining the difference in cost between the two accounts.

While AWS offers many conveniences, it is more expensive than other providers. The main reason is the data transfer cost AWS charges. The majority of other providers do not charge data transfer costs, thus offering more affordable options. AWS does offer various savings plans to mitigate data transfer costs. However, with the decline of traffic in the Harmony network, we are not able to fully utilize the plan and make long-term commitments.

The reasonable choice was to migrate our services from AWS to other providers. Hetzner, Digital Ocean, Latitude, and Contabo are the alternatives Harmony has chosen. These offer a more appropriate pricing model at the current state of Harmony, allowing us to be prudent with our resources. This allows us to diversify and adopt a multi-cloud strategy. Harmony plans to fully migrate out of AWS and utilize the aforementioned providers.

Multi-Cloud Strategy

The main reason for hosting our services in multiple cloud providers is to prevent a single point of failure. Though the chosen platforms are reliable providers, the chance of failure is never 0%. To prevent services from going down all at once and causing a network outage, Harmony’s services have been distributed to various data centers in different providers.

The breakdown of Mainnet services is shown below:

As shown above, Harmony still hosts a considerable amount of services in AWS despite the effort to migrate away from AWS. This is because Harmony purchased a yearly subscription to Compute Savings Plan and Reserved Instances in 2022 with the cost of approximately $12,000 per subscription per month. Last year, when Harmony was handling an immense amount of traffic, purchasing the subscriptions was a reasonable choice. The plans were utilized to the fullest extent and Harmony was able to reduce expenses. However, with the decreased usage and traffic, Harmony cannot fully utilize the subscriptions.

The Compute Savings Plan and Reserved Instances end in April and June of 2023 respectively. Because Harmony is fully committed to the savings plan until April, there is a limitation on the actions Harmony can take. Thus, services affected by it will be the latter priorities in migration. On the other hand, Harmony is putting significant effort into handling the Reserved Instances.

Termination of Underutilized Services

Unlike the Savings Plan which targets the majority of the instances, the Reserved Instances were mainly for elastic RPC clusters. As Harmony was handling a significant volume of requests in the previous years, there was a demand for elastic RPC clusters. Twenty i3en.xlarge were purchased as Reserved Instances and the elastic clusters were established to improve the performance and scalability of the network, despite their greater cost. The clusters were fully utilized and Harmony was able to serve the network requests in a more efficient manner.

However, with the decline of traffic, elastic RPC clusters are no longer needed. The clusters were not fully utilized thus maintaining the services would be simply wasting resources. Harmony has retired elastic RPC clusters and fully transitioned to the legacy RPC nodes (at the time of publishment, Harmony is supporting three full RPC nodes). Along with the clusters, more than a hundred shard 0 explorer RPC nodes have also been retired as they were underutilized as well. Mind that Harmony will always transition and scale to fulfill the network needs. However, with the current situation, the legacy RPC nodes can perfectly handle requests of the network.

The elastic RPC clusters occupied the majority (17 of 20) of the Reserved Instances. Now that the clusters have been terminated, Harmony has listed the Reserved Instances for sale. This would reduce the economic burden of resources that are not being utilized. One might wonder whether the Reserved Instances can be occupied by other services. However, currently, there are no services in Harmony that require as much redundancy and computing power. Thus, the only reasonable option is to terminate the services and list the Reserved Instances for sale. Additionally, listed instances will still incur data transfer costs.

Along with the elastic RPC clusters, Harmony terminated other underutilized instances and cut redundancies. One major redundancy we have reduced is the archival nodes. Originally Harmony supported four archival nodes, but Harmony has decided to maintain only three nodes. An archival node from Hetzner has been terminated because the four original archival nodes were minimally utilized. Currently, three nodes are enough to handle the functionality while preventing a single point of failure. With other required services, we have exercised compute instance resizing activities to suit the network need.

Harmony is consistently monitoring and inspecting the health of the Harmony network. We are striving to ensure the stability and robustness of the network. Harmony will work full hands to scale and provide the necessary services whenever more demands are required.

Security Practices and Measures

Harmony has been striving to improve and reinforce the security of our cloud infrastructures. Infrastructure changes, mainly enforcing stronger imposition of Single Sign-On (SSO), Multi-Factor Authentication (MFA), and Virtual Private Networks (VPN) have been executed. SSO allows users to sign in to multiple applications with a single set of credentials, preventing multiple credentials from being created. MFA adds an extra layer of security by requiring users to provide additional authentication factors, for example, codes sent to their personal mobile devices. VPNs provide secure access to our cloud resources by creating an encrypted tunnel between the network and the cloud environment. All these methods provide extra security layers to the ones already supplied by the cloud providers.

On top of the multiple security layers, we have been actively securing our passwords and secret keys. We have implemented an even more aggressive password and secret rotation, and API clean-up process. This helps prevent unauthorized access to our cloud resources. Passwords and secrets will be outdated quickly due to the active process, making it even more difficult for attackers to compromise the resources.

While the security layers and practices have been present in Harmony, we have reinforced the measures with even greater emphasis. The implementation of countermeasures at the protocol level and the adoption of these security strategies will further enhance the robustness and security of the Harmony network.

Harmony’s Roadmap

In order to provide a more secure, decentralized, and self-sustaining network, Harmony will be working on several key initiatives. First and foremost, we plan to complete our migration from AWS as previously mentioned. We will be able to reduce costs and enforce our multi-cloud strategy. This will help us avoid vendor lock-in and be prudent with our resources, as well as to prevent a single point of failure.

As we are migrating away from AWS, we hope to segregate the nodes and infrastructure in an orderly, secure manner. Currently, there exists only a simple distinction between the different services. This distinction is enough for us to organize our services. However, we hope to improve this organization method and create a clear distinction between the nodes and infrastructure of different projects. This would help our multi-cloud infrastructure become even more organized and secure.

Another key initiative is to create a centralized cost service for the costs of multiple cloud providers. One downside of a multi-cloud strategy is the difference in the pricing models of each provider. Due to the variation of the invoices, mainly the way items are categorized and priced, forming a single source of information in a uniform manner may be inefficient and confusing. Harmony plans to create a centralized cost service that will automate this process in an efficient and clear manner. This will not only enable both the internal and the community members to clearly visualize Harmony’s expenses. Transparency is a key mission Harmony believes in. We hope that the service will bring transparency to our community.

Last but not least, Harmony anticipates various protocol enhancements and the establishment of partnerships and collaborations that will allow the network to fully externalize. As stated, Harmony’s ultimate goal is to create a completely decentralized and self-sustaining network. These initiatives will bring us a step closer to those goals.

Conclusion

Open access is a key component in creating a decentralized community. Harmony hopes to bring more transparency to the internal works, projections, and costs to the community in order to collaborate for a self-sustaining network. Kindly stay tuned for further updates!

Open Access to Harmony was originally published in Harmony on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction

Harmony is a blockchain designed to provide fast, secure, and scalable infrastructure. This article will cover the current state and plans for Harmony’s architecture.

RPC Full Node

RPC full nodes serve as the entry point to the Harmony network. They handle requests for querying information in the blockchain or processing transactions. In 2020, Harmony transitioned from original RPC nodes to elastic RPC clusters to handle the increasing volume of RPC requests. The transition was made to improve the performance and scalability of the network. However, now that the traffic amount has decreased, we have transitioned back to the legacy RPC nodes.

The motive for this transition is to prevent resources from being wasted and extra costs from occurring. We have fully terminated our ERPC clusters and have set up the required nodes to properly handle RPC requests. Harmony currently has three full nodes fully utilized in serving requests.

Harmony will constantly monitor the health and traffic of the RPC nodes. If more or less traffic occurs, the team will scale the nodes for the robustness of the network. Harmony’s ultimate goal is to scale the network using light clients alongside full nodes. For detailed information regarding light clients, please check out the following article:

Scaling the Harmony Protocol with Light Clients

Internal Validator Node

Leader node is responsible for constructing new blocks and broadcasting them to validators. Validator nodes verify the validity of blocks to ensure that the ones from malicious leaders do not get appended to the chain. Currently, leaders are chosen from the twenty internal validator nodes hosted by Harmony (five validator nodes per shard). Each validator node contains five BLS keys each, which totals twenty-five BLS keys per shard. These keys are the actual identity that gets selected as the leader.

As of now, only the internal validators are considered when choosing leaders for each shard. To ensure the continuity of the network, if a leader is not fully functioning, a view change will kick in and another leader will be chosen. Fully terminating all the internal validators is not viable at the moment because there will be no possible leader candidates to produce blocks. Introducing external validators as possible leader candidates is a feature Harmony is currently developing and planning to complete by Q1 of 2023. Once completed, external validators will be part of the leader rotation process, allowing the community to play a more active role in block validation.

Harmony’s plan is to fully externalize the voting slots as our end goal is a fully decentralized network. The plan is as follows:

The governance for the stages of termination is happening currently: HIP-29. If the governance is approved, two nodes will be terminated per shard, making it three nodes per shard. Each node will have one less slot making it four slots per node and twelve slots per shard. Once the change is implemented, the network will have increased external voting power.

Harmony has implemented various measures to ensure the stability and availability of the network, including the use of multiple consensus mechanisms (FBFT and EPoS), automatic failover mechanisms, and real-time monitoring and alerting systems. These already guarantee the robustness of our network, however as the network becomes more decentralized, dependency on the external validator community is bound to grow. Harmony will continue to coordinate with the community to further assure the resilience of the network.

The multiple stages will allow Harmony to cooperate with the community on finding a robust solution for a decentralized, secure network.

Concern for Centralization

Though not specific to the HIP-29 proposal, there is one concern we need to address as external voting power greatens: centralization. Increasing the external voting power may also increase the centralization of the network as larger stakeholders can possibly absorb the voting slots. A smaller number of participants with larger stakes may have more influence over the network’s decision.

Previously, a hard fork was implemented in order to place a limit of BLS keys per shard. The motivation was to prevent the likelihood of a shard being taken over by a small group of misbehaving validators. Each validator would have a limit on the number of keys they possess per shard.

Regardless, there are more measures Harmony needs to take in order to further push towards decentralization. There are several proposals, such as limiting the total number of keys per validator, limiting the number of rewards per validator, and limiting the max a “whale” can delegate per validator. Note that these are just proposals that may or may not be implemented. Regardless, Harmony will work with the community in search of the most appropriate approach to achieving decentralization. Harmony will provide further updates on addressing the issue of centralization.

Bootnode

Bootnodes are responsible for bootstrapping nodes in the network. When a new node is introduced to the network, bootnode provides a set of nodes for the new nodes to sync with. The sync would be done in a peer-to-peer manner. This will allow the new node to have the necessary data to participate in the network.

Harmony is currently hosting four bootnodes. Bootnodes are relatively cheaper than other services as lightweight instances can handle the functionality. Harmony plans on fully externalizing all services but as of now, bootnodes are a critical service that needs to be part of our network.

DNS (Sync) Node

DNS nodes are another solution for syncing nodes. They are full nodes that Harmony designates to handle syncing. If nodes do not want to sync from peers bootstrapped by bootnodes, they can sync from the DNS nodes run by Harmony. The interaction between DNS nodes and nodes requiring sync is similar to the server and client model. They do not communicate with each other. Rather, new nodes call out to DNS nodes and they simply respond with the appropriate data.

Maintaining DNS nodes is expensive due to the requirement for high bandwidth and large storage. In order to optimize our infrastructure and cost, Harmony plans to retire the DNS nodes and operate the syncing functionality in a different form: stream sync.

Stream Sync

Nodes must be in sync with the latest state of the blockchain network. For this reason, stream sync is implemented as a new method for Harmony nodes to sync their states between nodes.

Stream sync involves a set of steps that each node follows to ensure its state is up to date. First, the node finds a set of neighboring nodes during the bootstrapping process. The node uses a connection manager to manage all of these connections. Then, the node will create stream channels for each neighbor through a stream manager. Once the stream channels have been established, the node can begin to sync its state.

The node starts by asking each neighbor for their current block height through a request manager. Once the requests have been fulfilled, the node can calculate the maximum block height. With the maximum block height, the node then asks each neighbor to share a piece of their state. The sharing process continues until the node fetches all states up to the current block.

Harmony stream sync uses a unique ID to identify nodes in the same shards. Nodes in the non-beacon chain use a downloader to fetch the last blocks of each epoch from the beacon chain. These blocks are essential for the nodes in the non-beacon chain as they enable retrieval of shard states and committee lists.

The benefit of stream sync is that it provides a faster state sync time with minimal network bandwidth. The process makes the entire network more decentralized as well because nodes will not be relying on designated sets of nodes. Rather, nodes are able to sync from anyone within the same shard. Stream sync ultimately can be seen as an upgrade to not only the traditional peer-to-peer sync but also the DNS sync. It removes the dependency on DNS nodes and eliminates the high overload on the few designated syncing nodes. Stream sync is planned to release during Q1 of 2023 and once the feature is fully functioning, Harmony will retire DNS nodes.

Snapshot Node

Snapshot nodes are used to store backups of the database. Snapshot nodes repeat the process of syncing and capturing snapshots in decentralized storage, Storj. The snapshots will allow new nodes to efficiently catch up to the current state of the network.

Without snapshot nodes, a new node would have to sync all the data from other peer nodes or DNS nodes. However, due to the immense amount of data, the syncing process would take an unreasonable amount of time and put a burden on both nodes. In order to avoid this complication, new nodes would first obtain data from snapshot nodes. Once sync from the snapshot node is completed, new nodes can then sync the rest of the data from other full nodes, whether peer or DNS nodes. This would offload the amount of data for other full nodes and make the network efficient overall.

Harmony currently operates one snapshot node per shard, making it four services within our Mainnet. Snapshot nodes can only be retired if Harmony’s sync is reasonably fast. Until then, the service will be operated to ensure the convenience of the network.

Archival Node

Archival nodes contain all the data of the Harmony blockchain. Currently, a single archival node in shard 0 contains 22TB of data. Spinning up a new archival node would take approximately three weeks or more depending on the node’s connectivity. Due to its immense size, it is crucial for us to maintain a clean copy.

Harmony is currently maintaining three archival nodes. The purpose of redundancy is to prevent a single point of failure. For example, if one archival node is dysfunctional and requires a new setup, another archival will be occupied with syncing the dysfunctional one. At that stage, only one archival node will be available to handle external requests. In order to prevent network stoppage, Harmony is maintaining more than two archival nodes.

At the current state of Harmony, three archival nodes are sufficient to handle traffic and provide network robustness. If more demands are required, Harmony will scale the archival nodes to fulfill the network needs.

Conclusion

Harmony’s focus is to make substantial advancements and the appropriate changes in our infrastructure. We look forward to witnessing the impact they will bring to our network. Kindly stay tuned for more infrastructure and protocol updates!

2023 Harmony Architecture was originally published in Harmony on Medium, where people are continuing the conversation by highlighting and responding to this story.

Harmony is continuing development towards uniform scaling through 1-block composable cross-shard transactions. In our newest update, Mainnet Release v2023.1.0, the hardfork allows for Native ONE Cross-Shard transfer precompile, allowing users to easily send ONE from one shard to another using MetaMask. With 100% of traffic currently on Shard 0, Cross-Shard transactions will enable dApps to launch and drive transaction volume to shards 1, 2, & 3.

In the tutorial below, users will learn how to easily send ONE tokens between shards using MetaMask browser extension wallet.

Step 1: Create a MetaMask wallet and add the extension to your browser of choice. MetaMask currently supports Chrome, Firefox, Brave, Edge, & Opera.

Step 2: Add the Harmony Mainnet Shard 0 network on MetaMask. There are currently two ways to do this:

1. You can add the Mainnet Shard 0 network directly through the new Cross-Shard Bridge (details below).
2. You can follow the directions shown in Harmony’s Technical Documentation here. The parameters set should reflect the following:

Harmony custom RPC for Shard 0 on mainnet

• Network Name: Harmony Mainnet Shard 0
• New RPC URL: https://api.harmony.one
• Chain ID: 1666600000
• Currency symbol (optional): ONE
• Block Explorer URL (optional): https://explorer.harmony.one

Step 3: Add Harmony Shard 1 Network manually. The parameters set should reflect the following:

Harmony custom RPC for Shard 1 on mainnet

• Network Name: Harmony Mainnet Shard 1
• New RPC URL: https://s1.api.harmony.one
• Chain ID: 1666600001
• Currency symbol (optional): ONE
• Block Explorer URL (optional): https://explorer.harmony.one

Step 4: Purchase ONE tokens using the new Harmony.one/buy page.

Step 5: Navigate to our Cross Shard Bridge.

Step 6: Connect your MetaMask wallet with the Cross Bridge Application.

Step 7: From this screen, you can opt to add the Harmony Shard 0 network if you have not already done so manually. To do this, simply click “Switch MetaMask to Harmony Shard 0 (mainnet)” and follow the prompts. If you have already added the Shard 0 and Shard 1 networks, ensure you have switched MetaMask networks to Shard 0 mainnet and enter your MetaMask address in the “Address” field by clicking “use my address”.

Step 8: Enter the amount of ONE tokens you would like to transfer to Shard 1. I always opt to perform a test transaction before sending the remainder of the funds.

Step 9: Once the transaction has been completed, you can view the details by clicking on the transaction hash to navigate to the Harmony Block Explorer.

Step 10: You can view the completed transaction in your MetaMask by toggling between the Shard 0 and Shard 1 networks.

Simple as that! In the coming weeks, we will add support for Shards 2 & 3. Follow Harmony Protocol on Twitter for more updates.

How to Transfer ONE Tokens From Shard 0 to Shard 1 Using MetaMask was originally published in Harmony on Medium, where people are continuing the conversation by highlighting and responding to this story.

The Harmony Protocol is a high-performance, decentralized blockchain that utilizes a unique consensus algorithm called “sharding.” One of the future vital features of the Harmony Protocol will be account abstraction, which allows for creating multiple accounts, each with its unique functionality and features. In Q2 of 2023, the Harmony Protocol will implement account abstraction to provide greater flexibility and functionality for users and developers.

Background

The current implementation of the Harmony Protocol utilizes a single type of account called a “base account.” These accounts are limited in their functionality, as they can only be used for sending and receiving transactions and cannot execute any smart contracts. This lack of flexibility has led to the development of other types of accounts, such as smart contracts and token contracts, which can be used for more advanced functionality, such as the creation of decentralized applications (dApps) and the issuance of tokens.

Account Abstraction

Account abstraction is a feature that allows for the creation of multiple types of accounts, each with unique functionality and features. The Harmony Protocol will implement account abstraction to provide greater flexibility and functionality for users and developers. This feature will be implemented by creating a new smart contract that acts as a factory for creating new accounts. Based on the parameters passed to the factory contract, the factory contract will allow for the creation of different account types, such as base accounts, token contract accounts, and smart contract accounts.

Account Types

The Harmony Protocol will be introducing the following types of accounts:

• Base Account: This is the standard account type and will be the default account type for all users. Base accounts can be used for sending and receiving transactions and participating in the consensus process. They cannot execute any smart contract functions. Base accounts will be created with a unique address, which will be derived from the user’s private key and the Harmony Protocol’s address generation algorithm.
• Token Contract Account: This type of account is used for issuing tokens and managing token transfers. Token contract accounts can be used to create and manage custom tokens and track the total supply and individual token balances. These accounts can execute pre-defined smart contract functions specific to token management, such as minting new tokens and transferring tokens to other accounts. Token contract accounts will be created by deploying a smart contract that implements the ERC-20 standard on the Harmony Protocol.
• Smart Contract Account: This type of account is used for the creation and execution of smart contracts. Smart contract accounts will be created by deploying a smart contract on the Harmony Protocol. Smart contract accounts can create decentralized applications (dApps) and interact with other smart contract accounts. These accounts can execute any smart contract function that the contract owner can define.

Account Creation

Creating a new account on the Harmony Protocol is done through a smart contract. The following is an example of how to create a new base account using the account factory contract:

pragma solidity ^0.8.0;

contract AccountFactory {

    function createAccount() public returns (address) {
        return new BaseAccount();
    }
}

In this example, a new base account is created by calling the “createAccount” function, which creates a new instance of the “BaseAccount” contract. This contract can then be used for sending and receiving transactions, as well as participating in the consensus process. The function returns the address of the new account, which can be used to interact with it.

Creating a new token contract account or smart contract account is done in a similar manner, but by calling the appropriate functions in the account factory contract. Here is an example of creating a new token contract account:

pragma solidity ^0.8.0;

contract AccountFactory {

    function createTokenContract(string memory name, string memory symbol, uint256 totalSupply) public returns (address) {
        return new TokenContract(name, symbol, totalSupply);
    }
}

In this example, the “createTokenContract” function is called to create a new instance of the “TokenContract” contract, which is used for managing tokens. The function takes in three arguments, the name of the token, the symbol of the token, and the total supply of tokens. It returns the address of the new token contract, which can be used to interact with it.

Similarly, here is an example of creating a new smart contract account:

pragma solidity ^0.8.0;

contract AccountFactory {

    function createSmartContract(string memory contractName, bytes memory contractCode) public returns (address) {
        return new SmartContract(contractName, contractCode);
    }
}

In this example, the “createSmartContract” function is called to create a new instance of the “SmartContract” contract. The function takes in two arguments, the smart contract’s name, the smart contract, and the actual code of the smart contract in bytes. It returns the address of the new smart contract, which can be used to interact with it.

Account Management

Once an account is created, it can be managed and interacted with through smart contracts. The following is an example of how to transfer tokens from one token contract account to another:

pragma solidity ^0.8.0;

contract TokenTransfer {

    function transfer(address from, address to, uint256 value) public {
        TokenContract(from).transfer(to, value);
    }
}

In this example, the “transfer” function transfers tokens from one token contract account to another. The function uses the token contract’s “transfer” function to perform the transfer. The function takes in three arguments, the address of the sender, the address of the recipient, and the number of tokens to be transferred.

The same approach can also be used to manage smart contract accounts by calling appropriate smart contract functions to interact with the smart contract.

Core Changes

Account abstraction is a novel way of improving the transaction validity conditions in Ethereum Virtual Machine (EVM) by allowing any arbitrary EVM bytecode to be executed. This expands the range of possible valid transactions beyond those limited by existing standards, providing contracts with more expressive and secure means of coding entire applications on top of the blockchain. To demonstrate this concept, we propose a new EVM opcode called PAYGAS. This opcode allows accounts to signal their willingness to pay specific amounts for specific types of transactions — permitting tighter control over exactly which transactions will execute and which will not proceed further. We have split account abstraction into two tiers: single-tenant AA, which is intended to support wallets or other use cases with few participants; and multi-tenant AA, which is intended to support applications with many participants like tornado.cash or Uniswap. The primary benefit here is its ability to enforce additional rules for the determination of transaction validity in order to ensure improved security and scalability across various industries such as payment systems, games, supply chain management and many others within blockchain development. For example, a PAYGAS implementation could prevent malicious actors from wasting resources by submitting arbitrarily large transaction fees in order to carry out expensive computations or accidentally executing duplicate payments since it requires every transaction to pay a set amount before being allowed to proceed further.

A new EIP-2718 transaction with type AA_TX_TYPE is introduced, referred to as “AA transactions”. This type of transaction requires its payload to be interpreted as rlp([nonce, target, data]), and has its base gas cost set at 15000 (instead of 21000) due to the lack of “intrinsic” ECDSA and signature. Additionally, nonces are processed similarly to existing transactions — if tx.nonce == tx.target.nonce fails then the transaction is invalid — otherwise proceeding with setting tx.nonce += 1 immediately afterwards. Furthermore, the PAYGAS opcode sets the gas price and gas limit for each contract willing to pay for specific transactions — preventing malicious actors from wasting resources by submitting large fees or executing duplicate payments since it requires every transaction to have an associated cost in order for it to be validly processed. Lastly, introducing a few new transaction-wide global variables works similarly (in particular, have similar reversion logic) to SSTORE refunds counter.

Single Tenant Account Abstraction introduces a new transaction type that requires its payload to be interpreted as rlp([nonce, target, data]), and has its base gas cost set at 15000 (instead of 21000) due to the lack of “intrinsic” ECDSA and signature. Additionally, introducing a few new transaction-wide global variables are used alongside the PAYGAS opcode. These include:

The NONCE (0x48) Opcode is also introduced with gas cost G_base which pushes the nonce of the callee onto the stack. The PAYGAS (0x49) Opcode is utilized for setting the gas price and gas limit for each contract willing to pay for specific transactions. In particular, it takes two arguments off the stack — version number and memory start — where it performs an assertion on version number being equal to zero before reading: bytes_to_int(vm.memory[memory_start: memory_start + 32]) for gas price and bytes to int(vm.memory[memory_start + 32: memory _start + 64]) for gas limit in order to prevent malicious actors from wasting resources or executing duplicate payments by requiring associated costs on each transaction accepted.

The PAYGAS (0x49) opcode is introduced to help set the gas price and gas limit for each contract willing to pay for transactions. The similarity in both reads to MLOAD and CALL means that memory will be expanded if needed. In accordance with future hard forks, different version numbers can be added which will cause the opcode to take different-sized memory slices and interpret them differently — two particular potential use cases are EIP-1559 and the escalator mechanism. The PAYGAS opcode works as follows: if the three conditions below are met, along with the version number check mentioned previously, then it sets the parameters accordingly. — The account’s balance must be greater than or equal to gas_price * gas_limit — globals.transaction_fee_paid should be False — In a top level AA execution frame, such that if the EVM execution of the entire transaction exits or reverts then it has finished its purpose. Once all these conditions are met, it will subtract (gas_price * gas_limit) from the account’s balance, setting globals.transaction_fee_paid to True and adjusting globals.gas_price and globals.gas_limit according to their respective arguments before setting the remaining gas in the current context equal to (gas_limit minus already consumed gas).

To ensure replay protection is effective and to prevent malicious actors from carrying out replay attacks, two key approaches must be implemented. The first involves requiring SET_INDESTRUCTIBLE when nonce values are checked or updated; this means the value cannot be reverted, forcing any subsequent transactions using the same nonce value to fail. The second approach requires the use of a new opcode called NonceVerify that will verify if a particular transaction has been executed before, i.e. if it uses an already used nonce value, it will simply fail and not be accepted on the blockchain. These two approaches are meant to work in tandem with one another to help ensure replay protection is effective against potentially malicious actors.

Requiring that contracts targeted by AA transactions begin with EIP-2937’s SET_INDESTRUCTIBLE opcode is essential for a successful transaction. If the contract does not include this code, then any transactions involving it will be invalid and cannot be accepted onto the blockchain. To ensure success, AA_PREFIX must be modified to include this opcode as well. An additional defensive measure against replay attacks involves preserving contract nonces across SELFDESTRUCT invocations instead of setting the nonce value to zero. This guards against malicious actors trying to manipulate the nonce value in order to perform an attack on the system. Miscellaneous considerations include requiring CALLER (0x33) to return AA_ENTRY_POINT if it is invoked in the first frame of execution of a call initiated by an AA transaction and ORIGIN (0x32) returning AA_ENTRY_POINT when invoked in any frame of execution of an AA transaction. Lastly, GASPRICE (0x3A) now pushes the value globals.gas_price, thus making it easier for users to monitor gas prices more closely.

In order to ensure efficient and secure transactions, miners and validating nodes have adopted strategies to determine if a transaction is worthy of being included in blocks or rebroadcast. Miners must first perform a small amount of processing to check that the transaction will pay the fee required for them to include it in blocks; this protects against DoS attacks. Similarly, validating nodes also need to verify that there is an appropriate fee before rebroadcasting it. The EIP on account abstraction keeps these consensus changes minimal, enabling miners and validating nodes to gradually introduce support for AA mempool with an initial focus on simple use cases. As more complex use cases are developed, additional steps are taken accordingly, allowing for seamless integration into the blockchain network. In all scenarios, earlier stages are given more attention due to their importance and complexity compared to later stages.

When a node receives an AA transaction, they process it against the current chain head’s post-state to determine its validity. If the code of the target is not prefixed with AA_PREFIX, execution will immediately exit with failure. Additionally, if environment opcodes such as BLOCKHASH, COINBASE, TIMESTAMP, and GASLIMIT are encountered or BALANCE is read from any account (including the target itself), then execution will also fail. Furthermore, any external call/create that changes the callee to anything but the target or a precompile will result in failure. Lastly, if more gas is consumed than VERIFICATION_GAS_CAP (specified above) or more gas than is available in the block, then exit with failure. If PAYGAS is reached then success or failure depends on whether or not the balance is sufficient (e.g. balance >= gas_price * gas_limit).

Nodes strictly adhere to the rule of only keeping transactions with valid nonces in the mempool. This means that if a new transaction to a specific contract and with the same nonce is encountered, then it will replace the original one (if its gas price is higher). Thus, at any given time the mempool contains at most one pending transaction per account. When a new block is processed, all accounts which were targeted by an AA transaction (around 833 when considering each block currently has 12500000 gas and an AA transaction costs >= 15000 ) should be noted and all pending transactions targeting those accounts should be dropped. All other transactions remain in the mempool.

Single & Multiple Tenants

If the Indestructible Contracts EIP is adopted inside the Harmony core code, Single Tenant AA can be adapted to allow for DELEGATECALL during transaction verification. The external state access of any contract whose first byte is SET_INDESTRUCTIBLE will no longer be banned, however calls that change the callee other than CALLCODE and DELEGATECALL are still not permitted. Furthermore, if the IS_STATIC EIP is implemented then the list of allowed prefixes can be extended to enable incoming static calls without changing the state. This could also be used for logging incoming payments as well as other benign use cases.

External calls into AA accounts can be allowed with the help of the opcode RESERVE_GAS, which takes an argument value N and burns N gas and adds it to the refund. An AA_PREFIX that reserves >= AA_BASE_GAS_COST * 2 gas can be added as well in order for at least AA_BASE_GAS_COST gas to be spent when calling into an account. Accounts can also opt for a higher RESERVE_GAS value in order to have a higher VERIFICATION_GAS_CAP. This will preserve a VERIFICATION_GAS_MULTIPLIER-to-1 ratio between the minimum gas cost to edit an account and the set VERIFICATION_GAS_CAP while protecting against excessive reverification gas consumption.

To address the potential for a single transaction to invalidate other transactions targeting the same account, an EIP-2930-style access list must be included in incoming transactions. This access list will make it binding that any accesses outside of the list are invalid. Additionally, incoming transactions can only be included in the mempool if their access list is disjoint from those of other transactions or if their gasprice is higher. An alternative approach would be to create per-storage-slot mempools instead of per-account mempools, allowing a single transaction to be part of multiple storage slots’ mempools (capped at five). In order to use multi-tenant AA, miners may need to edit nonces of incoming transactions which would make their final hash unpredictable at publication time and require clients to explicitly work around this issue. To refine these ideas, more research is necessary and this will be left for later work.

Backwards Compatibility

User-defined accounts (AA) are a type of Harmony account that allows users to extend their abilities beyond those offered by an externally owned account (EOA). While AA transactions have the same format as EOA transactions, they will always have origin == AA_ENTRY_POINT instead of assert origin == caller. Unfortunately, using single-tenant AA contracts can break the transaction non-malleability invariant since it is possible for a modified version of the transaction to still be valid. To prevent this from happening, AA accounts must be designed carefully. Meanwhile, multi-tenant AA contracts break the transaction non-malleability invariant more thoroughly, making the transaction hash unpredictable even for legitimate applications that use multi-tenant AA features. To ensure replay protection on AA contracts, developers must explicitly implement it with the CHAINID (0x46) opcode introduced in EIP-1344.

Security Considerations

When a transaction is added to the mempool, it is quickly verified by the client to ensure validity. However, an attacker may still submit a transaction that invalidates all other transactions from the same account and requires the network to perform additional recomputation — costing more gas. The EIP ensures that recomputation cannot exceed six times the block gas limit in one block, making this slightly more expensive but not drastically different than before.

Account Abstraction (AA) does not introduce any new risks for peer Denial-of-Service attacks. Such attacks can already be accomplished against existing clients by sending invalid transactions. To limit the amount of computation an adversary can force a client to expend, miners should follow the recommended mining strategies. Doing so will help ensure that malicious actors are unable to exploit Account Abstraction technology and cause disruption on the network.

When implementing account abstraction on the Harmony Protocol, it’s important to consider security measures to ensure the safety of users’ funds and data. The account factory contract should be audited for security vulnerabilities and properly deployed on the Harmony Protocol’s mainnet. Additionally, all smart contract code should be thoroughly tested and reviewed before deployment to prevent any potential exploitations. Users should also practice safe account management by keeping their private keys secure and using secure methods for account creation and management.

Conclusion

The Harmony Protocol’s account abstraction feature will be implemented in Q2 of 2023, providing greater flexibility and functionality for users and developers. With the introduction of multiple account types, including base accounts, token contract accounts, and smart contract accounts, developers will have more options for creating and managing their projects on the Harmony Protocol. This will also make it easier for users to participate in and interact with these projects, as they will be able to use different types of accounts based on their needs. The added feature of account abstraction to the Harmony Protocol is a significant step forward in providing a more versatile and user-friendly blockchain platform. By introducing new account types and smart contract functionality, the Harmony Protocol will open up new opportunities for decentralized applications, decentralized finance, and many more possibilities. The introduction of account abstraction will also make it easier for developers to create and manage token-based projects, such as decentralized exchanges and prediction markets, on the Harmony Protocol.

In addition to the benefits for developers and users, account abstraction also improves the overall security of the Harmony Protocol by implementing a more secure account creation process. The account factory contract, which is responsible for creating new accounts, can be designed with security measures to ensure the safety of users’ funds and data.

Overall, the implementation of account abstraction on the Harmony Protocol in Q2 of 2023 will bring a new level of flexibility and functionality to the platform, allowing for a wider range of decentralized applications and use cases. The Harmony Protocol’s team and community should continue to work on ensuring the security and reliability of the feature for the better user experience and overall ecosystem.

Continued Reading

• EIP-2938: Account Abstraction — https://eips.ethereum.org/EIPS/eip-2938
• Ethereum Magicians: EIP-2938 — https://ethereum-magicians.org/t/eip-2938-account-abstraction/4630
• Account Abstraction with EVM Object Format — https://notes.ethereum.org/@axic/account-abstraction-with-eof
• 2022 Harmony Technical Review — https://medium.com/harmony-one/2022-harmony-technical-review-88462efba368

Pull Request

• https://github.com/harmony-one/harmony/pull/4205

Implementing Account Abstraction on the Harmony Protocol in Q2 was originally published in Harmony on Medium, where people are continuing the conversation by highlighting and responding to this story.