Intro to testing modular blockchain software

Naïve bugs and common issues in production networks are problems that threaten the resiliency of decentralized blockchains. This is an especially important problem given the number of users and applications that are dependent on these blockchains. We want to highlight a topic that is under-discussed and more importantly, underutilized as a critical component in ensuring blockchain stability.

This blog post will discuss the importance of testing blockchains and the strategies to test them effectively.

TL;DR

• Robustness of blockchain networks involves rigorous testing of node types, customizations, and network-wide performance for security and reliability.
• With the increased flexibility of node setups in modular blockchains, it's essential to strengthen testing practices that maintain network security and dependability.
• Effective testing strategies include unit, integration, and system testing.
• Challenges in testing modular blockchains include complexity, interoperability issues, and the lack of standardized frameworks.

The importance of testing blockchains

Testing blockchain networks is crucial to ensure their security, reliability, and long-term stability. Testing ensures that if the network is not functioning as expected, issues are identified and resolved before deployment.

In the case of modular blockchains, the testing process is more complex as there are more node types and possible setups.

Each node type must be tested individually and then as part of the network as a whole. Since each node type can be customized, the testing process must also include testing of customizations.

Testing strategies for blockchains

What to remember when conducting testing processes

All teams, Celestia Labs is no exception, have limited human capacity to spread across each aspect of software development. Since blockchain development is fast-paced, testing is considered a lower priority compared to shipping new features. Even with constraints in testing time, teams should remember that testing can be efficient if they follow simple guidelines:

1. Exhaustive testing is impossible. You can't cover everything, but you can cover a lot by identifying and prioritizing testing of high risk elements.
2. Find and fix bugs as early as possible. As a bug progresses from specs* to testnet to mainnet, it becomes progressively costlier to fix.
3. Any automation is better than none. When conducting large network end-to-end testing, automation becomes crucial.
4. Test software on internal devnets before releasing to the public. By ensuring software is stable before shipping to testnet, you reduce work and frustration for the community.
5. Manual testing is better than no testing. Though manual testing is not a scalable long-term solution, it is still useful for short-term goals. You can always introduce automation later on.

Note*: Here is a practical example of how NashQueue (Team Lead and Researcher at Rollkit) uncovered a bug in specs.

The blockchain testing pyramid

As with any software testing, the testing process for blockchains should be both functional and non-functional. Functional testing focuses on examining individual modules and their interplay with other modules. On the other hand, non-functional testing evaluates the network's performance, security, and scalability to ensure a robust and reliable blockchain system.

The following strategies can be used to test blockchains effectively:

1. Unit Testing

Unit testing involves testing each module separately to ensure it is functioning correctly. An example of a module in this context is celestia-node’s header that encapsulates how headers are propagated across different nodes in the network. It is crucial for blockchains as it allows for the testing of each module's customization.

This testing strategy helps identify any issues within a module before it is integrated into the network. As modular blockchains are coupled with modular software design, testing each component in isolation is very beneficial.

2. Integration Testing

Integration testing is essential for blockchains to ensure the modules interact as expected. This testing strategy involves testing the modules in combination with other modules to ensure their interoperability.

An example of modules-to-modules testing is when a node is tested with another node. In Celestia’s case, we usually test a light node syncing and sampling from a single bridge node. Both of them are written from 1 code repository, but bundled differently.

Integration testing helps identify any issues with the interaction between modules and the network.

Often in this level of testing, it is expected to prepare a small network of nodes to test on the same machine.

3. System Testing

System testing involves testing the network as a whole to ensure that it meets the necessary requirements. This testing strategy involves testing the network's performance, scalability, and security. System testing helps identify issues with the network's overall functionality and performance.

In system testing, a larger network of nodes is often used to simulate a real-world environment. This testing strategy helps identify any issues that may arise due to the decentralized nature of the network.

A good example of bugs found in system level for Celestia was keys mismatch during transaction submission or simple balance checks by DA nodes. The functional bug was discovered during load testing of the PayForBlob feature. System tests are often exposing bugs that unit/integration can’t in spite of the scenario’s simplicity and sometimes unrelated to the testing objective of the test plan.

Another example is a false positive found during system release testing, where the DA node was operating as usual, but after rigorous review, it appeared it wasn't.

Additionally, system testing helps ensure that the network is capable of handling a large number of transactions and users.

System testing is a critical part of the testing process for blockchains. It helps ensure that the network functions as expected and that any issues are identified and resolved before deployment. By testing blockchains at the system level, we can ensure their successful adoption and use by communities.

This level of testing is regarded as the most expensive and should not have the same coverage as Unit/Integration. This is especially true for modular designed blockchains as more context should be accounted for when designing system tests.

We will cover this level of testing in more detail in Part 2 of this series.

Future of testing modular blockchains

As blockchains continue to evolve, testing must also evolve. New testing strategies and frameworks will emerge, and existing ones will be refined. The testing process will become more automated and streamlined, reducing testing time and cost.

As blockchains become more widely adopted, the need for standardized testing frameworks will become more apparent. Standardized frameworks will help testing scale to more types of networks and use cases. If you enjoyed this introductory post, in part 2 we will explain how Celestia Labs uses a tool called Testground to run large scale network tests.