Onchain fuzzing benchmark: running echidna on mainnet

In this article, we compare echidna's performance on mainnet in optimization mode. Our analysis finds that using a local node can be 2x-10x more effective than a RPC provider.

In July 2023, Trail of Bits released an exciting update for Echidna, their Ethereum smart contract fuzzer. Version 2.1.0 introduced the capability to fuzz on-chain code, potentially replicating real-world hacks with minimal guidance. This feature opened new avenues for testing smart contracts under more realistic conditions.

Key Feature: Optimization Mode

One notable feature is Echidna's “optimization mode.” Here, an attacker contract is designed to maximize a profit function, effectively extracting value from a victim contract. Here’s how the function looks:

    // The optimization function
    function echidna_optimize_extracted_profit() public returns (int256) {
        return (int256(StaxLP.balanceOf(address(this))) -
            int256(initialAmount));
    }

This function calculates the profit by comparing the current balance of the contract with an initial amount, simulating an attack scenario where the contract tries to maximize its balance.

An intriguing question arises: How does this mode perform against mainnet contracts, and how can its performance be maximized to either break or validate protocol invariants, as an attempt to protect mainnet funds?

Our approach

To evaluate this, we set up a benchmark comparing the performance of Echidna when running on a local node versus an RPC provider.

The full benchmark repository is available here, and it contains the same target contract as the one from Trail of Bits’ original blog post.

Tools and setup

We utilize reth, one of the fastest implementations of the Ethereum protocol, for our local node testing.

For the remote node, our initial trials with Ankr and Tenderly did not yield great results, showing significantly lower performance than what we expected. This led us to believe that they were using a different node implementation. After some research, we opted for llamanodes, which also used reth, ensuring consistency in our benchmark.

Running a node on the cloud

A public repository to launch a reth node on AWS is available at aviggiano/ethereum-node-cloud. This was used to ensure echidna could run on the same instance as the Ethereum node, which was not possible with existing managed solutions from AWS.

Benchmark results

After running Echidna in optimization mode, targeting both a local and a remote reth node on a m7i.xlarge instance on AWS, with N=10, we arrive at the following results:

This chart shows the Fuzzing percentage out of a test limit of 1,000,000 over Time.

As we can see, the local node consistently outperforms the RPC provider across all test runs. The degree of improvement ranges from 2x to as much as 10x faster than the remote setup. As expected, the performance varies between runs, as it depends on the randomness within the fuzzing process.

Another observation is regarding the consistency of each setup. The local node shows a much lower variance in performance across all runs than the remote node. This means finding and fixing mainnet bugs can be more reliable and faster.

Continuous fuzzing on mainnet

If you are interested in on-chain fuzzing, reach out to Recon so that you can continuously check your invariants on mainnet 24/7.