Stateless vs. Stateful Fuzzing in Smart Contract Security

In this blog post, we will learn about the difference between stateless and stateful fuzzing, each with its own set of characteristics and benefits.

In this blog post, we will learn about the difference between stateless and stateful fuzzing, each with its own set of characteristics and benefits.

What is Stateless & Stateful Fuzzing?

• Stateless fuzzing operates without taking into account the system's internal state between test cases. Each test is run in isolation, with no memory of previous runs. This method focuses on functions or components separately, making it suitable for standalone unit testing.
  
  

• Stateful fuzzing maintains an awareness of the system's internal state across multiple test cases. It more closely resembles real-world scenarios by taking into account the cumulative impact of previous inputs. This approach seeks to investigate the interconnectedness of various system functions.
  

Advantages and disadvantages of stateless and stateful fuzzing

Aspect

Stateless Fuzzing

Stateful Fuzzing

Pros

Simple and quick implementation.

Well-suited for isolated function testing.

Mimics real-world scenarios.

Reveals vulnerabilities from complex interactions.

Cons

Limited in uncovering complex interactions.

Might miss vulnerabilities stemming from sequential operations.

Requires more extensive implementation effort.

Longer execution times due to cumulative state considerations.

Kinds of Bugs Detected

Bugs in isolated functions (e.g., arithmetic bugs).

Sequential dependencies (e.g., actions out of order), invariant violations, global state issues.

Why it's Preferred

Quick implementation for isolated testing.

Real-world simulation, holistic bug detection, comprehensive invariant testing.

Which should I choose?

• Stateful fuzzing is generally preferred when you want to simulate real-world usage scenarios, as it results in a more accurate representation of potential vulnerabilities that may arise during practical deployment. It investigates the complex interactions between functions, allowing it to detect bugs that span multiple components and improve the overall security posture. Invariant testing, a subset of stateful fuzzing, focuses on preserving system invariants, ensuring that critical properties remain constant across multiple operations.

• Stateless fuzzing is generally preferred to test libraries or small portions of your code, especially when it contains few possible states or scenarios.

Example: 

Detecting Sequential Dependencies

Consider a smart contract voting system in which a user must first register before voting. Individual issues in the registration or voting functions may be detected by stateless fuzzing. It may, however, overlook a bug in which voting occurs without prior registration. Stateful fuzzing would detect the sequential dependency violation by preserving the system state.

Solidity Code:

Let's implement a simple smart contract to illustrate the importance of stateful fuzzing. The contract maintains a counter that increments on each successful transaction.

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.0;

contract Counter {
  uint256 internal value;

  constructor() {
    value = 0;
  }

  function increment() public virtual {
    value += 1;
  }

  function double() public virtual {
    value *= 2;
  }
}

Counter Contract: This smart contract is a simple counter that allows users to increment the counter value by 1. It also includes a function to double the counter value. 

The contract has the following components:

1. value: A variable that stores the current value of the counter.

2. constructor: Initializes the value to 0 when the contract is deployed.

3. increment: A public function that increments the value by 1.

4. double: A public function that doubles the current value of the counter.

Echidna Invariant Fuzz Test:

Now, let's write an Echidna test case for the Counter contract to check the invariant "The counter is always incremented by 1" and catch the introduced bug. 

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.0;

import "./Counter.sol";

contract TestCounter is Counter {
  uint256 private lastValue;
  
  function increment() public override {
    lastValue = value;
    super.increment();
    // increment always increases 1 by 1
    assert(value == lastValue + 1);
  }

  function double() public override {
    lastValue = value;
    super.double();
    // double always multiplies by 2
    assert(value == lastValue * 2);
  }

  function echidna_counter_always_increases_value_one_by_one() public returns(bool) {
    return value > 0 ? value == lastValue + 1 : true;
  }
}

Terminal Command to Run:

echidna TestCounter.sol --contract TestCounter --test-mode property

Output:

Observation:

• The Counter contract was designed to maintain a value that can be incremented by 1 through the increment function. Additionally, it includes a double function that introduces a bug by doubling the counter value. 

• The TestCounter contract, which inherits from Counter, is used for testing with Echidna. The echidna_counter_always_increases_value_one_by_one function checks whether the counter is always incremented by 1, which is violated when the double function is called.

• This setup allows Echidna to identify the deviation from the expected behavior, showcasing the power of invariant testing in detecting subtle bugs in smart contracts.

Conclusion:

The choice between stateless and stateful fuzzing in smart contract security is determined by the specific use case of your test. While stateless fuzzing is useful for isolated testing, stateful fuzzing is a better choice in general due to its real-world simulation and comprehensive bug detection capabilities. A nuanced understanding of these fuzzing methodologies is required for a thorough and effective defense against potential vulnerabilities on the path to robust smart contract security.