Testing smart contracts: unit, fuzz, invariants and static analysis

A contract with money cannot be "released and fixed later": after deployment the code cannot be corrected, and a bug costs real funds. So testing takes more time than writing code. Let us go through the layers of verification using our escrow as the example - the Foundry toolkit, static analysers and a way of thinking.

Layer 1. Unit tests

The most basic layer is a test for every scenario and every error. In Foundry tests are written in Solidity. Below are simplified examples in the spirit of our tests:

function test_ConfirmDelivery_PaysSellerMinusFees() public {
    uint256 id = _createAndAccept(1_000e6);   // a 1000 USDT deal

    vm.prank(buyer);
    escrow.confirmDelivery(id);

    // platform 1% = 10, guarantor 1% of 990 = 9.9, seller 980.1
    assertEq(usdt.balanceOf(seller), 980_100_000);
    assertEq(usdt.balanceOf(arbiter), 9_900_000);
}

function test_ConfirmDelivery_RevertsForNonBuyer() public {
    uint256 id = _createAndAccept(1_000e6);

    vm.prank(seller);
    vm.expectRevert(Escrow.NotAuthorized.selector);
    escrow.confirmDelivery(id);
}

Read them as a specification: what must happen and what must not. The more "negative" tests - "wrong caller", "wrong status", "zero amount" - the more reliable the contract.

Layer 2. Fuzzing (fuzz tests)

A human thinks of a limited number of cases. A fuzzer generates hundreds of random inputs itself and looks for one on which the test fails. In Foundry it is enough to declare arguments on the test function:

function testFuzz_SplitPartsSumToWhole(
    uint96 amount, uint16 arbiterBps, uint16 buyerBps
) public view {
    arbiterBps = uint16(bound(arbiterBps, 0, 3000));   // allowed rates
    buyerBps   = uint16(bound(buyerBps, 0, 10_000));

    (uint256 b, uint256 s, uint256 a, uint256 p) = harness.split(amount, arbiterBps, buyerBps);

    assertEq(b + s + a + p, amount);   // the parts always equal the whole
}

Fuzzing is exactly what catches edge cases: amounts of 1 wei, maximum values, rounding. In our project every such test runs 512 times on random data on every run.

Layer 3. Invariants

Fuzzing checks one function. But real bugs live in the interaction of functions: "created, accepted, disputed, cancelled, withdrew…". An invariant test gives the fuzzer a set of permitted actions and calls them randomly, in random order and with different participants, checking after every step the properties that must never change.

The escrow's main invariant is solvency:

function invariant_ContractIsSolvent() public view {
    assertGe(token.balanceOf(address(escrow)), escrow.reserved(address(token)));
}

"The contract's balance is never less than the sum of all its obligations." Whatever the participants do, in whatever order they call functions, this must stay true. Our invariant tests run 256 series of 128 calls each - tens of thousands of random actions.

Layer 4. "Evil" environments

To check protection against non-standard tokens and recipients, mocks are written - stand-in contracts with hostile behaviour:

• a token without a returned bool (like USDT);
• a token with a fee on transfer;
• a token with an address blacklist;
• a recipient whose receive() reverts or burns gas;
• a recipient re-entering the contract (a reentrancy check).

If the escrow survived all these mocks, then the defence from the article on non-standard tokens works on more than paper.

Layer 5. Regressions and exploits

When a bug is found it is first reproduced with a test that fails, then the code is fixed, and the test stays forever. This way the bug does not return. Our repository also has tests for the first version's vulnerabilities, run right on the EVM: they show clearly what was broken and how it was fixed.

Layer 6. Static analysis

Analysers read code without running it and look for known bug patterns.

Slither is a fast analyser: it finds reentrancy, uninitialised variables, dangerous calls, order-of-action violations. It produces many findings, some of which are false positives. Each finding must be examined and justified: fixed, or explained why it is safe.

Mythril does symbolic execution: instead of concrete numbers it explores execution paths with "abstract" values and looks for inputs that break the code. Slower, but it finds what others miss.

The results of the review are written down in the repository - a document showing the train of thought, not just a "passed" tick.

Layer 7. Gas and bytecode checks

Separate tests measure the gas of key operations so a stray edit does not make the service costlier. And an opcode-checking script ensures the bytecode has no instructions newer than the chosen EVM version: otherwise the contract would not work on older networks (more in the article on CREATE2).

What tests do not prove

An honest word on the limits.

• Tests do not prove the absence of bugs. They show that the checked scenarios work. Unforeseen ones may lurk.
• Tools are limited. Static analysis does not understand business logic: it will not notice that the guarantor gets the wrong amount if the code is technically correct.
• The specification may be wrong. Tests check that code does what is written in them. If the requirements are wrong, the tests will be wrong too.

So before working with large sums an independent audit is necessary - review by other eyes - followed by bug-bounty programmes. These are separate stages, and no set of automatic checks replaces them.

How to think when designing

A good contract starts with the question not "how do I write this" but "what must always be true". From such statements come invariants, from invariants tests, from tests confidence.

Examples of statements for escrow:

• a deal's payout happens exactly once;
• the sum of payouts always equals the deal amount;
• the platform cannot take more than accumulated fees;
• the guarantor cannot be a participant of the deal;
• from the "Settled" state there is no exit.

What next

Tests do not replace independent review: audits and bug bounties. For practice - the Foundry tutorial. You have walked the whole path: from "what is a blockchain" to the virtual machine, the escrow code and verification methods. Now is the time to apply the knowledge: read the guides by role and create your first deal.