Last updated on

The Swiss Cheese Model of Web3 Security: Where Mutation Testing Fits

The Swiss cheese model comes from safety engineering and risk management. You don’t rely on one perfect barrier. Instead, you stack multiple imperfect layers – processes, tools, and controls. Each has holes: limitations, blind spots, human error. Incidents happen when the holes line up, and a hazard passes through all the slices.

Recently, I posted a tweet about how the Swiss cheese model maps beautifully to Web3 security and testing. The next day, I opened Mastering Ethereum, 2nd Edition, and the first line I hit was:

Think of robust development as the first layer in a ‘Swiss cheese model’…

I smiled. Different context, same intuition: security is layered, and development/testing is one of those slices.

This post is about what that means for Web3, and where mutation testing and Mutorium Labs fit into the picture.

The Swiss Cheese Model in One Paragraph

In the Swiss cheese model:

  • Each slice represents a defensive layer (a process, a tool, a control).
  • Each slice has holes – it fails in specific ways.
  • A bad outcome occurs when a threat trajectory lines up with holes in every slice.

Translated to Web3:

  • The threat might be “loss of funds”, “broken invariant”, or “invalid proof accepted”.
  • The slices include specs, tests, audits, and monitoring.
  • An exploit happens when one specific bug slips through all of them.

The takeaway is uncomfortable and straightforward:

No single technique – not tests, not audits, not formal proofs – is enough on its own.

Layers of Swiss Cheese in Web3

If we sketch a realistic stack of slices for a serious Web3 / ZK project, it might look like this:

  • Specification & threat modeling
    Clear invariants, assumptions, and attacker capabilities.

  • Code review & pair programming
    Human reasoning, fresh eyes, and shared understanding.

  • Unit & integration tests
    Check intended behavior on known paths and edge cases.

  • Property-based tests & fuzzing
    Generate many weird inputs you wouldn’t think of manually.

  • Mutation testing & test-quality tooling
    Evaluate how effective your tests and fuzzers really are, catch certain classes of issues, and force you to close blind spots.

  • Static analysis & linters
    Catch common bug patterns and dangerous constructs automatically.

  • Formal verification (where it makes sense)
    Prove specific properties for critical components.

  • Independent audits
    External teams with different mental models and threat intuitions.

  • Bug bounties & production monitoring
    The final net once code meets the real world.

Each of these layers is valuable. Each also has holes: things it doesn’t see, doesn’t model, or gets wrong.

In the Swiss cheese view, unit & integration tests, fuzzing, and mutation testing are three closely related slices in the “testing family”, but they still fail in different ways. Mutation testing is the only one that explicitly probes the weaknesses of the other two – and that’s the area we at Mutorium Labs are obsessed with.

The Problem With “Tests Are Green”

Most teams treat testing as binary:

  • Red = bad, fix it.
  • Green = good, ship it.

But “all tests are green” doesn’t tell you how strong those tests are.

Some of the holes in the testing slices:

  • Large parts of the code are never exercised.
  • Tests assert that a function returns, but not that it returns the right thing.
  • Fuzzers explore inputs but don’t actually fail when behavior changes in subtle ways.
  • Tests encode the same misconceptions as the code itself.

In Swiss cheese terms: the testing-related slices might have much bigger holes than you think, aligned perfectly with certain classes of bugs.

We need a way to measure those holes, not just hope they’re small.

Mutation Testing: Measuring the Holes

This is where mutation testing comes in.

The basic idea:

  1. Take your contract or circuit.
  2. Make a small, mechanical change – a mutant:
    • Flip a comparison (>= → >),
    • Remove a condition,
    • Change a constant,
    • Swap an operation, etc.
  3. Run your existing test suite (and fuzzers) against the mutant.
  4. If the tests fail, the mutant is killed. If they still pass, the mutant survives.

A surviving mutant means:

We changed the behavior of the program, but none of our tests noticed.

That is literally a hole in one of the testing slices.

Aggregate this over many mutants, and you get a mutation score – a rough measure of how porous your testing layer is for that kind of defect.

  • High mutation score → tests kill most mutants → smaller and/or fewer holes.
  • Low mutation score → many mutants survive → bigger and/or more holes.

Mutation testing doesn’t replace tests, fuzzing, or audits.
It tells you how much trust you can put in your tests and fuzzers, and where they are weakest.

So in the Swiss cheese model, mutation testing earns its own slice:

  • As a defense, because it can reveal real issues that existing tests and fuzzers overlook.
  • And as a feedback mechanism, because it actively helps you shrink the holes in the neighbouring testing slices.

Where Mutorium Labs Fits Into the Swiss Cheese Stack

We started Mutorium Labs because we don’t just want more slices.
We want better slices – especially in the ZK/Web3 testing layer – and a dedicated mutation-testing slice that keeps those layers accountable.

Concretely, that means three things we’re working on:

1. noir-metrics – Understanding the Shape of the Slice

noir-metrics gives you structural insight into your Noir projects:

  • How big is the codebase?
  • How complex are the circuits?
  • How are tests organized relative to the code?

Think of it as a way to measure the slice before we start poking holes and inserting mutants. Metrics don’t secure anything by themselves, but they give you a map.

2. Coverage for Noir – Seeing Which Parts Are Touched

Coverage tooling (and the coverage PR work towards Noir itself) is about answering:

  • Which constraints, functions, and branches do our tests ever touch?
  • Which ones are never executed?

Uncovered code is like a region of the slice that hasn’t even been illuminated yet.
We don’t see the holes there because light never hits them.

3. zk-mutant – Generating and Tracking Mutants

zk-mutant is where the Swiss cheese metaphor really comes into play.

For Noir circuits and ZK applications, zk-mutant will:

  • Generate systematic mutants at the circuit/constraint level.
  • Run your existing tests & fuzzing harnesses against them.
  • Track which mutants are killed, which survive, and where they live in your code.
  • Surface weak spots in your testing strategy (and by extension, your security posture).

In other words, we’re building tools that help teams see and shrink the holes in their testing and mutation-testing slices inside the broader Swiss cheese stack.

Back to That Line in Mastering Ethereum

When I saw the line:

Think of robust development as the first layer in a ‘Swiss cheese model’…

Right after tweeting about the same model for Web3 security, it felt like a small confirmation.

Different authors, different backgrounds, same structure of thought:

  • Security is layered.
  • No single technique is enough.
  • Development and testing are critical slices, but they also have holes.

Mutorium Labs lives exactly in that gap: we’re not promising a magic bullet.
We’re working on making one specific area – testing and mutation testing – sharper, more measurable, and less porous.

Designing for Misalignment, Not Perfection

The goal of the Swiss cheese model is not to create a perfect slice.

The goal is to build:

  • Enough slices,
  • With holes in different places,
  • So that no single bug can line up all the way through.

Mutation testing fits that philosophy perfectly:

  • It doesn’t pretend to eliminate all bugs.
  • It provides feedback on where your testing assumptions are incorrect.
  • It nudges you toward deeper, property-driven tests and better fuzzing harnesses.

If you’re building serious ZK or Web3 systems, think in slices:

  • Specs, reviews, tests, fuzzers, mutation testing, static analysis, audits, bounties.
  • And within the testing-related slices, ask: how big are the holes, really?

That’s the question we’re trying to help answer at Mutorium Labs – with metrics, coverage, and mutation testing tools explicitly designed for Web3 and ZK.