Model based testing in C#

In my continuous journey to design and write better code I try and experiment with multiple testing techniques. A while ago I wrote a whole series of blog posts about Property based testing, an addition to the traditional example-based testing.

A property based test is meant to be fairly succinct and verify that simple properties hold true for all possible inputs given a set of preconditions. Libraries like FSCheck and CSCheck by randomizing the inputs for a particular operation.

But what if we need to test a more complicated scenario where it is not so easy to identify the separate properties of our system? This is where model based testing can help us. Instead of randomizing the input for one operation, we execute an arbitrary combination of operations against our system and compare it to a simplified model.

Let me show you how to do this using CsCheck.

I have created a small Counter with two operations: Increase() and Decrease():

	public class Counter
	{
	private int _currentValue;
	public int CurrentValue { get => _currentValue; }
	public Counter(int initialValue = 0)
	{
	_currentValue = initialValue;
	}
	public int Increment()
	{
	_currentValue+=1;
	return _currentValue;
	}
	public int Decrement()
	{
	_currentValue -= 1;
	return _currentValue;
	}
	public void Reset()
	{
	_currentValue = 0;
	}
	public override string ToString()
	{
	return $"Counter = {CurrentValue}";
	}
	}

view raw Counter.cs hosted with ❤ by GitHub

We now define a “simplified” model based on our mental model on how we think the system should work:

	public class CounterModel
	{
	public int Value { get; private set; }
	public CounterModel(int initialValue = 0)
	{
	Value = initialValue;
	}
	public void Increment()
	{
	Value++;
	}
	public void Decrement()
	{
	if (Value > 0)
	{
	Value--;
	}
	else
	{
	throw new InvalidOperationException("Counter cannot be decremented below zero.");
	}
	}
	public void Reset()
	{
	Value = 0;
	}
	public override string ToString() => Value.ToString();
	}

view raw CounterModel.cs hosted with ❤ by GitHub

Remark: The example I using is quite contrived  as the real “complex” system and the “simplified” model are in this case the same.

Now we need to specify a list of operations that can be executed in arbitrary combinations:

	// Define the operations that can be applied to the Counter and the model.
	var operations = new List<GenOperation<Counter, CounterModel>>{
	Gen.Operation<Counter, CounterModel>(
	(counter, model) =>
	{
	counter.Increment();
	model.Increment();
	}),
	Gen.Operation<Counter, CounterModel>(
	(counter, model) =>
	{
	try
	{
	counter.Decrement();
	model.Decrement();
	}
	catch (InvalidOperationException)
	{
	// Handle the exception thrown by Decrement() when n <= 0
	}
	}),
	Gen.Operation<Counter, CounterModel >(
	(counter, model) =>
	{
	counter.Reset();
	model.Reset();
	})
	};

view raw Operations.cs hosted with ❤ by GitHub

The last step is configure CSCheck to execute the operations against both the real system and  the model and compare the results.

	// Generate and apply random sequences of operations to both the Counter and the model.
	// The test will check that the Counter's state matches the model's state after each operation.
	Check.SampleModelBased<Counter, CounterModel>(
	Gen.Int.Select(initial => (new Counter(initial), new CounterModel(initial))),
	operations.ToArray(),
	(counter, model) => model.Value == counter.CurrentValue,
	iter: 10000); // Adjust the number of iterations as needed

view raw Sample.cs hosted with ❤ by GitHub

As you can see the test fails because there is a difference between how the real system and the model handles negative values:

The full example can be found here: wullemsb/modelbasedtesting: A small example demonstrating how to use model based testing with CSCheck (github.com)

For a more realistic example using FsCheck, check out this blog post by Arron Stannard:

https://aaronstannard.com/fscheck-property-testing-csharp-part2/