Skip to main content

Model based testing in C#

In my continuous journey to design and write better code I try and experiment with multiple testing techniques. A while ago I wrote a whole series of blog posts about Property based testing, an addition to the traditional example-based testing.

A property based test is meant to be fairly succinct and verify that simple properties hold true for all possible inputs given a set of preconditions. Libraries like FSCheck and CSCheck by randomizing the inputs for a particular operation.

But what if we need to test a more complicated scenario where it is not so easy to identify the separate properties of our system? This is where model based testing can help us. Instead of randomizing the input for one operation, we execute an arbitrary combination of operations against our system and compare it to a simplified model.

Let me show you how to do this using CsCheck.

I have created a small Counter with two operations: Increase() and Decrease():

public class Counter
{
private int _currentValue;
public int CurrentValue { get => _currentValue; }
public Counter(int initialValue = 0)
{
_currentValue = initialValue;
}
public int Increment()
{
_currentValue+=1;
return _currentValue;
}
public int Decrement()
{
_currentValue -= 1;
return _currentValue;
}
public void Reset()
{
_currentValue = 0;
}
public override string ToString()
{
return $"Counter = {CurrentValue}";
}
}
view raw Counter.cs hosted with ❤ by GitHub

We now define a “simplified” model based on our mental model on how we think the system should work:

public class CounterModel
{
public int Value { get; private set; }
public CounterModel(int initialValue = 0)
{
Value = initialValue;
}
public void Increment()
{
Value++;
}
public void Decrement()
{
if (Value > 0)
{
Value--;
}
else
{
throw new InvalidOperationException("Counter cannot be decremented below zero.");
}
}
public void Reset()
{
Value = 0;
}
public override string ToString() => Value.ToString();
}
view raw CounterModel.cs hosted with ❤ by GitHub

Remark: The example I using is quite contrived  as the real “complex” system and the “simplified” model are in this case the same.

Now we need to specify a list of operations that can be executed in arbitrary combinations:

// Define the operations that can be applied to the Counter and the model.
var operations = new List<GenOperation<Counter, CounterModel>>{
Gen.Operation<Counter, CounterModel>(
(counter, model) =>
{
counter.Increment();
model.Increment();
}),
Gen.Operation<Counter, CounterModel>(
(counter, model) =>
{
try
{
counter.Decrement();
model.Decrement();
}
catch (InvalidOperationException)
{
// Handle the exception thrown by Decrement() when n <= 0
}
}),
Gen.Operation<Counter, CounterModel >(
(counter, model) =>
{
counter.Reset();
model.Reset();
})
};
view raw Operations.cs hosted with ❤ by GitHub

The last step is configure CSCheck to execute the operations against both the real system and  the model and compare the results.

// Generate and apply random sequences of operations to both the Counter and the model.
// The test will check that the Counter's state matches the model's state after each operation.
Check.SampleModelBased<Counter, CounterModel>(
Gen.Int.Select(initial => (new Counter(initial), new CounterModel(initial))),
operations.ToArray(),
(counter, model) => model.Value == counter.CurrentValue,
iter: 10000); // Adjust the number of iterations as needed
view raw Sample.cs hosted with ❤ by GitHub

As you can see the test fails because there is a difference between how the real system and the model handles negative values:

The full example can be found here: wullemsb/modelbasedtesting: A small example demonstrating how to use model based testing with CSCheck (github.com)

For a more realistic example using FsCheck, check out this blog post by Arron Stannard:

https://aaronstannard.com/fscheck-property-testing-csharp-part2/

Popular posts from this blog

Kubernetes–Limit your environmental impact

Reducing the carbon footprint and CO2 emission of our (cloud) workloads, is a responsibility of all of us. If you are running a Kubernetes cluster, have a look at Kube-Green . kube-green is a simple Kubernetes operator that automatically shuts down (some of) your pods when you don't need them. A single pod produces about 11 Kg CO2eq per year( here the calculation). Reason enough to give it a try! Installing kube-green in your cluster The easiest way to install the operator in your cluster is through kubectl. We first need to install a cert-manager: kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.5/cert-manager.yaml Remark: Wait a minute before you continue as it can take some time before the cert-manager is up & running inside your cluster. Now we can install the kube-green operator: kubectl apply -f https://github.com/kube-green/kube-green/releases/latest/download/kube-green.yaml Now in the namespace where we want t...

Azure DevOps/ GitHub emoji

I’m really bad at remembering emoji’s. So here is cheat sheet with all emoji’s that can be used in tools that support the github emoji markdown markup: All credits go to rcaviers who created this list.

DevToys–A swiss army knife for developers

As a developer there are a lot of small tasks you need to do as part of your coding, debugging and testing activities.  DevToys is an offline windows app that tries to help you with these tasks. Instead of using different websites you get a fully offline experience offering help for a large list of tasks. Many tools are available. Here is the current list: Converters JSON <> YAML Timestamp Number Base Cron Parser Encoders / Decoders HTML URL Base64 Text & Image GZip JWT Decoder Formatters JSON SQL XML Generators Hash (MD5, SHA1, SHA256, SHA512) UUID 1 and 4 Lorem Ipsum Checksum Text Escape / Unescape Inspector & Case Converter Regex Tester Text Comparer XML Validator Markdown Preview Graphic Col...