Skip to main content

Property based testing in C#–Part 3

In this multipart blog post I want to introduce you in the world of property-based testing and how to do this in C#.

In the first part, I gave you an introduction on what property-based testing is, why it useful and how it can help you write better code. In the second post, I showed you a concrete example on how to start writing property-based tests in C# using FsCheck.

Today I show you how I use property-based tests to find edge cases and can help to understand a codebase. Along the way I’ll share some of the other features that FsCheck has to offer.

And to give you a more realistic example, I will use an open source library created by a colleague(thanks Willy for allowing me to use your library as an example); https://github.com/WilvanBil/NationalRegisterNumber.

National Register Number is a package that can generate and validate Belgian national register numbers. The logic is based on Official Documentation by the Belgian Government

The library is small and offers 2 API’s:

  • A NationalRegisterNumberGenerator.IsValid() method that allows you to check if the provided string is a national register number.
  • A Generate() that will return a random valid national register number.

When I opened the codebase, I immediately noticed the tests:

You see that these are typical examples(no pun intended) of example based tests. Some magical values where used as input to validate the test, but are we sure that we covered all edge cases? Let’s find out…

Willy used NUnit in his codebase. Luckily FsCheck has support for NUnit, so before we write our first property-based test let’s add the FsCheck.Nunit NuGet package:

dotnet add package FsCheck.NUnit

Let us first focus on the IsValid() method. When using FsCheck with NUnit, we don’t use the [Test] attribute but the [FsCheck.NUnit.Property] attribute:

This tests succeeds which is a good started but not unexpected. It would be quite a coincidence if we provide a pseudo-random string that is also a valid national registration number. Here as well you can use the Verbose property on the attribute to see the values used.

You can write the same test also like this:

As we now that the national register number is in fact a number, let’s give it another try with a numeric value:

Still succeeds! Let’s try some other values by specifying an Arbitrary but therefore I need to first explain what it is.

What are Arbitraries?

FsCheck uses a combination of generators and shrinkers to produce test data. Generators make a random choice of a value from an interval, with a uniform distribution.

Shrinkers come into the picture when FsCheck finds a set of values that falsify a given property. In that case it will try to make that value smaller than the original (random) value by getting the shrinks for the value and trying each one in turn to check that the property is still false. If it is, the smaller value becomes the new counter example and the shrinking process continues with that value.

Arbitraries brings a generator and shrinker together to be used in properties. FsCheck defines default arbitraries for some often used types. Some examples:

  • NegativeInt
  • NonNegativeInt
  • NonEmptyString
  • IntWithMinMax
  • NonEmptyArray

Check the code on Github for the full list.

To use a specific arbitrary we can pass it to the Prop.ForAll method:

In the example above we used an Arbitrary that includes the Int.MinValue and Int.MaxValue in our test data.

So far, so good. Nothing unexpected with the IsValid() method. Let’s now switch our focus to the Generate() method.

The Generate() method has multiple overloads:

Generate()
Generate(DateTime birthDate)
Generate(BiologicalSex sex)
Generate(DateTime birthDate, BiologicalSex sex)
Generate(DateTime minDate, DateTime maxDate)
Generate(DateTime minDate, DateTime maxDate, BiologicalSex sex)
Generate(DateTime birthDate, int followNumber)

Let us first focus on the one that allows us to specify a DateTime.

Our first test fails because the Generate method expects DateTime values before today.

We'll fix our property-based test to filter out these DateTime values in the future:

However this time when we run our tests, it still fails:

If we have a look at the full test output, we see the concept of shrinking in action:

You can see that we start with a full datetime including date and time and the shrinker will set first the seconds, minutes and then hours to 0.

Mmmh. This seems to be a bug. I can use the NationalRegisterNumberGenerator to generate an invalid number. Just to be sure I created an example based test that reproduces the problem I found:

Time to contact Willy to investigate the root cause of the bug!

This post is already quite long, so I’ll show you how to build your own generators in a next post.

Popular posts from this blog

XUnit - Assert.Collection

A colleague asked me to take a look at the following code inside a test project: My first guess would be that this code checks that the specified condition(the contains) is true for every element in the list.  This turns out not to be the case. The Assert.Collection expects a list of element inspectors, one for every item in the list. The first inspector is used to check the first item, the second inspector the second item and so on. The number of inspectors should match the number of elements in the list. An example: The behavior I expected could be achieved using the Assert.All method:

Azure DevOps/ GitHub emoji

I’m really bad at remembering emoji’s. So here is cheat sheet with all emoji’s that can be used in tools that support the github emoji markdown markup: All credits go to rcaviers who created this list.

Angular --deploy-url and --base-href

As long you are running your Angular application at a root URL (e.g. www.myangularapp.com ) you don’t need to worry that much about either the ‘--deploy-url’ and ‘--base-href’ parameters. But once you want to serve your Angular application from a server sub folder(e.g. www.mywebsite.com/angularapp ) these parameters become important. --base-href If you deploy your Angular app to a subfolder, the ‘--base-href’ is important to generate the correct routes. This parameter will update the <base href> tag inside the index.html. For example, if the index.html is on the server at /angularapp/index.html , the base href should be set to <base href="/angularapp/"> . More information: https://angular.io/guide/deployment --deploy-url A second parameter that is important is ‘--deploy-url’. This parameter will update the generated url’s for our assets(scripts, css) inside the index.html. To make your assets available at /angularapp/, the deploy url should