Skip to main content

Property based testing in C#–Part 3

In this multipart blog post I want to introduce you in the world of property-based testing and how to do this in C#.

In the first part, I gave you an introduction on what property-based testing is, why it useful and how it can help you write better code. In the second post, I showed you a concrete example on how to start writing property-based tests in C# using FsCheck.

Today I show you how I use property-based tests to find edge cases and can help to understand a codebase. Along the way I’ll share some of the other features that FsCheck has to offer.

And to give you a more realistic example, I will use an open source library created by a colleague(thanks Willy for allowing me to use your library as an example);

National Register Number is a package that can generate and validate Belgian national register numbers. The logic is based on Official Documentation by the Belgian Government

The library is small and offers 2 API’s:

  • A NationalRegisterNumberGenerator.IsValid() method that allows you to check if the provided string is a national register number.
  • A Generate() that will return a random valid national register number.

When I opened the codebase, I immediately noticed the tests:

You see that these are typical examples(no pun intended) of example based tests. Some magical values where used as input to validate the test, but are we sure that we covered all edge cases? Let’s find out…

Willy used NUnit in his codebase. Luckily FsCheck has support for NUnit, so before we write our first property-based test let’s add the FsCheck.Nunit NuGet package:

dotnet add package FsCheck.NUnit

Let us first focus on the IsValid() method. When using FsCheck with NUnit, we don’t use the [Test] attribute but the [FsCheck.NUnit.Property] attribute:

This tests succeeds which is a good started but not unexpected. It would be quite a coincidence if we provide a pseudo-random string that is also a valid national registration number. Here as well you can use the Verbose property on the attribute to see the values used.

You can write the same test also like this:

As we now that the national register number is in fact a number, let’s give it another try with a numeric value:

Still succeeds! Let’s try some other values by specifying an Arbitrary but therefore I need to first explain what it is.

What are Arbitraries?

FsCheck uses a combination of generators and shrinkers to produce test data. Generators make a random choice of a value from an interval, with a uniform distribution.

Shrinkers come into the picture when FsCheck finds a set of values that falsify a given property. In that case it will try to make that value smaller than the original (random) value by getting the shrinks for the value and trying each one in turn to check that the property is still false. If it is, the smaller value becomes the new counter example and the shrinking process continues with that value.

Arbitraries brings a generator and shrinker together to be used in properties. FsCheck defines default arbitraries for some often used types. Some examples:

  • NegativeInt
  • NonNegativeInt
  • NonEmptyString
  • IntWithMinMax
  • NonEmptyArray

Check the code on Github for the full list.

To use a specific arbitrary we can pass it to the Prop.ForAll method:

In the example above we used an Arbitrary that includes the Int.MinValue and Int.MaxValue in our test data.

So far, so good. Nothing unexpected with the IsValid() method. Let’s now switch our focus to the Generate() method.

The Generate() method has multiple overloads:

Generate(DateTime birthDate)
Generate(BiologicalSex sex)
Generate(DateTime birthDate, BiologicalSex sex)
Generate(DateTime minDate, DateTime maxDate)
Generate(DateTime minDate, DateTime maxDate, BiologicalSex sex)
Generate(DateTime birthDate, int followNumber)

Let us first focus on the one that allows us to specify a DateTime.

Our first test fails because the Generate method expects DateTime values before today.

We'll fix our property-based test to filter out these DateTime values in the future:

However this time when we run our tests, it still fails:

If we have a look at the full test output, we see the concept of shrinking in action:

You can see that we start with a full datetime including date and time and the shrinker will set first the seconds, minutes and then hours to 0.

Mmmh. This seems to be a bug. I can use the NationalRegisterNumberGenerator to generate an invalid number. Just to be sure I created an example based test that reproduces the problem I found:

Time to contact Willy to investigate the root cause of the bug!

This post is already quite long, so I’ll show you how to build your own generators in a next post.

Popular posts from this blog

DevToys–A swiss army knife for developers

As a developer there are a lot of small tasks you need to do as part of your coding, debugging and testing activities.  DevToys is an offline windows app that tries to help you with these tasks. Instead of using different websites you get a fully offline experience offering help for a large list of tasks. Many tools are available. Here is the current list: Converters JSON <> YAML Timestamp Number Base Cron Parser Encoders / Decoders HTML URL Base64 Text & Image GZip JWT Decoder Formatters JSON SQL XML Generators Hash (MD5, SHA1, SHA256, SHA512) UUID 1 and 4 Lorem Ipsum Checksum Text Escape / Unescape Inspector & Case Converter Regex Tester Text Comparer XML Validator Markdown Preview Graphic Color B

Help! I accidently enabled HSTS–on localhost

I ran into an issue after accidently enabling HSTS for a website on localhost. This was not an issue for the original website that was running in IIS and had a certificate configured. But when I tried to run an Angular app a little bit later on http://localhost:4200 the browser redirected me immediately to https://localhost . Whoops! That was not what I wanted in this case. To fix it, you need to go the network settings of your browser, there are available at: chrome://net-internals/#hsts edge://net-internals/#hsts brave://net-internals/#hsts Enter ‘localhost’ in the domain textbox under the Delete domain security policies section and hit Delete . That should do the trick…

Azure DevOps/ GitHub emoji

I’m really bad at remembering emoji’s. So here is cheat sheet with all emoji’s that can be used in tools that support the github emoji markdown markup: All credits go to rcaviers who created this list.