The art of simplicity

In JUnit you have the @RepeatedTest annotation. This annotation allows you to run a single test method multiple times with different execution contexts. Unlike simply calling a method in a loop, each repetition is treated as a separate test execution with its own lifecycle. Although it can be useful to discover and investigate race conditions, I never had a good reason to start using this kind of functionality. But with the introduction of AI workloads inside my applications, times have changed. As AI output is less deterministic, it now starts to make sense to run the same test multiple times as the AI output could differ from test run to test run. Let me show you how to do this using NUnit and XUnit... NUnit: Built-in Repeat Attribute NUnit provides the most elegant solution with its built-in [Repeat] attribute: XUnit: Using Theory and Custom Attributes XUnit doesn't provide an out-of-the-box equivalent to the @RepeatedTest annotation but you can build your own ...

After installing .NET 7 on my development machine, the NUnit tests for some of my projects started to fail. A look at the test logs showed the following error message: ========== Starting test discovery ========== NUnit Adapter 4.2.0.0: Test discovery starting Exception System.TypeInitializationException, Exception thrown discovering tests in C:\Users\bawu\source\repos\NationalRegisterNumber\NationalRegisterNumberTests\bin\Debug\net48\NationalRegisterNumber.UnitTests.dll The type initializer for 'NUnit.Engine.Services.RuntimeFrameworkService' threw an exception.    at NUnit.Engine.Services.RuntimeFrameworkService.ApplyImageData(TestPackage package)    at NUnit.Engine.Services.RuntimeFrameworkService.SelectRuntimeFramework(TestPackage package)    at NUnit.Engine.Runners.MasterTestRunner.GetEngineRunner()    at NUnit.Engine.Runners.MasterTestRunner.Explore(TestFilter filter)    at NUnit.VisualStudio.TestAdapter.NUnitEng...

In this multipart blog post I want to introduce you in the world of property-based testing and how to do this in C#. In the first part , I gave you an introduction on what property-based testing is, why it useful and how it can help you write better code. In the second post , I showed you a concrete example on how to start writing property-based tests in C# using FsCheck . In a third post I explained how property-based tests can help  to find edge cases and to understand a codebase better. In this post I continue the journey by having a look at how to write our own generators. If you didn’t read my previous post, generators are the tool that helps you to select a value from an interval. For some of the available types in .NET, a default generator (and shrinker) exists but sometimes it is necessary to create your own generators. Create your own FsCheck generator in C# Creating your own generator for FsCheck is easy in C#, the only thing you need is a public static class wi...

In this multipart blog post I want to introduce you in the world of property-based testing and how to do this in C#. In the first part , I gave you an introduction on what property-based testing is, why it useful and how it can help you write better code. In the second post , I showed you a concrete example on how to start writing property-based tests in C# using FsCheck . Today I show you how I use property-based tests to find edge cases and can help to understand a codebase. Along the way I’ll share some of the other features that FsCheck has to offer. And to give you a more realistic example, I will use an open source library created by a colleague(thanks Willy for allowing me to use your library as an example); https://github.com/WilvanBil/NationalRegisterNumber . National Register Number is a package that can generate and validate Belgian national register numbers. The logic is based on Official Documentation by the Belgian Government The library is small and offers 2 AP...

In this multipart blog post I want to introduce you in the world of property-based testing and how to do this in C#. In the first part(this one), I’ll give you an introduction on what property-based testing is, why it useful and how it can help you write better code. In a second post, I’ll show you a concrete example on how to start writing Property based tests in C#. Disclaimer: Property-based testing is hard! And although I would strongly recommend it for critical parts of your codebase, I would certainly not use it everywhere. What is Property-based testing? Property-based testing was introduced in 2000 by Koen Claessen and John Hughes via the Haskell library QuickCheck . Mark Seemann uses the following definition in his Pluralsight course on Property based testing: Property-based Testing is an automated testing technique where you incrementally zero in on the correct behavior of a system by describing its properties or qualities in general terms and then use randomly...

NUnit allows you to bootstrap individual tests through the SetUp and TearDown attributes. I typically try to keep my test classes clean and one way to do this is by moving logic to a test base class. NUnit will walk through the inheritance tree and call all Setup and TearDown methods. Setup methods (both types) are called on base classes first, then on derived classes. If any setup method throws an exception, no further setups are called. Something I noticed in the documentation : Teardown methods (again, both types) are called on derived classes first, then on the base class. The teardown methods at any level in the inheritance hierarchy will be called only if a setup method at the same level was called. The following example is illustrates the difference. So if you have a test structure like below, only the base SetUp and TearDown are called: This is a breaking change from NUnit 2.x and something to be aware of…

The default rule when writing tests is ‘one test, one assert’. There are multiple reasons why this is a good idea: It becomes very clear what your test is doing and what the success or failure condition is If there are multiple asserts in your test, the test will fail on the first assert that returns false. The other assertions will never be validated. However this sometimes lead to duplication where you are writing the same test multiple times to assert different aspects. In NUnit you can avoid the second reason by using ‘ Assert.Multiple() ’. NUnit will store any failures encountered in the Multiple block and report all of them together. If multiple asserts failed, then all will be reported.

When trying to review some code from a colleague, I first tried to run the available unit tests in Visual Studio. The tests were successfully discovered but when I tried to execute them, nothing happened. In the test output window I noticed the following message: No test matches the given testcase filter This error made me think that somewhere a testcase filter was set. But where? I started searching inside the available settings and config files but I couldn’t find anything special. The error message mislead me, the problem was not related to any kind of testcase filter. It turned out that the problem was caused by the fact that the NUnit Test Adapter wasn’t installed. I opened the Package Explorer and installed the NUnit3TestAdapter .

I recently started using XUnit . I’m still discovering what is possible using this unit test framework. Before I was using NUnit, although I really liked it I always forgot what exact attribute I needed to control the lifecycle of my tests. I had to look up in the documentation if I needed a SetupFixture , a OneTimeSetup or a Setup attribute. In XUnit there is a lot less magic going on and you can fallback to standard .NET idioms like the usage of the constructor and the IDisposable.Dispose() method. With these 2 you can already get quite far, but if necessary you can also use Class Fixtures : shared object instance across tests in a single class Collection Fixtures : shared object instances across multiple test classes

Inside my (.NET Core) unit tests I wanted to load my configuration. I created a small helper method that loads my configuration from a json file: To make this code compile, you have to add 2 NuGet packages to your test project: To enable the SetBasePath method: https://www.nuget.org/packages/Microsoft.Extensions.Configuration.FileExtensions/ To enable the AddJsonFile method: https://www.nuget.org/packages/Microsoft.Extensions.Configuration.Json/ Now you can invoke this method inside your test setup (I’m using NUnit so I use the OneTimeSetUp method) and pass on the TestDirectory (which is through TestContext.CurrentContext.TestDirectory for NUnit): Remark: Don’t forget to set the appsettings.json file properties to ‘Copy Always’

NUnit supports parameterized tests through the TestCase attribute. This allows you to specify multiple sets of arguments and will create multiple tests behind the scenes: However the kind of information that you can pass on through an attribute is rather limited. If you want to pass on complex objects you need a different solution. In that case (no pun intended) you can use the TestCaseSource attribute: TestCaseSourceAttribute is used on a parameterized test method to identify the source from which the required arguments will be provided. The attribute additionally identifies the method as a test method. The data is kept separate from the test itself and may be used by multiple test methods. An example:

I’m currently working on a project where I’m using Akka.NET and the Actor model. I’m amazed by the ease-of-use and how the actor model itself makes complex problems easy to implement. Anyway, I lost some time today investigating a problem that a colleague had. After installing TestKit and writing his first actor based unit test, the Test Runner explorer refused to work and the only output we got was: No tests found to run. After a few minutes I realized our mistake, we downloaded the Akka.TestKit.NUnit which is suitable for NUnit 2 and older. However we are using NUnit3, which requires a different NuGet package; Akka.TestKit.NUnit3 .  After installing the correct NuGet package, our tests appeared and the problem was gone. Kind of stupid to stumble over this problem because I had the same issue before (and even mentioned it explicitly in a previous blog post about Akka.NET; http://bartwullems.blogspot.be/2016/07/testing-your-akkanet-actors-using.html )

After upgrading to NUnit 3, I noticed that no Tracing information was captured and written to the Test Output. (Note: I didn’t verify but I’m quiet sure it worked when using NUnit 2.x) After searching through the documentation and Issues list on GitHub , I found the following quote: “Currently, the "channels" we capture are Console.Out, Console.Error and TestContext.Out. ” So indeed no Trace.Write(Line) in the list. Let’s check if this statement is correct… Here is my test code: And let’s now have a look at the test results: Trace.WriteLine – no output available : TestContext.WriteLine – with output :

When trying to run my NUnit tests on the build server, my build turned red with the following error message inside the Test step: 2016-04-28T08:46:53.3339665Z ##[error]Error: Exception System.ArgumentException, Exception thrown executing tests 2016-04-28T08:46:53.3339665Z ##[error] 2016-04-28T08:46:53.3349431Z ##[error]Error: Illegal characters in path. 2016-04-28T08:46:53.3349431Z ##[error] This problem is caused by a bug in the NUnit 3.0.8 test adapter when used through the NuGet package(and not through the VSIX extension). Luckily a fix is already available, so upgrade to the latest version of the NUnit test adapter and the problem should disappear.

After upgrading to NUnit 3.0, which became recently available, some of my tests started to fail. When I look at the test results, they all failed with the following error message: SetUpAttribute attribute not allowed in a SetUpFixture In NUnit 3.0, there are some breaking changes regarding the usage of the SetUpAttribute and the SetUpFixture.  You can no longer use the SetUpAttribute and TearDownAttribute inside a SetUpFixture. Instead you have to use the OneTimeSetUpAttribute and OneTimeTearDownAttribute. In NUnit 2.0, I had the following code: In NUnit 3.0, this has to change to: More information can be found here: https://github.com/nunit/nunit/wiki/SetUp-and-TearDown-Changes https://github.com/nunit/nunit/wiki/Breaking-Changes