The art of simplicity

I typically write my unit tests using the AAA(Arrange-Act-Assert) pattern. This pattern splits a test in 3 sections: The Arrange section of a unit test method initializes objects and sets the value of the data that is passed to the method under test. The Act section invokes the method under test with the arranged parameters. The Assert section verifies that the action of the method under test behaves as expected. Here is an example from one of my projects using XUnit : In the example above you can see that I include the 3 sections of the AAA pattern inside the test method itself. Recently I was reading a blog post by Jeremy Miller where I noticed he was using a different approach to separate the 3 sections: In the example above, Jeremy is using the IAsyncLifetime feature of XUnit to split the 3 sections: This also works when you don't need async logic by using the constructor and the regular IDisposable interface: What I like about this approach is...

When executing my XUnit tests on the build server, it failed with the following message: System.IO.FileNotFoundException : Could not load file or assembly 'Microsoft.VisualStudio.CodeCoverage.Shim, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified. Inside my csproj file following packages were referenced: <PackageReference Include="xunit" Version="2.4.0" /> <PackageReference Include="xunit.runner.visualstudio" Version="2.4.0" /> <PackageReference Include="coverlet.collector" Version="1.0.1" /> The ‘xunit.runner.visualstudio’ implicitly has a dependency on the Microsoft.NET.Test.Sdk(at minimum version 15.0) what could explain why he tried to load the assembly mentioned above. To get rid of this error, I had to explicitly include a reference to the ‘Microsoft.NET.Test.Sdk’ : <PackageReference Include="Mic...

So far I’ve never used the ‘using static’ directive introduced in C# 6. To simplify the assertions of my tests I created a static TestHelper: I’m using this TestHelper to simplify my NetArchTests (but more about that in another blog post). Inside my tests I can now do the following: Neat!

Integration Testing can be quite cumbersome, especially when you have a lot of moving parts involved. To test my ElasticSearch code I was used to spin up a docker instance and discard it after the tests ran. Recently I changed my approach after receiving the following tip from a colleague(thanks Jasper!): https://github.com/poulfoged/elasticsearch-inside   ElasticSearch Inside is  a fully embedded version of Elasticsearch for integration tests. When the instance is created both the JVM and Elasticsearch itself is extracted to a temporary location and started. Once disposed everything is removed again. And despite what you may think, this happens really fast(a few seconds on my machine). How to Install the Nuget package: Install-Package elasticsearch-inside After that you have to create a new instance of the ElasticSearch class and wait for the Ready() method. using (var elasticsearch = new Elasticsearch()) { await elasticsearch.Ready(); } ...

On one of my projects, we noticed flaky test behavior where the test task somethings succeeded but sometimes failed to correctly initialize and run the test adapters. As the issue seemed related not to a specific test, but to the setup of the test task itself we had no clue what whas going on. By comparing the succeeded and failed cases, we noticed that when the test task failed, we had the following warning in our build log: Agent 'AgentXXX' is using a deprecated version '1.105.7'. You will not be able to use this version of agent with the next major version of Team Foundation Server. Migrate to the latest 2.x version of agent. For more information, see https://go.microsoft.com/fwlink/?linkid=851067 . This brought us to the solution. We had 2 build agents running on our build server. One agent was an older 1.x version  where the other one was using the 2.x version. Every time the build was triggered on the new agent it succeeded, where it failed on the old on...

I’ve always been an NUnit or XUnit user, but from time to time I take another look at MSTest to see where Microsoft is going with their Test framework. With the latest version MS-Test 2, Microsoft introduced a new feature: parametrized tests (Do I have to mention that this feature exists for a long time in NUnit and XUnit? ). Let’s try it: Create a new Test project in Visual Studio 2017. Th test Project template already includes the MsTest.TestAdapter and MsTest.TestFramework NuGet packages. Right click on your project, select Manage NuGet packages… Update the test packages to the latest version. The default packages are a little bit outdated. Let’s write our parameterized test: Notice that there are some small differences: Instead of the TestMethod attribute, we are using the DataTestMethod attribute Instead of a method without parameters, we expect one parameter as part of our test. Now we have to provide the different possible values ...

After upgrading our unit tests to MSTest v2 we noticed that our tests were no longer discovered by the VSTest task on our build agent. As a solution, we decided to invoke the test execution ourself. Therefore I added 2 tasks to our Build definition: One command line task to execute dotnet test One task to collect and publish the test results In the command line task I configured the following settings: To execute the dotnet command we specify ‘dotnet’ as the Tool We also specify the following arguments: test: we want to execute the test commando --no-restore: the package restore already happened in a previous build step and shouldn’t be re-executed here --no-build: assembly compilation already happened in a previous build step and shouldn’t be re-executed here --logger:trx: output the test results in the trx format A last important setting that we change is the ‘Continue on error’ setting is set to true. If we don’t do t...

Thanks to the Chutzpah test adapter , you can run your QUnit, Jasmine or Mocha unit tests inside Visual Studio using the built-in tools like the Unit Test Explorer. I’m using this VSIX for years, and it works great. But what if you want to run these same tests as part of your build process? I’ll explain the steps to get it up and running inside TFS Build vNext. Start by opening the solution that contains the test project in Visual Studio. Add the NuGet package for the Chutzpah test runner to your solution, this is a solution level package, so does not need to be associated with any project. Don’t forget to check-in the change into source control Now go to the TFS web portal. Open your build definition and add a Visual Studio Test build task: Configure the Test Assembly parameter inside the task to search for your JavaScript tests. All my JavaScript tests have ‘test’ as part of their filename so I used the following pattern; **\$(BuildConfiguration)\*...

To run our JavaScript unit tests(you have them right?) inside Visual Studio we are using Chutzpah . Chutpzah is a JavaScript test runner which enables you to run unit tests using QUnit, Jasmine, Mocha, CoffeeScript and TypeScript(!). Chutzpah has a test adapter for the Visual Studio Unit Test Explorer which makes it possible to run your JavaScript unit tests among your other tests. This week I spent some time in making these tests also work on the build server. Here are some of the lessons I learned among the way: Tip 1  - Add the Chutzpah nuget package as a reference to your unit test project Add the Chutzpah nuget package to the project that contains your JavaScript unit tests. This NuGet package contains all required DLL’s and tooling to be able to run your JavaScript tests. By using this approach you don’t need to install the Chutzpah test adapter on your build server or specify the location of your test adapter. The TFS Build testing tools will automatically detect...

When trying to run my NUnit tests on the build server, my build turned red with the following error message inside the Test step: 2016-04-28T08:46:53.3339665Z ##[error]Error: Exception System.ArgumentException, Exception thrown executing tests 2016-04-28T08:46:53.3339665Z ##[error] 2016-04-28T08:46:53.3349431Z ##[error]Error: Illegal characters in path. 2016-04-28T08:46:53.3349431Z ##[error] This problem is caused by a bug in the NUnit 3.0.8 test adapter when used through the NuGet package(and not through the VSIX extension). Luckily a fix is already available, so upgrade to the latest version of the NUnit test adapter and the problem should disappear.

After upgrading to NUnit 3.0, which became recently available, some of my tests started to fail. When I look at the test results, they all failed with the following error message: SetUpAttribute attribute not allowed in a SetUpFixture In NUnit 3.0, there are some breaking changes regarding the usage of the SetUpAttribute and the SetUpFixture.  You can no longer use the SetUpAttribute and TearDownAttribute inside a SetUpFixture. Instead you have to use the OneTimeSetUpAttribute and OneTimeTearDownAttribute. In NUnit 2.0, I had the following code: In NUnit 3.0, this has to change to: More information can be found here: https://github.com/nunit/nunit/wiki/SetUp-and-TearDown-Changes https://github.com/nunit/nunit/wiki/Breaking-Changes