Friday, July 19, 2019

Azure DevOps Pipeline Build error: Unable to determine the location of vstest.console.exe

After replacing the existing build servers at one of my customers, some build started to fail with the following error message:

Unable to determine the location of vstest.console.exe ---> System.IO.FileNotFoundException: Unable to determine the location of vstest.console.exe

When looking at the build tasks, I noticed they were still trying to use Visual Studio 2015 to run their tests:

On this new server we only had Visual Studio 2017 and 2019 installed, so Visual Studio 2015 was a ‘no-go’. To fix it, we changed the VSTest version to ‘Latest’:

Thursday, July 18, 2019

Azure DevOps–Why is my database so big?

When preparing an Azure DevOps Server migration, the customer asked me why there Azure Devops databases grew so fast.

There are 2 queries that can help you answer this question.

  • The first query gives a general overview of the different parts:

The output of the first query returned the following results in this case:

Owner BlobSizeInMb
VersionCVersionControl 64992.008339881835
FileContainer 59599.479797363281
TeamTest 44243.018857001953
WorkItemTracking 23674.237445831054
CodeSense 989.945262908203
ProcessTemplate 125.949053763671
Generic 54.288738250000
  • The second query goes in more detail to differentiate between source control and build:

The output of the second query returned the following results in this case:

FileContainerOwner TotalSizeInMb
Build 56681.754142760742

Git

875.809092521484

vstfs:///Rele

683.627062796875

pipelines://b

271.543574333007

DistributedTask

1.143143653320

They started storing there build artifacts in the database, this explained why there Azure DevOps databases exploded so much.

Wednesday, July 17, 2019

Azure DevOps Pipelines–How the build agent detects Visual Studio related capabilities

Detecting if Visual Studio is installed on the build server can be quite painful. I’ve seen a few times where the build agent didn’t pick up the Visual Studio installation, making the build server useless. As Visual Studio is not added to the capabilities of the build agent, the specific agent is never used to conduct a build.

If everything is OK, you should see something like this in your Build Agent capabilities:

But how does the build agent detects that Visual Studio is there?

This is all done through vswhere.exe, a tool I blogged about before. Vswhere is installed as one of the tools of the build agent, but in case of trouble you can run it yourself to see what is going wrong.

Let’s try this:

  • Login on your build server and open the installation folder of one of your build agents.
  • Go to the externals\vswhere  subfolder. Here you should find the vswhere.exe
  • Let’s see what options are available by calling vswhere.exe –help:

PS D:\builds\dev-agent-1\externals\vswhere> .\vswhere.exe -help

Visual Studio Locator version 2.6.13+a6d40ba5f4 [query version 2.1.1046.44959]

Copyright (C) Microsoft Corporation. All rights reserved.

Usage: vswhere.exe [options]

Options:

-all           Finds all instances even if they are incomplete and may not launch.

  -prerelease    Also searches prereleases. By default, only releases are searched.

  -products arg  One or more product IDs to find. Defaults to Community, Professional, and Enterprise.

                 Specify "*" by itself to search all product instances installed.

                 See https://aka.ms/vs/workloads for a list of product IDs.

  -requires arg  One or more workload or component IDs required when finding instances.

                 All specified IDs must be installed unless -requiresAny is specified.

                 See https://aka.ms/vs/workloads for a list of workload and component IDs.

  -requiresAny   Find instances with any one or more workload or components IDs passed to -requires.

  -version arg   A version range for instances to find. Example: [15.0,16.0) will find versions 15.*.

  -latest        Return only the newest version and last installed.

  -sort          Sorts the instances from newest version and last installed to oldest.

                 When used with "find", first instances are sorted then files are sorted lexigraphically.

  -legacy        Also searches Visual Studio 2015 and older products. Information is limited.

                 This option cannot be used with either -products or -requires.

  -format arg    Return information about instances found in a format described below.

  -property arg  The name of a property to return. Defaults to "value" format.

                 Use delimiters ".", "/", or "_" to separate object and property names.

                 Example: "properties.nickname" will return the "nickname" property under "properties".

  -find arg      Returns matching file paths under the installation path. Defaults to "value" format.

                 The following patterns are supported:

                 ?  Matches any one character except "\".

                 *  Matches zero or more characters except "\".

                 ** Searches the current directory and subdirectories for the remaining search pattern.

  -nologo        Do not show logo information. Some formats noted below will not show a logo anyway.

  -utf8          Use UTF-8 encoding (recommended for JSON).

  -?, -h, -help  Display this help message.

Formats:

  json           An array of JSON objects for each instance (no logo).

  text           Colon-delimited properties in separate blocks for each instance (default).

  value          A single property specified by the -property parameter (no logo).

  xml            An XML data set containing instances (no logo).

  • Ok now that we know the available options, let’s have a look at the installed versions using vswhere.exe -all:

PS D:\builds\dev-agent-1\externals\vswhere> .\vswhere.exe -all

Visual Studio Locator version 1.0.62 [query version 2.1.1046.44959]

Copyright (C) Microsoft Corporation. All rights reserved.

instanceId: 1e1d4042

installDate: 14/06/2019

installationName: VisualStudio/16.1.2+29001.49

installationPath: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise

installationVersion: 16.1.29001.49

displayName: Visual Studio Enterprise 2019

description: Microsoft DevOps solution for productivity and coordination across teams of any size

updateDate: 2019-06-14T06:17:25.7201719Z

enginePath: C:\Program Files (x86)\Microsoft Visual Studio\Installer\resources\app\ServiceHub\Services\Microsoft.VisualS

tudio.Setup.Service

layoutPath: x:\vs2019

channelId: VisualStudio.16.Release

channelUri: https://aka.ms/vs/16/release/channel

installChannelUri: x:\vs2019\ChannelManifest.json

releaseNotes: https://go.microsoft.com/fwlink/?LinkId=660893#16.1.2

thirdPartyNotices: https://go.microsoft.com/fwlink/?LinkId=660909

  • That’s good news, it seems that a version of Visual Studio is installed. Unfortunately the build agent didn’t pick it up. So what is going wrong?
  • Let’s try to use the exact query that the build agent itself is using: vswhere.exe -version '[16.0,17.0)' -latest -format json

PS D:\builds\dev-agent-1\externals\vswhere> .\vswhere.exe -version '[16.0,17.0)' -latest -format json

[]

  • This returns an empty array. Strange! Let’s download and install the latest vswhere version from GitHub. This version has some extra features and can give us more details.
  • After downloading the latest vswhere.exe, let’s run vswhere.exe –all again:

PS D:\soft> .\vswhere.exe -all

Visual Studio Locator version 2.6.13+a6d40ba5f4 [query version 2.1.1046.44959]

Copyright (C) Microsoft Corporation. All rights reserved.

instanceId: 1e1d4042

installDate: 14/06/2019 8:17:25

installationName: VisualStudio/16.1.2+29001.49

installationPath: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise

installationVersion: 16.1.29001.49

productId: Microsoft.VisualStudio.Product.Enterprise

productPath: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\devenv.exe

state: 13

isComplete: 0

isLaunchable: 0

isPrerelease: 0

isRebootRequired: 0

displayName: Visual Studio Enterprise 2019

description: Microsoft DevOps solution for productivity and coordination across teams of any size

channelId: VisualStudio.16.Release

channelUri: https://aka.ms/vs/16/release/channel

enginePath: C:\Program Files (x86)\Microsoft Visual Studio\Installer\resources\app\ServiceHub\Services\Microsoft.VisualStudio.Setup.Service

installChannelUri: x:\vs2019\ChannelManifest.json

layoutPath: x:\vs2019

releaseNotes: https://go.microsoft.com/fwlink/?LinkId=660893#16.1.2

thirdPartyNotices: https://go.microsoft.com/fwlink/?LinkId=660909

updateDate: 2019-06-14T06:17:25.7201719Z

catalog_buildBranch: d16.1

catalog_buildVersion: 16.1.29001.49

catalog_id: VisualStudio/16.1.2+29001.49

catalog_localBuild: build-lab

catalog_manifestName: VisualStudio

catalog_manifestType: installer

catalog_productDisplayVersion: 16.1.2

catalog_productLine: Dev16

catalog_productLineVersion: 2019

catalog_productMilestone: RTW

catalog_productMilestoneIsPreRelease: False

catalog_productName: Visual Studio

catalog_productPatchVersion: 2

catalog_productPreReleaseMilestoneSuffix: 1.0

catalog_productSemanticVersion: 16.1.2+29001.49

catalog_requiredEngineVersion: 2.1.3095.63072

properties_campaignId: 1079615330.1554282809

properties_canceled: 1

properties_channelManifestId: VisualStudio.16.Release/16.1.2+29001.49

properties_nickname:

properties_operationMode: 0

properties_setupEngineFilePath: C:\Program Files (x86)\Microsoft Visual Studio\Installer\vs_installershell.exe

  • We get a lot more detail when using the latest version. One thing you should notice is the isComplete value which is 0. Something is definitely wrong with our installation.
  • Let’s compare the output with a server where everything is running fine:

D:\Builds\releases-agent-1\externals\vswhere>vswhere -all

Visual Studio Locator version 1.0.62 [query version 2.1.1046.44959]

Copyright (C) Microsoft Corporation. All rights reserved.

instanceId: 7a2855e5

installDate: 16/04/2019

installationName: VisualStudio/16.1.2+29001.49

installationPath: d:\program files\Visual Studio

installationVersion: 16.1.29001.49

displayName: Visual Studio Enterprise 2019

description: Microsoft DevOps solution for productivity and coordination across teams of any size

updateDate: 2019-06-17T13:57:50.6461374Z

enginePath: C:\Program Files (x86)\Microsoft Visual Studio\Installer\resources\app\ServiceHub\Services\Microsoft.VisualStudio.Setup.Service

layoutPath: X:\VS2019

channelId: VisualStudio.16.Release

channelUri: https://aka.ms/vs/16/release/channel

installChannelUri: D:\Soft\vs2019\ChannelManifest.json

releaseNotes: https://go.microsoft.com/fwlink/?LinkId=660893#16.1.2

thirdPartyNotices: https://go.microsoft.com/fwlink/?LinkId=660909

Use latest vswhere version returns the following on server where everything is ok:

D:\Soft>vswhere -all

Visual Studio Locator version 2.6.13+a6d40ba5f4 [query version 2.1.1046.44959]

Copyright (C) Microsoft Corporation. All rights reserved.

instanceId: 7a2855e5

installDate: 16/04/2019 16:02:38

installationName: VisualStudio/16.1.2+29001.49

installationPath: d:\program files\Visual Studio

installationVersion: 16.1.29001.49

productId: Microsoft.VisualStudio.Product.Enterprise

productPath: d:\program files\Visual Studio\Common7\IDE\devenv.exe

state: 4294967295

isComplete: 1

isLaunchable: 1

isPrerelease: 0

isRebootRequired: 0

displayName: Visual Studio Enterprise 2019

description: Microsoft DevOps solution for productivity and coordination across teams of any size

channelId: VisualStudio.16.Release

channelUri: https://aka.ms/vs/16/release/channel

enginePath: C:\Program Files (x86)\Microsoft Visual Studio\Installer\resources\app\ServiceHub\Services\Microsoft.VisualStudio.Setup.Service

installChannelUri: D:\Soft\vs2019\ChannelManifest.json

layoutPath: X:\VS2019

releaseNotes: https://go.microsoft.com/fwlink/?LinkId=660893#16.1.2

thirdPartyNotices: https://go.microsoft.com/fwlink/?LinkId=660909

updateDate: 2019-06-17T13:57:50.6461374Z

catalog_buildBranch: d16.1

catalog_buildVersion: 16.1.29001.49

catalog_id: VisualStudio/16.1.2+29001.49

catalog_localBuild: build-lab

catalog_manifestName: VisualStudio

catalog_manifestType: installer

catalog_productDisplayVersion: 16.1.2

catalog_productLine: Dev16

catalog_productLineVersion: 2019

catalog_productMilestone: RTW

catalog_productMilestoneIsPreRelease: False

catalog_productName: Visual Studio

catalog_productPatchVersion: 2

catalog_productPreReleaseMilestoneSuffix: 1.0

catalog_productSemanticVersion: 16.1.2+29001.49

catalog_requiredEngineVersion: 2.1.3095.63072

properties_campaignId: 1079615330.1554282809

properties_channelManifestId: VisualStudio.16.Release/16.1.2+29001.49

properties_nickname:

properties_setupEngineFilePath: C:\Program Files (x86)\Microsoft Visual Studio\Installer\vs_installershell.exe

  • This confirms our assumptions. It seems that the Visual Studio installation did not complete.

I opened up the Visual Studio installer and the Visual Studio installation continued. Problem solved!

Tuesday, July 16, 2019

GraphQL–Use GraphQL playground to authenticate requests

I really like GraphiQL to play around and test my GraphQL endpoints in the browser(try a live demo here). There is only one thing I really miss and that is the option to add extra headers when sending requests to your GraphQL endpoint.

Why do I need this?

One word; security. Most of my GraphQL endpoints are secured using oAuth. This makes using GraphiQL no longer an option as every request is answered by a 401 Unauthorized response.

Luckily there are a lot of alternative libraries that offer similar functionality as GraphiQL. The one I like is GraphQL Playground.

From the documentation:

GraphQL Playground uses components of GraphiQL under the hood but is meant as a more powerful GraphQL IDE enabling better (local) development workflows. Compared to GraphiQL, the GraphQL Playground ships with the following additional features:

  • Interactive, multi-column schema documentation
  • Automatic schema reloading
  • Support for GraphQL Subscriptions
  • Query history
  • Configuration of HTTP headers
  • Tabs

To add a security header, open your GraphQL Playground endpoint(I’m hosting it inside my ASP.NET Core app using GraphQL.Server.UI.Playground), go to the HTTP headers section and add the following JSON (every header value is a new json property):

Now when you send a request, the headers will be included...

Monday, July 15, 2019

GraphQL Documentary

Did I mention I’m a big fan of GraphQL? Now they even released a documentary:

“Starring Lee Byron, Dan Schafer and Nick Schrock (co-creators of GraphQL) and other big names from the #GraphQL community, "GraphQL: The Documentary" explores the story of why and how GraphQL came to be and the impact it's having on big #tech companies worldwide, including Facebook, Twitter, Airbnb and Github.”

Friday, July 12, 2019

Selenium - OpenQA.Selenium.DriverServiceNotFoundException

With the announcement from Microsoft to stop supporting Coded UI tests, I’m back in Selenium land.

To get it back in my fingers, I started to write a first simple Selenium test:

But although this test looked quite simple, it failed with the following exception:

Test Name:        TestMethod1

Test FullName:  SeleniumTestProject.UnitTest1.TestMethod1

Test Source:       C:\projects\test\SeleniumTestProject\UnitTest1.cs : line 10

Test Outcome:  Failed

Test Duration:   0:00:00,2912324

Result StackTrace:         

at OpenQA.Selenium.DriverService.FindDriverServiceExecutable(String executableName, Uri downloadUrl)

   at OpenQA.Selenium.Chrome.ChromeDriverService.CreateDefaultService()

   at OpenQA.Selenium.Chrome.ChromeDriver..ctor(ChromeOptions options)

   at SeleniumTestProject.UnitTest1.TestMethod1() in C:\projects\test\SeleniumTestProject\UnitTest1.cs:line 15

Result Message:             

Test method SeleniumTestProject.UnitTest1.TestMethod1 threw exception:

OpenQA.Selenium.DriverServiceNotFoundException: The chromedriver.exe file does not exist in the current directory or in a directory on the PATH environment variable. The driver can be downloaded at http://chromedriver.storage.googleapis.com/index.html.

The exception was easy to understand, I needed the chromedriver.exe to spin up a chrome instance. I could follow the suggestions in the exception and download it from http://chromedriver.storage.googleapis.com/index.html. A simpler alternative is adding the Selenium.Chrome.WebDriver NuGet package to your project.

This package does all the work for you: tt installs Chrome Driver(Win32) for Selenium WebDriver into your Unit Test Project and copies the "chromedriver.exe" to the bin folder when building your project.

Thursday, July 11, 2019

Tips from NDC Oslo 2019 - AZX

If you are using a Mac or Linux and want to use Azure, here is a tip for your: azx.ms

From the Github repo:

This utility is intended to help you create the scripts you need for Microsoft Azure. You can browse them online or install a quick utility function.

Wednesday, July 10, 2019

Microsoft Cloud Workshop

Interested in organizing your own Cloud workshop? Take a look at Microsoft Cloud Workshop, it contains a big library of workshop materials including presentation decks, trainer and student guides, and hands-on labs.

Tuesday, July 9, 2019

TFS - XAML Build controller stops working–Part 2

Yesterday I blogged about a problem that I had with the XAML Build controllers. I reconfigured the /queue what seemed to solve the problem.

Unfortunately the problem reappeared after a few hours. Instead of getting a 404 error, this time the details exposed a 500 error code.

In the logs I noticed the following error message:

WebHost failed to process a request.

Sender Information: System.ServiceModel.ServiceHostingEnvironment+HostingManager/17729775

Exception: System.ServiceModel.ServiceActivationException: The service '/queue/SampleCollection/Services/v4.0/MessageQueueService2.svc' cannot be activated due to an exception during compilation.  The exception message is: Memory gates checking failed because the free memory (419909632 bytes) is less than 5% of total memory.  As a result, the service will not be available for incoming requests.  To resolve this, either reduce the load on the machine or adjust the value of minFreeMemoryPercentageToActivateService on the serviceHostingEnvironment config element.. ---> System.InsufficientMemoryException: Memory gates checking failed because the free memory (419909632 bytes) is less than 5% of total memory.  As a result, the service will not be available for incoming requests.  To resolve this, either reduce the load on the machine or adjust the value of minFreeMemoryPercentageToActivateService on the serviceHostingEnvironment config element.

   at System.ServiceModel.Activation.ServiceMemoryGates.Check(Int32 minFreeMemoryPercentage, Boolean throwOnLowMemory, UInt64& availableMemoryBytes)

   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.CheckMemoryCloseIdleServices(EventTraceActivity eventTraceActivity)

   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(String normalizedVirtualPath, EventTraceActivity eventTraceActivity)

   --- End of inner exception stack trace ---

   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(String normalizedVirtualPath, EventTraceActivity eventTraceActivity)

   at System.ServiceModel.ServiceHostingEnvironment.EnsureServiceAvailableFast(String relativeVirtualPath, EventTraceActivity eventTraceActivity)

Process Name: w3wp

Process ID: 2216

Whoops! It seems that our TFS server is desperate for some memory. And indeed, after adding some extra memory and a reboot it finally started to work.

Let’s move on to the next problem…

Tips from NDC Oslo 2019 - Polly.NET policy wrapping

I do like Polly.NET, the “resilience and transient-fault-handling library that allows developers to express policies such as Retry, Circuit Breaker, Timeout, Bulkhead Isolation, and Fallback in a fluent and thread-safe manner”(quoted from the website).

One of the nice things you can do in Polly is combining multiple policies to achieve more complex functionality:

This works but requires some ugly nesting. A better approach I learned at NDC was through PolicyWrap:

More information here: https://github.com/App-vNext/Polly/wiki/PolicyWrap

Monday, July 8, 2019

TFS - XAML Build Controller stops working

When I started a XAML Build Controller on the build server, I noticed the following behavior:

  • The controller and agents start up and change to the Ready state while the icons went green
  • A few seconds later the icons turn red again, although the controller and agents remain in the same state.
  • On the Build Controller, a details link appear that shows the following error message:

TF400324: Team Foundation services are not available from server <Server/CollectionName>.
Page not found

So what was going on?

Everything was caused by a change in IIS I did. Before the TFS server was hosted on https://mytfs.company.be/tfs but they wanted to get rid of the /tfs part. So I removed the tfs application from IIS and moved all services to the root of the site. This worked perfectly for TFS itself but caused issues for the build controllers.

Why?

Because there are in fact 2 applications that should be hosted:

  1. the root TFS webservices now hosted at https://mytfs.company.be/
  2. a /queue application that should be hosted at https://mytfs.company.be/queue

By removing the /tfs, the /queue application was gone as well. I added the /queue application and my build controllers and agents turned green again…

Friday, July 5, 2019

Killing a hanging Windows Service

While testing a newly created Windows Service, the service failed on startup. As a consequence, I couldn’t start/stop/delete the service anymore.

To fix it I had to jump through some hoops:

  • First I had to query for all services to find the correct process id. This can be done using the sc queryex command
  • This returned a list of all services:

SERVICE_NAME: WpnUserService_ce0fb

DISPLAY_NAME: Windows Push Notifications User Service_ce0fb

        TYPE               : f0   ERROR

        STATE              : 4  RUNNING

                                (STOPPABLE, NOT_PAUSABLE, ACCEPTS_PRESHUTDOWN)

        WIN32_EXIT_CODE    : 0  (0x0)

        SERVICE_EXIT_CODE  : 0  (0x0)

        CHECKPOINT         : 0x0

        WAIT_HINT          : 0x0

        PID                : 8740

        FLAGS              :

SERVICE_NAME: FailingService.Host

DISPLAY_NAME: FailingService.Host

        TYPE               : 10  WIN32_OWN_PROCESS

        STATE              : 4  RUNNING

                                (STOPPABLE, NOT_PAUSABLE, ACCEPTS_SHUTDOWN)

        WIN32_EXIT_CODE    : 0  (0x0)

        SERVICE_EXIT_CODE  : 0  (0x0)

        CHECKPOINT         : 0x0

        WAIT_HINT          : 0x0

        PID                : 18836

FLAGS              :

  • Now that I have the PID, I can try to kill the windows service using taskkill /pid 18836
  • I got the following error message when trying to execute this:

ERROR: The process with PID 18836 could not be terminated.

Reason: This process can only be terminated forcefully (with /F option).

  • Let’s try this again with the ‘/F’ option: taskkill /pid 18836 /F

SUCCESS: The process with PID 18836 has been terminated.

  • After killing the service, I was finally able to delete it.

Thursday, July 4, 2019

Azure Artifacts - Views

A nice feature of Azure Artifacts(part of Azure DevOps) is the support for Views. By using Views you can split out a feed into multiple parts. This allows you to do a controlled rollout of a new package. By default, Azure Artifacts provide 3 views: @local, @prerelease, and @release.

This allows you to do package promotion:

  • We’ll start with an automated deployment to the @local feed.
  • When a package is ready for adoption, we can promote it to the @prerelease view:

  • When the package is validated and ready for prime time, we promote it into the @release view.

More information:

Wednesday, July 3, 2019

Tips from NDC Oslo 2019–F# Adding overloads to a discriminated union

A neat trick I noticed while watching the Dungeons, Dragons and Functions talk by Mathias Brandewinder is adding overloads to a discriminated union.

In his talk, he is modelling Dungeons and Dragons using F#. One of the things he needs to model is a dice roll. In D&D you have uncommon dice shapes and inside the rule book you’ll find dice rules like:  4d6+2d10+8.

What does this formula means? 4 roles of a 6-sided dice + 2 roles of a 10-sided dice+ 8

To model this in F#, Mathias created the following discriminated union:

You can then use this Roll type to create the formula above:

This works but is not very readible. To fix this, Mathias introduced some overloads on the Roll type:

This nicely cleans up the formula to:

More information can be found in Mathias blog post here: https://brandewinder.com/2018/07/31/give-me-monsters-part-3/

Tuesday, July 2, 2019

Tips from NDC Oslo 2019–F# Interactive built-in constants

Reading other files through F# Interactive can be tricky when using relative paths. It’s hard to guess what F# Interactive will use as the working directory. Of course you can start using absolute paths but a better alternative is using the built-in constants;

__SOURCE_DIRECTORY__
__SOURCE_FILE__

This allows you to do the following:

Monday, July 1, 2019

Tips from NDC Oslo 2019–Grid Garden

I have to admit that I’m really bad in CSS (although I was really proud on the linear-gradient I created 2 weeks ago). Before I always used the grid layout system of Bootstrap and didn’t worry to much about it. But I feel that the time is right to really start understanding Flexbox and the newer Grid Template.

And what is a better way than through gamification. So stop disturbing me as I’m taking care of my carrots