The art of simplicity

A long time ago, we made the decision to implement some features using batch processes that were scheduled to run at night (most of them as scheduled tasks). Over time the list of batch processes has further grown, with 85 processes running almost every night today. A sad record... Our business has further evolved and were it originally made sense to schedule all this work outside the regular business hours, we now encounter the following issues: The list of remaining free time slots is getting smaller and smaller, making it almost impossible to schedule a new task without interfering with other tasks. As our datasets have grown over the years, these processes typically also take longer and longer to execute. Our customers are interacting with our services more and more outside the regular business hours. It is no longer acceptable for our business to have to wait for some data. Realtime info is key to take correct business decisions. Some of these batch processes ar...

Today I investigated an issue we had in one of our build pipelines. For an unknown reason the build suddenly started to fail. In the build logs we found the following error message: ============================================================================== Generating script. Arguments passed sanitization without change. Formatted command: . 'D:\b\2\_work\135\s\Certificates\Import-PfxCertificate.ps1' -PfxFilePath ***2024.pfx -Password *** ========================== Starting Command Output =========================== "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command ". 'D:\b\2\_work\_temp\e3fa7d9e-bd6e-490f-8f71-182a958e03d7.ps1'" Importing store certificate 'D:\b\2\_work\135\s\Certificates\***2024.pfx'... Exception calling "Import" with "3" argument(s): "The profile for the user is a temporary profile. ...

At one of my clients we have multiple ADFS instances, one for testing purposes and one for production usage. The information on both servers is almost the same, only the endpoints for each relying party are different. Before we typically copied information from one server to another manually, typing over all the information. Of course this is a a cumbersome and error-prone process. I decided to simplify and automate this process with the help of some Powershell. The good news was that the hard work was already done for me, as Brad Held already created a script for exactly that purpose: PowerShell Gallery | Copy-RelyingPartyTrust 1.1 Here is how to use this script: Copy-RelyingPartyTrust.ps1 -sourceRPID testing:saml:com -path C:\Folder -filename SamlTest.json -import false As I found the script a little bit confusing I took the freedom to adapt the code and split it out in 2 separate scripts. Here is the export script: You can use this script like this: export.ps1 -rp...

If you follow security best practices, you are not re-using the same password for multiple purposes. As a consequence you end up with a long list of passwords that you need to secure and manage. Although the use of a password vault certainly improved the experience, I still try to avoid the usage of passwords as much as possible. Today I want to share a ‘trick’ I discovered that allows you to share(export/import) a PFX file without using passwords. No clue what a pfx is? Let me explain that first… A PFX file, also known as a PKCS#12 file , is a binary format used to store certificates and their associated private keys . It combines 2 parts: A certificate part : A certificate is a digital document that contains information about an entity (such as a person, organization, or server). It is used for authentication, encryption, and secure communication. Certificates are issued by a Certificate Authority (CA) . A private key part : The private key is a cryptographic key...

Recently some of our Azure DevOps pipelines started to fail. We noticed that this only happened on pipelines where we had tasks that used Powershell behind the scenes. The error message we got looked like this: Cannot process an element with node type "Text" A search on the Internet brought us to a solution . Disabling the “Run PowerShell in Parallel” setting solved the problem. Anyone with a better solution?

After upgrading your Azure DevOps Server instance, you should also upgrade your ElasticSearch instance. If the instance is running on the same server as Azure DevOps, it can be automatically updated using the Configure Search feature in the Azure DevOps Administration console. In case you are running ElasticSearch on a different machine (what I would recommend) you need to copy the Search service package to the target server. You can find the link in the wizard that is used to Configure Search: On the remote server extract the installer files and execute the following command: \Configure-TFSSearch.ps1 --operation update The installer should be able to find the existing ElasticSearch instance and update it for you. More information: https://docs.microsoft.com/en-us/azure/devops/project/search/administration

After sharing a script to be able to upload build tasks to your on premise TFS instance, here is a second script to remove build tasks:

Inside a Powershell script I’m using I have the following line: $headers = @{ "Accept" = "application/json; api-version=2.0-preview"; "X-TFS-FedAuthRedirect" = "Suppress" } Invoke-RestMethod -Uri $url -Credential $Credential -Headers $headers -ContentType application/octet-stream -Method Put -InFile $taskZipItem I’m using the Invoke-RestMethod cmdlet to call a REST api with some parameters. Although this worked perfectly on my machine(it always does isn’t it ), it failed on a colleagues machine with the following error message: Invoke-RestMethod : The 'Accept' header must be modified using the appropriate property or method. Parameter name: name At C:\uploadtask.ps1:31 char:1 + Invoke-RestMethod -Uri $url -Credential $Credential -Headers $headers -ContentTy ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~     + CategoryInfo     ...

While creating some custom Powershell commandlets using C# we were wondering how we could load these using a relative path. Starting from Powershell 3.0 this became easy thanks to the introduction of the $PSScriptRoot variable. This variable always points to the folder the current script is located in. An example: Import-Module $PSScriptRoot\..\BuildScripts\CopyFile.dll

One of the nice things you can do with NuGet is to trigger a Powershell script (Install.ps1) when the NuGet package is installed or removed. From the documentation : A package can include PowerShell scripts that automatically run when the package is installed or removed. NuGet automatically runs scripts based on their file names using the following conventions: Init.ps1 runs the first time a package is installed in a solution. If the same package is installed into additional projects in the solution, the script is not run during those installations. The script also runs every time the solution is opened (Package Manager Console window has to be open at the same for the script to run). For example, if you install a package, close Visual Studio, and then start Visual Studio and open the solution with Package Manager Console window, the Init.ps1 script runs again. Install.ps1 runs when a package is installed in a project. ...

Automation is a key piece in creating and maintaining your Azure environment. This month, Microsoft introduced a graphical tool and programming model to author and manage your automation scripts. So you don’t have to be a Powershell guru to start using the full power of Azure Automation.  Simply insert activities from the library to the canvas, link them together into a workflow, and configure the properties in order to create useful runbooks that automate your IT processes.  More information: https://azure.microsoft.com/en-gb/blog/azure-automation-graphical-and-textual-runbook-authoring/

Before decommissioning a Sharepoint server, I needed a list of all users on the different team sites. Time for some Powershell magic! On the Technet site I found a script that promised to do exactly what I needed: This script enumerates SharePoint 2010 or 2013 permissions across the entire farm down to the site (SPWeb) level. It also recursively expands the membership of any AD group and lists the assignment role binding on the permission. The output is an XML format . So I copied the script over, logged in on the Sharepoint server and opened up a Powershell command prompt. However when I tried to execute the script it failed with the following error message: Exception has been thrown by the target of an invocation + CategoryInfo : NotSpecified: (:) [], TargetInvocationExcept + FullyQualifiedErrorId : System.Reflection.TargetInvocationException So I opened up the script to find the root cause of the error and saw that it failed on the call to AllWebs : On...

While converting and merging some SharePoint sites, I created some PowerShell scripts to speed up the process. In case I’ll ever need them again, I post them here: Script 1: Mount a Content database using PowerShell Script 2: Generate a sub site using PowerShell Script 3: Generate a Document Library using PowerShell Script 4: Remove a site collection(without asking for confirmation) using PowerShell

After migrating our TFS SharePoint Portal to SharePoint 2013, our users started to complain that they couldn’t login. However when I took a look at the different site permissions, everything looked fine. All users were there as expected… Why can’t users login? This is caused by the new Claims based model in SharePoint 2013. In SharePoint 2010 you still had the choice between claims and the classic security model. in SharePoint 2013 this is no longer the case and therefore  you have to upgrade your users to their claims alternative. As long as you don’t do this, users will not be able to access your sites. To fix this, open the SharePoint PowerShell command line and execute the following script:

You want to write Powershell scripts inside Visual Studio? Thanks to the  PowerGUI Visual Studio Extension this becomes very easy. Here is the list of available features: PowerGUI Console Tool Window PowerShell Project Type IntelliSense support through a custom PowerShell editor Syntax highlighting and script analysis Supports PowerGUI settings Supports PowerGUI imported modules Supports PowerGUI Snippets PowerShell debugging PowerGUI Console Tool Window PowerShell Project Type

Last week I had to run a maintenance script on a server. But it gave me the following error: File cleanup.ps1 cannot be loaded because the execution of scripts is disabled on this system. Turns out that by default the security setting on the server(the so-called Execution Policy) does not allows us to execute a script. By default, the Execution Policy is set to Restricted. This setting means that you may not run any PS1 script at all. You can choose between the following Execution Policy levels: Restricted This is the default configuration in PowerShell. This setting means that no script can run, regardless of its signature. The only thing that can be run in PowerShell with this setting are individual commands. AllSigned This setting does allow scripts to run in PowerShell. The script must have an associated digital signature from a trusted publisher. There will be a prompt before you run the scripts from trusted publishers. This exposes you to running si...

You already have a build server up and running? Your code gets compiled and tested in a continuous way? You now want to take this one step further and automate your deployment? Let me introduce you TFS Deployer : TFS Deployer is a free tool available at Codeplex. It must be installed as an agent in your test and production environments and supports the execution of PowerShell scripts when an event happens in TFS. In particular it listens to build quality change notifications which occur when you change the quality of a build under the Team Builds node of Team Explorer. How does it work? The way it works is that when TFS Deployer starts up, the service subscribes to the build quality change event notification (1). Then the release manager, using Team Explorer updates the build quality indicator in the build store via Team Explorer (2), the build quality change event is fired (3), the TFS event service then looks up who is subscribed to the event (4) and then TFS Event Service no...

Do you want to learn Powershell in a fun and easy way? Have a look at the  "Climbing Mt. PowerShell" technology comic book. It"’s very well written and the comic book style makes it attractive and an enjoyable reading experience. Next time, I have to write some documentation, I’ll will be doing it in comic style too   Download Climbing Mt. PowerShell (2.2 MB)

A late Christmas gift for everyone,  Keith Hill has turned a set of blog posts into an 61 pages long PowerShell ebook. Download your own copy here .   Table of content: Introduction Item 1: Four Cmdlets that are the Keys to Discovery within PowerShell Key #1: Get-Command #2: Get-Help Key #3: Get-Member Key #4: Get-PSDrive PowerShell 2.0 Update Item 2: Understanding Output Output is Always a .NET Object Function Output Consists of Everything That Isn't Captured Other Types of Output That Can't Be Captured Item 3: Know What Objects Are Flowing Down the Pipeline Item 4: Output Cardinality - Scalars, Collections and Empty Sets - Oh My! Working with Scalars Working with Collections Working with Empty Sets Item 5: Use the Objects, Luke. Use the Objects! Item 6: Know Your Output Formatters Item 7: Understanding PowerShell Parsing Modes Item 8: Understandin...

When configuring a new lab environment for a client, I encountered the following error: Environment message: Type=Error; Message=TF266044: One or more machines are not ready to run workflows. For more information, see the individual machine errors.; Machine messages: Machine name: XXXXXX Machine message: Type=Error; Message=Error occurred while configuring TFSBuildServiceHost with Lab configuration. ExceptionType:Microsoft.TeamFoundation.Build.Client.BuildServiceHostAlreadyExistsException. ExceptionMessage: A build service host already exists for computer XXXX.domain.net. Specify a different computer name and try again. I found the solution in the following blog post ( http://blogs.msdn.com/b/lab_management/archive/2010/02/09/additional-upgrade-script-for-fixing-build-agent-registration.aspx ). Although our TFSenvironment wasn’t upgraded from the Beta 2, the fix still works if you encounter this error message.