Jekyll2020-01-03T17:14:22+00:00https://mathieubuisson.github.io/feed.xmlTheShellNutPowerShell, Automation, From Dev to Ops and Everything in BetweenMathieu BuissonUsing Azure Monitor Workbooks to document your Azure resources2020-01-03T00:00:00+00:002020-01-03T00:00:00+00:00https://mathieubuisson.github.io/azure-workbooks-inventory-resources<aside class="sidebar__right">
<nav class="toc">
<header><h4 class="nav__title"><i class="fa fa-file-text"></i> In This Article</h4></header>
<ul class="toc__menu" id="markdown-toc">
<li><a href="#the-problem" id="markdown-toc-the-problem">The Problem</a></li>
<li><a href="#introducing-azure-monitor-workbooks" id="markdown-toc-introducing-azure-monitor-workbooks">Introducing Azure Monitor Workbooks</a></li>
<li><a href="#creating-the-workbook-resource" id="markdown-toc-creating-the-workbook-resource">Creating the Workbook Resource</a></li>
<li><a href="#building-the-azure-resources-inventory-report" id="markdown-toc-building-the-azure-resources-inventory-report">Building the Azure Resources Inventory Report</a></li>
<li><a href="#sharing-the-azure-resources-inventory-report" id="markdown-toc-sharing-the-azure-resources-inventory-report">Sharing the Azure Resources Inventory Report</a></li>
</ul>
</nav>
</aside>
<h2 id="the-problem">The Problem</h2>
<p>Azure makes it very easy to deploy resources in various ways and contexts :</p>
<ul>
<li>deployed by humans or automated processes</li>
<li>run/managed by different teams</li>
<li>to support different applications/workloads</li>
<li>billed to different cost centers or subscriptions</li>
<li>in different geographical locations</li>
<li>in different environments/purposes (dev, QA, staging, production…)</li>
</ul>
<p>It is <strong>so easy</strong> to spin up resources that, without appropriate governance, this can quickly lead to <strong>resource sprawl</strong>.<br />
An aspect of Azure governance which is often overlooked is :</p>
<blockquote>
<p>Which resources are running and where ?<br />
What is the type, configuration, SKU of these resources ?</p>
</blockquote>
<p>In other words, how do we build and document an <strong>inventory of Azure resources</strong> ?<br />
Sure, this can be done in a Confluence page or any kind of wiki, but as resources and environments are created, modified, deleted by different teams and processes, how can we keep this documentation up-to-date ?</p>
<p>Any document which relies on human intervention is unlikely to keep up with the rate and variety of change, and it will eventually become stale.</p>
<p>A solution is to build a self-updating report about the Azure resources we have running, regardless of how/why they were provisioned and the Azure region and subscription in which they are located.</p>
<h2 id="introducing-azure-monitor-workbooks">Introducing Azure Monitor Workbooks</h2>
<p><strong><a href="https://docs.microsoft.com/en-us/azure/azure-monitor/platform/workbooks-overview">Azure Workbooks</a></strong> is a service which allows to build reports using text (in markdown) and a number of data visualization types (tables, charts, tiles, etc…). The data can come from various sources (logs, metrics, alerts, etc…) and is obtained using <abbr title="Kusto Query Language">KQL</abbr> queries.</p>
<p>As its name implies, this service lives under the <strong>Azure Monitor</strong> umbrella and it is typically used to create reports like :</p>
<ul>
<li>performance analysis</li>
<li>incident reviews</li>
<li>troubleshooting guides</li>
</ul>
<p>But because <strong>Azure Monitor Workbooks</strong> can also run <abbr title="Kusto Query Language">KQL</abbr> queries against the <strong><a href="https://azure.microsoft.com/en-us/features/resource-graph/">Azure Resource Graph</a></strong>, we can also use it to build an inventory of our Azure resources.</p>
<p><strong>Azure Monitor Workbooks</strong> provides templates readily available in the portal, to get started quickly. The built-in templates cover a good number of scenarios, but not for our specific use case, so <strong>we are going to create a workbook from scratch</strong>.</p>
<p>This workbook will enable us to gather a few relevant properties on the following types of resources, across <strong>all environments</strong> and <strong>all subscriptions</strong> :</p>
<ul>
<li>App Services</li>
<li>SQL Databases</li>
<li>Virtual Machines</li>
</ul>
<p>Let’s do this.</p>
<h2 id="creating-the-workbook-resource">Creating the Workbook Resource</h2>
<ol>
<li>In the Azure Portal, go to the <strong>Monitor</strong> service</li>
<li>In the <strong>Monitor</strong> menu click on <strong>Workbooks</strong></li>
<li>
<p>From here, we can view the built-in templates, but in our case, we select <strong>Empty</strong> :</p>
<p><img src="https://mathieubuisson.github.io/images/2020-01-03-azure-workbooks-inventory-resources-empty.png" alt="Empty" /></p>
</li>
</ol>
<p>We add an introductory section for the report, just to have some content to save.</p>
<ol>
<li>Click on <strong>Add text</strong></li>
<li>
<p>Add the following markdown :</p>
<div class="language-md highlighter-rouge"><div class="highlight"><pre class="highlight"><code> # Environments Inventory
⚠️ _Select a <span class="gs">**Resource Group**</span> in the drop-down below_
</code></pre></div> </div>
<p>The first line acts as the report title. The second line is a note on how to interact with the report (more on that later).</p>
</li>
<li>Click on <strong>Done Editing</strong></li>
<li>Verify that the rendered markdown displays as expected in the report</li>
</ol>
<p>Now, we save the report as a <strong>workbook</strong> resource.</p>
<ol>
<li>Click on the <strong>Save As</strong> button</li>
<li>
<p>Fill out the following fields :</p>
<ul>
<li><strong>Title</strong> : This is the user-friendly name of the workbook</li>
<li><strong>Subscription</strong> : The Azure subscription where we want to save the workbook resource</li>
<li><strong>Resource group</strong> : The resource group where we want to save the workbook resource.<br />
Make sure that users and stakeholders who need to view this inventory report have at least the <strong>Reader</strong> role on this resource group.</li>
<li><strong>Location</strong> : It defaults to the location of the destination resource group.<br />
I would recommend to leave it that way.</li>
<li><strong>Save To</strong> : After selecting a resource group, we have the ability to choose <strong>Shared reports</strong>.<br />
This is what we want, in order to make our report available to others.</li>
</ul>
</li>
<li>
<p>Click on <strong>Save</strong></p>
<p>Now, if we go to the destination resource group, we can see the corresponding resource :</p>
<p><img src="https://mathieubuisson.github.io/images/2020-01-03-azure-workbooks-inventory-resources-resource.png" alt="Workbook resource" /></p>
<p class="notice--info"><strong>Note:</strong><br />
Clicking on this resource merely provides access to the resource metadata.<br />
To view or edit the workbook, we need to access it from <strong>Azure Monitor</strong>.</p>
</li>
<li>In the Azure Portal, go back to the <strong>Monitor</strong> service</li>
<li>
<p>You should see the saved workbook in <strong>Recently modified workbooks</strong> :</p>
<p><img src="https://mathieubuisson.github.io/images/2020-01-03-azure-workbooks-inventory-resources-recently.png" alt="Recently modified" /></p>
</li>
<li>Click on the saved workbook to open it</li>
<li>Click on the <strong>Edit</strong> button to switch to editing mode</li>
</ol>
<p>Next, we are going to flesh out our Azure resources inventory report.</p>
<h2 id="building-the-azure-resources-inventory-report">Building the Azure Resources Inventory Report</h2>
<p>In our scenario, Azure environments have a 1-to-1 relationship with a resource group and these resource groups follow a basic naming convention :<br />
<code class="language-plaintext highlighter-rouge">{Environment_Type}-corp-{Environment_Location}</code></p>
<p>For example, the resource groups names are :</p>
<ul>
<li>dev-corp-europe</li>
<li>dev-corp-us</li>
<li>test-corp-us</li>
</ul>
<p>In the workbook, we want the user to be able to interactively choose which environment to look at.<br />
We can achieve this with a parameter.</p>
<h3 id="parameterizing-the-report">Parameterizing the Report</h3>
<ol>
<li>Click on <strong>Add parameters</strong></li>
<li>Click on <strong>Add parameter</strong> on the top left corner on the section editing box</li>
<li>
<p>Specify the parameter properties like so :</p>
<p><img src="https://mathieubuisson.github.io/images/2020-01-03-azure-workbooks-inventory-resources-parameter.png" alt="Parameter" /></p>
<p>Some of these fields warrant an explanation.</p>
<ul>
<li><strong>Parameter type</strong> : The type of UI to allow the user to specify a value for this parameter.
In this case, we want a drop down menu.</li>
<li><strong>Get data from</strong> : Where the possible values in the drop down menu should come from.<br />
This can be static data (a JSON array), or dynamic data (based on a <abbr title="Kusto Query Language">KQL</abbr> query).<br />
In this case, it is based on a query, since we are going to query the <strong>Azure Resource Graph</strong> to get a list of resource groups.</li>
<li><strong>Data source</strong> : Azure Resource Graph</li>
<li>
<p><strong>Subscriptions</strong> : Use all subscriptions, unless you want to restrict this query to specific subscriptions.</p>
<p class="notice--warning"><strong>Warning:</strong><br />
All <abbr title="Kusto Query Language">KQL</abbr> queries in a workbook are executed whenever we interact with the workbook.<br />
They are run under the credentials of whoever is viewing the workbook, so they return only resource groups and resources that the user has access to (at least <strong>Read</strong> permission).</p>
</li>
<li>
<p><strong>The actual query</strong> :</p>
<p class="notice--info"><strong>Note:</strong><br />
While teaching <abbr title="Kusto Query Language">KQL</abbr> is out of scope for this article, we are going to expand on some noteworthy elements of the queries, just so we understand what is happening.<br />
For more information on <abbr title="Kusto Query Language">KQL</abbr>, please refer to <strong><a href="https://docs.microsoft.com/en-us/azure/kusto/query/">the documentation</a></strong>.</p>
<p>This query gets all the resource groups, then it filters them based on our naming convention.<br />
Then, we <code class="language-plaintext highlighter-rouge">project</code> their <code class="language-plaintext highlighter-rouge">name</code> property, because this is the only property we need for the drop down menu items.</p>
</li>
</ul>
</li>
<li>Click <strong>Save</strong> to save the settings for this parameter</li>
<li>Click <strong>Done Editing</strong></li>
</ol>
<p>Now, we can test the UI for this parameter :</p>
<p><img src="https://mathieubuisson.github.io/images/2020-01-03-azure-workbooks-inventory-resources-dropdown.png" alt="dropdown" /></p>
<h3 id="adding-a-section-for-app-services-inventory">Adding a Section for App Services Inventory</h3>
<p>We start off with a short text block, containing the section title and a bit of usage information.</p>
<ol>
<li>Click on <strong>Add text</strong></li>
<li>
<p>Add the following markdown :</p>
<div class="language-md highlighter-rouge"><div class="highlight"><pre class="highlight"><code> ## App Services :
💡 _Click on a resource name to go to the resource in the portal, or click on a URL to go to the web app in your browser._
</code></pre></div> </div>
</li>
<li>Click on <strong>Done Editing</strong></li>
</ol>
<p>Now, we gather the data we need regarding the Azure App Services (in the resource group specified via the <code class="language-plaintext highlighter-rouge">ResourceGroupeName</code> parameter) and present the data in a table.</p>
<ol>
<li>Click on <strong>Add query</strong></li>
<li>Change the <strong>Data source</strong> to <strong>Azure Resource Graph</strong></li>
<li>For <strong>Subscriptions</strong>, select <strong>Use all Subscriptions</strong></li>
<li>Leave <strong>Visualization</strong> and <strong>Size</strong> at their default value</li>
<li>
<p>In the query text box, add the following query :</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> Resources
| where resourceGroup == '{ResourceGroupeName}'
| where type == 'microsoft.web/sites'
| project ["Web App"] = id,
["App Service Plan"] = tostring(properties.serverFarmId),
Kind = kind,
URL = strcat("https://", properties.defaultHostName),
["SKU Name"] = properties.sku
</code></pre></div> </div>
<p>Remember our <code class="language-plaintext highlighter-rouge">ResourceGroupeName</code> parameter ?<br />
The second line above shows how to reference the value of this parameter from queries in the workbook.</p>
<p>In this section we are only interested in <strong>App Services</strong>, so we filter on the type <code class="language-plaintext highlighter-rouge">microsoft.web/sites</code>.</p>
<p>Then, we use <code class="language-plaintext highlighter-rouge">project</code> to define the columns we want. We have to use square brackets and quotes whenever a column name contains spaces.</p>
<p>As you might have guessed, <code class="language-plaintext highlighter-rouge">strcat()</code> is a function which allows to concatenate 2 or more strings. Here, we use it to derive the web app URL from its <strong>defaultHostName</strong> property.</p>
</li>
<li>
<p>Click <strong>Done Editing</strong></p>
<p>We can now admire the result of our query :</p>
<p><img src="https://mathieubuisson.github.io/images/2020-01-03-azure-workbooks-inventory-resources-appservices-table.png" alt="App Services Table" /></p>
<p>Note that values in the <strong>Web App</strong> and <strong>App Service Plan</strong> columns are automatically displayed as links and clicking on any of these brings us to the underlying resource in the Azure Portal.<br />
Unfortunately, this is not the case for the <strong>URL</strong> column. But we can easily remediate that.</p>
</li>
<li>Click the <strong>Edit</strong> button at the bottom right of the App Service table</li>
<li>Click on <strong>Column Settings</strong></li>
<li>Select <strong>URL (Automatic)</strong></li>
<li>Set <strong>Column renderer</strong> to <strong>Link</strong></li>
<li>Set <strong>View to open</strong> to <strong>Url</strong></li>
<li>Click <strong>Save and Close</strong></li>
</ol>
<p>Now, we can browse to the actual application by clicking on its URL value in the table.</p>
<h3 id="adding-a-section-for-sql-databases-inventory">Adding a Section for SQL Databases Inventory</h3>
<p>We start off with a short text block, containing the section title and a bit of usage information.</p>
<ol>
<li>Click on <strong>Add text</strong></li>
<li>
<p>Add the following markdown :</p>
<div class="language-md highlighter-rouge"><div class="highlight"><pre class="highlight"><code> ## SQL Databases :
💡 _Click on a resource name to go to the resource in the portal_
</code></pre></div> </div>
</li>
<li>Click on <strong>Done Editing</strong></li>
</ol>
<p>Now, we gather the Azure SQL Databases information (from the specified resource group) and present it in a table.</p>
<ol>
<li>Click on <strong>Add query</strong></li>
<li>Change the <strong>Data source</strong> to <strong>Azure Resource Graph</strong></li>
<li>For <strong>Subscriptions</strong>, select <strong>Use all Subscriptions</strong></li>
<li>Leave <strong>Visualization</strong> at its default value</li>
<li>Set <strong>Size</strong> to <strong>Small</strong>, unless you have about 6 or more databases per environment</li>
<li>
<p>In the query text box, add the following query :</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> Resources
| where resourceGroup == '{ResourceGroupeName}'
| where type == 'microsoft.sql/servers/databases'
| where name != 'master'
| project Database = id,
["Elastic Pool"] = tostring(properties.elasticPoolId),
["SKU Name"] = sku.name,
["Pricing Tier"] = sku.tier,
Status = properties.status
</code></pre></div> </div>
<p>In this section we are interested in <strong>SQL Databases</strong>, this is why we filter on the type <code class="language-plaintext highlighter-rouge">microsoft.sql/servers/databases</code>.<br />
We filter out the <strong>master</strong> database because it is not relevant to our applications.</p>
<p>Then, we use <code class="language-plaintext highlighter-rouge">project</code> to expose the database properties we want and define the table’s columns.</p>
</li>
<li>Click <strong>Done Editing</strong></li>
</ol>
<p>The resulting table might not look as cute as Baby Yoda, but still looks pretty nice :</p>
<p><img src="https://mathieubuisson.github.io/images/2020-01-03-azure-workbooks-inventory-resources-databases-table.png" alt="SQL Databases table" /></p>
<p>Again, the <strong>Database</strong> and <strong>Elastic Pool</strong> columns allow us to browse to the underlying resource in the portal, since their values are displayed as links.</p>
<h3 id="adding-a-section-for-virtual-machines-inventory">Adding a Section for Virtual Machines Inventory</h3>
<p>We start off with a short text block, containing the section title and a bit of usage information.</p>
<ol>
<li>Click on <strong>Add text</strong></li>
<li>
<p>Add the following markdown :</p>
<div class="language-md highlighter-rouge"><div class="highlight"><pre class="highlight"><code> ## Virtual Machines :
💡 _Click on a Virtual Machine name to go to the resource in the portal_
</code></pre></div> </div>
</li>
<li>Click on <strong>Done Editing</strong></li>
</ol>
<p>Now, we gather the data relevant to Virtual Machines (in the specified resource group/environment) and present it in a table.</p>
<ol>
<li>Click on <strong>Add query</strong></li>
<li>Change the <strong>Data source</strong> to <strong>Azure Resource Graph</strong></li>
<li>For <strong>Subscriptions</strong>, select <strong>Use all Subscriptions</strong></li>
<li>Leave <strong>Visualization</strong> at its default value</li>
<li>Set <strong>Size</strong> to <strong>Small</strong>, unless you have about 6 or more VMs per environment</li>
<li>
<p>In the query text box, add the following query :</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> Resources
| where resourceGroup == '{ResourceGroupeName}'
| where type == 'microsoft.compute/virtualmachines'
| project Name = id,
Size = properties.hardwareProfile.vmSize,
["OS Image"] = strcat(properties.storageProfile.imageReference.offer, " - ", properties.storageProfile.imageReference.sku),
associated_nic = extract(".*/(.*)\"}$", 1, tostring(properties.networkProfile.networkInterfaces[0]))
| join kind = leftouter (
Resources
| where resourceGroup == '{ResourceGroupeName}'
| where type == 'microsoft.network/publicipaddresses'
| project ["Public IP Address"] = properties.ipAddress,
associated_nic = extract(".*/networkInterfaces/(.*?)/.*", 1, tostring(properties.ipConfiguration.id))
) on associated_nic
| project-away associated_nic, associated_nic1
</code></pre></div> </div>
<p>In this section we are interested in <strong>Virtual Machines</strong>, so we filter on the type <code class="language-plaintext highlighter-rouge">microsoft.compute/virtualmachines</code>.</p>
<p>Then, we use <code class="language-plaintext highlighter-rouge">project</code> to define the table’s columns.<br />
For the <strong>OS Image</strong> column, we use <code class="language-plaintext highlighter-rouge">strcat()</code> to construct a user-friendly OS image name, based on the image <code class="language-plaintext highlighter-rouge">offer</code> and <code class="language-plaintext highlighter-rouge">sku</code> properties.</p>
<p>Also, for VMs which have a public IP address, we want to show that IP address in a <strong>Public IP Address</strong> column.<br />
Sadly, the Virtual Machine object returned by the query doesn’t contain any reference to the Public IP, so we need to run another query specifically for <strong>Public IP addresses</strong> and then <code class="language-plaintext highlighter-rouge">join</code> its results with the <strong>Virtual Machines</strong> query.</p>
<p>In the <strong>Virtual Machines</strong> query, we use the <code class="language-plaintext highlighter-rouge">networkInterfaces</code> nested property to get the first NIC. This is safe in this case, because we know that our VMs only have 1 NIC.<br />
Then, we use a regex to <code class="language-plaintext highlighter-rouge">extract()</code> the name of the network interface.</p>
<p>So, the <strong>Public IP addresses</strong> query needs to have 2 columns :</p>
<ul>
<li>1 column containing the information we ultimately want : <strong>the Public IP address</strong></li>
<li>1 column that we can <code class="language-plaintext highlighter-rouge">join</code> on : <strong>the name of the network interface</strong></li>
</ul>
<p>We use the <code class="language-plaintext highlighter-rouge">ipConfiguration</code> nested property to get the resource ID of the network interface which is associated with the Public IP.<br />
Then, we use a regex to <code class="language-plaintext highlighter-rouge">extract()</code> the name of the network interface.</p>
<p>Now, we have 2 queries which have a column in common (same name : <code class="language-plaintext highlighter-rouge">associated_nic</code> and same value), so we can join these 2 queries on this column.<br />
Note that this is a <strong>left outer join</strong> because our left query (<strong>Virtual Machines</strong> query) is the main one : we keep the Virtual Machine data even if there is no match in the <strong>Public IP addresses</strong> query.</p>
</li>
<li>Click <strong>Done Editing</strong></li>
</ol>
<p>Here is what the VMs table looks like :</p>
<p><img src="https://mathieubuisson.github.io/images/2020-01-03-azure-workbooks-inventory-resources-vms-table.png" alt="Virtual Machines table" /></p>
<p>Again, clicking on a VM name brings us to the underlying resource in the Azure Portal.</p>
<p>We can add more sections to inventory other types of resources : Azure Storage accounts, Virtual Networks and subnets, Automation runbooks, the list goes on…<br />
At this point, we have a fairly good grasp of the basic building blocks :</p>
<ul>
<li>Markdown blocks to add structure and any additional information to the report</li>
<li>Azure Resource Graph queries to get and present data about our Azure resources</li>
</ul>
<p>So let’s consider our workbook done and save it.</p>
<ol>
<li>Click on the <strong>Done Editing</strong> button at the top of the screen</li>
<li>Click the <strong>Save</strong> button</li>
</ol>
<h2 id="sharing-the-azure-resources-inventory-report">Sharing the Azure Resources Inventory Report</h2>
<p>We have 2 options to share the report with others users and stakeholders :</p>
<ul>
<li>Pinning some (or all) of the sections to an Azure Portal dashboard</li>
<li>Sharing a link to the workbook</li>
</ul>
<p>To pin all sections of the workbook to a dashboard :</p>
<ol>
<li>Click the <strong>Edit</strong> button at the top of the workbook</li>
<li>Click the <strong>Pin</strong> button</li>
<li>Click the <strong>Pin All</strong> button</li>
<li>Click the <strong>Done Pinning</strong> button</li>
</ol>
<p>To share the link to the workbook :</p>
<ol>
<li>Click the <strong>Share</strong> button at the top of the workbook</li>
<li>Below the <strong>Link to share</strong> field, click the <strong>Copy</strong> button</li>
<li>Send the copied URL to any users who may be interested in viewing this report.<br />
It is a crazy long URL, so you might want to use a URL shortener service and share the shortened URL instead.</li>
</ol>
<p>That’s it, we now have an inventory report of our Azure resources for multiple environments, across all subscriptions.<br />
<strong>This report is interactive</strong> : the user can choose which environment/resource group to look at, and sort the data on any table’s column.
More importantly, the report <strong>will always be up-to-date</strong> since the queries to get the data are executed whenever we view the report.</p>
<p>Off course, this was just a fairly simple example, the report can be added to and customized at will.<br />
With just 2 types of building blocks (markdown and <abbr title="Kusto Query Language">KQL</abbr> queries), the possibilities are almost endless.</p>
<p>To learn more on data sources, visualization types and capabilities offered by <strong>Azure Monitor Workbooks</strong>, please refer to the excellent <strong><a href="https://docs.microsoft.com/en-us/azure/azure-monitor/platform/workbooks-overview">documentation</a></strong>.</p>TheShellNutIn this post, we demonstrate how to build a self-updating inventory of Azure resources in any Resource Group, across multiple subscriptions. Then, we share the workbook to make it available to colleagues and other stakeholders.Making Azure infrastructure code more composable and maintainable with Terraform modules2019-04-23T00:00:00+01:002019-04-23T00:00:00+01:00https://mathieubuisson.github.io/composable-infrastructure-code-terraform-modules<aside class="sidebar__right">
<nav class="toc">
<header><h4 class="nav__title"><i class="fa fa-file-text"></i> In This Article</h4></header>
<ul class="toc__menu" id="markdown-toc">
<li><a href="#the-problem" id="markdown-toc-the-problem">The Problem</a></li>
<li><a href="#breaking-down-the-problem-into-smaller-parts" id="markdown-toc-breaking-down-the-problem-into-smaller-parts">Breaking Down the Problem Into Smaller Parts</a></li>
<li><a href="#composing-a-solution-for-the-overall-problem" id="markdown-toc-composing-a-solution-for-the-overall-problem">Composing a Solution For the Overall Problem</a></li>
</ul>
</nav>
</aside>
<h2 id="the-problem">The Problem</h2>
<h3 id="modeling-complex-infrastructure-with-a-declarative-dsl">Modeling complex infrastructure with a declarative DSL</h3>
<p>There are a number of <strong><a href="https://pascalnaber.wordpress.com/2018/11/11/stop-using-arm-templates-use-the-azure-cli-instead/">issues and limitations with Azure ARM templates</a></strong>, and to be fair, many of them are not specific to ARM templates but pertain to declarative <abbr title="Domain Specific Languages">DSLs</abbr> in general.</p>
<p>The first and most painfully obvious issue with declarative <abbr title="Domain Specific Languages">DSLs</abbr> is their limited support for <strong>logic</strong>. Constructs which are trivial in any general purpose programming language (iteration, conditions, etc…) tend to be clunky and limited in <abbr title="Domain Specific Languages">DSLs</abbr>.</p>
<blockquote>
<p>Have you ever tried a <code class="language-plaintext highlighter-rouge">if {...} elseif {...} else {...}</code> in a YAML-based DSL ?</p>
</blockquote>
<blockquote>
<p>How about populating an array variable from the iteration over another array variable, in an ARM template ?</p>
</blockquote>
<p>It is feasible, but :</p>
<ul>
<li>Is the syntax clean and readable ?</li>
<li>Is it properly explained in the documentation (with examples) ?</li>
<li>Is it easy to debug when it blows up in your face ?</li>
</ul>
<p>I’ll let you answer that yourselves.</p>
<p>In some cases, we can get away with not using any logic and stick to a purely declarative approach, but this implies one of 2 things :</p>
<ul>
<li>The scenario is very simple</li>
<li>It’s a trade-off with <strong><abbr title="Don't Repeat Yourself">DRY</abbr></strong> and may result in <strong>CPDD</strong> (Copy Paste Driven Development)</li>
</ul>
<p>YAML and JSON were intended as data serialization formats, not to represent complex infrastructure stacks, and that is the root of this problem (<a href="https://blog.atomist.com/in-defense-of-yaml/">eloquently explained here</a>).</p>
<h3 id="large-monolithic-arm-templates">Large monolithic ARM templates</h3>
<p>It is not uncommon for Azure ARM templates to grow to 500, 600, or even 1000+ lines. Don’t take my word for it, just browse the <strong><a href="https://github.com/Azure/azure-quickstart-templates">Azure Quickstart Templates repo</a></strong>.</p>
<p>There are 2 main reasons for that :</p>
<ul>
<li>The ARM template syntax is quite verbose</li>
<li>Splitting large templates into smaller ones is not always possible or easy</li>
</ul>
<p>Sure, we can use <a href="https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-linked-templates">linked templates</a>, but the only way to specify a linked template is a URL, and it has to be publicly accessible.</p>
<p>Many coding standards, in any programming language have guidelines in terms of lines code per function, method, class, or file. A basic principle is :</p>
<blockquote>
<p>To be easy to read and understand, it should fit into a screen</p>
</blockquote>
<p>This is definitely not the case with most ARM templates, we often scroll down, and up, and down again… to understand what is going on.</p>
<p>This also impacts dev workflow/collaboration because some source management tools may not display the diffs for such long files. This makes reviewing pull requests more complicated than it should be.</p>
<p>In most programming contexts, <strong>we would never tolerate such large and monolithic code</strong>, so should we tolerate this for our infrastructure code ?</p>
<p>I don’t think so.<br />
So let’s see how <strong><a href="https://www.terraform.io/">Terraform</a></strong> can help, with an example based on this ARM template :<br />
<strong><a href="https://github.com/Azure/azure-quickstart-templates/tree/master/201-jenkins-acr">201-jenkins-acr</a></strong></p>
<h2 id="breaking-down-the-problem-into-smaller-parts">Breaking Down the Problem Into Smaller Parts</h2>
<p><strong>Decomposition</strong> is a crucial aspect of Software Engineering (and problem-solving in general). It is the practice of breaking a problem down into smaller parts to make it easier to tackle.</p>
<p>This manifests in code organized in units, which have clear, logical boundaries and which are named to express their purpose. These units of code can be functions, or classes in object-oriented languages.<br />
In <strong>Terraform</strong>, these units are <strong>modules</strong>.</p>
<blockquote>
<p>How do we decide where to split the code ?<br />
In other words, how do we define the boundaries of our Terraform modules ?</p>
</blockquote>
<h3 id="a-module-should-have-a-single-purpose-or-a-clear-focus">A module should have a single purpose, or a clear focus</h3>
<p>This is typically reflected in the module’s naming. If we find it hard to find a meaningful and concise name for our module, this is a sign that the module may be doing more than 1 thing.</p>
<p>Our example template defines the following resources :</p>
<ul>
<li>A storage account</li>
<li>An Azure container registry</li>
<li>Network-related resources (virtual network, subnet, NSG, etc.)</li>
<li>A Linux VM where we run Jenkins</li>
</ul>
<p>At the most basic level, a <strong>Terraform</strong> module is just a directory with a bunch of <code class="language-plaintext highlighter-rouge">.tf</code> files. So the way we split our code into modules should manifest in the directory structure of the repo. Here is what it looks like :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\git\tf-jenkins-acr</span><span class="w">
</span><span class="err">├───</span><span class="o">.</span><span class="nf">vscode</span><span class="w">
</span><span class="err">└───</span><span class="nx">modules</span><span class="w">
</span><span class="err">├───</span><span class="nf">acr</span><span class="w">
</span><span class="err">├───</span><span class="nf">jenkins</span><span class="w">
</span><span class="err">├───</span><span class="nf">networking</span><span class="w">
</span><span class="err">└───</span><span class="nf">storage</span><span class="w">
</span></code></pre></div></div>
<p>Inside the <code class="language-plaintext highlighter-rouge">modules</code> directory, each folder is a module.<br />
They all have a single-word name, which is an indication that they have a single purpose. Also, these names are reasonably explicit to convey the module’s responsibility.</p>
<p>By the way, all the code we are talking about here is available <strong><a href="https://github.com/MathieuBuisson/tf-jenkins-acr">in this GitHub repository</a></strong>.</p>
<h3 id="a-module-should-be-generic-enough-to-allow-for-reuse">A module should be generic enough to allow for reuse</h3>
<p>A <strong>Terraform</strong> module is only a part of a solution to a particular problem, and it is likely that the problem may change in the future. So, when designing a module, we should try to think about what is likely to change, and what is unlikely to change.</p>
<p>Then, having determined the appropriate level of flexibility for our context, we can decide what can be hard-coded and what should be parameterized.</p>
<p>For example in the storage account definition of the original ARM template, almost everything is hard-coded :</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Microsoft.Storage/storageAccounts"</span><span class="p">,</span><span class="w">
</span><span class="nl">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"[variables('acrStorageAccountName')]"</span><span class="p">,</span><span class="w">
</span><span class="nl">"apiVersion"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2016-01-01"</span><span class="p">,</span><span class="w">
</span><span class="nl">"location"</span><span class="p">:</span><span class="w"> </span><span class="s2">"[parameters('location')]"</span><span class="p">,</span><span class="w">
</span><span class="nl">"sku"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Standard_LRS"</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="nl">"kind"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Storage"</span><span class="p">,</span><span class="w">
</span><span class="nl">"properties"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"encryption"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"services"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"blob"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"enabled"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="nl">"keySource"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Microsoft.Storage"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>To make this into something which can adapt easily to future requirements or even different scenarios, the <code class="language-plaintext highlighter-rouge">storage</code> module has everything parameterized :</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">resource</span> <span class="dl">"</span><span class="s2">azurerm_storage_account</span><span class="dl">"</span> <span class="dl">"</span><span class="s2">sa</span><span class="dl">"</span> <span class="p">{</span>
<span class="nx">name</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.storage_account_name}</span><span class="dl">"</span>
<span class="nx">resource_group_name</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.resource_group_name}</span><span class="dl">"</span>
<span class="nx">location</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.location}</span><span class="dl">"</span>
<span class="nx">account_tier</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.storage_account_tier}</span><span class="dl">"</span>
<span class="nx">account_replication_type</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.storage_replication_type}</span><span class="dl">"</span>
<span class="nx">account_kind</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.storage_account_kind}</span><span class="dl">"</span>
<span class="nx">enable_blob_encryption</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.blob_encryption}</span><span class="dl">"</span>
<span class="nx">enable_file_encryption</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.file_encryption}</span><span class="dl">"</span>
<span class="nx">account_encryption_source</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.encryption_key_source}</span><span class="dl">"</span>
<span class="p">}</span>
</code></pre></div></div>
<div class="notice--info">
<h4>Note :</h4>
<p><code class="language-plaintext highlighter-rouge">${var.*}</code> is <strong>Terraform</strong> interpolation syntax to get variable values. <em>Variables</em> are more accurately described as the module’s <strong>input parameters</strong>.</p>
</div>
<p>To avoid having to specify values for these variables when using the module, but keep the flexibility to specify values whenever we want to, we populate the <code class="language-plaintext highlighter-rouge">variables.tf</code> file with default values :</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">variable</span> <span class="dl">"</span><span class="s2">storage_account_name</span><span class="dl">"</span> <span class="p">{</span>
<span class="nx">description</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">Name of the storage account</span><span class="dl">"</span>
<span class="nx">type</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">string</span><span class="dl">"</span>
<span class="p">}</span>
<span class="nx">variable</span> <span class="dl">"</span><span class="s2">resource_group_name</span><span class="dl">"</span> <span class="p">{</span>
<span class="nx">description</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">Name of the resource group where the storage account belongs</span><span class="dl">"</span>
<span class="nx">type</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">string</span><span class="dl">"</span>
<span class="p">}</span>
<span class="nx">variable</span> <span class="dl">"</span><span class="s2">location</span><span class="dl">"</span> <span class="p">{</span>
<span class="nx">description</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">Azure region where the storage account will be located</span><span class="dl">"</span>
<span class="nx">type</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">string</span><span class="dl">"</span>
<span class="p">}</span>
<span class="nx">variable</span> <span class="dl">"</span><span class="s2">storage_account_tier</span><span class="dl">"</span> <span class="p">{</span>
<span class="nx">description</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">Tier to use for this storage account. Valid values are : 'Standard' and 'Premium'.</span><span class="dl">"</span>
<span class="k">default</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">Standard</span><span class="dl">"</span>
<span class="p">}</span>
<span class="nx">variable</span> <span class="dl">"</span><span class="s2">storage_replication_type</span><span class="dl">"</span> <span class="p">{</span>
<span class="nx">description</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">Type of replication to use for this storage account. Valid values are : 'LRS', 'GRS', 'RAGRS' and 'ZRS'.</span><span class="dl">"</span>
<span class="k">default</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">LRS</span><span class="dl">"</span>
<span class="p">}</span>
<span class="nx">variable</span> <span class="dl">"</span><span class="s2">storage_account_kind</span><span class="dl">"</span> <span class="p">{</span>
<span class="nx">description</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">Kind of storage account. Valid values are : 'Storage', 'StorageV2' and 'BlobStorage'.</span><span class="dl">"</span>
<span class="k">default</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">Storage</span><span class="dl">"</span>
<span class="p">}</span>
<span class="nx">variable</span> <span class="dl">"</span><span class="s2">blob_encryption</span><span class="dl">"</span> <span class="p">{</span>
<span class="nx">description</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">Whether Encryption Services should be enabled for Blob storage</span><span class="dl">"</span>
<span class="k">default</span> <span class="o">=</span> <span class="kc">true</span>
<span class="p">}</span>
<span class="nx">variable</span> <span class="dl">"</span><span class="s2">file_encryption</span><span class="dl">"</span> <span class="p">{</span>
<span class="nx">description</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">Whether Encryption Services should be enabled for file storage</span><span class="dl">"</span>
<span class="k">default</span> <span class="o">=</span> <span class="kc">false</span>
<span class="p">}</span>
<span class="nx">variable</span> <span class="dl">"</span><span class="s2">encryption_key_source</span><span class="dl">"</span> <span class="p">{</span>
<span class="nx">description</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">Encryption key source for the storage account. Valid values are : 'Microsoft.Keyvault' and 'Microsoft.Storage'.</span><span class="dl">"</span>
<span class="k">default</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">Microsoft.Storage</span><span class="dl">"</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="a-module-should-group-tightly-coupled-resources-together">A module should group tightly coupled resources together</h3>
<p>When considering where to set modules boundaries, we should take into account dependencies between resources.<br />
This means we should try to group dependent resources in the same module as much as possible/practical.</p>
<p>In our example ARM template, the subnet resource is tightly coupled with the virtual network resource (it’s even nested into it). And then, the subnet resource is associated with both the network interface and the network security group.</p>
<p>To make this dependency chain easier to manage, we group these resources in the <code class="language-plaintext highlighter-rouge">networking</code> module. In this module, there is a nice example of how a resource can reference other resources from the same module:</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">resource</span> <span class="dl">"</span><span class="s2">azurerm_subnet_network_security_group_association</span><span class="dl">"</span> <span class="dl">"</span><span class="s2">nsglink</span><span class="dl">"</span> <span class="p">{</span>
<span class="nx">subnet_id</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${azurerm_subnet.subnet.id}</span><span class="dl">"</span>
<span class="nx">network_security_group_id</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${azurerm_network_security_group.nsg.id}</span><span class="dl">"</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Here, we define a <code class="language-plaintext highlighter-rouge">azurerm_subnet_network_security_group_association</code> resource which depends on a subnet resource and a network security group. The interpolation syntax to reference another resource is : <code class="language-plaintext highlighter-rouge">resource_type.resource_instance_name.attribute</code>.</p>
<h3 id="a-module-should-be-loosely-coupled-with-other-modules">A module should be loosely coupled with other modules</h3>
<p>If multiple modules are used as part of the same overall solution, there will be cases where a resource from a module needs to reference a resource from another module. This <strong>inter-module coupling should be loose</strong>.</p>
<p>What this means is that <strong>Terraform</strong> modules should not have direct knowledge of other modules. Instead, they take their dependencies via <strong>input parameters</strong> (i.e. variables) and they expose information needed by other modules via <strong>outputs</strong>.</p>
<p>In our example scenario, the <code class="language-plaintext highlighter-rouge">jenkins</code> module needs to know the login server name of the Azure container registry, to be able to configure Jenkins to push to this registry. So the <code class="language-plaintext highlighter-rouge">acr</code> module exposes this information via its <code class="language-plaintext highlighter-rouge">login_server</code> output :</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">output</span> <span class="dl">"</span><span class="s2">login_server</span><span class="dl">"</span> <span class="p">{</span>
<span class="nx">value</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${azurerm_container_registry.acr.login_server}</span><span class="dl">"</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This information is <em>returned</em> to the caller (root module or <strong>Terraform</strong> configuration) and accessible via the interpolation syntax <code class="language-plaintext highlighter-rouge">module.module_name.output_name</code>, like so :</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">module</span> <span class="dl">"</span><span class="s2">jenkins</span><span class="dl">"</span> <span class="p">{</span>
<span class="p">...</span>
<span class="p">...</span>
<span class="nx">registry_login_server</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${module.acr.login_server}</span><span class="dl">"</span>
<span class="p">...</span>
<span class="p">...</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Then, the caller passes this value to the <code class="language-plaintext highlighter-rouge">jenkins</code> module via its <code class="language-plaintext highlighter-rouge">registry_login_server</code> parameter.<br />
This parameter is defined in the <code class="language-plaintext highlighter-rouge">jenkins</code> module as a variable, like so :</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">variable</span> <span class="dl">"</span><span class="s2">registry_login_server</span><span class="dl">"</span> <span class="p">{</span>
<span class="nx">description</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">URL to specify to login to the container registry</span><span class="dl">"</span>
<span class="nx">type</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">string</span><span class="dl">"</span>
<span class="p">}</span>
</code></pre></div></div>
<p>In this arrangement, the <code class="language-plaintext highlighter-rouge">jenkins</code> module doesn’t need to know where this information comes from.<br />
Similarly, the <code class="language-plaintext highlighter-rouge">acr</code> module doesn’t need to know what is consuming its outputs, or even, whether its outputs are being used at all.</p>
<h2 id="composing-a-solution-for-the-overall-problem">Composing a Solution For the Overall Problem</h2>
<p>When we are done splitting our <strong>Infrastructure as Code</strong> template into smaller, primitive building blocks, and each one of these is responsible for its own specific concern, it’s time to tie them together to form an overall solution.</p>
<p>To that end, we write a <strong>Terraform</strong> configuration which calls the modules and combines them together by accessing their outputs and passing them to other module’s parameters as needed.</p>
<div class="notice--info">
<h4>Note :</h4>
<p>As alluded to earlier, this <strong>Terraform</strong> configuration can also be called a <em>“root module”</em>. This is not really different, since a module is just a directory with a bunch of <code class="language-plaintext highlighter-rouge">.tf</code> files.</p>
</div>
<p>In our case, this refers to the <code class="language-plaintext highlighter-rouge">.tf</code> files located at the root of the repository :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">PS</span><span class="w"> </span><span class="nx">C:\</span><span class="err">></span><span class="w"> </span><span class="nx">dir</span><span class="w"> </span><span class="o">*.</span><span class="nf">tf</span><span class="w">
</span><span class="nx">Directory:</span><span class="w"> </span><span class="nx">C:\git\tf-jenkins-acr</span><span class="w">
</span><span class="nf">Mode</span><span class="w"> </span><span class="nx">LastWriteTime</span><span class="w"> </span><span class="nx">Length</span><span class="w"> </span><span class="nx">Name</span><span class="w">
</span><span class="nf">----</span><span class="w"> </span><span class="nf">-------------</span><span class="w"> </span><span class="nf">------</span><span class="w"> </span><span class="nf">----</span><span class="w">
</span><span class="nt">-a</span><span class="nf">----</span><span class="w"> </span><span class="nx">23/04/2019</span><span class="w"> </span><span class="nx">09:51</span><span class="w"> </span><span class="nx">2666</span><span class="w"> </span><span class="nx">main.tf</span><span class="w">
</span><span class="nt">-a</span><span class="nf">----</span><span class="w"> </span><span class="nx">23/04/2019</span><span class="w"> </span><span class="nx">09:51</span><span class="w"> </span><span class="nx">294</span><span class="w"> </span><span class="nx">outputs.tf</span><span class="w">
</span><span class="nt">-a</span><span class="nf">----</span><span class="w"> </span><span class="nx">23/04/2019</span><span class="w"> </span><span class="nx">09:51</span><span class="w"> </span><span class="nx">330</span><span class="w"> </span><span class="nx">providers.tf</span><span class="w">
</span><span class="nt">-a</span><span class="nf">----</span><span class="w"> </span><span class="nx">23/04/2019</span><span class="w"> </span><span class="nx">09:51</span><span class="w"> </span><span class="nx">1807</span><span class="w"> </span><span class="nx">variables.tf</span><span class="w">
</span></code></pre></div></div>
<p>The main file (as its name implies) is the <code class="language-plaintext highlighter-rouge">main.tf</code>, as this is the one which defines all resources. Actually, it doesn’t define resources directly, it does so <strong>indirectly</strong> by invoking the modules which take care of managing their own set of resources.</p>
<p>So here is our newly modularized <strong>Infrastructure as Code</strong> solution (<code class="language-plaintext highlighter-rouge">main.tf</code>):</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">resource</span> <span class="dl">"</span><span class="s2">random_string</span><span class="dl">"</span> <span class="dl">"</span><span class="s2">suffix</span><span class="dl">"</span> <span class="p">{</span>
<span class="nx">length</span> <span class="o">=</span> <span class="mi">10</span>
<span class="nx">upper</span> <span class="o">=</span> <span class="kc">false</span>
<span class="nx">special</span> <span class="o">=</span> <span class="kc">false</span>
<span class="p">}</span>
<span class="nx">locals</span> <span class="p">{</span>
<span class="nx">resource_prefix</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">jenkins</span><span class="dl">"</span>
<span class="nx">resource_suffix</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${random_string.suffix.result}</span><span class="dl">"</span>
<span class="nx">default_storage_account_name</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">registry${local.resource_suffix}</span><span class="dl">"</span>
<span class="nx">storage_account_name</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.storage_account_name != </span><span class="dl">""</span><span class="s2"> ? var.storage_account_name : local.default_storage_account_name}</span><span class="dl">"</span>
<span class="nx">default_acr_name</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">acr${local.resource_suffix}</span><span class="dl">"</span>
<span class="nx">acr_name</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.registry_name != </span><span class="dl">""</span><span class="s2"> ? var.registry_name : local.default_acr_name}</span><span class="dl">"</span>
<span class="p">}</span>
<span class="nx">resource</span> <span class="dl">"</span><span class="s2">azurerm_resource_group</span><span class="dl">"</span> <span class="dl">"</span><span class="s2">rg</span><span class="dl">"</span> <span class="p">{</span>
<span class="nx">name</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.resource_group_name}</span><span class="dl">"</span>
<span class="nx">location</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.location}</span><span class="dl">"</span>
<span class="p">}</span>
<span class="nx">module</span> <span class="dl">"</span><span class="s2">storage</span><span class="dl">"</span> <span class="p">{</span>
<span class="nx">source</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">./modules/storage</span><span class="dl">"</span>
<span class="nx">storage_account_name</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${local.storage_account_name}</span><span class="dl">"</span>
<span class="nx">resource_group_name</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${azurerm_resource_group.rg.name}</span><span class="dl">"</span>
<span class="nx">location</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${azurerm_resource_group.rg.location}</span><span class="dl">"</span>
<span class="nx">storage_account_tier</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.storage_account_tier}</span><span class="dl">"</span>
<span class="p">}</span>
<span class="nx">module</span> <span class="dl">"</span><span class="s2">acr</span><span class="dl">"</span> <span class="p">{</span>
<span class="nx">source</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">./modules/acr</span><span class="dl">"</span>
<span class="nx">registry_name</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${local.acr_name}</span><span class="dl">"</span>
<span class="nx">resource_group_name</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${azurerm_resource_group.rg.name}</span><span class="dl">"</span>
<span class="nx">location</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${azurerm_resource_group.rg.location}</span><span class="dl">"</span>
<span class="nx">storage_account_id</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${module.storage.storage_account_id}</span><span class="dl">"</span>
<span class="p">}</span>
<span class="nx">module</span> <span class="dl">"</span><span class="s2">networking</span><span class="dl">"</span> <span class="p">{</span>
<span class="nx">source</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">./modules/networking</span><span class="dl">"</span>
<span class="nx">public_ip_name</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${local.resource_prefix}-publicip</span><span class="dl">"</span>
<span class="nx">resource_group_name</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${azurerm_resource_group.rg.name}</span><span class="dl">"</span>
<span class="nx">location</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${azurerm_resource_group.rg.location}</span><span class="dl">"</span>
<span class="nx">public_ip_dns_label</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.jenkins_vm_dns_prefix}</span><span class="dl">"</span>
<span class="nx">nsg_name</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${local.resource_prefix}-nsg</span><span class="dl">"</span>
<span class="nx">vnet_name</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${local.resource_prefix}-vnet</span><span class="dl">"</span>
<span class="nx">subnet_name</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${local.resource_prefix}-subnet</span><span class="dl">"</span>
<span class="nx">nic_name</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${local.resource_prefix}-nic</span><span class="dl">"</span>
<span class="p">}</span>
<span class="nx">module</span> <span class="dl">"</span><span class="s2">jenkins</span><span class="dl">"</span> <span class="p">{</span>
<span class="nx">source</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">./modules/jenkins</span><span class="dl">"</span>
<span class="nx">vm_name</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.jenkins_vm_dns_prefix}</span><span class="dl">"</span>
<span class="nx">resource_group_name</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${azurerm_resource_group.rg.name}</span><span class="dl">"</span>
<span class="nx">location</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${azurerm_resource_group.rg.location}</span><span class="dl">"</span>
<span class="nx">nic_id</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${module.networking.nic_id}</span><span class="dl">"</span>
<span class="nx">vm_size</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.vm_size}</span><span class="dl">"</span>
<span class="nx">admin_username</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.admin_username}</span><span class="dl">"</span>
<span class="nx">admin_password</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.admin_password}</span><span class="dl">"</span>
<span class="nx">public_ip_fqdn</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${module.networking.public_ip_fqdn}</span><span class="dl">"</span>
<span class="nx">git_repository</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.git_repository}</span><span class="dl">"</span>
<span class="nx">registry_login_server</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${module.acr.login_server}</span><span class="dl">"</span>
<span class="nx">service_principal_id</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.service_principal_id}</span><span class="dl">"</span>
<span class="nx">service_principal_secret</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${var.service_principal_secret}</span><span class="dl">"</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">locals</code> block is not necessary but it is a flexible way of setting a default naming convention for resources, if names are not specified via input variables.</p>
<p>The only resource which is defined directly from here is the resource group, everything else is done by the modules.</p>
<p>This is <strong>63 lines</strong> of code, which is a pretty massive improvement over the original ARM template (<strong>294 lines</strong>). This means reading (and more importantly, understanding) what is going on in this configuration requires much less scrolling.</p>
<p>Also, the fact that this is now split into smaller pieces and that everything is parameterized, makes it easier to maintain and <strong>more adaptable to changing requirements</strong> (and remember, the only constant is change).</p>
<p>Again, the code is <strong><a href="https://github.com/MathieuBuisson/tf-jenkins-acr">here</a></strong>.<br />
Feel free to take a look at it, get an understanding of how the parts fit together, and since this is a <strong>working</strong> example, you can even try it for yourselves.</p>TheShellNutIn this post, we show how to make Azure "Infrastructure as Code" more concise and modular. We take an ARM template from the QuickStart Templates repo, we break it down into small, tightly focused units and we combine them as a Terraform configurationDeploying a production-ready Azure Kubernetes (AKS) cluster with PSAksDeployment2019-02-04T00:00:00+00:002019-02-04T00:00:00+00:00https://mathieubuisson.github.io/deploying-aks-cluster-psaksdeployment<aside class="sidebar__right">
<nav class="toc">
<header><h4 class="nav__title"><i class="fa fa-file-text"></i> In This Article</h4></header>
<ul class="toc__menu" id="markdown-toc">
<li><a href="#introducing-psaksdeployment" id="markdown-toc-introducing-psaksdeployment">Introducing PSAksDeployment</a></li>
<li><a href="#installing-psaksdeployment" id="markdown-toc-installing-psaksdeployment">Installing PSAksDeployment</a></li>
<li><a href="#deploying-an-aks-cluster" id="markdown-toc-deploying-an-aks-cluster">Deploying an AKS Cluster</a></li>
<li><a href="#deleting-an-aks-cluster" id="markdown-toc-deleting-an-aks-cluster">Deleting an AKS Cluster</a></li>
<li><a href="#zooming-in-on-a-few-terraform-resources" id="markdown-toc-zooming-in-on-a-few-terraform-resources">Zooming In On a Few Terraform Resources</a></li>
</ul>
</nav>
</aside>
<h2 id="introducing-psaksdeployment">Introducing PSAksDeployment</h2>
<p><strong><a href="https://azure.microsoft.com/en-us/services/kubernetes-service/">Azure Kubernetes Service</a></strong> (AKS) makes provisioning <strong><a href="https://kubernetes.io/">Kubernetes</a></strong> clusters very easy.<br />
Unfortunately, the examples we can find out there, be it in official documentation or blog posts, are more “<strong>Hello World!</strong>” than “<strong>real world</strong>”.</p>
<p>Deploying a <strong>production-ready</strong> Kubernetes cluster requires additional components and considerations, like :</p>
<ul>
<li>Monitoring</li>
<li><a href="https://kubernetes.io/docs/reference/kubectl">Kubectl</a> configuration</li>
<li>How to deploy resources (<a href="https://helm.sh/">Helm</a> and Tiller)</li>
<li>Routing requests from the outside world to services in the cluster (ingress controller)</li>
<li>Issuing and managing TLS certificates for HTTPS endpoints</li>
</ul>
<p><code class="language-plaintext highlighter-rouge">PSAksDeployment</code> aims to bridge the gap between a “<strong>Hello World!</strong>” AKS cluster and a cluster on which we can run production apps.</p>
<p>It is an opinionated implementation, in the sense that :</p>
<ul>
<li>The monitoring solution is <strong><a href="https://azure.microsoft.com/en-us/services/monitor/">Azure Monitor</a></strong> (with Log Analytics)</li>
<li>The ingress controller is <strong><a href="https://kubernetes.github.io/ingress-nginx/">NGINX Ingress Controller</a></strong></li>
<li>Management Kubernetes resources are deployed into a “management” namespace</li>
<li>The solution to manage TLS certificates is <strong><a href="https://github.com/jetstack/cert-manager">cert-manager</a></strong> (with Let’s Encrypt)</li>
<li>The ingress controller TLS certificate is propagated to other namespaces (including namespaces created at a later point), to allow ingresses in any namespace to use it, using a custom tool : <strong><a href="https://github.com/MathieuBuisson/PSAksDeployment/tree/master/PSAksDeployment/Assets/secret-propagator">secret-propagator</a></strong></li>
</ul>
<h2 id="installing-psaksdeployment">Installing PSAksDeployment</h2>
<p>To use <code class="language-plaintext highlighter-rouge">PSAksDeployment</code>, you need :</p>
<ul>
<li>Windows PowerShell 5.1</li>
<li><a href="https://dotnet.microsoft.com/download/dotnet-framework-runtime">.Net Framework 4.7.2</a> (required by the “Az” PowerShell modules)</li>
</ul>
<p>This means it is only supported on Windows at this point in time. I may make it work with PowerShell Core if there is a need for it.</p>
<p><code class="language-plaintext highlighter-rouge">PSAksDeployment</code> is available on the <a href="https://www.powershellgallery.com/packages/PSAksDeployment/">PowerShell Gallery</a>, so installing it is easy as :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">Install-Module</span><span class="w"> </span><span class="nt">-Name</span><span class="w"> </span><span class="s1">'PSAksDeployment'</span><span class="w"> </span><span class="nt">-Repository</span><span class="w"> </span><span class="s1">'PSGallery'</span><span class="w">
</span></code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">PSAksDeployment</code> leverages a few extra tools under the hood :</p>
<ul>
<li><a href="https://www.terraform.io/">Terraform</a></li>
<li><a href="https://kubernetes.io/docs/reference/kubectl">Kubectl</a></li>
<li><a href="https://helm.sh/">Helm</a></li>
</ul>
<p>So <code class="language-plaintext highlighter-rouge">PSAksDeployment</code> provides the command <code class="language-plaintext highlighter-rouge">Install-PSAksPrerequisites</code>, which downloads and installs these tools (if they are not already installed) in a location specified via the <code class="language-plaintext highlighter-rouge">InstallationFolder</code> parameter, like so :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">Install-PSAksPrerequisites</span><span class="w"> </span><span class="nt">-InstallationFolder</span><span class="w"> </span><span class="s1">'C:\Tools'</span><span class="w">
</span></code></pre></div></div>
<p>If the specified installation folder is not in the <code class="language-plaintext highlighter-rouge">PATH</code> environment variable, it also takes care of adding it.</p>
<div class="notice--warning">
<h4>Note :</h4>
<p>Be patient, the file downloads may take a while.</p>
</div>
<p>As soon as <code class="language-plaintext highlighter-rouge">Install-PSAksPrerequisites</code> completes, we can start deploying stuff.</p>
<h2 id="deploying-an-aks-cluster">Deploying an AKS Cluster</h2>
<p>This is where the command <code class="language-plaintext highlighter-rouge">Invoke-PSAksDeployment</code> comes in.</p>
<p>It deploys the following :</p>
<ul>
<li>an <strong><a href="https://azure.microsoft.com/en-us/services/kubernetes-service/">Azure Kubernetes Service</a></strong> (AKS) instance</li>
<li>an Azure Log Analytics workspace with the ContainerInsights solution</li>
<li>a Public IP address for the ingress controller</li>
<li>a “management” namespace</li>
<li>Tiller</li>
<li><strong><a href="https://kubernetes.github.io/ingress-nginx/">NGINX Ingress Controller</a></strong></li>
<li><strong><a href="https://github.com/jetstack/cert-manager">cert-manager</a></strong></li>
<li>a TLS certificate (to support HTTPS ingresses)</li>
<li><strong><a href="https://github.com/MathieuBuisson/PSAksDeployment/tree/master/PSAksDeployment/Assets/secret-propagator">secret-propagator</a></strong></li>
</ul>
<p>It primarily acts as an input validation and orchestration layer. Under the hood, most of the work is done by <strong><a href="https://www.terraform.io/">Terraform</a></strong>.</p>
<p>Due to the nature of what it does, <code class="language-plaintext highlighter-rouge">Invoke-PSAksDeployment</code> takes in a large number of parameters :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">PS</span><span class="w"> </span><span class="nx">C:\</span><span class="err">></span><span class="w"> </span><span class="nx">Import-Module</span><span class="w"> </span><span class="nt">-Name</span><span class="w"> </span><span class="s1">'PSAksDeployment'</span><span class="w">
</span><span class="nf">PS</span><span class="w"> </span><span class="nx">C:\</span><span class="err">></span><span class="w"> </span><span class="p">(</span><span class="nf">Get-Command</span><span class="w"> </span><span class="nt">-Name</span><span class="w"> </span><span class="s1">'Invoke-PSAksDeployment'</span><span class="p">)</span><span class="o">.</span><span class="nf">Parameters</span><span class="o">.</span><span class="nf">Keys</span><span class="w">
</span><span class="nx">ServicePrincipalID</span><span class="w">
</span><span class="nx">ServicePrincipalSecret</span><span class="w">
</span><span class="nf">AzureTenantID</span><span class="w">
</span><span class="nf">Subscription</span><span class="w">
</span><span class="nf">ClusterName</span><span class="w">
</span><span class="nf">ClusterLocation</span><span class="w">
</span><span class="nf">LogAnalyticsWorkspaceLocation</span><span class="w">
</span><span class="nf">KubernetesVersion</span><span class="w">
</span><span class="nf">NodeCount</span><span class="w">
</span><span class="nf">NodeSize</span><span class="w">
</span><span class="nf">OSDiskSizeGB</span><span class="w">
</span><span class="nf">MaxPodsPerNode</span><span class="w">
</span><span class="nf">Environment</span><span class="w">
</span><span class="nf">LetsEncryptEmail</span><span class="w">
</span><span class="nf">TerraformOutputFolder</span><span class="w">
</span><span class="nf">ConfigPath</span><span class="w">
</span></code></pre></div></div>
<p>You may be wondering :</p>
<blockquote>
<p>What are these for ?<br />
Which ones are mandatory ?<br />
What is the default value ?<br />
What are the possible values ?</p>
</blockquote>
<p>We can get this information with the cmdlet help, for example :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">PS</span><span class="w"> </span><span class="nx">C:\</span><span class="err">></span><span class="w"> </span><span class="nx">Get-Help</span><span class="w"> </span><span class="s1">'Invoke-PSAksDeployment'</span><span class="w"> </span><span class="nt">-Parameter</span><span class="w"> </span><span class="s1">'ServicePrincipalID'</span><span class="w">
</span><span class="nt">-ServicePrincipalID</span><span class="w"> </span><span class="err"><</span><span class="nf">String</span><span class="err">></span><span class="w">
</span><span class="nf">The</span><span class="w"> </span><span class="nx">application</span><span class="w"> </span><span class="nx">ID</span><span class="w"> </span><span class="nx">of</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">Service</span><span class="w"> </span><span class="nx">Principal</span><span class="w"> </span><span class="nx">used</span><span class="w"> </span><span class="nx">by</span><span class="w"> </span><span class="nx">Terraform</span><span class="w"> </span><span class="p">(</span><span class="nf">and</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">AKS</span><span class="w"> </span><span class="nx">cluster</span><span class="p">)</span><span class="w"> </span><span class="nf">to</span><span class="w"> </span><span class="nx">access</span><span class="w"> </span><span class="nx">Azure.</span><span class="w">
</span><span class="nf">Required</span><span class="err">?</span><span class="w"> </span><span class="nx">true</span><span class="w">
</span><span class="nf">Position</span><span class="err">?</span><span class="w"> </span><span class="nx">1</span><span class="w">
</span><span class="nf">Default</span><span class="w"> </span><span class="nx">value</span><span class="w">
</span><span class="nf">Accept</span><span class="w"> </span><span class="nx">pipeline</span><span class="w"> </span><span class="nx">input</span><span class="err">?</span><span class="w"> </span><span class="nx">false</span><span class="w">
</span><span class="nf">Accept</span><span class="w"> </span><span class="nx">wildcard</span><span class="w"> </span><span class="nx">characters</span><span class="err">?</span><span class="w"> </span><span class="nx">false</span><span class="w">
</span></code></pre></div></div>
<p>But doing this for each parameter can be tedious, so there is another way to specify all inputs to <code class="language-plaintext highlighter-rouge">Invoke-PSAksDeployment</code> : a configuration file.</p>
<p><code class="language-plaintext highlighter-rouge">New-PSAksDeploymentConfig</code> scaffolds this configuration file, with helpful information for each parameter :</p>
<ul>
<li>a description</li>
<li>the data type</li>
<li>valid values</li>
<li>the default value</li>
</ul>
<p>Here is an example :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">PS</span><span class="w"> </span><span class="nx">C:\</span><span class="err">></span><span class="w"> </span><span class="nv">$ScaffoldParams</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">@{</span><span class="w">
</span><span class="err">>></span><span class="w"> </span><span class="nx">ServicePrincipalID</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'29x1ecd3-190f-42c9-8660-088f69d121zn'</span><span class="w">
</span><span class="err">>></span><span class="w"> </span><span class="nx">ServicePrincipalSecret</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'tsWpRr6/YCxNyh8efMvjWbe5JoOiOw03xR1o9S5CLhZ='</span><span class="w">
</span><span class="err">>></span><span class="w"> </span><span class="nx">AzureTenantID</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'96v3b174-9c1p-4a5e-9177-18c3bccc87cb'</span><span class="w">
</span><span class="err">>></span><span class="w"> </span><span class="nx">Subscription</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'DevOps'</span><span class="w">
</span><span class="err">>></span><span class="w"> </span><span class="nx">ClusterLocation</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'North Europe'</span><span class="w">
</span><span class="err">>></span><span class="w"> </span><span class="nx">Path</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'.\my-k8s-prod.psd1'</span><span class="w">
</span><span class="err">>></span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="nf">PS</span><span class="w"> </span><span class="nx">C:\</span><span class="err">></span><span class="w"> </span><span class="nx">New-PSAksDeploymentConfig</span><span class="w"> </span><span class="err">@</span><span class="nx">ScaffoldParams</span><span class="w">
</span></code></pre></div></div>
<p>We still need to provide quite a few bits of information, but this is needed so that the tool can connect to Azure and fetch more information, like :</p>
<ul>
<li>Subscriptions the specified Azure Service Principal has access to</li>
<li>Azure regions where AKS is available</li>
<li>Kubernetes versions available in the specified region</li>
<li>Azure regions where a Log Analytics workspace can be provisioned</li>
</ul>
<p>Here is what the generated file looks like :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">@{</span><span class="w">
</span><span class="err"><#</span><span class="w">
</span><span class="nx">The</span><span class="w"> </span><span class="nx">name</span><span class="w"> </span><span class="nx">of</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">Azure</span><span class="w"> </span><span class="nx">subscription</span><span class="w"> </span><span class="nx">where</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">AKS</span><span class="w"> </span><span class="nx">instance</span><span class="w"> </span><span class="err">(</span><span class="nx">and</span><span class="w"> </span><span class="nx">other</span><span class="w"> </span><span class="nx">Azure</span><span class="w"> </span><span class="nx">resources</span><span class="err">)</span><span class="w"> </span><span class="nx">will</span><span class="w"> </span><span class="nx">be</span><span class="w"> </span><span class="nx">deployed</span><span class="err">.</span><span class="w">
</span><span class="nx">Type</span><span class="w"> </span><span class="err">:</span><span class="w"> </span><span class="nx">String</span><span class="w">
</span><span class="nx">Valid</span><span class="w"> </span><span class="nx">values</span><span class="w"> </span><span class="err">:</span><span class="w"> </span><span class="s2">"DevOps"</span><span class="p">,</span><span class="w"> </span><span class="s2">"ANY OTHER SUBSCRIPTION WHICH CAN BE ACCESSED BY THE SERVICE PRINCIPAL"</span><span class="w">
</span><span class="err">#></span><span class="w">
</span><span class="nx">Subscription</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">""</span><span class="w">
</span><span class="err"><#</span><span class="w">
</span><span class="nx">The</span><span class="w"> </span><span class="nx">Azure</span><span class="w"> </span><span class="nx">region</span><span class="w"> </span><span class="nx">where</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">AKS</span><span class="w"> </span><span class="nx">cluster</span><span class="w"> </span><span class="err">(</span><span class="nx">and</span><span class="w"> </span><span class="nx">other</span><span class="w"> </span><span class="nx">Azure</span><span class="w"> </span><span class="nx">resources</span><span class="err">)</span><span class="w"> </span><span class="nx">will</span><span class="w"> </span><span class="nx">be</span><span class="w"> </span><span class="nx">deployed</span><span class="err">.</span><span class="w">
</span><span class="nx">Type</span><span class="w"> </span><span class="err">:</span><span class="w"> </span><span class="nx">String</span><span class="w">
</span><span class="nx">Valid</span><span class="w"> </span><span class="nx">values</span><span class="w"> </span><span class="err">:</span><span class="w"> </span><span class="s2">"East US"</span><span class="p">,</span><span class="w"> </span><span class="s2">"West Europe"</span><span class="p">,</span><span class="w"> </span><span class="s2">"Central US"</span><span class="p">,</span><span class="w"> </span><span class="s2">"Canada Central"</span><span class="p">,</span><span class="w"> </span><span class="s2">"Canada East"</span><span class="p">,</span><span class="w"> </span><span class="s2">"UK South"</span><span class="p">,</span><span class="w"> </span><span class="s2">"West US"</span><span class="p">,</span><span class="w"> </span><span class="s2">"West US 2"</span><span class="p">,</span><span class="w"> </span><span class="s2">"Australia East"</span><span class="p">,</span><span class="w"> </span><span class="s2">"North Europe"</span><span class="p">,</span><span class="w"> </span><span class="s2">"Japan East"</span><span class="p">,</span><span class="w"> </span><span class="s2">"East US 2"</span><span class="p">,</span><span class="w"> </span><span class="s2">"Southeast Asia"</span><span class="p">,</span><span class="w"> </span><span class="s2">"UK West"</span><span class="p">,</span><span class="w"> </span><span class="s2">"South India"</span><span class="p">,</span><span class="w"> </span><span class="s2">"East Asia"</span><span class="w">
</span><span class="err">#></span><span class="w">
</span><span class="nx">ClusterLocation</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">""</span><span class="w">
</span><span class="err"><#</span><span class="w">
</span><span class="nx">The</span><span class="w"> </span><span class="nx">Azure</span><span class="w"> </span><span class="nx">region</span><span class="w"> </span><span class="nx">where</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">Log</span><span class="w"> </span><span class="nx">Analytics</span><span class="w"> </span><span class="nx">workspace</span><span class="w"> </span><span class="nx">will</span><span class="w"> </span><span class="nx">be</span><span class="w"> </span><span class="nx">deployed</span><span class="err">.</span><span class="w">
</span><span class="nx">This</span><span class="w"> </span><span class="nx">might</span><span class="w"> </span><span class="nx">not</span><span class="w"> </span><span class="nx">be</span><span class="w"> </span><span class="nx">possible</span><span class="w"> </span><span class="nx">to</span><span class="w"> </span><span class="nx">provision</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">Log</span><span class="w"> </span><span class="nx">Analytics</span><span class="w"> </span><span class="nx">workspace</span><span class="w"> </span><span class="nx">in</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">same</span><span class="w"> </span><span class="nx">region</span><span class="w"> </span><span class="nx">as</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">AKS</span><span class="w"> </span><span class="nx">cluster</span><span class="p">,</span><span class="w"> </span><span class="nx">because</span><span class="w"> </span><span class="nx">Log</span><span class="w"> </span><span class="nx">Analytics</span><span class="w"> </span><span class="nx">is</span><span class="w"> </span><span class="nx">available</span><span class="w"> </span><span class="nx">in</span><span class="w"> </span><span class="nx">a</span><span class="w"> </span><span class="nx">limited</span><span class="w"> </span><span class="nx">set</span><span class="w"> </span><span class="nx">of</span><span class="w"> </span><span class="nx">regions</span><span class="err">.</span><span class="w">
</span><span class="nx">Type</span><span class="w"> </span><span class="err">:</span><span class="w"> </span><span class="nx">String</span><span class="w">
</span><span class="nx">Valid</span><span class="w"> </span><span class="nx">values</span><span class="w"> </span><span class="err">:</span><span class="w"> </span><span class="s2">"East US"</span><span class="p">,</span><span class="w"> </span><span class="s2">"West Europe"</span><span class="p">,</span><span class="w"> </span><span class="s2">"Southeast Asia"</span><span class="p">,</span><span class="w"> </span><span class="s2">"Australia Southeast"</span><span class="p">,</span><span class="w"> </span><span class="s2">"West Central US"</span><span class="p">,</span><span class="w"> </span><span class="s2">"Japan East"</span><span class="p">,</span><span class="w"> </span><span class="s2">"UK South"</span><span class="p">,</span><span class="w"> </span><span class="s2">"Central India"</span><span class="p">,</span><span class="w"> </span><span class="s2">"Canada Central"</span><span class="p">,</span><span class="w"> </span><span class="s2">"West US 2"</span><span class="w">
</span><span class="err">#></span><span class="w">
</span><span class="nx">LogAnalyticsWorkspaceLocation</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">""</span><span class="w">
</span><span class="err"><#</span><span class="w">
</span><span class="nx">The</span><span class="w"> </span><span class="nx">application</span><span class="w"> </span><span class="nx">ID</span><span class="w"> </span><span class="nx">of</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">Service</span><span class="w"> </span><span class="nx">Principal</span><span class="w"> </span><span class="nx">used</span><span class="w"> </span><span class="nx">by</span><span class="w"> </span><span class="nx">Terraform</span><span class="w"> </span><span class="err">(</span><span class="nx">and</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">AKS</span><span class="w"> </span><span class="nx">cluster</span><span class="err">)</span><span class="w"> </span><span class="nx">to</span><span class="w"> </span><span class="nx">access</span><span class="w"> </span><span class="nx">Azure</span><span class="err">.</span><span class="w">
</span><span class="nx">Type</span><span class="w"> </span><span class="err">:</span><span class="w"> </span><span class="nx">String</span><span class="w">
</span><span class="nx">Valid</span><span class="w"> </span><span class="nx">values</span><span class="w"> </span><span class="err">:</span><span class="w">
</span><span class="err">#></span><span class="w">
</span><span class="nx">ServicePrincipalID</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">""</span><span class="w">
</span><span class="err"><#</span><span class="w">
</span><span class="nx">The</span><span class="w"> </span><span class="nx">password</span><span class="w"> </span><span class="nx">of</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">Service</span><span class="w"> </span><span class="nx">Principal</span><span class="w"> </span><span class="nx">used</span><span class="w"> </span><span class="nx">by</span><span class="w"> </span><span class="nx">Terraform</span><span class="w"> </span><span class="err">(</span><span class="nx">and</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">AKS</span><span class="w"> </span><span class="nx">cluster</span><span class="err">)</span><span class="w"> </span><span class="nx">to</span><span class="w"> </span><span class="nx">access</span><span class="w"> </span><span class="nx">Azure</span><span class="err">.</span><span class="w">
</span><span class="nx">Type</span><span class="w"> </span><span class="err">:</span><span class="w"> </span><span class="nx">String</span><span class="w">
</span><span class="nx">Valid</span><span class="w"> </span><span class="nx">values</span><span class="w"> </span><span class="err">:</span><span class="w">
</span><span class="err">#></span><span class="w">
</span><span class="nx">ServicePrincipalSecret</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">""</span><span class="w">
</span><span class="err"><#</span><span class="w">
</span><span class="nx">The</span><span class="w"> </span><span class="nx">ID</span><span class="w"> </span><span class="nx">of</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">Azure</span><span class="w"> </span><span class="nx">AD</span><span class="w"> </span><span class="nx">tenant</span><span class="w"> </span><span class="nx">where</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">Terraform</span><span class="w"> </span><span class="nx">Service</span><span class="w"> </span><span class="nx">Principal</span><span class="w"> </span><span class="err">(</span><span class="nx">and</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">target</span><span class="w"> </span><span class="nx">subscription</span><span class="err">)</span><span class="w"> </span><span class="nx">live</span><span class="err">.</span><span class="w">
</span><span class="nx">Type</span><span class="w"> </span><span class="err">:</span><span class="w"> </span><span class="nx">String</span><span class="w">
</span><span class="nx">Valid</span><span class="w"> </span><span class="nx">values</span><span class="w"> </span><span class="err">:</span><span class="w">
</span><span class="err">#></span><span class="w">
</span><span class="nx">AzureTenantID</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">""</span><span class="w">
</span><span class="err"><#</span><span class="w">
</span><span class="nx">The</span><span class="w"> </span><span class="nx">name</span><span class="w"> </span><span class="nx">of</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">AKS</span><span class="w"> </span><span class="nx">cluster</span><span class="err">.</span><span class="w">
</span><span class="nx">The</span><span class="w"> </span><span class="nx">name</span><span class="w"> </span><span class="nx">of</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">resource</span><span class="w"> </span><span class="nx">group</span><span class="w"> </span><span class="nx">and</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">cluster</span><span class="w"> </span><span class="nx">DNS</span><span class="w"> </span><span class="nx">prefix</span><span class="w"> </span><span class="nx">are</span><span class="w"> </span><span class="nx">derived</span><span class="w"> </span><span class="nx">from</span><span class="w"> </span><span class="nx">this</span><span class="w"> </span><span class="nx">value</span><span class="err">.</span><span class="w">
</span><span class="nx">Type</span><span class="w"> </span><span class="err">:</span><span class="w"> </span><span class="nx">String</span><span class="w">
</span><span class="nx">Valid</span><span class="w"> </span><span class="nx">values</span><span class="w"> </span><span class="err">:</span><span class="w"> </span><span class="nx">The</span><span class="w"> </span><span class="nx">name</span><span class="w"> </span><span class="nx">can</span><span class="w"> </span><span class="nx">contain</span><span class="w"> </span><span class="nx">only</span><span class="w"> </span><span class="nx">letters</span><span class="p">,</span><span class="w"> </span><span class="nx">numbers</span><span class="p">,</span><span class="w"> </span><span class="nx">and</span><span class="w"> </span><span class="nx">hyphens</span><span class="err">.</span><span class="w"> </span><span class="nx">The</span><span class="w"> </span><span class="nx">name</span><span class="w"> </span><span class="nx">must</span><span class="w"> </span><span class="nx">start</span><span class="w"> </span><span class="nx">with</span><span class="w"> </span><span class="nx">a</span><span class="w"> </span><span class="nx">letter</span><span class="w"> </span><span class="nx">and</span><span class="w"> </span><span class="nx">must</span><span class="w"> </span><span class="nx">end</span><span class="w"> </span><span class="nx">with</span><span class="w"> </span><span class="nx">a</span><span class="w"> </span><span class="nx">letter</span><span class="w"> </span><span class="nx">or</span><span class="w"> </span><span class="nx">a</span><span class="w"> </span><span class="nx">number</span><span class="err">.</span><span class="w">
</span><span class="err">#></span><span class="w">
</span><span class="nx">ClusterName</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">""</span><span class="w">
</span><span class="err"><#</span><span class="w">
</span><span class="nx">The</span><span class="w"> </span><span class="nx">version</span><span class="w"> </span><span class="nx">of</span><span class="w"> </span><span class="nx">Kubernetes</span><span class="w"> </span><span class="nx">software</span><span class="w"> </span><span class="nx">running</span><span class="w"> </span><span class="nx">in</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">AKS</span><span class="w"> </span><span class="nx">Cluster</span><span class="err">.</span><span class="w">
</span><span class="nx">Type</span><span class="w"> </span><span class="err">:</span><span class="w"> </span><span class="nx">String</span><span class="w">
</span><span class="nx">Valid</span><span class="w"> </span><span class="nx">values</span><span class="w"> </span><span class="err">:</span><span class="w"> </span><span class="s2">"1.8.14"</span><span class="p">,</span><span class="w"> </span><span class="s2">"1.8.15"</span><span class="p">,</span><span class="w"> </span><span class="s2">"1.9.10"</span><span class="p">,</span><span class="w"> </span><span class="s2">"1.9.11"</span><span class="p">,</span><span class="w"> </span><span class="s2">"1.10.8"</span><span class="p">,</span><span class="w"> </span><span class="s2">"1.10.9"</span><span class="p">,</span><span class="w"> </span><span class="s2">"1.11.4"</span><span class="p">,</span><span class="w"> </span><span class="s2">"1.11.5"</span><span class="w">
</span><span class="err">#></span><span class="w">
</span><span class="nx">KubernetesVersion</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"1.11.5"</span><span class="w">
</span><span class="err"><#</span><span class="w">
</span><span class="nx">The</span><span class="w"> </span><span class="nx">number</span><span class="w"> </span><span class="nx">of</span><span class="w"> </span><span class="nx">worker</span><span class="w"> </span><span class="nx">nodes</span><span class="w"> </span><span class="nx">in</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">AKS</span><span class="w"> </span><span class="nx">cluster</span><span class="err">.</span><span class="w">
</span><span class="nx">Type</span><span class="w"> </span><span class="err">:</span><span class="w"> </span><span class="nx">Int32</span><span class="w">
</span><span class="nx">Valid</span><span class="w"> </span><span class="nx">values</span><span class="w"> </span><span class="err">:</span><span class="w"> </span><span class="nx">Between</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nx">and</span><span class="w"> </span><span class="nx">100</span><span class="err">.</span><span class="w">
</span><span class="err">#></span><span class="w">
</span><span class="nx">NodeCount</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nx">3</span><span class="w">
</span><span class="err"><#</span><span class="w">
</span><span class="nx">The</span><span class="w"> </span><span class="nx">VM</span><span class="w"> </span><span class="nx">size</span><span class="w"> </span><span class="nx">for</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">AKS</span><span class="w"> </span><span class="nx">cluster</span><span class="w"> </span><span class="nx">nodes</span><span class="err">.</span><span class="w">
</span><span class="nx">This</span><span class="w"> </span><span class="nx">is</span><span class="w"> </span><span class="nx">more</span><span class="w"> </span><span class="nx">descriptive</span><span class="w"> </span><span class="nx">version</span><span class="w"> </span><span class="nx">of</span><span class="w"> </span><span class="nx">Azure</span><span class="w"> </span><span class="nx">VM</span><span class="w"> </span><span class="nx">sizes</span><span class="p">,</span><span class="w"> </span><span class="nx">it</span><span class="w"> </span><span class="nx">follows</span><span class="w"> </span><span class="nx">a</span><span class="w"> </span><span class="nx">naming</span><span class="w"> </span><span class="nx">convention</span><span class="w"> </span><span class="nx">as</span><span class="w"> </span><span class="err">:</span><span class="w">
</span><span class="err">{</span><span class="nx">VM</span><span class="w"> </span><span class="nx">Family</span><span class="p">}</span><span class="nf">_</span><span class="p">{</span><span class="nf">Number</span><span class="w"> </span><span class="nx">of</span><span class="w"> </span><span class="nx">vCPUs</span><span class="p">}</span><span class="nf">_</span><span class="p">{</span><span class="nf">Number</span><span class="w"> </span><span class="nx">of</span><span class="w"> </span><span class="nx">GB</span><span class="w"> </span><span class="nx">of</span><span class="w"> </span><span class="nx">RAM</span><span class="p">}</span><span class="w">
</span><span class="kr">Type</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nf">String</span><span class="w">
</span><span class="nf">Valid</span><span class="w"> </span><span class="nx">values</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="s2">"B_2vCPU_8GB"</span><span class="p">,</span><span class="w"> </span><span class="s2">"B_4vCPU_16GB"</span><span class="p">,</span><span class="w"> </span><span class="s2">"D_2vCPU_8GB"</span><span class="p">,</span><span class="w"> </span><span class="s2">"D_4vCPU_16GB"</span><span class="p">,</span><span class="w"> </span><span class="s2">"D_8vCPU_32GB"</span><span class="p">,</span><span class="w"> </span><span class="s2">"E_2vCPU_16GB"</span><span class="p">,</span><span class="w"> </span><span class="s2">"E_4vCPU_32GB"</span><span class="p">,</span><span class="w"> </span><span class="s2">"F_2vCPU_4GB"</span><span class="p">,</span><span class="w"> </span><span class="s2">"F_4vCPU_8GB"</span><span class="p">,</span><span class="w"> </span><span class="s2">"DS_2vCPU_7GB"</span><span class="p">,</span><span class="w"> </span><span class="s2">"DS_4vCPU_14GB"</span><span class="w">
</span><span class="c">#></span><span class="w">
</span><span class="nf">NodeSize</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"D_2vCPU_8GB"</span><span class="w">
</span><span class="cm"><#
The OS disk size (GB) for the cluster nodes.
If set to 0, the default osDisk size for the specified vmSize is applied.
Type : Int32
Valid values : Between 0 and 1024.
#></span><span class="w">
</span><span class="nf">OSDiskSizeGB</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">30</span><span class="w">
</span><span class="o">...</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p class="notice--info">(Output cut for brevity)</p>
<p>As we can see above, values are already set when there is a default value, otherwise, it’s an empty string.<br />
We can populate and change the values to our needs, and when the file is ready, we can feed it to <code class="language-plaintext highlighter-rouge">Invoke-PSAksDeployment</code>, like so :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">PS</span><span class="w"> </span><span class="nx">C:\</span><span class="err">></span><span class="w"> </span><span class="nx">Invoke-PSAksDeployment</span><span class="w"> </span><span class="nt">-ConfigPath</span><span class="w"> </span><span class="s1">'.\my-k8s-prod.psd1'</span><span class="w">
</span></code></pre></div></div>
<p>While the deployment is in progress, there is quite a lot of logging written to the console, some of which will look very familiar to those who use <code class="language-plaintext highlighter-rouge">Terraform</code>.</p>
<p>The overall deployment duration depends on many variables, some of which pertain to the Azure infrastructure/platform.<br />
That being said, in my experience, it takes between 20 and 40 minutes.</p>
<p>When it completes, we can take a look at the deployed resources in the Azure portal, but what we can see in the resource group (<code class="language-plaintext highlighter-rouge">my-k8s-prod-rg</code> in this case) is somewhat deceptive :</p>
<p><img src="https://mathieubuisson.github.io/images/2019-02-04-deploying-aks-cluster-psaksdeployment-resourcegroup.png" alt="my-k8s-prod-rg" /></p>
<p>Azure AKS creates another resource group (<code class="language-plaintext highlighter-rouge">MC_my-k8s-prod-rg_my-k8s-prod_northeurope</code> in this case), which contains the <strong>infrastructure resources</strong> associated with the cluster : Kubernetes node VMs, virtual network, load balancer, storage, etc :</p>
<p><img src="https://mathieubuisson.github.io/images/2019-02-04-deploying-aks-cluster-psaksdeployment-infra-resourcegroup.png" alt="Infra resource group" /></p>
<p>Also, the usual <strong>Kubernetes</strong> management tools are ready to work with our new cluster.<br />
For example, we can use our trusty <code class="language-plaintext highlighter-rouge">kubectl</code> to list the deployments in the “management” namespace :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">PS</span><span class="w"> </span><span class="nx">C:\</span><span class="err">></span><span class="w"> </span><span class="nx">kubectl</span><span class="w"> </span><span class="nx">get</span><span class="w"> </span><span class="nx">deployment</span><span class="w"> </span><span class="nt">-n</span><span class="w"> </span><span class="nx">management</span><span class="w">
</span><span class="nf">NAME</span><span class="w"> </span><span class="nx">DESIRED</span><span class="w"> </span><span class="nx">CURRENT</span><span class="w"> </span><span class="nx">UP-TO-DATE</span><span class="w"> </span><span class="nx">AVAILABLE</span><span class="w"> </span><span class="nx">AGE</span><span class="w">
</span><span class="nf">cert-manager</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nx">33m</span><span class="w">
</span><span class="nf">nginx-ingress-controller</span><span class="w"> </span><span class="nx">2</span><span class="w"> </span><span class="nx">2</span><span class="w"> </span><span class="nx">2</span><span class="w"> </span><span class="nx">2</span><span class="w"> </span><span class="nx">40m</span><span class="w">
</span><span class="nf">nginx-ingress-default-backend</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nx">40m</span><span class="w">
</span><span class="nf">secret-propagator</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nx">32m</span><span class="w">
</span></code></pre></div></div>
<p>Or use a <code class="language-plaintext highlighter-rouge">helm</code> command to list all Helm releases :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">PS</span><span class="w"> </span><span class="nx">C:\</span><span class="err">></span><span class="w"> </span><span class="nx">helm</span><span class="w"> </span><span class="nx">ls</span><span class="w">
</span><span class="nf">NAME</span><span class="w"> </span><span class="nx">REVISION</span><span class="w"> </span><span class="nx">UPDATED</span><span class="w"> </span><span class="nx">STATUS</span><span class="w"> </span><span class="nx">CHART</span><span class="w"> </span><span class="nx">APP</span><span class="w"> </span><span class="nx">VERSION</span><span class="w"> </span><span class="nx">NAMESPACE</span><span class="w">
</span><span class="nf">cert-manager</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nx">Mon</span><span class="w"> </span><span class="nx">Jan</span><span class="w"> </span><span class="nx">28</span><span class="w"> </span><span class="nx">10:35:08</span><span class="w"> </span><span class="nx">2019</span><span class="w"> </span><span class="nx">DEPLOYED</span><span class="w"> </span><span class="nx">cert-manager-v0.5.2</span><span class="w"> </span><span class="nx">v0.5.2</span><span class="w"> </span><span class="nx">management</span><span class="w">
</span><span class="nf">cluster-issuer</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nx">Mon</span><span class="w"> </span><span class="nx">Jan</span><span class="w"> </span><span class="nx">28</span><span class="w"> </span><span class="nx">10:35:18</span><span class="w"> </span><span class="nx">2019</span><span class="w"> </span><span class="nx">DEPLOYED</span><span class="w"> </span><span class="nx">cluster-issuer-0.1.0</span><span class="w"> </span><span class="nx">1.0</span><span class="w"> </span><span class="nx">default</span><span class="w">
</span><span class="nf">nginx-ingress</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nx">Mon</span><span class="w"> </span><span class="nx">Jan</span><span class="w"> </span><span class="nx">28</span><span class="w"> </span><span class="nx">10:28:41</span><span class="w"> </span><span class="nx">2019</span><span class="w"> </span><span class="nx">DEPLOYED</span><span class="w"> </span><span class="nx">nginx-ingress-1.1.5</span><span class="w"> </span><span class="nx">0.21.0</span><span class="w"> </span><span class="nx">management</span><span class="w">
</span><span class="nf">secret-propagator</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nx">Mon</span><span class="w"> </span><span class="nx">Jan</span><span class="w"> </span><span class="nx">28</span><span class="w"> </span><span class="nx">10:36:32</span><span class="w"> </span><span class="nx">2019</span><span class="w"> </span><span class="nx">DEPLOYED</span><span class="w"> </span><span class="nx">secret-propagator-0.1.0</span><span class="w"> </span><span class="nx">1.0</span><span class="w"> </span><span class="nx">default</span><span class="w">
</span></code></pre></div></div>
<h2 id="deleting-an-aks-cluster">Deleting an AKS Cluster</h2>
<p>An AKS cluster deployed with <code class="language-plaintext highlighter-rouge">Invoke-PSAksDeployment</code> may need to be later deprovisioned.</p>
<p>In this case, the cmdlet <code class="language-plaintext highlighter-rouge">Remove-PSAksDeployment</code> automates tearing down the Azure Kubernetes Service instance and all associated resources, to stop incurring any Azure charges.</p>
<div class="notice--warning">
<h4>Warning :</h4>
<p>This deletes <strong>all resources in both resource groups</strong> : the target resource group and the infrastructure resource group created by AKS.<br />
Keep this in mind, especially if any resource(s) were added outside of <code class="language-plaintext highlighter-rouge">PSAksDeployment</code>’s purview.</p>
</div>
<p>Here is an example usage :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">PS</span><span class="w"> </span><span class="nx">C:\</span><span class="err">></span><span class="w"> </span><span class="nv">$DestroyParams</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">@{</span><span class="w">
</span><span class="err">>></span><span class="w"> </span><span class="nx">ServicePrincipalID</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'29x1ecd3-190f-42c9-8660-088f69d121zn'</span><span class="w">
</span><span class="err">>></span><span class="w"> </span><span class="nx">ServicePrincipalSecret</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'tsWpRr6/YCxNyh8efMvjWbe5JoOiOw03xR1o9S5CLhZ='</span><span class="w">
</span><span class="err">>></span><span class="w"> </span><span class="nx">AzureTenantID</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'96v3b174-9c1p-4a5e-9177-18c3bccc87cb'</span><span class="w">
</span><span class="err">>></span><span class="w"> </span><span class="nx">Subscription</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'DevOps'</span><span class="w">
</span><span class="err">>></span><span class="w"> </span><span class="nx">ClusterName</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'my-k8s-prod'</span><span class="w">
</span><span class="err">>></span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="nf">PS</span><span class="w"> </span><span class="nx">C:\</span><span class="err">></span><span class="w"> </span><span class="nx">Remove-PSAksDeployment</span><span class="w"> </span><span class="err">@</span><span class="nx">DestroyParams</span><span class="w">
</span></code></pre></div></div>
<h2 id="zooming-in-on-a-few-terraform-resources">Zooming In On a Few Terraform Resources</h2>
<p>Now that we know how to use <code class="language-plaintext highlighter-rouge">PSAksDeployment</code>, let’s take a look at a few sections of <code class="language-plaintext highlighter-rouge">Terraform</code> configurations which are worthy of mention/explanation.</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">resource</span> <span class="dl">"</span><span class="s2">null_resource</span><span class="dl">"</span> <span class="dl">"</span><span class="s2">helm_init</span><span class="dl">"</span> <span class="p">{</span>
<span class="nx">provisioner</span> <span class="dl">"</span><span class="s2">local-exec</span><span class="dl">"</span> <span class="p">{</span>
<span class="nx">command</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">helm init --wait --replicas ${var.tiller_replica_count} --tiller-namespace kube-system --service-account=${kubernetes_service_account.tiller.metadata.0.name}</span><span class="dl">"</span>
<span class="p">}</span>
<span class="nx">depends_on</span> <span class="o">=</span> <span class="p">[</span><span class="dl">"</span><span class="s2">kubernetes_cluster_role_binding.tiller</span><span class="dl">"</span><span class="p">]</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This resource is just running a command on the local machine.<br />
The purpose of the <code class="language-plaintext highlighter-rouge">helm init</code> command is to install and configure <code class="language-plaintext highlighter-rouge">tiller</code> (the server-side component of <code class="language-plaintext highlighter-rouge">Helm</code>) into the Kubernetes cluster.</p>
<p>After this command completes, we are ready to use <code class="language-plaintext highlighter-rouge">Helm</code> against our Kubernetes cluster to deploy components and applications via <code class="language-plaintext highlighter-rouge">Helm</code> charts.<br />
<code class="language-plaintext highlighter-rouge">Helm</code> charts are essentially packages describing Kubernetes resources to deploy (pods, services, etc…) and how to deploy them. We actually use <code class="language-plaintext highlighter-rouge">Helm</code> charts in subsequent steps.</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">resource</span> <span class="dl">"</span><span class="s2">helm_release</span><span class="dl">"</span> <span class="dl">"</span><span class="s2">nginx_ingress</span><span class="dl">"</span> <span class="p">{</span>
<span class="nx">name</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">nginx-ingress</span><span class="dl">"</span>
<span class="nx">chart</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">stable/nginx-ingress</span><span class="dl">"</span>
<span class="nx">namespace</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${kubernetes_namespace.management.metadata.0.name}</span><span class="dl">"</span>
<span class="c1">// Giving Azure 10min to create a load-balancer and assign the Public IP to it</span>
<span class="nx">timeout</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">600</span><span class="dl">"</span>
<span class="nx">depends_on</span> <span class="o">=</span> <span class="p">[</span><span class="dl">"</span><span class="s2">null_resource.helm_init</span><span class="dl">"</span><span class="p">]</span>
<span class="nx">values</span> <span class="o">=</span> <span class="p">[</span><span class="o"><<</span><span class="nx">EOF</span>
<span class="nx">controller</span><span class="p">:</span>
<span class="nx">replicaCount</span><span class="p">:</span> <span class="nx">$</span><span class="p">{</span><span class="kd">var</span><span class="p">.</span><span class="nx">ingressctrl_replica_count</span><span class="p">}</span>
<span class="nl">service</span><span class="p">:</span>
<span class="nl">loadBalancerIP</span><span class="p">:</span> <span class="dl">"</span><span class="s2">${var.ingressctrl_ip_address}</span><span class="dl">"</span>
<span class="nx">EOF</span>
<span class="p">]</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">helm_release</code> Terraform resource allows to deploy <code class="language-plaintext highlighter-rouge">Helm</code> charts into Kubernetes.</p>
<p>Here, the <code class="language-plaintext highlighter-rouge">Helm</code> chart being deployed is “nginx-ingress”.<br />
The <strong>NGINX Ingress Controller</strong> is a popular solution to manage access to services running inside the cluster from the outside world.</p>
<p>The <code class="language-plaintext highlighter-rouge">loadBalancerIP</code> value is interesting :<br />
it tells Kubernetes to create a service and expose it externally via a load-balancer. Then, Kubernetes asks the underlying cloud provider (Azure, in this case) to provision a load-balancer and attach it to the specified IP address.</p>
<p>The value of this IP address comes from a Public IP Azure resource which is created at an earlier step.</p>
<p>Sometimes, the provisioning of the Azure load-balancer takes more than 5 minutes (the default <code class="language-plaintext highlighter-rouge">Helm</code> timeout), this is why we set the <code class="language-plaintext highlighter-rouge">timeout</code> value to <code class="language-plaintext highlighter-rouge">600</code> (10 minutes).</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">resource</span> <span class="dl">"</span><span class="s2">helm_release</span><span class="dl">"</span> <span class="dl">"</span><span class="s2">cert_manager</span><span class="dl">"</span> <span class="p">{</span>
<span class="nx">name</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">cert-manager</span><span class="dl">"</span>
<span class="nx">chart</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">stable/cert-manager</span><span class="dl">"</span>
<span class="c1">// Since v0.6.0, cert-manager Helm chart doesn't provide</span>
<span class="c1">// a good way of installing the cert-manager CRDs</span>
<span class="nx">version</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">v0.5.2</span><span class="dl">"</span>
<span class="nx">namespace</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${kubernetes_namespace.management.metadata.0.name}</span><span class="dl">"</span>
<span class="nx">timeout</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">540</span><span class="dl">"</span>
<span class="nx">depends_on</span> <span class="o">=</span> <span class="p">[</span><span class="dl">"</span><span class="s2">helm_release.nginx_ingress</span><span class="dl">"</span><span class="p">]</span>
<span class="nx">values</span> <span class="o">=</span> <span class="p">[</span><span class="o"><<</span><span class="nx">EOF</span>
<span class="nx">ingressShim</span><span class="p">:</span>
<span class="nx">defaultIssuerName</span><span class="p">:</span> <span class="nx">letsencrypt</span><span class="o">-</span><span class="nx">$</span><span class="p">{</span><span class="kd">var</span><span class="p">.</span><span class="nx">letsencrypt_environment</span><span class="p">}</span>
<span class="nl">defaultIssuerKind</span><span class="p">:</span> <span class="nx">ClusterIssuer</span>
<span class="nx">EOF</span>
<span class="p">]</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This time, we deploy the <strong>cert-manager</strong> <code class="language-plaintext highlighter-rouge">Helm</code> chart.<br />
<strong><a href="https://github.com/jetstack/cert-manager">Cert-manager</a></strong> is a very cool tool which automates the issuance and renewal of TLS certificates needed by HTTPS-based services. The resulting certificates are ultimately stored in Kubernetes as <strong>secret</strong> resources.</p>
<p><strong>cert-manager</strong> extends Kubernetes with custom resources, like : Certificate, Issuer, etc…<br />
These <strong>CustomResourceDefinition</strong> (CRDs) are not shipped with the <code class="language-plaintext highlighter-rouge">Helm</code> chart anymore, which means we need to apply a separate YAML manifest prior to using <strong>cert-manager</strong> <code class="language-plaintext highlighter-rouge">Helm</code> chart. Besides, the URL of this manifest varies based on the <strong>cert-manager</strong> version.</p>
<p>So as a (hopefully temporary) workaround, we pin the <strong>cert-manager</strong> <code class="language-plaintext highlighter-rouge">Helm</code> chart version to the latest one which ships with the CRDs.</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">resource</span> <span class="dl">"</span><span class="s2">helm_release</span><span class="dl">"</span> <span class="dl">"</span><span class="s2">cluster_issuer</span><span class="dl">"</span> <span class="p">{</span>
<span class="nx">name</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">cluster-issuer</span><span class="dl">"</span>
<span class="nx">chart</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">..</span><span class="se">\\</span><span class="s2">..</span><span class="se">\\</span><span class="s2">Assets</span><span class="se">\\</span><span class="s2">cluster-issuer</span><span class="dl">"</span>
<span class="nx">depends_on</span> <span class="o">=</span> <span class="p">[</span><span class="dl">"</span><span class="s2">helm_release.cert_manager</span><span class="dl">"</span><span class="p">]</span>
<span class="nx">values</span> <span class="o">=</span> <span class="p">[</span><span class="o"><<</span><span class="nx">EOF</span>
<span class="nx">email</span><span class="p">:</span> <span class="nx">$</span><span class="p">{</span><span class="kd">var</span><span class="p">.</span><span class="nx">letsencrypt_email_address</span><span class="p">}</span>
<span class="nl">environment</span><span class="p">:</span> <span class="nx">$</span><span class="p">{</span><span class="kd">var</span><span class="p">.</span><span class="nx">letsencrypt_environment</span><span class="p">}</span>
<span class="nx">EOF</span>
<span class="p">]</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Again, we deploy a <code class="language-plaintext highlighter-rouge">Helm</code> chart, but this time we are not pulling the chart from the stable repository but from a local directory.<br />
This is a custom chart to create a cluster-wide issuer resource for <strong>cert-manager</strong>.</p>
<p><strong>cert-manager</strong> can talk to different certificate authorities, but in this case, we configure it to talk to <strong>Let’s Encrypt</strong>. <strong><a href="https://letsencrypt.org/">Let’s Encrypt</a></strong> is easy to use, free, and suitable for production certificates.</p>
<p>The caveat to that is :<br />
<strong>Let’s Encrypt</strong> certificates are only valid for 90 days, but this is not a problem here, because <strong>cert-manager</strong> takes care of renewing them automatically.</p>
<p>That’s pretty much it for now.<br />
For more information about <code class="language-plaintext highlighter-rouge">PSAksDeployment</code> and a dive into the code, head over to <strong><a href="https://github.com/MathieuBuisson/PSAksDeployment">the project on GitHub</a></strong>.<br />
If you have any question, remark, issue, or feature request, feel free to open an issue.</p>TheShellNutIn this post, we introduce PSAksDeployment: a tool which deploys an AKS cluster to a "ready-to-use" state in a few PowerShell commands. We also take a peek into how it uses Terraform and Helm under the hood.Adding PowerShell code quality gates in VSTS using the PSCodeHealth extension2018-05-30T00:00:00+01:002018-05-30T00:00:00+01:00https://mathieubuisson.github.io/powershell-code-quality-gate-vsts<aside class="sidebar__right">
<nav class="toc">
<header><h4 class="nav__title"><i class="fa fa-file-text"></i> In This Article</h4></header>
<ul class="toc__menu" id="markdown-toc">
<li><a href="#introducing-the-pscodehealth-vsts-extension" id="markdown-toc-introducing-the-pscodehealth-vsts-extension">Introducing the PSCodeHealth VSTS Extension</a></li>
<li><a href="#installing-the-extension-in-vsts" id="markdown-toc-installing-the-extension-in-vsts">Installing the Extension in VSTS</a></li>
<li><a href="#using-the-task-in-a-build-definition" id="markdown-toc-using-the-task-in-a-build-definition">Using the Task in a Build Definition</a></li>
<li><a href="#enforcing-the-quality-gate" id="markdown-toc-enforcing-the-quality-gate">Enforcing the Quality Gate</a></li>
<li><a href="#conclusion" id="markdown-toc-conclusion">Conclusion</a></li>
</ul>
</nav>
</aside>
<h2 id="introducing-the-pscodehealth-vsts-extension">Introducing the PSCodeHealth VSTS Extension</h2>
<p>We have already taken <a href="https://mathieubuisson.github.io/powershell-code-quality-pscodehealth/">a detailed look at PSCodeHealth</a> and how we can really <a href="https://mathieubuisson.github.io/pscodehealth-release-pipeline/">unleash its usefulness in a release pipeline</a>.</p>
<p>So if you care about coding standards and/or quality of your PowerShell code and you use <a href="https://www.visualstudio.com/team-services/">Visual Studio Team Services</a> to build/release your PowerShell projects, you will probably be interested in the <code class="language-plaintext highlighter-rouge">PSCodeHealth</code> VSTS extension.</p>
<p>The extension provides a Build/Release task to gather <strong>PowerShell</strong> code quality metrics. This task also allows to define (and optionally enforce) quality gates based on these code metrics.</p>
<p>If you already use the <code class="language-plaintext highlighter-rouge">PSCodeHealth</code> PowerShell module, the VSTS extension doesn’t really add anything new in terms of functionality.<br />
What it does bring to the table is : <strong>ease of use</strong> and seamless <strong>integration with VSTS</strong>.</p>
<h2 id="installing-the-extension-in-vsts">Installing the Extension in VSTS</h2>
<ul>
<li>Browse to the <a href="https://marketplace.visualstudio.com/items?itemName=MathieuBuisson.MB-PSCodeHealth-task">extension page in the Visual Studio marketplace</a></li>
<li>Click on the “<strong>Get it free</strong>” button</li>
<li>Select your VSTS account</li>
<li>If you are an admin of the VSTS account :
<ul>
<li>Click “<strong>Install</strong>”</li>
<li>After a few seconds, you should see a message telling that you are all set</li>
</ul>
</li>
<li>If you are not an admin of the VSTS account :
<ul>
<li>Click “<strong>Request</strong>”</li>
<li>The extension will be added automatically as soon as the request is approved</li>
</ul>
</li>
</ul>
<p>Once added to your VSTS account, the <code class="language-plaintext highlighter-rouge">PSCodeHealth</code> task will be available for :</p>
<ul>
<li>Build definitions</li>
<li>Release definitions</li>
</ul>
<h2 id="using-the-task-in-a-build-definition">Using the Task in a Build Definition</h2>
<p>To illustrate the usage of the <code class="language-plaintext highlighter-rouge">PSCodeHealth</code> task, we’ll create a brand new build definition for the following GitHub repository : <a href="https://github.com/MathieuBuisson/PSGithubSearch">PSGithubSearch</a>.</p>
<h3 id="creating-a-new-build-definition">Creating a new build definition</h3>
<ul>
<li>In the “<strong>Builds</strong>” hub, click on “<strong>New</strong>”</li>
<li>In the “<strong>Select your repository</strong>” page, select :
<ul>
<li>“<strong>GitHub</strong>” if you have a <strong>GitHub</strong> service endpoint to access the repo</li>
<li>“<strong>VSTS Git</strong>” if you have imported the repo to the VSTS “<strong>Code</strong>” hub using the clone URL</li>
</ul>
</li>
<li>Set the default branch to “master”</li>
<li>In the “<strong>Choose a template</strong>” page, click on “<strong>Empty process</strong>”</li>
<li>In the “<strong>Process</strong>” tab, we’ll use <code class="language-plaintext highlighter-rouge">PSGithubSearch-CI</code> here</li>
<li>Choose an Agent queue :
<ul>
<li>For hosted agents, use “<strong>Hosted VS2017</strong>”, because it has PowerShell 5.1 and even <code class="language-plaintext highlighter-rouge">Pester</code> is pre-installed (albeit an old version)</li>
<li>For private agents, ensure they have PowerShell 5.x. If they don’t have <code class="language-plaintext highlighter-rouge">Pester</code>, you can either install it permanently or use a build task to always get the latest version</li>
</ul>
</li>
<li>In the “<strong>Phase 1</strong>” tab, you might want to change the phase display name to something meaningful</li>
</ul>
<h3 id="adding-the-pscodehealth-task-to-the-build-definition">Adding the PSCodeHealth task to the build definition</h3>
<ul>
<li>In the “<strong>Phase</strong>” tab, click on the “<strong>Plus</strong>” button</li>
<li>Type <code class="language-plaintext highlighter-rouge">PSCodeHealth</code></li>
<li>Click on “<strong>Add</strong>” to add the task</li>
<li>Click on the task name (“<strong>PowerShell Code Analysis</strong>” by default) to get to the task configuration</li>
</ul>
<h3 id="configuring-the-task">Configuring the task</h3>
<h4 id="specifying-the-paths-of-powershell-code-and-tests">Specifying the paths of PowerShell code and tests</h4>
<p>The field labeled “<strong>Path to Analyze</strong>” allows you to specify the location of the PowerShell files (.ps1, .psm1 and .psd1) to analyze. This is a path relative to the root of the repository.</p>
<p>The easy and recommended way of filling out this path is to click the ellipsis button on the right, to browse the files and folders in the repository.</p>
<p><img src="https://mathieubuisson.github.io/images/2018-05-30-powershell-code-quality-gate-vsts-code-tests-path.png" alt="Source code and tests paths" /></p>
<p>If “<strong>Path to Analyze</strong>” is a folder, which is the case here, the checkbox “<strong>Search PowerShell files in subdirectories</strong>” allows to analyze PowerShell code in subfolders, recursively.</p>
<p>The field labeled “<strong>Pester Tests Path</strong>” allows to specify the location of the tests, which are going to be executed by <code class="language-plaintext highlighter-rouge">Pester</code>. Again, I would recommended to click the ellipsis button to browse the files and folders in the repo.</p>
<p>In this case, we chose the <code class="language-plaintext highlighter-rouge">Tests/Unit</code> subfolder, which contains all the unit tests.</p>
<h4 id="generating-an-html-report">Generating an HTML report</h4>
<p>You may opt to generate an HTML report by checking the following checkbox :</p>
<p><img src="https://mathieubuisson.github.io/images/2018-05-30-powershell-code-quality-gate-vsts-html.png" alt="Generate an HTML Report" /></p>
<p>This enables the field labeled “<strong>HTML report path</strong>” to specify the path and file name of the report. Here, we save the file in the artifacts staging directory, in case we want to publish it as an artifact.</p>
<h4 id="selecting-specific-code-metrics">Selecting specific code metrics</h4>
<p>The quality gate compares actual code metrics of the analyzed PowerShell code with compliance rules for these metrics.<br />
If 1 or more evaluated code metric(s) doesn’t meet its compliance rule, the analyzed code doesn’t meet the quality gate.</p>
<p>By default, the task will evaluate <strong>all</strong> code metrics against their respective compliance rules.<br />
If you only care about specific metrics, you can specify which ones by checking the box labeled “<strong>Select metrics to evaluate</strong>”.</p>
<p>This enables additional checkboxes to choose which metrics to evaluate for compliance.</p>
<p><img src="https://mathieubuisson.github.io/images/2018-05-30-powershell-code-quality-gate-vsts-metrics.png" alt="Select Code Metrics" /></p>
<p>The label for each checkbox should hopefully be self-explanatory.<br />
For more information on the metrics, how they are measured, their meaning, their purpose, please refer to <a href="http://pscodehealth.readthedocs.io/en/latest/Metrics/#metrics-collected-for-the-overall-health-report">this documentation page</a>.</p>
<p>In this case, we unselected the total number of <code class="language-plaintext highlighter-rouge">PSScriptAnalyzer</code> findings, as we deemed the average more relevant than the total.<br />
Speaking of <code class="language-plaintext highlighter-rouge">PSScriptAnalyzer</code>, there is no need to install it since it is shipped with the extension.</p>
<h4 id="overriding-the-default-compliance-rules">Overriding the default compliance rules</h4>
<p><strong>PSCodeHealth</strong> comes with default compliance rules which determine, for their respective code metric, whether the analyzed code passes or fails.</p>
<p>These default rules can be overridden for some (or all) of the code metrics, so <strong>you can define the quality gate to suit your requirements</strong> or goals.</p>
<p>The field labeled “<strong>Override compliance rules with a custom settings file</strong>” allows to specify the JSON file where custom compliance rules are defined.<br />
Click the ellipsis button to look for the file in the repository.</p>
<p><img src="https://mathieubuisson.github.io/images/2018-05-30-powershell-code-quality-gate-vsts-custom-rules.png" alt="Custom Compliance Rules JSON File" /></p>
<p>Here, we specified a file named <code class="language-plaintext highlighter-rouge">ProjectRules.json</code> at the root of the repository. This file defines compliance rules for 2 metrics :</p>
<ul>
<li><strong>LinesOfCodeAverage</strong> : much higher <em>warning</em> and <em>fail</em> thresholds than the default, because the functions in this project are a bit monolithic and we are OK with that for now.</li>
<li><strong>TestCoverage</strong> : slightly more permissive than the default, because some of this code is difficult to unit test (but may very well be covered by integration tests).</li>
</ul>
<p>Here is what this looks like :</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
</span><span class="nl">"PerFunctionMetrics"</span><span class="p">:</span><span class="w"> </span><span class="p">[],</span><span class="w">
</span><span class="nl">"OverallMetrics"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"LinesOfCodeAverage"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"WarningThreshold"</span><span class="p">:</span><span class="w"> </span><span class="mi">115</span><span class="p">,</span><span class="w">
</span><span class="nl">"FailThreshold"</span><span class="p">:</span><span class="w"> </span><span class="mi">165</span><span class="p">,</span><span class="w">
</span><span class="nl">"HigherIsBetter"</span><span class="p">:</span><span class="w"> </span><span class="kc">false</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"TestCoverage"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"WarningThreshold"</span><span class="p">:</span><span class="w"> </span><span class="mi">75</span><span class="p">,</span><span class="w">
</span><span class="nl">"FailThreshold"</span><span class="p">:</span><span class="w"> </span><span class="mi">65</span><span class="p">,</span><span class="w">
</span><span class="nl">"HigherIsBetter"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>For more information on compliance rules, their <em>warning</em> and <em>fail</em> thresholds and how to customize them, please refer to <a href="http://pscodehealth.readthedocs.io/en/latest/HowDoI/CustomizeComplianceRules/">this documentation page</a>.</p>
<h3 id="running-the-build">Running the build</h3>
<p>Now, we are done with the task configuration. We can save the build definition and a queue a new build.</p>
<p>When the task is running, the <code class="language-plaintext highlighter-rouge">PSCodeHealth</code> module analyzes the PowerShell files found in the specified path, generates code metrics, runs any tests file(s) using <code class="language-plaintext highlighter-rouge">Pester</code> and creates a <code class="language-plaintext highlighter-rouge">[PSCodeHealth.Overall.HealthReport]</code> object.</p>
<p>Then, the metrics values in the report are evaluated for compliance, based on their respective rules (rules from the JSON file override the defaults).</p>
<p>Here is the build console output we got :</p>
<p><img src="https://mathieubuisson.github.io/images/2018-05-30-powershell-code-quality-gate-vsts-warning.png" alt="Build console output warning" /></p>
<p>This output provides :</p>
<ul>
<li>Overall view of the <code class="language-plaintext highlighter-rouge">PSCodeHealth</code> report</li>
<li>Report broken down by function</li>
<li>Some variables values based on the task configuration (for instance, we get the user-defined compliance rules from the specified JSON file)</li>
<li>Compliance result for each evaluated metric</li>
<li>Task result</li>
</ul>
<p>As seen in the screenshot, the analyzed code didn’t meet our quality gate because 1 metric failed compliance.<br />
Despite this compliance failure, the build was successful. This is because the default compliance failure action is to just <strong>log a warning</strong> for each rule which failed compliance.</p>
<p>Notice that the failing metric was <strong>TestsPassRate</strong>, which means that some unit tests have failed. Failing tests are a serious matter so we certainly want to fail the build whenever this occurs.<br />
In other words, we want to <strong>enforce our quality gate</strong>.</p>
<h2 id="enforcing-the-quality-gate">Enforcing the Quality Gate</h2>
<p>To choose whether (and how) to enforce our lovingly crafted quality gate, we use the field “<strong>Action to take in case of compliance failure(s)</strong>” :</p>
<p><img src="https://mathieubuisson.github.io/images/2018-05-30-powershell-code-quality-gate-vsts-failure-action.png" alt="Compliance Failure Actions" /></p>
<p>Here is an explanation of each possible choice in this dropdown :</p>
<ul>
<li><strong>Silently continue</strong> : no action is taken</li>
<li><strong>Log warning</strong> : a warning is logged for each metric failing compliance</li>
<li><strong>Log error and set task result to ‘Failed’</strong> :
<ul>
<li>a warning is logged for each failing metric</li>
<li>an error is logged with the names of metrics failing compliance</li>
<li>the task result is set to ‘Failed’, which also sets the overall build result to ‘Failed’</li>
</ul>
</li>
</ul>
<p>So let’s choose the last one to ensure the build fails whenever the code does not meet the quality gate.<br />
Then we save the build definition and queue a new build.</p>
<p>Here is the output of this new build :</p>
<p><img src="https://mathieubuisson.github.io/images/2018-05-30-powershell-code-quality-gate-vsts-fail.png" alt="Build failure console output" /></p>
<p>As expected, in addition to the warning, we get an error telling us which metric(s) failed compliance.<br />
Also, the task result is set to “<strong>Failed</strong>”, which in turn, sets the overall build status to “<strong>Failed</strong>”.</p>
<h2 id="conclusion">Conclusion</h2>
<p>I believe that making the build fail is crucial for a quality gate, because a build failure should trigger <strong>2 fundamental practices of Continuous Integration</strong> :</p>
<ul>
<li>The CI system notifies the dev who made the code change and/or the team</li>
<li>Addressing the issue(s) surfaced by a failing build is the priority #1</li>
</ul>
<p>Keep in mind that there is no shame in breaking the build. A build failure is not a failure, <strong>it is feedback</strong>. Specific, timely, actionable feedback, to facilitate the development process.</p>
<p>I do hope that the <code class="language-plaintext highlighter-rouge">PSCodeHealth</code> VSTS extension can bring this kind of benefits, and help you build awesome PowerShell projects.</p>
<p>Naturally, this extension is open source and available <a href="https://github.com/MathieuBuisson/vsts-PSCodeHealth">on GitHub</a>.<br />
In addition, it may serve as an interesting example, in case you are curious about how to author a VSTS extension using <a href="https://github.com/Microsoft/vsts-task-lib/blob/master/powershell/Docs/README.md">the PowerShell SDK</a>.</p>TheShellNutIn this post, we introduce the PSCodeHealth extension for Visual Studio Team Services. We look at how to use it to assess the quality of PowerShell code and to define and enforce quality gates in VSTS build definitions.Triggering the removal of machines from Octopus Deploy on Azure VM deletion2018-03-12T00:00:00+00:002018-03-12T00:00:00+00:00https://mathieubuisson.github.io/trigger-octopus-removal-azure<aside class="sidebar__right">
<nav class="toc">
<header><h4 class="nav__title"><i class="fa fa-file-text"></i> In This Article</h4></header>
<ul class="toc__menu" id="markdown-toc">
<li><a href="#the-use-case" id="markdown-toc-the-use-case">The Use Case</a></li>
<li><a href="#runbook-to-remove-a-machine-from-an-octopus-server" id="markdown-toc-runbook-to-remove-a-machine-from-an-octopus-server">Runbook To Remove a Machine From an Octopus Server</a></li>
<li><a href="#runbook-to-be-triggered-on-azure-resource-deletion" id="markdown-toc-runbook-to-be-triggered-on-azure-resource-deletion">Runbook To Be Triggered on Azure Resource Deletion</a></li>
<li><a href="#adding-an-event-subscription" id="markdown-toc-adding-an-event-subscription">Adding an Event Subscription</a></li>
<li><a href="#testing-the-trigger-and-runbooks" id="markdown-toc-testing-the-trigger-and-runbooks">Testing The Trigger And Runbooks</a></li>
</ul>
</nav>
</aside>
<h2 id="the-use-case">The Use Case</h2>
<p>Let’s say we use <a href="https://octopus.com/">Octopus Deploy</a> to standardize our software deployments across many environments.<br />
These environments, especially the ones used for <strong>development</strong>, <strong>CI/CD</strong>, or <strong>QA</strong> purposes greatly benefit from being managed as transient <em>cattle</em> (as opposed to <em>pets</em>).</p>
<p>In a <em>cattle</em> mindset, we spin up environments only when they are needed and tear them down as soon as they are not needed.<br />
Coupled with automation, this provides <strong>speed</strong>, <strong>consistency</strong>, <strong>self-service</strong> and <strong>cost control</strong>. Even more so if we leverage cloud resources.</p>
<p>When spinning up environments in Azure, the provisioning of Octopus deployment targets (including registration of Octopus tentacles with the server) can be fully automated with ARM templates and the <a href="https://octopus.com/docs/infrastructure/windows-targets/azure-virtual-machines/via-an-arm-template">Tentacle Azure VM extension</a>.</p>
<p>This is nice, but the journey from <em>pet</em> environments to <em>cattle</em> requires the ability to kill our cattle in a quick and painless way. The deletion of resources from Azure is easy to automate but :</p>
<blockquote>
<p>What about the removal of Octopus deployment targets from the server ?</p>
</blockquote>
<p>The Octopus Tentacle Azure VM extension can register the tentacle with the Octopus server, but <strong>not unregister it</strong>. There is no off-the-shelf solution to automatically trigger the removal of an Octopus machine (i.e. deployment target) when the VM is deleted from Azure.</p>
<p>So in this article, we’ll look at how to do that, by leveraging 2 powerful Azure services :</p>
<ul>
<li><strong>Azure Automation</strong> : SASS offering to run and manage scripts, workflows and configurations</li>
<li><strong>Azure Event Grid</strong> : enables apps, Azure services, and 3rd-party services to emit, route and react to events</li>
</ul>
<h2 id="runbook-to-remove-a-machine-from-an-octopus-server">Runbook To Remove a Machine From an Octopus Server</h2>
<p>First, we need an Azure Automation account.<br />
Creating an Automation account is straightforward, as explained in <a href="https://docs.microsoft.com/en-us/azure/automation/automation-quickstart-create-account">this documentation page</a>.</p>
<h3 id="importing-the-octoposh-powershell-module-into-our-automation-account">Importing the Octoposh PowerShell module Into our Automation Account</h3>
<p>To send commands to our Octopus server, we could write our own calls to the <a href="https://octopus.com/docs/api-and-integration/api">Octopus REST API</a>, but why reinvent the wheel when we can use the excellent <a href="https://github.com/Dalmirog/OctoPosh">Octoposh</a> module right from Azure Automation ?</p>
<p>To import the module in our Automation account, select <strong>Modules</strong> and click on <strong>Browse Gallery</strong>. This provides access all the modules from the PowerShell Gallery.</p>
<p>Search for <code class="language-plaintext highlighter-rouge">Octoposh</code>, select the search result and then, click on <strong>Import</strong> and <strong>OK</strong>.</p>
<h3 id="adding-variables-to-connect-to-our-octopus-server">Adding Variables to Connect to our Octopus Server</h3>
<p>Our runbook will need to connect to the Octopus server. This requires 2 pieces of information :</p>
<ul>
<li>The server URL</li>
<li>An Octopus API key (for authentication/authorization)</li>
</ul>
<p>We’ll store these 2 values as variables in our Automation account so that the runbook can easily and securely access them.</p>
<p>Select <strong>Variables</strong> and click on <strong>Add a variable</strong>.</p>
<p>Create the following variables :</p>
<ul>
<li>The server URL :
<ul>
<li><strong>Name</strong> : OctopusURL</li>
<li><strong>Type</strong> : String</li>
<li><strong>Value</strong> : the URL of our Octopus Deploy server</li>
<li><strong>Encrypted</strong> : No</li>
</ul>
</li>
<li>Octopus API key :
<ul>
<li><strong>Name</strong> : OctopusAPIKey</li>
<li><strong>Type</strong> : String</li>
<li><strong>Value</strong> : API key from an account which has <strong>MachineDelete</strong> permission (preferably a service account)</li>
<li><strong>Encrypted</strong> : Yes</li>
</ul>
</li>
</ul>
<p>We are now ready to create the runbook used to remove machines from Octopus.</p>
<h3 id="creating-the-runbook-to-remove-a-machine-from-octopus">Creating The Runbook To Remove a Machine From Octopus</h3>
<p>Within our Automation account, select <strong>Runbooks</strong> and click on <strong>Add a Runbook</strong>.<br />
In this case, we name the runbook <code class="language-plaintext highlighter-rouge">Remove-MachineFromOctopusServer</code> and the runbook type is <strong>PowerShell</strong>.</p>
<p>As soon as the runbook is created, the Azure portal should switch to the runbook editor.</p>
<p>Our runbook will contain the following code :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kr">Param</span><span class="p">(</span><span class="w">
</span><span class="p">[</span><span class="kt">Parameter</span><span class="p">(</span><span class="kt">Mandatory</span><span class="p">)]</span><span class="w">
</span><span class="p">[</span><span class="kt">string</span><span class="p">]</span><span class="nv">$MachineName</span><span class="p">,</span><span class="w">
</span><span class="p">[</span><span class="kt">Parameter</span><span class="p">(</span><span class="kt">Mandatory</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$False</span><span class="p">)]</span><span class="w">
</span><span class="p">[</span><span class="kt">string</span><span class="p">]</span><span class="nv">$OctopusURL</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="nf">Get-AutomationVariable</span><span class="w"> </span><span class="nt">-Name</span><span class="w"> </span><span class="s1">'OctopusURL'</span><span class="p">),</span><span class="w">
</span><span class="p">[</span><span class="kt">Parameter</span><span class="p">(</span><span class="kt">Mandatory</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$False</span><span class="p">)]</span><span class="w">
</span><span class="p">[</span><span class="kt">string</span><span class="p">]</span><span class="nv">$OctopusAPIKey</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="nf">Get-AutomationVariable</span><span class="w"> </span><span class="nt">-Name</span><span class="w"> </span><span class="s1">'OctopusAPIKey'</span><span class="p">)</span><span class="w">
</span><span class="p">)</span><span class="w">
</span><span class="s2">"Machine Name : </span><span class="nv">$MachineName</span><span class="s2">"</span><span class="w">
</span><span class="s2">"Octopus Server URL : </span><span class="nv">$OctopusURL</span><span class="s2">"</span><span class="w">
</span><span class="nf">Import-Module</span><span class="w"> </span><span class="nt">-Name</span><span class="w"> </span><span class="s1">'OctoPosh'</span><span class="w"> </span><span class="nt">-Force</span><span class="w">
</span><span class="nf">Set-OctopusConnectionInfo</span><span class="w"> </span><span class="nt">-URL</span><span class="w"> </span><span class="nv">$OctopusURL</span><span class="w"> </span><span class="nt">-APIKey</span><span class="w"> </span><span class="nv">$OctopusAPIKey</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Out-Null</span><span class="w">
</span><span class="nv">$OctopusMachine</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">Get-OctopusMachine</span><span class="w"> </span><span class="nt">-MachineName</span><span class="w"> </span><span class="nv">$MachineName</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$OctopusMachine</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$OctopusMachine</span><span class="o">.</span><span class="nf">Resource</span><span class="w">
</span><span class="nv">$Result</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">Remove-OctopusResource</span><span class="w"> </span><span class="nt">-Resource</span><span class="w"> </span><span class="nv">$OctopusMachine</span><span class="o">.</span><span class="nf">Resource</span><span class="w">
</span><span class="nx">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$Result</span><span class="w"> </span><span class="o">-eq</span><span class="w"> </span><span class="nv">$True</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="s1">'The machine has been successfully removed from Octopus.'</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">Else</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$ResultString</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$Result</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Out-String</span><span class="w">
</span><span class="nf">Write-Error</span><span class="w"> </span><span class="nv">$ResultString</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>The runbook’s parameters deserve some explanation :</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">$MachineName</code> : Value passed from another runbook (more on that later)</li>
<li><code class="language-plaintext highlighter-rouge">$OctopusURL</code> : Value read from the <code class="language-plaintext highlighter-rouge">OctopusURL</code> Automation variable we created earlier</li>
<li><code class="language-plaintext highlighter-rouge">$OctopusAPIKey</code> : Value read from the <code class="language-plaintext highlighter-rouge">OctopusAPIKey</code> Automation variable</li>
</ul>
<p>Because we imported the <code class="language-plaintext highlighter-rouge">Octoposh</code> module into our Automation account, its cmdlets are readily available when we write runbooks. Here, the module is imported explicitly, but this is only a preference of mine, we could just use PowerShell module auto-loading.</p>
<p><code class="language-plaintext highlighter-rouge">Set-OctopusConnectionInfo</code> is a cmdlet which comes from <code class="language-plaintext highlighter-rouge">Octoposh</code>.<br />
The command is piped to <code class="language-plaintext highlighter-rouge">Out-Null</code> to avoid, <em>ahem</em>, having the API key exposed in plain text in the output.</p>
<p><code class="language-plaintext highlighter-rouge">Get-OctopusMachine</code> is used to query the Octopus server for the machine (by name).<br />
If the machine is present in Octopus, we use <code class="language-plaintext highlighter-rouge">Remove-OctopusResource</code> (another <code class="language-plaintext highlighter-rouge">Octoposh</code> cmdlet) to remove it.</p>
<h2 id="runbook-to-be-triggered-on-azure-resource-deletion">Runbook To Be Triggered on Azure Resource Deletion</h2>
<h3 id="creating-the-listener-runbook">Creating The “Listener” Runbook</h3>
<p>Now, we create a second runbook, the purpose of which is to listen to Azure resource deletion events.</p>
<p>We name this runbook <code class="language-plaintext highlighter-rouge">Watch-VMDeletion</code> and the runbook type is <strong>PowerShell</strong>.</p>
<p>This runbook will contain the following code :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kr">Param</span><span class="p">(</span><span class="w">
</span><span class="p">[</span><span class="kt">Object</span><span class="p">]</span><span class="nv">$WebhookData</span><span class="w">
</span><span class="p">)</span><span class="w">
</span><span class="nv">$RequestBody</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$WebhookData</span><span class="o">.</span><span class="nf">RequestBody</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">ConvertFrom-Json</span><span class="w">
</span><span class="nv">$Data</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$RequestBody</span><span class="o">.</span><span class="nf">data</span><span class="w">
</span><span class="c"># No point in proceeding if the operation is not a VM deletion</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$Data</span><span class="o">.</span><span class="nf">operationName</span><span class="w"> </span><span class="o">-ne</span><span class="w"> </span><span class="s1">'Microsoft.Compute/virtualMachines/delete'</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="kr">return</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nv">$PropertiesToLog</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">@(</span><span class="s1">'correlationId'</span><span class="p">,</span><span class="s1">'resourceProvider'</span><span class="p">,</span><span class="s1">'resourceUri'</span><span class="p">,</span><span class="s1">'operationName'</span><span class="p">,</span><span class="s1">'status'</span><span class="p">,</span><span class="s1">'subscriptionId'</span><span class="p">)</span><span class="w">
</span><span class="nv">$WebhookDataToLog</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$Data</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Select-Object</span><span class="w"> </span><span class="nt">-Property</span><span class="w"> </span><span class="nv">$PropertiesToLog</span><span class="w">
</span><span class="nv">$WebhookDataToLog</span><span class="w">
</span><span class="nv">$Resources</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$Data</span><span class="o">.</span><span class="nf">resourceUri</span><span class="o">.</span><span class="nf">Split</span><span class="p">(</span><span class="s1">'/'</span><span class="p">)</span><span class="w">
</span><span class="nv">$VMName</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$Resources</span><span class="p">[</span><span class="mi">8</span><span class="p">]</span><span class="o">.</span><span class="nf">Trim</span><span class="p">()</span><span class="w">
</span><span class="s2">"VMName from webhook data : </span><span class="nv">$VMName</span><span class="s2">"</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$Data</span><span class="o">.</span><span class="nf">status</span><span class="w"> </span><span class="o">-eq</span><span class="w"> </span><span class="s1">'Succeeded'</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="s1">'Invoking runbook : Remove-MachineFromOctopusServer'</span><span class="w">
</span><span class="o">.</span><span class="nf">\Remove-MachineFromOctopusServer.ps1</span><span class="w"> </span><span class="nt">-MachineName</span><span class="w"> </span><span class="nv">$VMName</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">Else</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="s1">'Deletion event did not succeed, skipping the removal from Octopus server'</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>The value of the <code class="language-plaintext highlighter-rouge">$WebhookData</code> parameter is automagically populated by the webhook triggering it.<br />
In turn, the webhook data is passed from the event which has triggered the webhook. This gives us access to rich information about the event and the deleted resource.</p>
<p>This runbook will be triggered every time an Azure resource is deleted, regardless of the resource type. This is because the event subscription filters are a bit limited.</p>
<p class="notice--info">I opened <a href="https://feedback.azure.com/forums/909934-azure-event-grid/suggestions/33594466-allow-wildcards-in-prefix-subjectbeginswith-an">this suggestion</a> on the Azure Event Grid Uservoice about this limitation.</p>
<p>So, we perform the filtering within this runbook :</p>
<ul>
<li>We check the value of the <code class="language-plaintext highlighter-rouge">operationName</code> property from the webhook data</li>
<li>If the operation was not on a <code class="language-plaintext highlighter-rouge">virtualMachines</code> resource, we are done</li>
<li>If it was, we proceed</li>
</ul>
<p>Then, we do a bit of string manipulation on a the <code class="language-plaintext highlighter-rouge">resourceUri</code> value to extract the name of the VM.</p>
<p>Getting the name of VM is crucial because we have to pass it to the <code class="language-plaintext highlighter-rouge">Remove-MachineFromOctopusServer</code> runbook.</p>
<p>As seen in the above code, calling a PowerShell runbook from another PowerShell runbook is easy. As long as the called runbook lives in the same Automation account, it can be invoked just like a script which would be in the current directory.</p>
<p>We pass the VM name extracted from the webhook data to <code class="language-plaintext highlighter-rouge">Remove-MachineFromOctopusServer</code> via its <code class="language-plaintext highlighter-rouge">$MachineName</code> parameter. Remember, it is a mandatory parameter.<br />
This other runbook takes care of the rest : the Octopus-related work.</p>
<h3 id="adding-a-webhook-to-the-listener-runbook">Adding a Webhook To The “Listener” Runbook</h3>
<p>The purpose of the <code class="language-plaintext highlighter-rouge">Watch-VMDeletion</code> runbook is to <strong>react to events</strong>. To enable this, the runbook will be triggered via webhook.</p>
<p>So let’s add a webhook to this runbook :</p>
<ul>
<li>Select the <code class="language-plaintext highlighter-rouge">Watch-VMDeletion</code> runbook</li>
<li>Click on <strong>Webhook</strong></li>
<li>Click on <strong>Create new webhook</strong></li>
<li>Name it <strong>WatchVMDeletion</strong> (or whatever makes sense to you)</li>
<li>Set a sensible expiry date (the maximum lifetime is 10 years)</li>
</ul>
<p><img src="https://mathieubuisson.github.io/images/2018-03-12-trigger-octopus-removal-azure-create-webhook.png" alt="Create new webhook" /></p>
<ul>
<li>The webhook URL is needed for later, so make sure to copy it somewhere safe (unless you can remember a 46-character token)</li>
<li>Click on <strong>Configure parameters and run settings</strong></li>
<li>Leave the default webhook data blank, just click on <strong>OK</strong></li>
<li>Click on <strong>Create</strong></li>
</ul>
<p>With this webhook, we now have an automated way of triggering our runbook, but at this point, the webhook doesn’t know anything about resource deletion events … yet.</p>
<p>To remediate this, we are going to add an event subscription to our Automation account, and configure it so that it routes events to the webhook we just created.</p>
<h2 id="adding-an-event-subscription">Adding an Event Subscription</h2>
<p>The webhook is just a relatively dumb event handler, the main glue between Azure Automation and Azure Event Grid is an <strong>event subscription</strong>.</p>
<p>So we need to add an event subscription to our Automation account :</p>
<ul>
<li>In our Automation account blade, select <strong>Event Grid</strong></li>
<li>Click on the <strong>+ Event Subscription</strong> button</li>
<li>Select the subscription where the cattle VMs will live (and die)</li>
<li>Give the event subscription a meaningful name, like <strong>OctoVMDeletion</strong></li>
<li>For topic type, select <strong>Azure Subscriptions</strong> (if the VMs will only ever be in 1 specific resource group, we could select <strong>Resource Groups</strong> instead)</li>
<li>Clear the <strong>Subscribe to all event types</strong> checkbox</li>
<li>Select only the <strong>Resource Delete Success</strong> event type, since we are only interested in deletions</li>
<li>For Subscriber type, choose <strong>Web Hook</strong></li>
<li>In Subscriber Endpoint, paste the webhook URL copied during the webhook creation</li>
</ul>
<p>It should look like this :</p>
<p><img src="https://mathieubuisson.github.io/images/2018-03-12-trigger-octopus-removal-azure-create-event.png" alt="Create an event subscription" /></p>
<ul>
<li>Click <strong>Create</strong></li>
</ul>
<div class="notice--info">
<h4>Update :</h4>
<p>Last time I checked, the <strong>Azure Subscriptions</strong> topic type was not visible in the Azure Portal anymore.<br />
Fortunately, we can still use PowerShell to create this event subscription.<br />
Here is how :</p>
</div>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">Select-AzureRmSubscription</span><span class="w"> </span><span class="nt">-SubscriptionId</span><span class="w"> </span><span class="nv">$TargetSubscriptionId</span><span class="w">
</span><span class="bp">$Event</span><span class="nf">GridSubParams</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">@{</span><span class="w">
</span><span class="nx">EventSubscriptionName</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'OctoVMDeletion'</span><span class="w">
</span><span class="nx">EndpointType</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'webhook'</span><span class="w">
</span><span class="nx">Endpoint</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$WebhookUrlFromWebhookCreationStep</span><span class="w">
</span><span class="nx">IncludedEventType</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'Microsoft.Resources.ResourceDeleteSuccess'</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nf">New-AzureRmEventGridSubscription</span><span class="w"> </span><span class="err">@</span><span class="nx">EventGridSubParams</span><span class="w">
</span></code></pre></div></div>
<p>Now, all the pieces are in place, we can delete deployment targets from Azure and watch them magically disappear from our Octopus Deploy server.</p>
<h2 id="testing-the-trigger-and-runbooks">Testing The Trigger And Runbooks</h2>
<p>We have a VM living in the Azure subscription specified in the Event Grid subscription. It is named <strong>CattleVM01</strong>, which is a pretty name for a <em>cattle</em> (not for a <em>pet</em>).</p>
<p>This VM is also a deployment target in our Octopus server :</p>
<p><img src="https://mathieubuisson.github.io/images/2018-03-12-trigger-octopus-removal-azure-octo-machine.png" alt="Octopus Deployment Target" /></p>
<p>Let’s delete this VM from Azure (or the resource group containing the VM) and see what happens.</p>
<p>As soon as the VM deletion completes, we can go back to our event subscription and click <strong>Metrics</strong>. Hovering over the time of deletion shows that there was indeed an event published, and delivered to a handler :</p>
<p><img src="https://mathieubuisson.github.io/images/2018-03-12-trigger-octopus-removal-azure-metrics.png" alt="Event Subscription Metrics" /></p>
<p>Now, let’s go to the <code class="language-plaintext highlighter-rouge">Watch-VMDeletion</code> runbook, click on <strong>Jobs</strong> and select the latest one.</p>
<p>By clicking on <strong>Input</strong>, we can view the webhook data, which contains all information about the event.</p>
<p>By clicking on <strong>Output</strong>, we can view the runbook output, which looks like this :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">correlationId</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">9610978a-5b35-40c5-8cac-e1f2814ffaee</span><span class="w">
</span><span class="nf">resourceProvider</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">Microsoft.Compute</span><span class="w">
</span><span class="nf">resourceUri</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">/subscriptions/08d7f375-8b5e-4f22-afa7-8f09d060bc11/resourcegroups/CattleVM01/providers/Microsoft.Compute/virtualMachines/CattleVM01</span><span class="w">
</span><span class="nf">operationName</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">Microsoft.Compute/virtualMachines/delete</span><span class="w">
</span><span class="nf">status</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">Succeeded</span><span class="w">
</span><span class="nf">subscriptionId</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">08d7f375-8b5e-4f22-afa7-8f09d060bc11</span><span class="w">
</span><span class="nf">VMName</span><span class="w"> </span><span class="nx">from</span><span class="w"> </span><span class="nx">webhook</span><span class="w"> </span><span class="nx">data</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">CattleVM01</span><span class="w">
</span><span class="nf">Invoking</span><span class="w"> </span><span class="nx">runbook</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">Remove-MachineFromOctopusServer</span><span class="w">
</span><span class="nf">Machine</span><span class="w"> </span><span class="nx">Name</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">CattleVM01</span><span class="w">
</span><span class="nf">Octopus</span><span class="w"> </span><span class="nx">Server</span><span class="w"> </span><span class="nx">URL</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">https://xxxxxxx.xxxxxxxxxx.com</span><span class="w">
</span><span class="nf">Name</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">CattleVM01</span><span class="w">
</span><span class="nf">Thumbprint</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">287CBDB08B9D3790B96EA1F945260F52026D3939</span><span class="w">
</span><span class="nf">Uri</span><span class="w"> </span><span class="p">:</span><span class="w">
</span><span class="nf">IsDisabled</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nf">EnvironmentIds</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="nf">Environments-141</span><span class="p">}</span><span class="w">
</span><span class="nf">Roles</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="nf">App</span><span class="p">,</span><span class="w"> </span><span class="nx">Core-API</span><span class="p">}</span><span class="w">
</span><span class="nf">MachinePolicyId</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">MachinePolicies-1</span><span class="w">
</span><span class="nf">TenantedDeploymentParticipation</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">Untenanted</span><span class="w">
</span><span class="nf">TenantIds</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="p">{}</span><span class="w">
</span><span class="nf">TenantTags</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="p">{}</span><span class="w">
</span><span class="nf">Status</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">Online</span><span class="w">
</span><span class="nf">HealthStatus</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">Healthy</span><span class="w">
</span><span class="nf">HasLatestCalamari</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">True</span><span class="w">
</span><span class="nf">StatusSummary</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">Octopus</span><span class="w"> </span><span class="nx">was</span><span class="w"> </span><span class="nx">able</span><span class="w"> </span><span class="nx">to</span><span class="w"> </span><span class="nx">successfully</span><span class="w"> </span><span class="nx">establish</span><span class="w"> </span><span class="nx">a</span><span class="w"> </span><span class="nx">connection</span><span class="w"> </span><span class="nx">with</span><span class="w"> </span><span class="nx">this</span><span class="w"> </span><span class="nx">machine</span><span class="w"> </span><span class="nx">on</span><span class="w"> </span><span class="nx">Sunday</span><span class="p">,</span><span class="w"> </span><span class="nx">March</span><span class="w"> </span><span class="nx">11</span><span class="p">,</span><span class="w"> </span><span class="nx">2018</span><span class="w"> </span><span class="nx">8:37</span><span class="w"> </span><span class="nx">PM</span><span class="w">
</span><span class="nf">IsInProcess</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nf">Endpoint</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">Octopus.Client.Model.Endpoints.PollingTentacleEndpointResource</span><span class="w">
</span><span class="nf">Id</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">Machines-383</span><span class="w">
</span><span class="nf">LastModifiedOn</span><span class="w"> </span><span class="p">:</span><span class="w">
</span><span class="nf">LastModifiedBy</span><span class="w"> </span><span class="p">:</span><span class="w">
</span><span class="nf">Links</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="p">{[</span><span class="kt">Self</span><span class="p">,</span><span class="w"> </span><span class="nf">/api/machines/Machines-383</span><span class="p">],</span><span class="w"> </span><span class="p">[</span><span class="kt">Connection</span><span class="p">,</span><span class="w"> </span><span class="nf">/api/machines/Machines-383/connection</span><span class="p">],</span><span class="w"> </span><span class="p">[</span><span class="kt">TasksTemplate</span><span class="p">,</span><span class="w"> </span><span class="nf">/api/machines/Machines-383/tasks</span><span class="p">{</span><span class="err">?</span><span class="kt">skip</span><span class="p">}]}</span><span class="w">
</span><span class="kt">The</span><span class="w"> </span><span class="kt">machine</span><span class="w"> </span><span class="kt">has</span><span class="w"> </span><span class="kt">been</span><span class="w"> </span><span class="kt">successfully</span><span class="w"> </span><span class="kt">removed</span><span class="w"> </span><span class="kt">from</span><span class="w"> </span><span class="no">Octopus.</span><span class="w">
</span></code></pre></div></div>
<p>We can go back the Octopus Deploy web portal and confirm that the deployment target <strong>CattleVM01</strong> is gone.</p>
<p>Now, development teams and QA folks can spin up and tear down environments to their heart’s content, without cluttering our Octopus server with stale deployment targets.</p>
<p>Besides, this offers a glimpse at how powerful the combination of <strong>Azure Automation</strong> and <strong>Azure Event Grid</strong> can be.</p>
<p>For more information on Azure Event Grid, head over to <a href="https://docs.microsoft.com/en-us/azure/event-grid/">the docs</a>.</p>TheShellNutIn this post, we take a look at leveraging Azure Automation and Azure Event Grid to ensure that deleting a VM from Azure will automatically trigger the removal of the corresponding machine from Octopus Deploy.Writing and Using Custom Assertions for Pester Tests2017-10-31T00:00:00+00:002017-10-31T00:00:00+00:00https://mathieubuisson.github.io/pester-custom-assertions<p><a href="https://github.com/pester/Pester">Pester</a>, the awesome PowerShell testing framework has recently introduced a capability which allows us to extend our test assertions. This allows to simplify our tests by abstracting custom or complex assertion logic away from the tests and into separate scripts or modules.</p>
<p>Keeping the test scripts as clean and readable as possible is central to leveraging the simplicity and elegance of Pester’s <abbr title="Domain Specific Language">DSL</abbr>. Also, I really like the idea that test scripts can act as an <strong>executable</strong> (potentially business-readable) <strong>specification</strong>.</p>
<p>Concretely, “<em>custom assertions</em>” means that we can plug additional operators into <strong>Pester</strong>’s assertion function : <code class="language-plaintext highlighter-rouge">Should</code>. Assuming we are using the <code class="language-plaintext highlighter-rouge">Pester</code> version 4.0.8, there are quite a few built-in <code class="language-plaintext highlighter-rouge">Should</code> operators :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\</span><span class="err">></span><span class="w"> </span><span class="nv">$PesterPath</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="nf">Get-Module</span><span class="w"> </span><span class="nt">-Name</span><span class="w"> </span><span class="s1">'Pester'</span><span class="w"> </span><span class="nt">-ListAvailable</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="nf">ModuleBase</span><span class="w">
</span><span class="nx">C:\</span><span class="err">></span><span class="w"> </span><span class="nv">$AllItems</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="nf">Get-ChildItem</span><span class="w"> </span><span class="s2">"</span><span class="nv">$PesterPath</span><span class="s2">\Functions\Assertions\"</span><span class="p">)</span><span class="o">.</span><span class="nf">BaseName</span><span class="w">
</span><span class="nx">C:\</span><span class="err">></span><span class="w"> </span><span class="nv">$AllItems</span><span class="o">.</span><span class="nf">Where</span><span class="p">({</span><span class="w"> </span><span class="bp">$_</span><span class="w"> </span><span class="o">-notmatch</span><span class="w"> </span><span class="s1">'Should|Set-'</span><span class="w"> </span><span class="p">})</span><span class="w">
</span><span class="nf">Be</span><span class="w">
</span><span class="nf">BeGreaterThan</span><span class="w">
</span><span class="nf">BeIn</span><span class="w">
</span><span class="nf">BeLessThan</span><span class="w">
</span><span class="nf">BeLike</span><span class="w">
</span><span class="nf">BeLikeExactly</span><span class="w">
</span><span class="nf">BeNullOrEmpty</span><span class="w">
</span><span class="nf">BeOfType</span><span class="w">
</span><span class="nf">Exist</span><span class="w">
</span><span class="nf">FileContentMatch</span><span class="w">
</span><span class="nf">FileContentMatchExactly</span><span class="w">
</span><span class="nf">FileContentMatchMultiline</span><span class="w">
</span><span class="nf">Match</span><span class="w">
</span><span class="nf">MatchExactly</span><span class="w">
</span><span class="nf">PesterThrow</span><span class="w">
</span></code></pre></div></div>
<p>But in some cases, it is possible that none of these operators fit the type of assertion/comparison needed in our tests.<br />
For example, let’s say we need to validate that a number is within an <strong>inclusive</strong> range. With the built-in operators we are limited to the <code class="language-plaintext highlighter-rouge">BeGreaterThan</code> and <code class="language-plaintext highlighter-rouge">BeLessThan</code>, so our test would look like this :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">Describe</span><span class="w"> </span><span class="s1">'Built-in assertions with numbers'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">It</span><span class="w"> </span><span class="s1">'55.5 should be in range [0-100]'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="mf">55.5</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Should</span><span class="w"> </span><span class="nt">-BeGreaterThan</span><span class="w"> </span><span class="nt">-0</span><span class="mf">.000001</span><span class="w">
</span><span class="mf">55.5</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Should</span><span class="w"> </span><span class="nt">-BeLessThan</span><span class="w"> </span><span class="nx">100.000001</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>First, it forces us to have 2 assertions in a single test, which is not ideal.<br />
Second, because we want an <strong>inclusive</strong> range and there is no inclusive operators like <code class="language-plaintext highlighter-rouge">BeGreaterThanOrEqualTo</code> or <code class="language-plaintext highlighter-rouge">BeLessThanOrEqualTo</code>, we have to modify the “<em>expected</em>” part of the assertion.</p>
<p>Instead of comparing the value to the <em>expected</em> low end of the range (0), we have to compare it with the “<em>expected</em>” minus “<em>just a little bit</em>” to ensure that the assertion passes if the value <strong>equals</strong> the <em>expected</em> low end of the range. A similar gymnastic is needed when asserting the high end of the range.</p>
<p>This is not only confusing, but also unreliable. The precision of the assertion depends on how small is the “<em>just a little bit</em>”, or in other words, how many decimal places we add/remove to the “<em>expected</em>” part of the assertion.</p>
<p>This whole thing feels wrong so let’s write a better assertion using a custom <code class="language-plaintext highlighter-rouge">Should</code> operator.</p>
<h2 id="writing-a-custom-should-operator">Writing a custom Should operator</h2>
<p>There isn’t much documentation nor examples out there yet, so for now, the best place to start is <a href="https://github.com/pester/Pester/wiki/Add-AssertionOperator">this example in the Pester wiki</a>.</p>
<p>Our custom operator is going to be named <code class="language-plaintext highlighter-rouge">BeInRange</code> and unlike the examples we can find, it will require 2 values to represent the “<em>expected</em>” part of the assertion :</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">$Min</code> for the low end of the range</li>
<li><code class="language-plaintext highlighter-rouge">$Max</code> for the high end of the range</li>
</ul>
<p>So here is what the assertion operator function <code class="language-plaintext highlighter-rouge">BeInRange</code> looks like :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kr">Function</span><span class="w"> </span><span class="nf">BeInRange</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="cm"><#
</span><span class="cs">.SYNOPSIS</span><span class="cm">
Tests whether a value is in a given inclusive range
#></span><span class="w">
</span><span class="p">[</span><span class="kt">CmdletBinding</span><span class="p">()]</span><span class="w">
</span><span class="kr">Param</span><span class="p">(</span><span class="w">
</span><span class="nv">$ActualValue</span><span class="p">,</span><span class="w">
</span><span class="nv">$Min</span><span class="p">,</span><span class="w">
</span><span class="nv">$Max</span><span class="p">,</span><span class="w">
</span><span class="p">[</span><span class="kt">switch</span><span class="p">]</span><span class="nv">$Negate</span><span class="w">
</span><span class="p">)</span><span class="w">
</span><span class="p">[</span><span class="kt">bool</span><span class="p">]</span><span class="nv">$Pass</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$ActualValue</span><span class="w"> </span><span class="o">-ge</span><span class="w"> </span><span class="nv">$Min</span><span class="w"> </span><span class="o">-and</span><span class="w"> </span><span class="nv">$ActualValue</span><span class="w"> </span><span class="o">-le</span><span class="w"> </span><span class="nv">$Max</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$Negate</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nv">$Pass</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">-not</span><span class="p">(</span><span class="nv">$Pass</span><span class="p">)</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="o">-not</span><span class="p">(</span><span class="nv">$Pass</span><span class="p">)</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$Negate</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$FailureMessage</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'Expected: value {{{0}}} to be outside the range {{{1}-{2}}} but it was in the range.'</span><span class="w"> </span><span class="nt">-f</span><span class="w"> </span><span class="nv">$ActualValue</span><span class="p">,</span><span class="w"> </span><span class="nv">$Min</span><span class="p">,</span><span class="w"> </span><span class="nv">$Max</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">Else</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$FailureMessage</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'Expected: value {{{0}}} to be in the range {{{1}-{2}}} but it was outside the range.'</span><span class="w"> </span><span class="nt">-f</span><span class="w"> </span><span class="nv">$ActualValue</span><span class="p">,</span><span class="w"> </span><span class="nv">$Min</span><span class="p">,</span><span class="w"> </span><span class="nv">$Max</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nv">$ObjProperties</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">@{</span><span class="w">
</span><span class="nx">Succeeded</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$Pass</span><span class="w">
</span><span class="nx">FailureMessage</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$FailureMessage</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">return</span><span class="w"> </span><span class="nf">New-Object</span><span class="w"> </span><span class="nx">PSObject</span><span class="w"> </span><span class="nt">-Property</span><span class="w"> </span><span class="nv">$ObjProperties</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>First thing to note : the parameter representing the “<em>actual</em>” part of the assertion has to be named <code class="language-plaintext highlighter-rouge">ActualValue</code>. If not, <strong>Pester</strong>’s internal function <code class="language-plaintext highlighter-rouge">Invoke-Assertion</code> blows up because it calls any assertion function with the <code class="language-plaintext highlighter-rouge">ActualValue</code> parameter to pass the asserted value.</p>
<p>All <code class="language-plaintext highlighter-rouge">Should</code> operators can be negated by inserting <code class="language-plaintext highlighter-rouge">-Not</code> before them. For our custom operator to respect this behavior, we need to implement the <code class="language-plaintext highlighter-rouge">Negate</code> switch parameter, as seen above.</p>
<blockquote>
<p>What’s the deal with the triple braces in the <code class="language-plaintext highlighter-rouge">$FailureMessage</code> string ?</p>
</blockquote>
<ul>
<li>Brace #1 : The string formatting operator (<code class="language-plaintext highlighter-rouge">-f</code>) uses <code class="language-plaintext highlighter-rouge">{}</code> as placeholders</li>
<li>Brace #2 : We enclose the currently asserted values in <code class="language-plaintext highlighter-rouge">{}</code> to be consistent with built-in assertion’s failure messages</li>
<li>Brace #3 : The string formatting operator freaks out if the string contains braces, so we double the second set of braces to escape them</li>
</ul>
<p>Our assertion operator function has a contractual obligation : it has to return an object of the type <code class="language-plaintext highlighter-rouge">[PSObject]</code> with a property named <code class="language-plaintext highlighter-rouge">Succeeded</code> and a property named <code class="language-plaintext highlighter-rouge">FailureMessage</code>. So we built <code class="language-plaintext highlighter-rouge">$ObjProperties</code> accordingly.</p>
<p>How do we know that ?</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\</span><span class="err">></span><span class="w"> </span><span class="nx">Get-Help</span><span class="w"> </span><span class="s1">'Add-AssertionOperator'</span><span class="w"> </span><span class="nt">-Parameter</span><span class="w"> </span><span class="s1">'Test'</span><span class="w">
</span><span class="nt">-Test</span><span class="w"> </span><span class="err"><</span><span class="nf">ScriptBlock</span><span class="err">></span><span class="w">
</span><span class="nf">The</span><span class="w"> </span><span class="nx">test</span><span class="w"> </span><span class="nx">function.</span><span class="w"> </span><span class="nx">The</span><span class="w"> </span><span class="nx">function</span><span class="w"> </span><span class="nx">must</span><span class="w"> </span><span class="nx">return</span><span class="w"> </span><span class="nx">a</span><span class="w"> </span><span class="nx">PSObject</span><span class="w"> </span><span class="nx">with</span><span class="w"> </span><span class="nx">a</span><span class="w"> </span><span class="p">[</span><span class="kt">Bool</span><span class="p">]</span><span class="nx">succeeded</span><span class="w">
</span><span class="nf">and</span><span class="w"> </span><span class="nx">a</span><span class="w"> </span><span class="p">[</span><span class="kt">string</span><span class="p">]</span><span class="nx">failureMessage</span><span class="w"> </span><span class="nx">property.</span><span class="w">
</span><span class="nf">Required</span><span class="err">?</span><span class="w"> </span><span class="nx">true</span><span class="w">
</span><span class="nf">Position</span><span class="err">?</span><span class="w"> </span><span class="nx">2</span><span class="w">
</span><span class="nf">Default</span><span class="w"> </span><span class="nx">value</span><span class="w">
</span><span class="nf">Accept</span><span class="w"> </span><span class="nx">pipeline</span><span class="w"> </span><span class="nx">input</span><span class="err">?</span><span class="w"> </span><span class="nx">false</span><span class="w">
</span><span class="nf">Accept</span><span class="w"> </span><span class="nx">wildcard</span><span class="w"> </span><span class="nx">characters</span><span class="err">?</span><span class="w"> </span><span class="nx">false</span><span class="w">
</span></code></pre></div></div>
<h2 id="using-a-custom-should-operator-in-our-tests">Using a custom Should operator in our tests</h2>
<p>We put our <code class="language-plaintext highlighter-rouge">BeInRange</code> function in a module named <code class="language-plaintext highlighter-rouge">CustomAssertions.psm1</code> to make it easy to reuse. We could even store the module within a location in our <code class="language-plaintext highlighter-rouge">$Env:PSModulePath</code> to allow importing it by name, instead of by path.</p>
<p>There is another way : put the custom operator function in a .ps1 file inside <strong>Pester</strong>’s assertions folder : <code class="language-plaintext highlighter-rouge">"$((Get-Module Pester).ModuleBase)\Functions\Assertions\"</code>.<br />
<strong>Pester</strong> picks up the operators automatically from this location but this might be less flexible, so choose a method based on your preference/scenario.</p>
<p>Now is the time to write tests to verify how much cleaner our assertions can be with our custom operator and ensure that it behaves as expected :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">Import-Module</span><span class="w"> </span><span class="s2">"</span><span class="bp">$PSScriptRoot</span><span class="s2">\CustomAssertions.psm1"</span><span class="w"> </span><span class="nt">-Force</span><span class="w">
</span><span class="nf">Add-AssertionOperator</span><span class="w"> </span><span class="nt">-Name</span><span class="w"> </span><span class="s1">'BeInRange'</span><span class="w"> </span><span class="nt">-Test</span><span class="w"> </span><span class="nv">$</span><span class="nn">Function</span><span class="p">:</span><span class="nv">BeInRange</span><span class="w">
</span><span class="nf">Describe</span><span class="w"> </span><span class="s1">'BeInRange assertions with numbers'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">It</span><span class="w"> </span><span class="s1">'55.5 should be in range [0-100]'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="mf">55.5</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Should</span><span class="w"> </span><span class="nt">-BeInRange</span><span class="w"> </span><span class="nt">-Min</span><span class="w"> </span><span class="nx">0</span><span class="w"> </span><span class="nt">-Max</span><span class="w"> </span><span class="nx">100</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nf">It</span><span class="w"> </span><span class="s1">'0 should be in range [0-100] (inclusive range)'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="mi">0</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Should</span><span class="w"> </span><span class="nt">-BeInRange</span><span class="w"> </span><span class="nt">-Min</span><span class="w"> </span><span class="nx">0</span><span class="w"> </span><span class="nt">-Max</span><span class="w"> </span><span class="nx">100</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nf">It</span><span class="w"> </span><span class="s1">'80 should not be in range [0-55.5]'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="mi">80</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Should</span><span class="w"> </span><span class="o">-Not</span><span class="w"> </span><span class="nt">-BeInRange</span><span class="w"> </span><span class="nt">-Min</span><span class="w"> </span><span class="nx">0</span><span class="w"> </span><span class="nt">-Max</span><span class="w"> </span><span class="nx">55.5</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nf">It</span><span class="w"> </span><span class="s1">'Should fail (to verify the failure message)'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="mi">1</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Should</span><span class="w"> </span><span class="nt">-BeInRange</span><span class="w"> </span><span class="nt">-Min</span><span class="w"> </span><span class="nx">10</span><span class="w"> </span><span class="nt">-Max</span><span class="w"> </span><span class="nx">20</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>The first line imports the module to make our <code class="language-plaintext highlighter-rouge">BeInRange</code> function visible in the global scope to ensure <code class="language-plaintext highlighter-rouge">Add-AssertionOperator</code> will see it.</p>
<p><code class="language-plaintext highlighter-rouge">Add-AssertionOperator</code> is the function from <strong>Pester</strong> which enables the magic. It registers a custom assertion operator function with <strong>Pester</strong> which integrates it with the <code class="language-plaintext highlighter-rouge">Should</code> function as a dynamic parameter. Clever stuff.</p>
<p>Unfortunately, as soon as we run this tests script more than once (without unloading the <code class="language-plaintext highlighter-rouge">Pester</code> module), we get this error :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">Executing</span><span class="w"> </span><span class="nx">script</span><span class="w"> </span><span class="o">.</span><span class="nx">\CustomAssertions.Tests.ps1</span><span class="w">
</span><span class="p">[</span><span class="nf">-</span><span class="p">]</span><span class="w"> </span><span class="kt">Error</span><span class="w"> </span><span class="kt">occurred</span><span class="w"> </span><span class="kt">in</span><span class="w"> </span><span class="kt">test</span><span class="w"> </span><span class="kt">script</span><span class="w"> </span><span class="s1">'.\CustomAssertions.Tests.ps1'</span><span class="w"> </span><span class="mi">35</span><span class="kt">ms</span><span class="w">
</span><span class="kt">RuntimeException</span><span class="p">:</span><span class="w"> </span><span class="kt">Assertion</span><span class="w"> </span><span class="kt">operator</span><span class="w"> </span><span class="kt">name</span><span class="w"> </span><span class="s1">'BeInRange'</span><span class="w"> </span><span class="kt">has</span><span class="w"> </span><span class="kt">been</span><span class="w"> </span><span class="kt">added</span><span class="w"> </span><span class="kt">multiple</span><span class="w"> </span><span class="no">times.</span><span class="w">
</span><span class="kt">at</span><span class="w"> </span><span class="kt">Assert</span><span class="nt">-AssertionOperatorNameIsUnique</span><span class="p">,</span><span class="w"> </span><span class="nf">C:\Program</span><span class="w"> </span><span class="nx">Files\WindowsPowerShell\Modules\Pester\4.0.8\Pester.psm1:</span><span class="w"> </span><span class="nx">line</span><span class="w"> </span><span class="nx">243</span><span class="w">
</span><span class="kt">at</span><span class="w"> </span><span class="kt">Add</span><span class="nt">-AssertionOperator</span><span class="p">,</span><span class="w"> </span><span class="nf">C:\Program</span><span class="w"> </span><span class="nx">Files\WindowsPowerShell\Modules\Pester\4.0.8\Pester.psm1:</span><span class="w"> </span><span class="nx">line</span><span class="w"> </span><span class="nx">211</span><span class="w">
</span><span class="kt">at</span><span class="w"> </span><span class="err"><</span><span class="kt">ScriptBlock</span><span class="err">></span><span class="p">,</span><span class="w"> </span><span class="nf">C:\CustomAssertions\CustomAssertions.Tests.ps1:</span><span class="w"> </span><span class="nx">line</span><span class="w"> </span><span class="nx">2</span><span class="w">
</span><span class="kt">at</span><span class="w"> </span><span class="err"><</span><span class="kt">ScriptBlock</span><span class="err">></span><span class="p">,</span><span class="w"> </span><span class="nf">C:\Program</span><span class="w"> </span><span class="nx">Files\WindowsPowerShell\Modules\Pester\4.0.8\Pester.psm1:</span><span class="w"> </span><span class="nx">line</span><span class="w"> </span><span class="nx">802</span><span class="w">
</span><span class="kt">at</span><span class="w"> </span><span class="kt">Invoke</span><span class="nt">-Pester</span><span class="err"><</span><span class="kt">End</span><span class="err">></span><span class="p">,</span><span class="w"> </span><span class="nf">C:\Program</span><span class="w"> </span><span class="nx">Files\WindowsPowerShell\Modules\Pester\4.0.8\Pester.psm1:</span><span class="w"> </span><span class="nx">line</span><span class="w"> </span><span class="nx">817</span><span class="w">
</span></code></pre></div></div>
<p>Thankfully <a href="https://github.com/pester/Pester/issues/891">this bug</a> is fixed in the master branch and slated for milestone 4.1.</p>
<p>As we can see above, the assertions using the <code class="language-plaintext highlighter-rouge">BeInRange</code> operator are simpler and more readable than the initial example using the built-in operators.</p>
<p>Now let’s see if it works :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\</span><span class="err">></span><span class="w"> </span><span class="nv">$TestsPath</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'C:\CustomAssertions\CustomAssertions.Tests.ps1'</span><span class="w">
</span><span class="nf">C:\</span><span class="err">></span><span class="w"> </span><span class="nx">Invoke-Pester</span><span class="w"> </span><span class="nt">-Script</span><span class="w"> </span><span class="nv">$TestsPath</span><span class="w">
</span><span class="nf">Executing</span><span class="w"> </span><span class="nx">all</span><span class="w"> </span><span class="nx">tests</span><span class="w"> </span><span class="nx">in</span><span class="w"> </span><span class="s1">'C:\CustomAssertions\CustomAssertions.Tests.ps1'</span><span class="w">
</span><span class="nf">Executing</span><span class="w"> </span><span class="nx">script</span><span class="w"> </span><span class="nx">C:\CustomAssertions\CustomAssertions.Tests.ps1</span><span class="w">
</span><span class="nf">Describing</span><span class="w"> </span><span class="nx">BeInRange</span><span class="w"> </span><span class="nx">assertions</span><span class="w"> </span><span class="nx">with</span><span class="w"> </span><span class="nx">numbers</span><span class="w">
</span><span class="p">[</span><span class="o">+</span><span class="p">]</span><span class="w"> </span><span class="mf">55.5</span><span class="w"> </span><span class="nf">should</span><span class="w"> </span><span class="nx">be</span><span class="w"> </span><span class="nx">in</span><span class="w"> </span><span class="nx">range</span><span class="w"> </span><span class="p">[</span><span class="mi">0</span><span class="nt">-100</span><span class="p">]</span><span class="w"> </span><span class="nx">94ms</span><span class="w">
</span><span class="p">[</span><span class="o">+</span><span class="p">]</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="nf">should</span><span class="w"> </span><span class="nx">be</span><span class="w"> </span><span class="nx">in</span><span class="w"> </span><span class="nx">range</span><span class="w"> </span><span class="p">[</span><span class="mi">0</span><span class="nt">-100</span><span class="p">]</span><span class="w"> </span><span class="p">(</span><span class="nf">inclusive</span><span class="w"> </span><span class="nx">range</span><span class="p">)</span><span class="w"> </span><span class="mi">53</span><span class="nf">ms</span><span class="w">
</span><span class="p">[</span><span class="o">+</span><span class="p">]</span><span class="w"> </span><span class="mi">80</span><span class="w"> </span><span class="nf">should</span><span class="w"> </span><span class="nx">not</span><span class="w"> </span><span class="nx">be</span><span class="w"> </span><span class="nx">in</span><span class="w"> </span><span class="nx">range</span><span class="w"> </span><span class="p">[</span><span class="mi">0</span><span class="nt">-55</span><span class="mf">.5</span><span class="p">]</span><span class="w"> </span><span class="nx">51ms</span><span class="w">
</span><span class="p">[</span><span class="nf">-</span><span class="p">]</span><span class="w"> </span><span class="kt">Should</span><span class="w"> </span><span class="kt">fail</span><span class="w"> </span><span class="p">(</span><span class="kt">to</span><span class="w"> </span><span class="kt">verify</span><span class="w"> </span><span class="kt">the</span><span class="w"> </span><span class="kt">failure</span><span class="w"> </span><span class="kt">message</span><span class="p">)</span><span class="w"> </span><span class="mi">38</span><span class="kt">ms</span><span class="w">
</span><span class="kt">Expected</span><span class="p">:</span><span class="w"> </span><span class="kt">value</span><span class="w"> </span><span class="p">{</span><span class="mi">1</span><span class="p">}</span><span class="w"> </span><span class="kt">to</span><span class="w"> </span><span class="kt">be</span><span class="w"> </span><span class="kt">in</span><span class="w"> </span><span class="kt">the</span><span class="w"> </span><span class="kt">range</span><span class="w"> </span><span class="p">{</span><span class="mi">10</span><span class="nt">-20</span><span class="p">}</span><span class="w"> </span><span class="kt">but</span><span class="w"> </span><span class="kt">it</span><span class="w"> </span><span class="kt">was</span><span class="w"> </span><span class="kt">outside</span><span class="w"> </span><span class="kt">the</span><span class="w"> </span><span class="no">range.</span><span class="w">
</span><span class="mi">15</span><span class="p">:</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="kt">Should</span><span class="w"> </span><span class="nt">-BeInRange</span><span class="w"> </span><span class="nt">-Min</span><span class="w"> </span><span class="mi">10</span><span class="w"> </span><span class="nt">-Max</span><span class="w"> </span><span class="mi">20</span><span class="w">
</span><span class="kt">at</span><span class="w"> </span><span class="kt">Invoke</span><span class="nt">-Assertion</span><span class="p">,</span><span class="w"> </span><span class="nf">C:\Program</span><span class="w"> </span><span class="nx">Files\WindowsPowerShell\Modules\Pester\4.0.8\Functions\Assertions\Should.ps1:</span><span class="w"> </span><span class="nx">line</span><span class="w"> </span><span class="nx">209</span><span class="w">
</span><span class="kt">at</span><span class="w"> </span><span class="err"><</span><span class="kt">ScriptBlock</span><span class="err">></span><span class="p">,</span><span class="w"> </span><span class="nf">C:\CustomAssertions\CustomAssertions.Tests.ps1:</span><span class="w"> </span><span class="nx">line</span><span class="w"> </span><span class="nx">15</span><span class="w">
</span><span class="kt">Tests</span><span class="w"> </span><span class="kt">completed</span><span class="w"> </span><span class="kt">in</span><span class="w"> </span><span class="mi">237</span><span class="kt">ms</span><span class="w">
</span><span class="kt">Tests</span><span class="w"> </span><span class="kt">Passed</span><span class="p">:</span><span class="w"> </span><span class="mi">3</span><span class="p">,</span><span class="w"> </span><span class="kt">Failed</span><span class="p">:</span><span class="w"> </span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="kt">Skipped</span><span class="p">:</span><span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="kt">Pending</span><span class="p">:</span><span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="kt">Inconclusive</span><span class="p">:</span><span class="w"> </span><span class="mi">0</span><span class="w">
</span></code></pre></div></div>
<p>This works as expected.</p>
<p>But does it work with objects of the type <code class="language-plaintext highlighter-rouge">[string]</code> ? This would allow us to do assertions related to alphabetical ordering.<br />
How about <code class="language-plaintext highlighter-rouge">[datetime]</code> objects ? This would allow to validate that a <code class="language-plaintext highlighter-rouge">[datetime]</code> value is within an expected date (or time) range.</p>
<p>We add the following tests to our tests script :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">Context</span><span class="w"> </span><span class="s1">'Assertions on [string] objects'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">It</span><span class="w"> </span><span class="s1">'"abcd" should be in range ["ab"-"yz"]'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="s1">'abcd'</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Should</span><span class="w"> </span><span class="nt">-BeInRange</span><span class="w"> </span><span class="nt">-Min</span><span class="w"> </span><span class="s1">'ab'</span><span class="w"> </span><span class="nt">-Max</span><span class="w"> </span><span class="s1">'yz'</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nf">It</span><span class="w"> </span><span class="s1">'"a" should be in range ["a"-"yz"]'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="s1">'a'</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Should</span><span class="w"> </span><span class="nt">-BeInRange</span><span class="w"> </span><span class="nt">-Min</span><span class="w"> </span><span class="s1">'a'</span><span class="w"> </span><span class="nt">-Max</span><span class="w"> </span><span class="s1">'yz'</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nf">It</span><span class="w"> </span><span class="s1">'"az" should not be in range ["ab"-"aefg"]'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="s1">'az'</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Should</span><span class="w"> </span><span class="o">-Not</span><span class="w"> </span><span class="nt">-BeInRange</span><span class="w"> </span><span class="nt">-Min</span><span class="w"> </span><span class="s1">'ab'</span><span class="w"> </span><span class="nt">-Max</span><span class="w"> </span><span class="s1">'aefg'</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nf">Context</span><span class="w"> </span><span class="s1">'Assertions on [datetime] objects'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$Now</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">[</span><span class="kt">datetime</span><span class="p">]::</span><span class="nf">Now</span><span class="w">
</span><span class="nf">It</span><span class="w"> </span><span class="s1">'The 1st of October 2017 should be in the range representing the current year'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$YearStart</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">Get-Date</span><span class="w"> </span><span class="nt">-Month</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nt">-Day</span><span class="w"> </span><span class="nx">1</span><span class="w">
</span><span class="nv">$YearEnd</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">Get-Date</span><span class="w"> </span><span class="nt">-Month</span><span class="w"> </span><span class="nx">12</span><span class="w"> </span><span class="nt">-Day</span><span class="w"> </span><span class="nx">31</span><span class="w">
</span><span class="nv">$FirstOct</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">Get-Date</span><span class="w"> </span><span class="nt">-Year</span><span class="w"> </span><span class="nx">2017</span><span class="w"> </span><span class="nt">-Month</span><span class="w"> </span><span class="nx">10</span><span class="w"> </span><span class="nt">-Day</span><span class="w"> </span><span class="nx">1</span><span class="w">
</span><span class="nv">$FirstOct</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Should</span><span class="w"> </span><span class="nt">-BeInRange</span><span class="w"> </span><span class="nt">-Min</span><span class="w"> </span><span class="nv">$YearStart</span><span class="w"> </span><span class="nt">-Max</span><span class="w"> </span><span class="nv">$YearEnd</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nf">It</span><span class="w"> </span><span class="s1">'Today at 10 AM should be between today at 01 AM and now'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$DayStart</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">Get-Date</span><span class="w"> </span><span class="nt">-Hour</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nt">-Minute</span><span class="w"> </span><span class="nx">0</span><span class="w">
</span><span class="nf">Get-Date</span><span class="w"> </span><span class="nt">-Hour</span><span class="w"> </span><span class="nx">10</span><span class="w"> </span><span class="nt">-Minute</span><span class="w"> </span><span class="nx">0</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Should</span><span class="w"> </span><span class="nt">-BeInRange</span><span class="w"> </span><span class="nt">-Min</span><span class="w"> </span><span class="nv">$DayStart</span><span class="w"> </span><span class="nt">-Max</span><span class="w"> </span><span class="nv">$Now</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nf">It</span><span class="w"> </span><span class="s1">'Now should not be in the range representing last month'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$LastMonthStart</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="nf">Get-Date</span><span class="w"> </span><span class="nt">-Day</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nt">-Hour</span><span class="w"> </span><span class="nx">0</span><span class="w"> </span><span class="nt">-Minute</span><span class="w"> </span><span class="nx">0</span><span class="p">)</span><span class="o">.</span><span class="nf">AddMonths</span><span class="p">(</span><span class="nt">-1</span><span class="p">)</span><span class="w">
</span><span class="nv">$LastMonthEnd</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="nf">Get-Date</span><span class="w"> </span><span class="nt">-Day</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nt">-Hour</span><span class="w"> </span><span class="nx">0</span><span class="w"> </span><span class="nt">-Minute</span><span class="w"> </span><span class="nx">0</span><span class="p">)</span><span class="o">.</span><span class="nf">AddMinutes</span><span class="p">(</span><span class="nt">-1</span><span class="p">)</span><span class="w">
</span><span class="nv">$Now</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Should</span><span class="w"> </span><span class="o">-Not</span><span class="w"> </span><span class="nt">-BeInRange</span><span class="w"> </span><span class="nt">-Min</span><span class="w"> </span><span class="nv">$LastMonthStart</span><span class="w"> </span><span class="nt">-Max</span><span class="w"> </span><span class="nv">$LastMonthEnd</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nf">It</span><span class="w"> </span><span class="s1">'Now should not be in a range located in the future'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$FutureStart</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">Get-Date</span><span class="w"> </span><span class="nt">-Year</span><span class="w"> </span><span class="nx">2117</span><span class="w">
</span><span class="nv">$FutureEnd</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">Get-Date</span><span class="w"> </span><span class="nt">-Year</span><span class="w"> </span><span class="nx">2217</span><span class="w">
</span><span class="nv">$Now</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Should</span><span class="w"> </span><span class="o">-Not</span><span class="w"> </span><span class="nt">-BeInRange</span><span class="w"> </span><span class="nt">-Min</span><span class="w"> </span><span class="nv">$FutureStart</span><span class="w"> </span><span class="nt">-Max</span><span class="w"> </span><span class="nv">$FutureEnd</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>Then, we run the tests script again :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\</span><span class="err">></span><span class="w"> </span><span class="nx">Invoke-Pester</span><span class="w"> </span><span class="nt">-Script</span><span class="w"> </span><span class="nv">$TestsPath</span><span class="w">
</span><span class="nf">Executing</span><span class="w"> </span><span class="nx">all</span><span class="w"> </span><span class="nx">tests</span><span class="w"> </span><span class="nx">in</span><span class="w"> </span><span class="s1">'C:\CustomAssertions\CustomAssertions.Tests.ps1'</span><span class="w">
</span><span class="nf">Executing</span><span class="w"> </span><span class="nx">script</span><span class="w"> </span><span class="nx">C:\CustomAssertions\CustomAssertions.Tests.ps1</span><span class="w">
</span><span class="nf">Describing</span><span class="w"> </span><span class="nx">BeInRange</span><span class="w"> </span><span class="nx">assertions</span><span class="w"> </span><span class="nx">with</span><span class="w"> </span><span class="nx">numbers</span><span class="w">
</span><span class="nf">Context</span><span class="w"> </span><span class="nx">Assertions</span><span class="w"> </span><span class="nx">on</span><span class="w"> </span><span class="nx">numbers</span><span class="w">
</span><span class="p">[</span><span class="o">+</span><span class="p">]</span><span class="w"> </span><span class="mf">55.5</span><span class="w"> </span><span class="nf">should</span><span class="w"> </span><span class="nx">be</span><span class="w"> </span><span class="nx">in</span><span class="w"> </span><span class="nx">range</span><span class="w"> </span><span class="p">[</span><span class="mi">0</span><span class="nt">-100</span><span class="p">]</span><span class="w"> </span><span class="nx">76ms</span><span class="w">
</span><span class="p">[</span><span class="o">+</span><span class="p">]</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="nf">should</span><span class="w"> </span><span class="nx">be</span><span class="w"> </span><span class="nx">in</span><span class="w"> </span><span class="nx">range</span><span class="w"> </span><span class="p">[</span><span class="mi">0</span><span class="nt">-100</span><span class="p">]</span><span class="w"> </span><span class="p">(</span><span class="nf">inclusive</span><span class="w"> </span><span class="nx">range</span><span class="p">)</span><span class="w"> </span><span class="mi">27</span><span class="nf">ms</span><span class="w">
</span><span class="p">[</span><span class="o">+</span><span class="p">]</span><span class="w"> </span><span class="mi">80</span><span class="w"> </span><span class="nf">should</span><span class="w"> </span><span class="nx">not</span><span class="w"> </span><span class="nx">be</span><span class="w"> </span><span class="nx">in</span><span class="w"> </span><span class="nx">range</span><span class="w"> </span><span class="p">[</span><span class="mi">0</span><span class="nt">-55</span><span class="mf">.5</span><span class="p">]</span><span class="w"> </span><span class="nx">28ms</span><span class="w">
</span><span class="p">[</span><span class="nf">-</span><span class="p">]</span><span class="w"> </span><span class="kt">Should</span><span class="w"> </span><span class="kt">fail</span><span class="w"> </span><span class="p">(</span><span class="kt">to</span><span class="w"> </span><span class="kt">verify</span><span class="w"> </span><span class="kt">the</span><span class="w"> </span><span class="kt">failure</span><span class="w"> </span><span class="kt">message</span><span class="p">)</span><span class="w"> </span><span class="mi">28</span><span class="kt">ms</span><span class="w">
</span><span class="kt">Expected</span><span class="p">:</span><span class="w"> </span><span class="kt">value</span><span class="w"> </span><span class="p">{</span><span class="mi">1</span><span class="p">}</span><span class="w"> </span><span class="kt">to</span><span class="w"> </span><span class="kt">be</span><span class="w"> </span><span class="kt">in</span><span class="w"> </span><span class="kt">the</span><span class="w"> </span><span class="kt">range</span><span class="w"> </span><span class="p">{</span><span class="mi">10</span><span class="nt">-20</span><span class="p">}</span><span class="w"> </span><span class="kt">but</span><span class="w"> </span><span class="kt">it</span><span class="w"> </span><span class="kt">was</span><span class="w"> </span><span class="kt">outside</span><span class="w"> </span><span class="kt">the</span><span class="w"> </span><span class="no">range.</span><span class="w">
</span><span class="mi">17</span><span class="p">:</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="kt">Should</span><span class="w"> </span><span class="nt">-BeInRange</span><span class="w"> </span><span class="nt">-Min</span><span class="w"> </span><span class="mi">10</span><span class="w"> </span><span class="nt">-Max</span><span class="w"> </span><span class="mi">20</span><span class="w">
</span><span class="kt">at</span><span class="w"> </span><span class="kt">Invoke</span><span class="nt">-Assertion</span><span class="p">,</span><span class="w"> </span><span class="nf">C:\Program</span><span class="w"> </span><span class="nx">Files\WindowsPowerShell\Modules\Pester\4.0.8\Functions</span><span class="w">
</span><span class="nf">\Assertions\Should.ps1:</span><span class="w"> </span><span class="nx">line</span><span class="w"> </span><span class="nx">209</span><span class="w">
</span><span class="kt">at</span><span class="w"> </span><span class="err"><</span><span class="kt">ScriptBlock</span><span class="err">></span><span class="p">,</span><span class="w"> </span><span class="nf">C:\CustomAssertions\CustomAssertions.Tests.ps1:</span><span class="w"> </span><span class="nx">line</span><span class="w"> </span><span class="nx">17</span><span class="w">
</span><span class="kt">Context</span><span class="w"> </span><span class="kt">Assertions</span><span class="w"> </span><span class="kt">on</span><span class="w"> </span><span class="p">[</span><span class="kt">string</span><span class="p">]</span><span class="w"> </span><span class="kt">objects</span><span class="w">
</span><span class="p">[</span><span class="o">+</span><span class="p">]</span><span class="w"> </span><span class="s2">"abcd"</span><span class="w"> </span><span class="kt">should</span><span class="w"> </span><span class="kt">be</span><span class="w"> </span><span class="kt">in</span><span class="w"> </span><span class="kt">range</span><span class="w"> </span><span class="p">[</span><span class="s2">"ab"</span><span class="nf">-</span><span class="s2">"yz"</span><span class="p">]</span><span class="w"> </span><span class="mi">65</span><span class="kt">ms</span><span class="w">
</span><span class="p">[</span><span class="o">+</span><span class="p">]</span><span class="w"> </span><span class="s2">"a"</span><span class="w"> </span><span class="kt">should</span><span class="w"> </span><span class="kt">be</span><span class="w"> </span><span class="kt">in</span><span class="w"> </span><span class="kt">range</span><span class="w"> </span><span class="p">[</span><span class="s2">"a"</span><span class="nf">-</span><span class="s2">"yz"</span><span class="p">]</span><span class="w"> </span><span class="mi">26</span><span class="kt">ms</span><span class="w">
</span><span class="p">[</span><span class="o">+</span><span class="p">]</span><span class="w"> </span><span class="s2">"az"</span><span class="w"> </span><span class="kt">should</span><span class="w"> </span><span class="kt">not</span><span class="w"> </span><span class="kt">be</span><span class="w"> </span><span class="kt">in</span><span class="w"> </span><span class="kt">range</span><span class="w"> </span><span class="p">[</span><span class="s2">"ab"</span><span class="nf">-</span><span class="s2">"aefg"</span><span class="p">]</span><span class="w"> </span><span class="mi">16</span><span class="kt">ms</span><span class="w">
</span><span class="kt">Context</span><span class="w"> </span><span class="kt">Assertions</span><span class="w"> </span><span class="kt">on</span><span class="w"> </span><span class="p">[</span><span class="kt">datetime</span><span class="p">]</span><span class="w"> </span><span class="kt">objects</span><span class="w">
</span><span class="p">[</span><span class="o">+</span><span class="p">]</span><span class="w"> </span><span class="kt">The</span><span class="w"> </span><span class="mi">1</span><span class="kt">st</span><span class="w"> </span><span class="kt">of</span><span class="w"> </span><span class="kt">October</span><span class="w"> </span><span class="mi">2017</span><span class="w"> </span><span class="kt">should</span><span class="w"> </span><span class="kt">be</span><span class="w"> </span><span class="kt">in</span><span class="w"> </span><span class="kt">the</span><span class="w"> </span><span class="kt">range</span><span class="w"> </span><span class="kt">representing</span><span class="w"> </span><span class="kt">the</span><span class="w"> </span><span class="kt">current</span><span class="w"> </span><span class="kt">year</span><span class="w"> </span><span class="mi">68</span><span class="kt">ms</span><span class="w">
</span><span class="p">[</span><span class="o">+</span><span class="p">]</span><span class="w"> </span><span class="kt">Today</span><span class="w"> </span><span class="kt">at</span><span class="w"> </span><span class="mi">10</span><span class="w"> </span><span class="kt">AM</span><span class="w"> </span><span class="kt">should</span><span class="w"> </span><span class="kt">be</span><span class="w"> </span><span class="kt">between</span><span class="w"> </span><span class="kt">today</span><span class="w"> </span><span class="kt">at</span><span class="w"> </span><span class="mi">01</span><span class="w"> </span><span class="kt">AM</span><span class="w"> </span><span class="kt">and</span><span class="w"> </span><span class="kt">now</span><span class="w"> </span><span class="mi">31</span><span class="kt">ms</span><span class="w">
</span><span class="p">[</span><span class="o">+</span><span class="p">]</span><span class="w"> </span><span class="kt">Now</span><span class="w"> </span><span class="kt">should</span><span class="w"> </span><span class="kt">not</span><span class="w"> </span><span class="kt">be</span><span class="w"> </span><span class="kt">in</span><span class="w"> </span><span class="kt">the</span><span class="w"> </span><span class="kt">range</span><span class="w"> </span><span class="kt">representing</span><span class="w"> </span><span class="kt">last</span><span class="w"> </span><span class="kt">month</span><span class="w"> </span><span class="mi">32</span><span class="kt">ms</span><span class="w">
</span><span class="p">[</span><span class="o">+</span><span class="p">]</span><span class="w"> </span><span class="kt">Now</span><span class="w"> </span><span class="kt">should</span><span class="w"> </span><span class="kt">not</span><span class="w"> </span><span class="kt">be</span><span class="w"> </span><span class="kt">in</span><span class="w"> </span><span class="nf">a</span><span class="w"> </span><span class="nx">range</span><span class="w"> </span><span class="nx">located</span><span class="w"> </span><span class="nx">in</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">future</span><span class="w"> </span><span class="nx">33ms</span><span class="w">
</span><span class="kt">Tests</span><span class="w"> </span><span class="kt">completed</span><span class="w"> </span><span class="kt">in</span><span class="w"> </span><span class="mi">436</span><span class="kt">ms</span><span class="w">
</span><span class="kt">Tests</span><span class="w"> </span><span class="kt">Passed</span><span class="p">:</span><span class="w"> </span><span class="mi">10</span><span class="p">,</span><span class="w"> </span><span class="kt">Failed</span><span class="p">:</span><span class="w"> </span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="kt">Skipped</span><span class="p">:</span><span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="kt">Pending</span><span class="p">:</span><span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="kt">Inconclusive</span><span class="p">:</span><span class="w"> </span><span class="mi">0</span><span class="w">
</span></code></pre></div></div>
<p>Indeed, it does work for these 3 types of objects.</p>
<p>Now that we know that the “<em>expected</em>” part of the assertion can be comprised of multiple values, this enables complex or specialized assertions. Let’s try a more complex assertion operator which may be useful in infrastructure validation scenarios.</p>
<h2 id="asserting-that-an-ip-address-is-in-a-given-subnet">Asserting that an IP address is in a given subnet</h2>
<p>We are going to write a new function named <code class="language-plaintext highlighter-rouge">BeInSubnet</code> containing logic to check if an IPv4 address is in the same subnet as a specified address (this can be the network ID but it doesn’t have to be) and a specified subnet mask. We’ll add it to our existing <strong>CustomAssertions</strong> module.</p>
<p>Here is the function :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kr">Function</span><span class="w"> </span><span class="nf">BeInSubnet</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="cm"><#
</span><span class="cs">.SYNOPSIS</span><span class="cm">
Tests whether an IPv4 address in the same subnet as a given address with a given subnet mask.
#></span><span class="w">
</span><span class="p">[</span><span class="kt">CmdletBinding</span><span class="p">()]</span><span class="w">
</span><span class="kr">Param</span><span class="p">(</span><span class="w">
</span><span class="nv">$ActualValue</span><span class="p">,</span><span class="w">
</span><span class="nv">$Network</span><span class="p">,</span><span class="w">
</span><span class="nv">$Mask</span><span class="p">,</span><span class="w">
</span><span class="p">[</span><span class="kt">switch</span><span class="p">]</span><span class="nv">$Negate</span><span class="w">
</span><span class="p">)</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$ActualValue</span><span class="w"> </span><span class="o">-isnot</span><span class="w"> </span><span class="p">[</span><span class="kt">ipaddress</span><span class="p">]</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$ActualValue</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$ActualValue</span><span class="w"> </span><span class="o">-as</span><span class="w"> </span><span class="p">[</span><span class="kt">ipaddress</span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$Network</span><span class="w"> </span><span class="o">-isnot</span><span class="w"> </span><span class="p">[</span><span class="kt">ipaddress</span><span class="p">]</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$Network</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$Network</span><span class="w"> </span><span class="o">-as</span><span class="w"> </span><span class="p">[</span><span class="kt">ipaddress</span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$Mask</span><span class="w"> </span><span class="o">-isnot</span><span class="w"> </span><span class="p">[</span><span class="kt">ipaddress</span><span class="p">]</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$Mask</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$Mask</span><span class="w"> </span><span class="o">-as</span><span class="w"> </span><span class="p">[</span><span class="kt">ipaddress</span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nv">$ActualNetworkBinary</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$ActualValue</span><span class="o">.</span><span class="nf">Address</span><span class="w"> </span><span class="o">-band</span><span class="w"> </span><span class="nv">$Mask</span><span class="o">.</span><span class="nf">Address</span><span class="w">
</span><span class="nv">$ExpectedNetworkBinary</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$Network</span><span class="o">.</span><span class="nf">Address</span><span class="w"> </span><span class="o">-band</span><span class="w"> </span><span class="nv">$Mask</span><span class="o">.</span><span class="nf">Address</span><span class="w">
</span><span class="p">[</span><span class="kt">bool</span><span class="p">]</span><span class="nv">$Pass</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$ActualNetworkBinary</span><span class="w"> </span><span class="o">-eq</span><span class="w"> </span><span class="nv">$ExpectedNetworkBinary</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$Negate</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nv">$Pass</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">-not</span><span class="p">(</span><span class="nv">$Pass</span><span class="p">)</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="o">-not</span><span class="p">(</span><span class="nv">$Pass</span><span class="p">)</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$ActualSubnetString</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="nv">$ActualNetworkBinary</span><span class="w"> </span><span class="o">-as</span><span class="w"> </span><span class="p">[</span><span class="kt">ipaddress</span><span class="p">])</span><span class="o">.</span><span class="nf">IPAddressToString</span><span class="w">
</span><span class="nx">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$Negate</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$FailureMessage</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'Expected: address {{{0}}} to be outside subnet {{{1}}} with mask {{{2}}} but was within it.'</span><span class="w"> </span><span class="nt">-f</span><span class="w"> </span><span class="nv">$ActualValue</span><span class="o">.</span><span class="nf">IPAddressToString</span><span class="p">,</span><span class="w"> </span><span class="nv">$Network</span><span class="o">.</span><span class="nf">IPAddressToString</span><span class="p">,</span><span class="w"> </span><span class="nv">$Mask</span><span class="o">.</span><span class="nf">IPAddressToString</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nx">Else</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$FailureMessage</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'Expected: address {{{0}}} to be in subnet {{{1}}} with mask {{{2}}} but was in subnet {{{3}}}.'</span><span class="w"> </span><span class="nt">-f</span><span class="w"> </span><span class="nv">$ActualValue</span><span class="o">.</span><span class="nf">IPAddressToString</span><span class="p">,</span><span class="w"> </span><span class="nv">$Network</span><span class="o">.</span><span class="nf">IPAddressToString</span><span class="p">,</span><span class="w"> </span><span class="nv">$Mask</span><span class="o">.</span><span class="nf">IPAddressToString</span><span class="p">,</span><span class="w"> </span><span class="nv">$ActualSubnetString</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nv">$ObjProperties</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">@{</span><span class="w">
</span><span class="nx">Succeeded</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$Pass</span><span class="w">
</span><span class="nx">FailureMessage</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$FailureMessage</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">return</span><span class="w"> </span><span class="nf">New-Object</span><span class="w"> </span><span class="nx">PSObject</span><span class="w"> </span><span class="nt">-Property</span><span class="w"> </span><span class="nv">$ObjProperties</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>The function’s parameters don’t enforce a specific <code class="language-plaintext highlighter-rouge">[type]</code>, this is to make the function more flexible. That way, the assertions using this operator will be able to pass <code class="language-plaintext highlighter-rouge">[string]</code> or <code class="language-plaintext highlighter-rouge">[ipaddress]</code> objects into it.</p>
<p>This is why we check the type of each value passed via the function’s parameters <code class="language-plaintext highlighter-rouge">$ActualValue</code>, <code class="language-plaintext highlighter-rouge">$Network</code>, <code class="language-plaintext highlighter-rouge">$Mask</code> and convert them to the type <code class="language-plaintext highlighter-rouge">[ipaddress]</code> for further manipulation.</p>
<p>The rest of the function is fairly similar to our previous custom operator function.</p>
<p>Then, we create a tests script (<code class="language-plaintext highlighter-rouge">BeInSubnet.Tests.ps1</code>) to verify that our <code class="language-plaintext highlighter-rouge">BeInSubnet</code> custom assertion operator behaves as expected :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">Import-Module</span><span class="w"> </span><span class="s2">"</span><span class="bp">$PSScriptRoot</span><span class="s2">\CustomAssertions.psm1"</span><span class="w"> </span><span class="nt">-Force</span><span class="w">
</span><span class="nf">Add-AssertionOperator</span><span class="w"> </span><span class="nt">-Name</span><span class="w"> </span><span class="s1">'BeInSubnet'</span><span class="w"> </span><span class="nt">-Test</span><span class="w"> </span><span class="nv">$</span><span class="nn">Function</span><span class="p">:</span><span class="nv">BeInSubnet</span><span class="w">
</span><span class="nf">Describe</span><span class="w"> </span><span class="s1">'BeInSubnet assertions'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">Context</span><span class="w"> </span><span class="s1">'Assertions on [string] objects'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">It</span><span class="w"> </span><span class="s1">'"10.1.5.193" should be in the same subnet as "10.1.5.0" with mask "255.255.255.0"'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="s1">'10.1.5.193'</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Should</span><span class="w"> </span><span class="nt">-BeInSubnet</span><span class="w"> </span><span class="nt">-Network</span><span class="w"> </span><span class="s1">'10.1.5.0'</span><span class="w"> </span><span class="nt">-Mask</span><span class="w"> </span><span class="s1">'255.255.255.0'</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nf">It</span><span class="w"> </span><span class="s1">'"10.1.5.0" should be in the same subnet as "10.1.5.0" with mask "255.255.255.0"'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="s1">'10.1.5.0'</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Should</span><span class="w"> </span><span class="nt">-BeInSubnet</span><span class="w"> </span><span class="nt">-Network</span><span class="w"> </span><span class="s1">'10.1.5.0'</span><span class="w"> </span><span class="nt">-Mask</span><span class="w"> </span><span class="s1">'255.255.255.0'</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nf">It</span><span class="w"> </span><span class="s1">'"10.1.5.193" should be in the same subnet as "10.1.5.0" with mask "255.0.0.0"'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="s1">'10.1.5.193'</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Should</span><span class="w"> </span><span class="nt">-BeInSubnet</span><span class="w"> </span><span class="nt">-Network</span><span class="w"> </span><span class="s1">'10.1.5.0'</span><span class="w"> </span><span class="nt">-Mask</span><span class="w"> </span><span class="s1">'255.0.0.0'</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nf">It</span><span class="w"> </span><span class="s1">'"10.1.5.193" should not be in the same subnet as "10.1.5.0" with mask "255.255.255.128"'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="s1">'10.1.5.193'</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Should</span><span class="w"> </span><span class="o">-Not</span><span class="w"> </span><span class="nt">-BeInSubnet</span><span class="w"> </span><span class="nt">-Network</span><span class="w"> </span><span class="s1">'10.1.5.0'</span><span class="w"> </span><span class="nt">-Mask</span><span class="w"> </span><span class="s1">'255.255.255.128'</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nf">It</span><span class="w"> </span><span class="s1">'Should fail (to verify the failure message)'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="s1">'10.1.5.193'</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Should</span><span class="w"> </span><span class="nt">-BeInSubnet</span><span class="w"> </span><span class="nt">-Network</span><span class="w"> </span><span class="s1">'10.1.5.0'</span><span class="w"> </span><span class="nt">-Mask</span><span class="w"> </span><span class="s1">'255.255.255.128'</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nf">Context</span><span class="w"> </span><span class="s1">'Assertions on [ipaddress] objects'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$Value</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'10.1.5.193'</span><span class="w"> </span><span class="o">-as</span><span class="w"> </span><span class="p">[</span><span class="kt">ipaddress</span><span class="p">]</span><span class="w">
</span><span class="nv">$Network</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'10.1.5.0'</span><span class="w"> </span><span class="o">-as</span><span class="w"> </span><span class="p">[</span><span class="kt">ipaddress</span><span class="p">]</span><span class="w">
</span><span class="nv">$SubnetMask</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'255.255.255.0'</span><span class="w"> </span><span class="o">-as</span><span class="w"> </span><span class="p">[</span><span class="kt">ipaddress</span><span class="p">]</span><span class="w">
</span><span class="nv">$LargeSubnetMask</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'255.0.0.0'</span><span class="w"> </span><span class="o">-as</span><span class="w"> </span><span class="p">[</span><span class="kt">ipaddress</span><span class="p">]</span><span class="w">
</span><span class="nv">$SmallSubnetMask</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'255.255.255.128'</span><span class="w"> </span><span class="o">-as</span><span class="w"> </span><span class="p">[</span><span class="kt">ipaddress</span><span class="p">]</span><span class="w">
</span><span class="nf">It</span><span class="w"> </span><span class="s1">'"10.1.5.193" should be in the same subnet as "10.1.5.0" with mask "255.255.255.0"'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$Value</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Should</span><span class="w"> </span><span class="nt">-BeInSubnet</span><span class="w"> </span><span class="nt">-Network</span><span class="w"> </span><span class="nv">$Network</span><span class="w"> </span><span class="nt">-Mask</span><span class="w"> </span><span class="nv">$SubnetMask</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nf">It</span><span class="w"> </span><span class="s1">'"10.1.5.0" should be in the same subnet as "10.1.5.0" with mask "255.255.255.0"'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$Network</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Should</span><span class="w"> </span><span class="nt">-BeInSubnet</span><span class="w"> </span><span class="nt">-Network</span><span class="w"> </span><span class="nv">$Network</span><span class="w"> </span><span class="nt">-Mask</span><span class="w"> </span><span class="nv">$SubnetMask</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nf">It</span><span class="w"> </span><span class="s1">'"10.1.5.193" should be in the same subnet as "10.1.5.0" with mask "255.0.0.0"'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$Value</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Should</span><span class="w"> </span><span class="nt">-BeInSubnet</span><span class="w"> </span><span class="nt">-Network</span><span class="w"> </span><span class="nv">$Network</span><span class="w"> </span><span class="nt">-Mask</span><span class="w"> </span><span class="nv">$LargeSubnetMask</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nf">It</span><span class="w"> </span><span class="s1">'"10.1.5.193" should not be in the same subnet as "10.1.5.0" with mask "255.255.255.128"'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$Value</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Should</span><span class="w"> </span><span class="o">-Not</span><span class="w"> </span><span class="nt">-BeInSubnet</span><span class="w"> </span><span class="nt">-Network</span><span class="w"> </span><span class="nv">$Network</span><span class="w"> </span><span class="nt">-Mask</span><span class="w"> </span><span class="nv">$SmallSubnetMask</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nf">It</span><span class="w"> </span><span class="s1">'Should fail (to verify the failure message)'</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$Value</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Should</span><span class="w"> </span><span class="nt">-BeInSubnet</span><span class="w"> </span><span class="nt">-Network</span><span class="w"> </span><span class="nv">$Network</span><span class="w"> </span><span class="nt">-Mask</span><span class="w"> </span><span class="nv">$SmallSubnetMask</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>The tests are clean and readable. Now, imagine if we had to do the same assertions with the built-in <code class="language-plaintext highlighter-rouge">Should</code> operators… The tests script would have been cluttered with a large amount of <code class="language-plaintext highlighter-rouge">[ipaddress]</code> manipulation code, unless this code was extracted into a helper function.</p>
<p>Let’s run these tests and check the result :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\</span><span class="err">></span><span class="w"> </span><span class="nv">$SubnetTestsPath</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'C:\CustomAssertions\BeInSubnet.Tests.ps1'</span><span class="w">
</span><span class="nf">C:\</span><span class="err">></span><span class="w"> </span><span class="nx">Invoke-Pester</span><span class="w"> </span><span class="nt">-Script</span><span class="w"> </span><span class="nv">$SubnetTestsPath</span><span class="w">
</span><span class="nf">Executing</span><span class="w"> </span><span class="nx">all</span><span class="w"> </span><span class="nx">tests</span><span class="w"> </span><span class="nx">in</span><span class="w"> </span><span class="s1">'C:\CustomAssertions\BeInSubnet.Tests.ps1'</span><span class="w">
</span><span class="nf">Executing</span><span class="w"> </span><span class="nx">script</span><span class="w"> </span><span class="nx">C:\CustomAssertions\BeInSubnet.Tests.ps1</span><span class="w">
</span><span class="nf">Describing</span><span class="w"> </span><span class="nx">BeInSubnet</span><span class="w"> </span><span class="nx">assertions</span><span class="w">
</span><span class="nf">Context</span><span class="w"> </span><span class="nx">Assertions</span><span class="w"> </span><span class="nx">on</span><span class="w"> </span><span class="p">[</span><span class="kt">string</span><span class="p">]</span><span class="w"> </span><span class="nx">objects</span><span class="w">
</span><span class="p">[</span><span class="o">+</span><span class="p">]</span><span class="w"> </span><span class="s2">"10.1.5.193"</span><span class="w"> </span><span class="nf">should</span><span class="w"> </span><span class="nx">be</span><span class="w"> </span><span class="nx">in</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">same</span><span class="w"> </span><span class="nx">subnet</span><span class="w"> </span><span class="nx">as</span><span class="w"> </span><span class="s2">"10.1.5.0"</span><span class="w"> </span><span class="nx">with</span><span class="w"> </span><span class="nx">mask</span><span class="w"> </span><span class="s2">"255.255.255.0"</span><span class="w"> </span><span class="nx">138ms</span><span class="w">
</span><span class="p">[</span><span class="o">+</span><span class="p">]</span><span class="w"> </span><span class="s2">"10.1.5.0"</span><span class="w"> </span><span class="nf">should</span><span class="w"> </span><span class="nx">be</span><span class="w"> </span><span class="nx">in</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">same</span><span class="w"> </span><span class="nx">subnet</span><span class="w"> </span><span class="nx">as</span><span class="w"> </span><span class="s2">"10.1.5.0"</span><span class="w"> </span><span class="nx">with</span><span class="w"> </span><span class="nx">mask</span><span class="w"> </span><span class="s2">"255.255.255.0"</span><span class="w"> </span><span class="nx">20ms</span><span class="w">
</span><span class="p">[</span><span class="o">+</span><span class="p">]</span><span class="w"> </span><span class="s2">"10.1.5.193"</span><span class="w"> </span><span class="nf">should</span><span class="w"> </span><span class="nx">be</span><span class="w"> </span><span class="nx">in</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">same</span><span class="w"> </span><span class="nx">subnet</span><span class="w"> </span><span class="nx">as</span><span class="w"> </span><span class="s2">"10.1.5.0"</span><span class="w"> </span><span class="nx">with</span><span class="w"> </span><span class="nx">mask</span><span class="w"> </span><span class="s2">"255.0.0.0"</span><span class="w"> </span><span class="nx">23ms</span><span class="w">
</span><span class="p">[</span><span class="o">+</span><span class="p">]</span><span class="w"> </span><span class="s2">"10.1.5.193"</span><span class="w"> </span><span class="nf">should</span><span class="w"> </span><span class="nx">not</span><span class="w"> </span><span class="nx">be</span><span class="w"> </span><span class="nx">in</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">same</span><span class="w"> </span><span class="nx">subnet</span><span class="w"> </span><span class="nx">as</span><span class="w"> </span><span class="s2">"10.1.5.0"</span><span class="w"> </span><span class="nx">with</span><span class="w"> </span><span class="nx">mask</span><span class="w"> </span><span class="s2">"255.255.255.128"</span><span class="w"> </span><span class="nx">23ms</span><span class="w">
</span><span class="p">[</span><span class="nf">-</span><span class="p">]</span><span class="w"> </span><span class="kt">Should</span><span class="w"> </span><span class="kt">fail</span><span class="w"> </span><span class="p">(</span><span class="kt">to</span><span class="w"> </span><span class="kt">verify</span><span class="w"> </span><span class="kt">the</span><span class="w"> </span><span class="kt">failure</span><span class="w"> </span><span class="kt">message</span><span class="p">)</span><span class="w"> </span><span class="mi">27</span><span class="kt">ms</span><span class="w">
</span><span class="kt">Expected</span><span class="p">:</span><span class="w"> </span><span class="kt">address</span><span class="w"> </span><span class="p">{</span><span class="mf">10.1.5.193</span><span class="p">}</span><span class="w"> </span><span class="kt">to</span><span class="w"> </span><span class="kt">be</span><span class="w"> </span><span class="kt">in</span><span class="w"> </span><span class="kt">subnet</span><span class="w"> </span><span class="p">{</span><span class="mf">10.1.5.0</span><span class="p">}</span><span class="w"> </span><span class="kt">with</span><span class="w"> </span><span class="kt">mask</span><span class="w"> </span><span class="p">{</span><span class="mf">255.255.255.128</span><span class="p">}</span><span class="w"> </span><span class="kt">but</span><span class="w"> </span><span class="kt">was</span><span class="w"> </span><span class="kt">in</span><span class="w"> </span><span class="kt">subnet</span><span class="w"> </span><span class="p">{</span><span class="mf">10.1.5.128</span><span class="p">}</span><span class="o">.</span><span class="w">
</span><span class="mi">20</span><span class="p">:</span><span class="w"> </span><span class="s1">'10.1.5.193'</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="kt">Should</span><span class="w"> </span><span class="nt">-BeInSubnet</span><span class="w"> </span><span class="nt">-Network</span><span class="w"> </span><span class="s1">'10.1.5.0'</span><span class="w"> </span><span class="nt">-Mask</span><span class="w"> </span><span class="s1">'255.255.255.128'</span><span class="w">
</span><span class="kt">at</span><span class="w"> </span><span class="kt">Invoke</span><span class="nt">-Assertion</span><span class="p">,</span><span class="w"> </span><span class="nf">C:\Program</span><span class="w"> </span><span class="nx">Files\WindowsPowerShell\Modules\Pester\4.0.8\Functions\Assertions\Should.ps1:</span><span class="w"> </span><span class="nx">line</span><span class="w"> </span><span class="nx">209</span><span class="w">
</span><span class="kt">at</span><span class="w"> </span><span class="err"><</span><span class="kt">ScriptBlock</span><span class="err">></span><span class="p">,</span><span class="w"> </span><span class="nf">C:\CustomAssertions\BeInSubnet.Tests.ps1:</span><span class="w"> </span><span class="nx">line</span><span class="w"> </span><span class="nx">20</span><span class="w">
</span><span class="kt">Context</span><span class="w"> </span><span class="kt">Assertions</span><span class="w"> </span><span class="kt">on</span><span class="w"> </span><span class="p">[</span><span class="kt">ipaddress</span><span class="p">]</span><span class="w"> </span><span class="kt">objects</span><span class="w">
</span><span class="p">[</span><span class="o">+</span><span class="p">]</span><span class="w"> </span><span class="s2">"10.1.5.193"</span><span class="w"> </span><span class="kt">should</span><span class="w"> </span><span class="kt">be</span><span class="w"> </span><span class="kt">in</span><span class="w"> </span><span class="kt">the</span><span class="w"> </span><span class="kt">same</span><span class="w"> </span><span class="kt">subnet</span><span class="w"> </span><span class="kt">as</span><span class="w"> </span><span class="s2">"10.1.5.0"</span><span class="w"> </span><span class="kt">with</span><span class="w"> </span><span class="kt">mask</span><span class="w"> </span><span class="s2">"255.255.255.0"</span><span class="w"> </span><span class="mi">80</span><span class="kt">ms</span><span class="w">
</span><span class="p">[</span><span class="o">+</span><span class="p">]</span><span class="w"> </span><span class="s2">"10.1.5.0"</span><span class="w"> </span><span class="kt">should</span><span class="w"> </span><span class="kt">be</span><span class="w"> </span><span class="kt">in</span><span class="w"> </span><span class="kt">the</span><span class="w"> </span><span class="kt">same</span><span class="w"> </span><span class="kt">subnet</span><span class="w"> </span><span class="kt">as</span><span class="w"> </span><span class="s2">"10.1.5.0"</span><span class="w"> </span><span class="kt">with</span><span class="w"> </span><span class="kt">mask</span><span class="w"> </span><span class="s2">"255.255.255.0"</span><span class="w"> </span><span class="mi">26</span><span class="kt">ms</span><span class="w">
</span><span class="p">[</span><span class="o">+</span><span class="p">]</span><span class="w"> </span><span class="s2">"10.1.5.193"</span><span class="w"> </span><span class="kt">should</span><span class="w"> </span><span class="kt">be</span><span class="w"> </span><span class="kt">in</span><span class="w"> </span><span class="kt">the</span><span class="w"> </span><span class="kt">same</span><span class="w"> </span><span class="kt">subnet</span><span class="w"> </span><span class="kt">as</span><span class="w"> </span><span class="s2">"10.1.5.0"</span><span class="w"> </span><span class="kt">with</span><span class="w"> </span><span class="kt">mask</span><span class="w"> </span><span class="s2">"255.0.0.0"</span><span class="w"> </span><span class="mi">28</span><span class="kt">ms</span><span class="w">
</span><span class="p">[</span><span class="o">+</span><span class="p">]</span><span class="w"> </span><span class="s2">"10.1.5.193"</span><span class="w"> </span><span class="kt">should</span><span class="w"> </span><span class="kt">not</span><span class="w"> </span><span class="kt">be</span><span class="w"> </span><span class="kt">in</span><span class="w"> </span><span class="kt">the</span><span class="w"> </span><span class="kt">same</span><span class="w"> </span><span class="kt">subnet</span><span class="w"> </span><span class="kt">as</span><span class="w"> </span><span class="s2">"10.1.5.0"</span><span class="w"> </span><span class="kt">with</span><span class="w"> </span><span class="kt">mask</span><span class="w"> </span><span class="s2">"255.255.255.128"</span><span class="w"> </span><span class="mi">18</span><span class="kt">ms</span><span class="w">
</span><span class="p">[</span><span class="nf">-</span><span class="p">]</span><span class="w"> </span><span class="kt">Should</span><span class="w"> </span><span class="kt">fail</span><span class="w"> </span><span class="p">(</span><span class="kt">to</span><span class="w"> </span><span class="kt">verify</span><span class="w"> </span><span class="kt">the</span><span class="w"> </span><span class="kt">failure</span><span class="w"> </span><span class="kt">message</span><span class="p">)</span><span class="w"> </span><span class="mi">29</span><span class="kt">ms</span><span class="w">
</span><span class="kt">Expected</span><span class="p">:</span><span class="w"> </span><span class="kt">address</span><span class="w"> </span><span class="p">{</span><span class="mf">10.1.5.193</span><span class="p">}</span><span class="w"> </span><span class="kt">to</span><span class="w"> </span><span class="kt">be</span><span class="w"> </span><span class="kt">in</span><span class="w"> </span><span class="kt">subnet</span><span class="w"> </span><span class="p">{</span><span class="mf">10.1.5.0</span><span class="p">}</span><span class="w"> </span><span class="kt">with</span><span class="w"> </span><span class="kt">mask</span><span class="w"> </span><span class="p">{</span><span class="mf">255.255.255.128</span><span class="p">}</span><span class="w"> </span><span class="kt">but</span><span class="w"> </span><span class="kt">was</span><span class="w"> </span><span class="kt">in</span><span class="w"> </span><span class="kt">subnet</span><span class="w"> </span><span class="p">{</span><span class="mf">10.1.5.128</span><span class="p">}</span><span class="o">.</span><span class="w">
</span><span class="mi">43</span><span class="p">:</span><span class="w"> </span><span class="nv">$Value</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="kt">Should</span><span class="w"> </span><span class="nt">-BeInSubnet</span><span class="w"> </span><span class="nt">-Network</span><span class="w"> </span><span class="nv">$Network</span><span class="w"> </span><span class="nt">-Mask</span><span class="w"> </span><span class="nv">$SmallSubnetMask</span><span class="w">
</span><span class="kt">at</span><span class="w"> </span><span class="kt">Invoke</span><span class="nt">-Assertion</span><span class="p">,</span><span class="w"> </span><span class="nf">C:\Program</span><span class="w"> </span><span class="nx">Files\WindowsPowerShell\Modules\Pester\4.0.8\Functions\Assertions\Should.ps1:</span><span class="w"> </span><span class="nx">line</span><span class="w"> </span><span class="nx">209</span><span class="w">
</span><span class="kt">at</span><span class="w"> </span><span class="err"><</span><span class="kt">ScriptBlock</span><span class="err">></span><span class="p">,</span><span class="w"> </span><span class="nf">C:\CustomAssertions\BeInSubnet.Tests.ps1:</span><span class="w"> </span><span class="nx">line</span><span class="w"> </span><span class="nx">43</span><span class="w">
</span><span class="kt">Tests</span><span class="w"> </span><span class="kt">completed</span><span class="w"> </span><span class="kt">in</span><span class="w"> </span><span class="mi">418</span><span class="kt">ms</span><span class="w">
</span><span class="kt">Tests</span><span class="w"> </span><span class="kt">Passed</span><span class="p">:</span><span class="w"> </span><span class="mi">8</span><span class="p">,</span><span class="w"> </span><span class="kt">Failed</span><span class="p">:</span><span class="w"> </span><span class="mi">2</span><span class="p">,</span><span class="w"> </span><span class="kt">Skipped</span><span class="p">:</span><span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="kt">Pending</span><span class="p">:</span><span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="kt">Inconclusive</span><span class="p">:</span><span class="w"> </span><span class="mi">0</span><span class="w">
</span></code></pre></div></div>
<p>Good, everything works as expected.</p>
<p>Also, we can see that the failure message is quite helpful. It tells us exactly what is going on, with the “<em>actual</em>” values and the “<em>expected</em>” values. This is a major advantage of custom assertions, it enables us to create more meaningful failure messages because these messages can be tailored for specific assertions.</p>
<h2 id="conclusion">Conclusion</h2>
<p>While there are a few quirks to iron out to make this feature fully usable and mature, the ability to extend <strong>Pester</strong>’s assertion operators with simple PowerShell functions is very powerful.</p>
<p>This allows to perform very complex or specialized assertions in our tests while keeping them relatively human-readable and remaining true to <strong>Pester</strong>’s <abbr title="Domain Specific Language">DSL</abbr>. This expands <strong>Pester</strong>’s assertions capabilities, as well as its use cases.</p>
<p>Don’t go too crazy, though. I would advise keeping the complexity of the assertion logic to a minimum, because the more complexity, the more chance that something may not work completely as expected. And we want assertions to be <strong>very reliable</strong>, we want tests to fail because the code under test doesn’t behave as intended, <strong>not the assertion logic</strong>.</p>TheShellNutPester, the awesome PowerShell testing framework has recently introduced a capability which allows us to extend our test assertions. This allows to simplify our tests by abstracting custom or complex assertion logic away from the tests and into separate scripts or modules.Adding a Quality Gate To a PowerShell Release Pipeline With PSCodeHealth2017-10-24T00:00:00+01:002017-10-24T00:00:00+01:00https://mathieubuisson.github.io/pscodehealth-release-pipeline<aside class="sidebar__right">
<nav class="toc">
<header><h4 class="nav__title"><i class="fa fa-file-text"></i> In This Article</h4></header>
<ul class="toc__menu" id="markdown-toc">
<li><a href="#pscodehealths-default-compliance-rules" id="markdown-toc-pscodehealths-default-compliance-rules">PSCodeHealth’s Default Compliance Rules</a></li>
<li><a href="#checking-a-powershell-project-against-pscodehealths-compliance-rules" id="markdown-toc-checking-a-powershell-project-against-pscodehealths-compliance-rules">Checking a PowerShell Project Against PSCodeHealth’s Compliance Rules</a></li>
<li><a href="#customizing-compliance-rules-to-fit-our-requirements" id="markdown-toc-customizing-compliance-rules-to-fit-our-requirements">Customizing Compliance Rules to Fit Our Requirements</a></li>
<li><a href="#using-pscodehealth-as-a-quality-gate-in-a-release-pipeline" id="markdown-toc-using-pscodehealth-as-a-quality-gate-in-a-release-pipeline">Using PSCodeHealth As a Quality Gate In a Release Pipeline</a></li>
<li><a href="#conclusion" id="markdown-toc-conclusion">Conclusion</a></li>
</ul>
</nav>
</aside>
<p>In <a href="https://mathieubuisson.github.io/powershell-code-quality-pscodehealth/">the previous article</a>, we had brief glances at some of <code class="language-plaintext highlighter-rouge">PSCodeHealth</code>’s compliance rules for a few code metrics. In this post, we are going to further cover this feature and how it can be leveraged within a release pipeline.</p>
<p><code class="language-plaintext highlighter-rouge">PSCodeHealth</code> collects quite a few metrics to quantify some aspects of code quality and most of these have <strong>compliance rules</strong> associated to them. The purpose of compliance rules is to tell, for a specific metric, how <em>good</em> or how <em>bad</em> it is.</p>
<h2 id="pscodehealths-default-compliance-rules">PSCodeHealth’s Default Compliance Rules</h2>
<p>First, to view all the compliance rules, we can use the command <code class="language-plaintext highlighter-rouge">Get-PSCodeHealthComplianceRule</code> without any parameter, like so :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\</span><span class="err">></span><span class="w"> </span><span class="nx">Get-PSCodeHealthComplianceRule</span><span class="w">
</span><span class="nf">Metric</span><span class="w"> </span><span class="nx">Name</span><span class="w"> </span><span class="nx">Metric</span><span class="w"> </span><span class="nx">Group</span><span class="w"> </span><span class="nx">Warning</span><span class="w"> </span><span class="nx">Fail</span><span class="w"> </span><span class="nx">Threshold</span><span class="w"> </span><span class="nx">Higher</span><span class="w"> </span><span class="nx">Is</span><span class="w">
</span><span class="nf">Threshold</span><span class="w"> </span><span class="nx">Better</span><span class="w">
</span><span class="nf">-----------</span><span class="w"> </span><span class="nf">------------</span><span class="w"> </span><span class="nf">--------------</span><span class="w"> </span><span class="nf">--------------</span><span class="w"> </span><span class="nf">--------------</span><span class="w">
</span><span class="nx">LinesOfCode</span><span class="w"> </span><span class="nx">PerFunctionMetrics</span><span class="w"> </span><span class="nx">30</span><span class="w"> </span><span class="nx">60</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nx">ScriptAnalyzerFindings</span><span class="w"> </span><span class="nx">PerFunctionMetrics</span><span class="w"> </span><span class="nx">7</span><span class="w"> </span><span class="nx">12</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nx">TestCoverage</span><span class="w"> </span><span class="nx">PerFunctionMetrics</span><span class="w"> </span><span class="nx">80</span><span class="w"> </span><span class="nx">70</span><span class="w"> </span><span class="nx">True</span><span class="w">
</span><span class="nx">CommandsMissed</span><span class="w"> </span><span class="nx">PerFunctionMetrics</span><span class="w"> </span><span class="nx">6</span><span class="w"> </span><span class="nx">12</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nx">Complexity</span><span class="w"> </span><span class="nx">PerFunctionMetrics</span><span class="w"> </span><span class="nx">15</span><span class="w"> </span><span class="nx">30</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nf">MaximumNestingDepth</span><span class="w"> </span><span class="nx">PerFunctionMetrics</span><span class="w"> </span><span class="nx">4</span><span class="w"> </span><span class="nx">8</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nf">LinesOfCodeTotal</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">1000</span><span class="w"> </span><span class="nx">2000</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nf">LinesOfCodeAverage</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">30</span><span class="w"> </span><span class="nx">60</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nf">ScriptAnalyzerFindingsTotal</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">30</span><span class="w"> </span><span class="nx">60</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nf">ScriptAnalyzerErrors</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nx">3</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nf">ScriptAnalyzerWarnings</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">10</span><span class="w"> </span><span class="nx">20</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nf">ScriptAnalyzerInformation</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">20</span><span class="w"> </span><span class="nx">40</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nf">ScriptAnalyzerFindingsAverage</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">7</span><span class="w"> </span><span class="nx">12</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nf">NumberOfFailedTests</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nx">3</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nf">TestsPassRate</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">99</span><span class="w"> </span><span class="nx">97</span><span class="w"> </span><span class="nx">True</span><span class="w">
</span><span class="nf">TestCoverage</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">80</span><span class="w"> </span><span class="nx">70</span><span class="w"> </span><span class="nx">True</span><span class="w">
</span><span class="nf">CommandsMissedTotal</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">200</span><span class="w"> </span><span class="nx">400</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nf">ComplexityAverage</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">15</span><span class="w"> </span><span class="nx">30</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nf">ComplexityHighest</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">30</span><span class="w"> </span><span class="nx">60</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nf">NestingDepthAverage</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">4</span><span class="w"> </span><span class="nx">8</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nf">NestingDepthHighest</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">8</span><span class="w"> </span><span class="nx">16</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span></code></pre></div></div>
<p>For example, regarding the <strong>NestingDepthAverage</strong> metric, the lower the value, the better it is.<br />
The <em>warning</em> threshold for this metric is 4, so a value of 4 or lower outputs a “<strong>Pass</strong>” compliance result. The <em>fail</em> threshold is 8, so a value between 5 and 8 outputs a “<strong>Warning</strong>” compliance result and a value of 9 or more is a “<strong>Fail</strong>”.</p>
<p>We follow the same principle for the <strong>NestingDepthHighest</strong> metric with 2 differences :</p>
<ul>
<li>Instead of the average, this is the <strong>highest</strong> value across all functions</li>
<li>The threshold values are the double of those for the average metric</li>
</ul>
<p>For more information of each of these metrics and what they attempt to measure, please refer to <a href="http://pscodehealth.readthedocs.io/en/latest/Metrics/">this documentation page</a>.</p>
<p>As we can see above, there are 2 metrics groups : <strong>OverallMetrics</strong> and <strong>PerFunctionMetrics</strong>. Metrics in the <strong>OverallMetrics</strong> group are values calculated by looking at all the analyzed code, so they always have a single value per <code class="language-plaintext highlighter-rouge">PSCodeHealth</code> report. As a result, they have a single compliance result per report.</p>
<p>On the other hand, metrics in the <strong>PerFunctionMetrics</strong> group have a value for every single function in the <code class="language-plaintext highlighter-rouge">PSCodeHealth</code> report. Therefore, compliance rules associated to these metrics can be used to check the compliance of each individual function.</p>
<p>We can filter the compliance rules by metric(s) with the <code class="language-plaintext highlighter-rouge">MetricName</code> parameter, like so :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\</span><span class="err">></span><span class="w"> </span><span class="nx">Get-PSCodeHealthComplianceRule</span><span class="w"> </span><span class="nt">-MetricName</span><span class="w"> </span><span class="s1">'LinesOfCode'</span><span class="p">,</span><span class="s1">'Complexity'</span><span class="p">,</span><span class="s1">'TestCoverage'</span><span class="w">
</span><span class="nf">Metric</span><span class="w"> </span><span class="nx">Name</span><span class="w"> </span><span class="nx">Metric</span><span class="w"> </span><span class="nx">Group</span><span class="w"> </span><span class="nx">Warning</span><span class="w"> </span><span class="nx">Fail</span><span class="w"> </span><span class="nx">Threshold</span><span class="w"> </span><span class="nx">Higher</span><span class="w"> </span><span class="nx">Is</span><span class="w">
</span><span class="nf">Threshold</span><span class="w"> </span><span class="nx">Better</span><span class="w">
</span><span class="nf">-----------</span><span class="w"> </span><span class="nf">------------</span><span class="w"> </span><span class="nf">--------------</span><span class="w"> </span><span class="nf">--------------</span><span class="w"> </span><span class="nf">-------------</span><span class="w">
</span><span class="nx">LinesOfCode</span><span class="w"> </span><span class="nx">PerFunctionMetrics</span><span class="w"> </span><span class="nx">30</span><span class="w"> </span><span class="nx">60</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nx">TestCoverage</span><span class="w"> </span><span class="nx">PerFunctionMetrics</span><span class="w"> </span><span class="nx">80</span><span class="w"> </span><span class="nx">70</span><span class="w"> </span><span class="nx">True</span><span class="w">
</span><span class="nx">Complexity</span><span class="w"> </span><span class="nx">PerFunctionMetrics</span><span class="w"> </span><span class="nx">15</span><span class="w"> </span><span class="nx">30</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nx">TestCoverage</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">80</span><span class="w"> </span><span class="nx">70</span><span class="w"> </span><span class="nx">True</span><span class="w">
</span></code></pre></div></div>
<p>Note that we got 2 compliance rules for <strong>TestCoverage</strong>. This is because this metric is measured at 2 different levels :</p>
<ul>
<li>For the entire set of analyzed PowerShell files (<strong>OverallMetrics</strong> group)</li>
<li>For each individual function (<strong>PerFunctionMetrics</strong> group)</li>
</ul>
<p>There is no need to memorize all the possible values for this <code class="language-plaintext highlighter-rouge">MetricName</code> parameter, they are discoverable via tab-completion.</p>
<h2 id="checking-a-powershell-project-against-pscodehealths-compliance-rules">Checking a PowerShell Project Against PSCodeHealth’s Compliance Rules</h2>
<p>Now that we know what <code class="language-plaintext highlighter-rouge">PSCodeHealth</code> compliance rules are, it’s time to run them against some PowerShell code. Just like in the <a href="https://mathieubuisson.github.io/powershell-code-quality-pscodehealth/">the previous article</a>, we’ll use the code in <a href="https://github.com/MathieuBuisson/PSGithubSearch">PSGithubSearch</a> as an example.</p>
<p>The command which does this is <code class="language-plaintext highlighter-rouge">Test-PSCodeHealthCompliance</code>. It expects an object with the type <code class="language-plaintext highlighter-rouge">[PSCodeHealth.Overall.HealthReport]</code>, so we create the health report first, and we pass it to <code class="language-plaintext highlighter-rouge">Test-PSCodeHealthCompliance</code> via its <code class="language-plaintext highlighter-rouge">HealthReport</code> parameter :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\PSGithubSearch</span><span class="err">></span><span class="w"> </span><span class="nv">$Params</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">@{</span><span class="nx">Path</span><span class="o">=</span><span class="s1">'.\PSGithubSearch\'</span><span class="err">;</span><span class="w"> </span><span class="nx">TestsPath</span><span class="o">=</span><span class="s1">'.\Tests\Unit\'</span><span class="p">}</span><span class="w">
</span><span class="nf">C:\PSGithubSearch</span><span class="err">></span><span class="w"> </span><span class="nv">$HealthReport</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">Invoke-PSCodeHealth</span><span class="w"> </span><span class="err">@</span><span class="nx">Params</span><span class="w">
</span><span class="nf">C:\PSGithubSearch</span><span class="err">></span><span class="w"> </span><span class="nx">Test-PSCodeHealthCompliance</span><span class="w"> </span><span class="nt">-HealthReport</span><span class="w"> </span><span class="nv">$HealthReport</span><span class="w">
</span><span class="nf">Metric</span><span class="w"> </span><span class="nx">Name</span><span class="w"> </span><span class="nx">Warning</span><span class="w"> </span><span class="nx">Fail</span><span class="w"> </span><span class="nx">Threshold</span><span class="w"> </span><span class="nx">Value</span><span class="w"> </span><span class="nx">Result</span><span class="w">
</span><span class="nf">Threshold</span><span class="w">
</span><span class="nf">-----------</span><span class="w"> </span><span class="nf">---------------</span><span class="w"> </span><span class="nf">--------------</span><span class="w"> </span><span class="nf">-----</span><span class="w"> </span><span class="nf">------</span><span class="w">
</span><span class="nx">LinesOfCode</span><span class="w"> </span><span class="nx">30</span><span class="w"> </span><span class="nx">60</span><span class="w"> </span><span class="nx">173</span><span class="w"> </span><span class="nx">Fail</span><span class="w">
</span><span class="nx">ScriptAnalyzerFindings</span><span class="w"> </span><span class="nx">7</span><span class="w"> </span><span class="nx">12</span><span class="w"> </span><span class="nx">0</span><span class="w"> </span><span class="nx">Pass</span><span class="w">
</span><span class="nx">TestCoverage</span><span class="w"> </span><span class="nx">80</span><span class="w"> </span><span class="nx">70</span><span class="w"> </span><span class="nx">41.67</span><span class="w"> </span><span class="nx">Fail</span><span class="w">
</span><span class="nx">CommandsMissed</span><span class="w"> </span><span class="nx">6</span><span class="w"> </span><span class="nx">12</span><span class="w"> </span><span class="nx">28</span><span class="w"> </span><span class="nx">Fail</span><span class="w">
</span><span class="nx">Complexity</span><span class="w"> </span><span class="nx">15</span><span class="w"> </span><span class="nx">30</span><span class="w"> </span><span class="nx">19</span><span class="w"> </span><span class="nx">Warning</span><span class="w">
</span><span class="nf">MaximumNestingDepth</span><span class="w"> </span><span class="nx">4</span><span class="w"> </span><span class="nx">8</span><span class="w"> </span><span class="nx">3</span><span class="w"> </span><span class="nx">Pass</span><span class="w">
</span><span class="nf">LinesOfCodeTotal</span><span class="w"> </span><span class="nx">1000</span><span class="w"> </span><span class="nx">2000</span><span class="w"> </span><span class="nx">552</span><span class="w"> </span><span class="nx">Pass</span><span class="w">
</span><span class="nf">LinesOfCodeAverage</span><span class="w"> </span><span class="nx">30</span><span class="w"> </span><span class="nx">60</span><span class="w"> </span><span class="nx">110.4</span><span class="w"> </span><span class="nx">Fail</span><span class="w">
</span><span class="nf">ScriptAnalyzerFindingsTotal</span><span class="w"> </span><span class="nx">30</span><span class="w"> </span><span class="nx">60</span><span class="w"> </span><span class="nx">0</span><span class="w"> </span><span class="nx">Pass</span><span class="w">
</span><span class="nf">ScriptAnalyzerErrors</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nx">3</span><span class="w"> </span><span class="nx">0</span><span class="w"> </span><span class="nx">Pass</span><span class="w">
</span><span class="nf">ScriptAnalyzerWarnings</span><span class="w"> </span><span class="nx">10</span><span class="w"> </span><span class="nx">20</span><span class="w"> </span><span class="nx">0</span><span class="w"> </span><span class="nx">Pass</span><span class="w">
</span><span class="nf">ScriptAnalyzerInformation</span><span class="w"> </span><span class="nx">20</span><span class="w"> </span><span class="nx">40</span><span class="w"> </span><span class="nx">0</span><span class="w"> </span><span class="nx">Pass</span><span class="w">
</span><span class="nf">ScriptAnalyzerFindingsAverage</span><span class="w"> </span><span class="nx">7</span><span class="w"> </span><span class="nx">12</span><span class="w"> </span><span class="nx">0</span><span class="w"> </span><span class="nx">Pass</span><span class="w">
</span><span class="nf">NumberOfFailedTests</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nx">3</span><span class="w"> </span><span class="nx">0</span><span class="w"> </span><span class="nx">Pass</span><span class="w">
</span><span class="nf">TestsPassRate</span><span class="w"> </span><span class="nx">99</span><span class="w"> </span><span class="nx">97</span><span class="w"> </span><span class="nx">100</span><span class="w"> </span><span class="nx">Pass</span><span class="w">
</span><span class="nf">TestCoverage</span><span class="w"> </span><span class="nx">80</span><span class="w"> </span><span class="nx">70</span><span class="w"> </span><span class="nx">77.1</span><span class="w"> </span><span class="nx">Warning</span><span class="w">
</span><span class="nf">CommandsMissedTotal</span><span class="w"> </span><span class="nx">200</span><span class="w"> </span><span class="nx">400</span><span class="w"> </span><span class="nx">60</span><span class="w"> </span><span class="nx">Pass</span><span class="w">
</span><span class="nf">ComplexityAverage</span><span class="w"> </span><span class="nx">15</span><span class="w"> </span><span class="nx">30</span><span class="w"> </span><span class="nx">11.6</span><span class="w"> </span><span class="nx">Pass</span><span class="w">
</span><span class="nf">ComplexityHighest</span><span class="w"> </span><span class="nx">30</span><span class="w"> </span><span class="nx">60</span><span class="w"> </span><span class="nx">19</span><span class="w"> </span><span class="nx">Pass</span><span class="w">
</span><span class="nf">NestingDepthAverage</span><span class="w"> </span><span class="nx">4</span><span class="w"> </span><span class="nx">8</span><span class="w"> </span><span class="nx">2.2</span><span class="w"> </span><span class="nx">Pass</span><span class="w">
</span><span class="nf">NestingDepthHighest</span><span class="w"> </span><span class="nx">8</span><span class="w"> </span><span class="nx">16</span><span class="w"> </span><span class="nx">3</span><span class="w"> </span><span class="nx">Pass</span><span class="w">
</span></code></pre></div></div>
<p>So we have a few “<strong>Fails</strong>”, but we knew that already because we saw red-colored items in <a href="https://mathieubuisson.github.io/assets/html/healthreport.html">the HTML report</a>. Indeed, the items color-coding is based on their compliance result :</p>
<ul>
<li>Red corresponds to a <strong>Fail</strong> compliance result</li>
<li>Yellow corresponds to a <strong>Warning</strong> compliance result</li>
<li>Green corresponds to a <strong>Pass</strong> compliance result</li>
</ul>
<p>You might be wondering why we have 2 instances of <strong>TestCoverage</strong> with different values. Here is the first half of the answer :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\PSGithubSearch</span><span class="err">></span><span class="w"> </span><span class="nx">Test-PSCodeHealthCompliance</span><span class="w"> </span><span class="nt">-HealthReport</span><span class="w"> </span><span class="nv">$HealthReport</span><span class="w"> </span><span class="nt">-SettingsGroup</span><span class="w"> </span><span class="s1">'OverallMetrics'</span><span class="w"> </span><span class="nt">-MetricName</span><span class="w"> </span><span class="s1">'TestCoverage'</span><span class="w">
</span><span class="nf">Metric</span><span class="w"> </span><span class="nx">Name</span><span class="w"> </span><span class="nx">Warning</span><span class="w"> </span><span class="nx">Fail</span><span class="w"> </span><span class="nx">Threshold</span><span class="w"> </span><span class="nx">Value</span><span class="w"> </span><span class="nx">Result</span><span class="w">
</span><span class="nf">Threshold</span><span class="w">
</span><span class="nf">-----------</span><span class="w"> </span><span class="nf">---------------</span><span class="w"> </span><span class="nf">--------------</span><span class="w"> </span><span class="nf">-----</span><span class="w"> </span><span class="nf">------</span><span class="w">
</span><span class="nx">TestCoverage</span><span class="w"> </span><span class="nx">80</span><span class="w"> </span><span class="nx">70</span><span class="w"> </span><span class="nx">77.1</span><span class="w"> </span><span class="nx">Warning</span><span class="w">
</span><span class="nx">C:\PSGithubSearch</span><span class="err">></span><span class="w"> </span><span class="nx">Test-PSCodeHealthCompliance</span><span class="w"> </span><span class="nt">-HealthReport</span><span class="w"> </span><span class="nv">$HealthReport</span><span class="w"> </span><span class="nt">-SettingsGroup</span><span class="w"> </span><span class="s1">'PerFunctionMetrics'</span><span class="w"> </span><span class="nt">-MetricName</span><span class="w"> </span><span class="s1">'TestCoverage'</span><span class="w">
</span><span class="nf">Metric</span><span class="w"> </span><span class="nx">Name</span><span class="w"> </span><span class="nx">Warning</span><span class="w"> </span><span class="nx">Fail</span><span class="w"> </span><span class="nx">Threshold</span><span class="w"> </span><span class="nx">Value</span><span class="w"> </span><span class="nx">Result</span><span class="w">
</span><span class="nf">Threshold</span><span class="w">
</span><span class="nf">-----------</span><span class="w"> </span><span class="nf">---------------</span><span class="w"> </span><span class="nf">--------------</span><span class="w"> </span><span class="nf">-----</span><span class="w"> </span><span class="nf">------</span><span class="w">
</span><span class="nx">TestCoverage</span><span class="w"> </span><span class="nx">80</span><span class="w"> </span><span class="nx">70</span><span class="w"> </span><span class="nx">41.67</span><span class="w"> </span><span class="nx">Fail</span><span class="w">
</span></code></pre></div></div>
<p>The first one is the <strong>overall</strong> test coverage and the second one the test coverage of one of the function in the project. When testing compliance against a metric in the <strong>PerFunctionMetrics</strong> group, the value from the worst function is retained. Here, <code class="language-plaintext highlighter-rouge">41.67</code> is the value from the function which has the lowest test coverage :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\PSGithubSearch</span><span class="err">></span><span class="w"> </span><span class="nv">$HealthReport</span><span class="o">.</span><span class="nf">FunctionHealthRecords</span><span class="w"> </span><span class="o">|</span><span class="w">
</span><span class="err">>></span><span class="w"> </span><span class="nf">Select-Object</span><span class="w"> </span><span class="nt">-ExpandProperty</span><span class="w"> </span><span class="s1">'TestCoverage'</span><span class="w"> </span><span class="o">|</span><span class="w">
</span><span class="err">>></span><span class="w"> </span><span class="nf">Sort-Object</span><span class="w">
</span><span class="mf">41.67</span><span class="w">
</span><span class="mf">69.57</span><span class="w">
</span><span class="mf">79.63</span><span class="w">
</span><span class="mf">85.42</span><span class="w">
</span><span class="mf">87.5</span><span class="w">
</span></code></pre></div></div>
<p>In case we don’t care about the compliance results details for every single metric and we just want a overall compliance result, we can use the <code class="language-plaintext highlighter-rouge">Summary</code> parameter :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\PSGithubSearch</span><span class="err">></span><span class="w"> </span><span class="nx">Test-PSCodeHealthCompliance</span><span class="w"> </span><span class="nt">-HealthReport</span><span class="w"> </span><span class="nv">$HealthReport</span><span class="w"> </span><span class="nt">-Summary</span><span class="w">
</span><span class="nf">Fail</span><span class="w">
</span></code></pre></div></div>
<p>The summary result is the worst compliance result across the evaluated compliance rules :</p>
<ul>
<li>If any evaluated metric has a <strong>Fail</strong> result, the summary result is <strong>Fail</strong></li>
<li>If any evaluated metric has a <strong>Warning</strong> result and none has <strong>Fail</strong>, the summary result is <strong>Warning</strong></li>
<li>If all evaluated metrics has a <strong>Pass</strong> result, the summary result is <strong>Pass</strong></li>
</ul>
<h2 id="customizing-compliance-rules-to-fit-our-requirements">Customizing Compliance Rules to Fit Our Requirements</h2>
<h3 id="how-pscodehealths-metrics-thresholds-were-set">How PSCodeHealth’s metrics thresholds were set</h3>
<p>You might think <code class="language-plaintext highlighter-rouge">PSCodeHealth</code> is a very opiniated and rigid tool, with all these rules telling you what is <em>good</em> and <em>bad</em>, based on thresholds set by the 1 guy who wrote the tool. Who the hell is this guy anyway ?<br />
I hear you.</p>
<p>First, I have tried to base compliance rules and metrics thresholds on 3 types of foundations, <strong>in that order</strong> :</p>
<ol>
<li>PowerShell community consensus (like in the <a href="https://github.com/PoshCode/PowerShellPracticeAndStyle">Best Practices and Style Guide</a>)</li>
<li>Practices/thresholds from reference PowerShell projects (like in the <a href="https://github.com/PowerShell/DscResources/blob/master/HighQualityModuleGuidelines.md">High Quality Module Guidelines</a>)</li>
<li>Metrics thresholds from <a href="https://www.ndepend.com/docs/code-metrics">other code quality tools</a> (which don’t support PowerShell and are generally geared towards Java, C#, Python…)</li>
</ol>
<p>The problem is that #1 was often out of the question because many of the metrics collected <code class="language-plaintext highlighter-rouge">PSCodeHealth</code> have never been measured or even discussed by the PowerShell Community. #2 being extremely rare, I had to resort to #3 most of the time. And this may be <em>okay</em> because :</p>
<blockquote>
<p>Dear PowerShell, you’re not <strong>that</strong> special</p>
</blockquote>
<p>It turns out that communities for many different languages have come up with very similar consensuses and metrics thresholds. If something holds true across many different languages, why would there be an exception for PowerShell ?</p>
<p>That said, these rules, just like the <code class="language-plaintext highlighter-rouge">PSScriptAnalyzer</code> rules, should be based on input from the PowerShell community. So I invite you to <a href="https://github.com/MathieuBuisson/PSCodeHealth/issues">open issues</a> so that we can open discussions, form consensuses and improve the tool.</p>
<p>Second, we can override <code class="language-plaintext highlighter-rouge">PSCodeHealth</code>’s default metrics thresholds with user-defined ones.</p>
<h3 id="customizing-metrics-thresholds">Customizing metrics thresholds</h3>
<p>Some PowerShell projects may have different requirements depending on their nature. It may be harder to reach a given test coverage for a project, not because it is poorly coded, but because it is altering external resources (a database for example). In this case, the <strong>Warning</strong> threshold (80%) and the <strong>Fail</strong> threshold (70%) may need to be adjusted.</p>
<p>The default compliance rules and their thresholds are stored in the file <code class="language-plaintext highlighter-rouge">PSCodeHealthSettings.json</code> in the module root. To customize them, it is NOT recommended to modify this file, but to create a new JSON file containing the rules we need to override.</p>
<p>For example, let’s say that for <code class="language-plaintext highlighter-rouge">PSGithubSearch</code>, we decide that we want to override the following metrics thresholds :</p>
<ul>
<li>OverallMetrics group :
<ul>
<li>LinesOfCodeAverage :
<ul>
<li>Warning threshold : 115</li>
<li>Fail threshold : 165</li>
</ul>
</li>
<li>TestCoverage :
<ul>
<li>Warning threshold : 75</li>
<li>Fail threshold : 65</li>
</ul>
</li>
</ul>
</li>
</ul>
<p>We create a new JSON file with the following content :</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
</span><span class="nl">"PerFunctionMetrics"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="w">
</span><span class="p">],</span><span class="w">
</span><span class="nl">"OverallMetrics"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"LinesOfCodeAverage"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"WarningThreshold"</span><span class="p">:</span><span class="w"> </span><span class="mi">115</span><span class="p">,</span><span class="w">
</span><span class="nl">"FailThreshold"</span><span class="p">:</span><span class="w"> </span><span class="mi">165</span><span class="p">,</span><span class="w">
</span><span class="nl">"HigherIsBetter"</span><span class="p">:</span><span class="w"> </span><span class="kc">false</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"TestCoverage"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"WarningThreshold"</span><span class="p">:</span><span class="w"> </span><span class="mi">75</span><span class="p">,</span><span class="w">
</span><span class="nl">"FailThreshold"</span><span class="p">:</span><span class="w"> </span><span class="mi">65</span><span class="p">,</span><span class="w">
</span><span class="nl">"HigherIsBetter"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>We can name the file however we want and save it wherever we want (the root of the project is a convenient place for use in a release pipeline).<br />
Any metric <strong>not</strong> specified in this file will use the default rule and thresholds.</p>
<p>To view the rules in effect (including custom rules), we run <code class="language-plaintext highlighter-rouge">Get-PSCodeHealthComplianceRule</code> and specify the path of the file containing user-defined rules via the <code class="language-plaintext highlighter-rouge">CustomSettingsPath</code> parameter :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\PSGithubSearch</span><span class="err">></span><span class="w"> </span><span class="nv">$Params</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">@{</span><span class="nx">CustomSettingsPath</span><span class="o">=</span><span class="s1">'.\ProjectRules.json'</span><span class="err">;</span><span class="w"> </span><span class="nx">SettingsGroup</span><span class="o">=</span><span class="s1">'OverallMetrics'</span><span class="p">}</span><span class="w">
</span><span class="nf">C:\PSGithubSearch</span><span class="err">></span><span class="w"> </span><span class="nx">Get-PSCodeHealthComplianceRule</span><span class="w"> </span><span class="err">@</span><span class="nx">Params</span><span class="w">
</span><span class="nf">Metric</span><span class="w"> </span><span class="nx">Name</span><span class="w"> </span><span class="nx">Metric</span><span class="w"> </span><span class="nx">Group</span><span class="w"> </span><span class="nx">Warning</span><span class="w"> </span><span class="nx">Fail</span><span class="w"> </span><span class="nx">Threshold</span><span class="w"> </span><span class="nx">Higher</span><span class="w"> </span><span class="nx">Is</span><span class="w">
</span><span class="nf">Threshold</span><span class="w"> </span><span class="nx">Better</span><span class="w">
</span><span class="nf">-----------</span><span class="w"> </span><span class="nf">------------</span><span class="w"> </span><span class="nf">--------------</span><span class="w"> </span><span class="nf">--------------</span><span class="w"> </span><span class="nf">--------------</span><span class="w">
</span><span class="nx">LinesOfCodeTotal</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">1000</span><span class="w"> </span><span class="nx">2000</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nx">LinesOfCodeAverage</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">115</span><span class="w"> </span><span class="nx">165</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nx">ScriptAnalyzerFindingsTotal</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">30</span><span class="w"> </span><span class="nx">60</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nx">ScriptAnalyzerErrors</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nx">3</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nx">ScriptAnalyzerWarnings</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">10</span><span class="w"> </span><span class="nx">20</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nf">ScriptAnalyzerInformation</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">20</span><span class="w"> </span><span class="nx">40</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nf">ScriptAnalyzerFindingsAverage</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">7</span><span class="w"> </span><span class="nx">12</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nf">NumberOfFailedTests</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">1</span><span class="w"> </span><span class="nx">3</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nf">TestsPassRate</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">99</span><span class="w"> </span><span class="nx">97</span><span class="w"> </span><span class="nx">True</span><span class="w">
</span><span class="nf">TestCoverage</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">75</span><span class="w"> </span><span class="nx">65</span><span class="w"> </span><span class="nx">True</span><span class="w">
</span><span class="nf">CommandsMissedTotal</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">200</span><span class="w"> </span><span class="nx">400</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nf">ComplexityAverage</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">15</span><span class="w"> </span><span class="nx">30</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nf">ComplexityHighest</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">30</span><span class="w"> </span><span class="nx">60</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nf">NestingDepthAverage</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">4</span><span class="w"> </span><span class="nx">8</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nf">NestingDepthHighest</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">8</span><span class="w"> </span><span class="nx">16</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span></code></pre></div></div>
<p>Now, let’s check the compliance of our project code against these customized rules. Remember, earlier we got a <strong>Fail</strong> for <strong>LinesOfCodeAverage</strong> and a <strong>Warning</strong> for <strong>TestCoverage</strong> but now with our custom thresholds, they should both pass :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\PSGithubSearch</span><span class="err">></span><span class="w"> </span><span class="nv">$Params</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">@{</span><span class="nx">CustomSettingsPath</span><span class="o">=</span><span class="s1">'.\ProjectRules.json'</span><span class="err">;</span><span class="w">
</span><span class="err">>></span><span class="w"> </span><span class="nx">SettingsGroup</span><span class="o">=</span><span class="s1">'OverallMetrics'</span><span class="err">;</span><span class="w">
</span><span class="err">>></span><span class="w"> </span><span class="nx">MetricName</span><span class="o">=</span><span class="s1">'LinesOfCodeAverage'</span><span class="p">,</span><span class="s1">'TestCoverage'</span><span class="err">;</span><span class="w">
</span><span class="err">>></span><span class="w"> </span><span class="nx">HealthReport</span><span class="o">=</span><span class="nv">$HealthReport</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="nf">C:\PSGithubSearch</span><span class="err">></span><span class="w"> </span><span class="nx">Test-PSCodeHealthCompliance</span><span class="w"> </span><span class="err">@</span><span class="nx">Params</span><span class="w">
</span><span class="nf">Metric</span><span class="w"> </span><span class="nx">Name</span><span class="w"> </span><span class="nx">Warning</span><span class="w"> </span><span class="nx">Fail</span><span class="w"> </span><span class="nx">Threshold</span><span class="w"> </span><span class="nx">Value</span><span class="w"> </span><span class="nx">Result</span><span class="w">
</span><span class="nf">Threshold</span><span class="w">
</span><span class="nf">-----------</span><span class="w"> </span><span class="nf">---------------</span><span class="w"> </span><span class="nf">--------------</span><span class="w"> </span><span class="nf">-----</span><span class="w"> </span><span class="nf">------</span><span class="w">
</span><span class="nx">LinesOfCodeAverage</span><span class="w"> </span><span class="nx">115</span><span class="w"> </span><span class="nx">165</span><span class="w"> </span><span class="nx">110.4</span><span class="w"> </span><span class="nx">Pass</span><span class="w">
</span><span class="nx">TestCoverage</span><span class="w"> </span><span class="nx">75</span><span class="w"> </span><span class="nx">65</span><span class="w"> </span><span class="nx">77.1</span><span class="w"> </span><span class="nx">Pass</span><span class="w">
</span></code></pre></div></div>
<p>We can see that the metrics thresholds are those from our <code class="language-plaintext highlighter-rouge">ProjectRules.json</code> file and that both metrics get a <strong>Pass</strong>.</p>
<h2 id="using-pscodehealth-as-a-quality-gate-in-a-release-pipeline">Using PSCodeHealth As a Quality Gate In a Release Pipeline</h2>
<p>The release pipeline for the <code class="language-plaintext highlighter-rouge">PSGithubSearch</code> project is triggered <a href="https://ci.appveyor.com/project/MathieuBuisson/psgithubsearch">in AppVeyor</a> by every code change in the GitHub repo. It is driven by the awesome task-based build automation tool : <a href="https://github.com/nightroman/Invoke-Build">Invoke-Build</a>.</p>
<h3 id="tests-related-tasks-in-the-current-pipeline">Tests-related tasks in the current pipeline</h3>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">task</span><span class="w"> </span><span class="nx">Unit_Tests</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">Write-TaskBanner</span><span class="w"> </span><span class="nt">-TaskName</span><span class="w"> </span><span class="nv">$Task</span><span class="o">.</span><span class="nf">Name</span><span class="w">
</span><span class="nv">$UnitTestSettings</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$Settings</span><span class="o">.</span><span class="nf">UnitTestParams</span><span class="w">
</span><span class="nv">$</span><span class="nn">Script</span><span class="p">:</span><span class="nv">UnitTestsResult</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">Invoke-Pester</span><span class="w"> </span><span class="err">@</span><span class="nx">UnitTestSettings</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>This task runs unit tests for the project. Pester output is stored in the variable named <code class="language-plaintext highlighter-rouge">UnitTestsResult</code>. This variable is script scoped because we need to access it from subsequent tasks.</p>
<p>In case you are curious about the arguments for <code class="language-plaintext highlighter-rouge">Invoke-Pester</code> (<code class="language-plaintext highlighter-rouge">$Settings.UnitTestParams</code>), they come from the file <code class="language-plaintext highlighter-rouge">PSGithubSearch.BuildSettings.ps1</code> :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">UnitTestParams</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">@{</span><span class="w">
</span><span class="nx">Script</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'.\Tests\Unit'</span><span class="w">
</span><span class="nx">CodeCoverage</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'.\PSGithubSearch\PSGithubSearch.psm1'</span><span class="w">
</span><span class="nx">OutputFile</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"</span><span class="bp">$PSScriptRoot</span><span class="s2">\BuildOutput\UnitTestsResult.xml"</span><span class="w">
</span><span class="nx">PassThru</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$True</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>This is also telling <code class="language-plaintext highlighter-rouge">Pester</code> to generate a NUnit-style test result file.</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">task</span><span class="w"> </span><span class="nx">Fail_If_Failed_Unit_Test</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">Write-TaskBanner</span><span class="w"> </span><span class="nt">-TaskName</span><span class="w"> </span><span class="nv">$Task</span><span class="o">.</span><span class="nf">Name</span><span class="w">
</span><span class="nv">$FailureMessage</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'{0} Unit test(s) failed. Aborting build'</span><span class="w"> </span><span class="nt">-f</span><span class="w"> </span><span class="nv">$UnitTestsResult</span><span class="o">.</span><span class="nf">FailedCount</span><span class="w">
</span><span class="nx">assert</span><span class="w"> </span><span class="p">(</span><span class="nv">$UnitTestsResult</span><span class="o">.</span><span class="nf">FailedCount</span><span class="w"> </span><span class="o">-eq</span><span class="w"> </span><span class="nx">0</span><span class="p">)</span><span class="w"> </span><span class="nv">$FailureMessage</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>This task ensures the build fails if there is any failed unit test. This is one of those tasks which needs to access the script scoped <code class="language-plaintext highlighter-rouge">$UnitTestsResult</code>. If its <code class="language-plaintext highlighter-rouge">FailedCount</code> property is not 0, it will make the build fail.</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">task</span><span class="w"> </span><span class="nx">Publish_Unit_Tests_Coverage</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">Write-TaskBanner</span><span class="w"> </span><span class="nt">-TaskName</span><span class="w"> </span><span class="nv">$Task</span><span class="o">.</span><span class="nf">Name</span><span class="w">
</span><span class="nv">$Coverage</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">Format-Coverage</span><span class="w"> </span><span class="nt">-PesterResults</span><span class="w"> </span><span class="nv">$UnitTestsResult</span><span class="w"> </span><span class="nt">-CoverallsApiToken</span><span class="w"> </span><span class="nv">$Settings</span><span class="o">.</span><span class="nf">CoverallsKey</span><span class="w"> </span><span class="nt">-BranchName</span><span class="w"> </span><span class="nv">$Settings</span><span class="o">.</span><span class="nf">Branch</span><span class="w">
</span><span class="nx">Publish-Coverage</span><span class="w"> </span><span class="nt">-Coverage</span><span class="w"> </span><span class="nv">$Coverage</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>This task uses the <a href="https://github.com/JanJoris/coveralls">Coveralls module</a> to publish the tests result to <a href="https://coveralls.io/github/MathieuBuisson/PSGithubSearch">coveralls.io</a>.</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">task</span><span class="w"> </span><span class="nx">Upload_Test_Results_To_AppVeyor</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">Write-TaskBanner</span><span class="w"> </span><span class="nt">-TaskName</span><span class="w"> </span><span class="nv">$Task</span><span class="o">.</span><span class="nf">Name</span><span class="w">
</span><span class="nv">$TestResultFiles</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="nf">Get-ChildItem</span><span class="w"> </span><span class="nt">-Path</span><span class="w"> </span><span class="nv">$Settings</span><span class="o">.</span><span class="nf">BuildOutput</span><span class="w"> </span><span class="nt">-Filter</span><span class="w"> </span><span class="s1">'*TestsResult.xml'</span><span class="p">)</span><span class="o">.</span><span class="nf">FullName</span><span class="w">
</span><span class="nx">Foreach</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$TestResultFile</span><span class="w"> </span><span class="nx">in</span><span class="w"> </span><span class="nv">$TestResultFiles</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="s2">"Uploading test result file : </span><span class="nv">$TestResultFile</span><span class="s2">"</span><span class="w">
</span><span class="p">(</span><span class="nf">New-Object</span><span class="w"> </span><span class="s1">'System.Net.WebClient'</span><span class="p">)</span><span class="o">.</span><span class="nf">UploadFile</span><span class="p">(</span><span class="nv">$Settings</span><span class="o">.</span><span class="nf">TestUploadUrl</span><span class="p">,</span><span class="w"> </span><span class="nv">$TestResultFile</span><span class="p">)</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>This task uploads test result file(s) to AppVeyor so that it can integrate the tests results in the build result page :<br />
<img src="https://mathieubuisson.github.io/images/2017-10-24-pscodehealth-release-pipeline-tests.png" alt="AppVeyor Tests tab" /></p>
<p>And finally :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">task</span><span class="w"> </span><span class="nx">Test</span><span class="w"> </span><span class="nx">Unit_Tests</span><span class="p">,</span><span class="w">
</span><span class="nf">Fail_If_Failed_Unit_Test</span><span class="p">,</span><span class="w">
</span><span class="nf">Publish_Unit_Tests_Coverage</span><span class="p">,</span><span class="w">
</span><span class="nf">Upload_Test_Results_To_AppVeyor</span><span class="w">
</span></code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">Test</code> task references all the tasks mentioned earlier. This acts as a logical grouping because whenever this <code class="language-plaintext highlighter-rouge">Test</code> task is invoked, it will invoke all referenced tasks <strong>in the specified order</strong>.</p>
<h3 id="code-analysis-tasks-in-the-current-pipeline">Code analysis tasks in the current pipeline</h3>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">task</span><span class="w"> </span><span class="nx">Analyze</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">Write-TaskBanner</span><span class="w"> </span><span class="nt">-TaskName</span><span class="w"> </span><span class="nv">$Task</span><span class="o">.</span><span class="nf">Name</span><span class="w">
</span><span class="nx">Add-AppveyorTest</span><span class="w"> </span><span class="nt">-Name</span><span class="w"> </span><span class="s1">'Code Analysis'</span><span class="w"> </span><span class="nt">-Outcome</span><span class="w"> </span><span class="nx">Running</span><span class="w">
</span><span class="nv">$AnalyzeSettings</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$Settings</span><span class="o">.</span><span class="nf">AnalyzeParams</span><span class="w">
</span><span class="nv">$</span><span class="nn">Script</span><span class="p">:</span><span class="nv">AnalyzeFindings</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">Invoke-ScriptAnalyzer</span><span class="w"> </span><span class="err">@</span><span class="nx">AnalyzeSettings</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$AnalyzeFindings</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$FindingsString</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$AnalyzeFindings</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Out-String</span><span class="w">
</span><span class="nf">Write-Warning</span><span class="w"> </span><span class="nv">$FindingsString</span><span class="w">
</span><span class="nf">Update-AppveyorTest</span><span class="w"> </span><span class="nt">-Name</span><span class="w"> </span><span class="s1">'Code Analysis'</span><span class="w"> </span><span class="nt">-Outcome</span><span class="w"> </span><span class="nx">Failed</span><span class="w"> </span><span class="nt">-ErrorMessage</span><span class="w"> </span><span class="nv">$FindingsString</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">Else</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">Update-AppveyorTest</span><span class="w"> </span><span class="nt">-Name</span><span class="w"> </span><span class="s1">'Code Analysis'</span><span class="w"> </span><span class="nt">-Outcome</span><span class="w"> </span><span class="nx">Passed</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>This task runs <code class="language-plaintext highlighter-rouge">PSScriptAnalyzer</code> against the project’s code and stores the output in the <code class="language-plaintext highlighter-rouge">AnalyzeFindings</code> variable. This variable is script scoped, which enables other tasks to access it. The commands <code class="language-plaintext highlighter-rouge">Add-AppveyorTest</code> and <code class="language-plaintext highlighter-rouge">Update-AppveyorTest</code> are provided by AppVeyor’s build agent to integrate with the <strong>Tests</strong> tab in the build result page.</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">task</span><span class="w"> </span><span class="nx">Fail_If_Analyze_Findings</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">Write-TaskBanner</span><span class="w"> </span><span class="nt">-TaskName</span><span class="w"> </span><span class="nv">$Task</span><span class="o">.</span><span class="nf">Name</span><span class="w">
</span><span class="nv">$FailureMessage</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'PSScriptAnalyzer found {0} issues. Aborting build'</span><span class="w"> </span><span class="nt">-f</span><span class="w"> </span><span class="nv">$AnalyzeFindings</span><span class="o">.</span><span class="nf">Count</span><span class="w">
</span><span class="nx">assert</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="o">-not</span><span class="p">(</span><span class="nv">$AnalyzeFindings</span><span class="p">)</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="nv">$FailureMessage</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>This task ensure the build fail if <code class="language-plaintext highlighter-rouge">PSScriptAnalyzer</code> outputs anything.</p>
<h3 id="expanding-the-code-analysis-tasks-into-a-full-blown-quality-gate">Expanding the code analysis tasks into a full-blown quality gate</h3>
<p>For the task <code class="language-plaintext highlighter-rouge">Fail_If_Failed_Unit_Test</code>, we could change the assertion to read the <code class="language-plaintext highlighter-rouge">NumberOfFailedTests</code> property of the <code class="language-plaintext highlighter-rouge">PSCodeHealth</code> report, to decide whether the build should fail. But we’ll leave the Tests-related task as they are because this is not where <strong>PSCodeHealth</strong> gives us the most value.</p>
<p>We are going to focus on the code analysis tasks : <code class="language-plaintext highlighter-rouge">Analyze</code> and <code class="language-plaintext highlighter-rouge">Fail_If_Analyze_Findings</code>. We’ll use <strong>PSCodeHealth</strong> to streamline and expand them to cover more than linting. We’ll create a quality gate based on the code metrics that we care about and compliance rules which reflect our project’s requirements.</p>
<p>To leverage <code class="language-plaintext highlighter-rouge">PSCodeHealth</code> in our release pipeline, the first thing to do is to add it as a build dependency in our settings file <code class="language-plaintext highlighter-rouge">PSGithubSearch.BuildSettings.ps1</code> :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$Settings</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">@{</span><span class="w">
</span><span class="nx">Dependency</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="err">@(</span><span class="s1">'Coveralls'</span><span class="p">,</span><span class="s1">'Pester'</span><span class="p">,</span><span class="s1">'PsScriptAnalyzer'</span><span class="p">,</span><span class="s1">'PSCodeHealth'</span><span class="err">)</span><span class="w">
</span><span class="err">...</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>As seen earlier, the <code class="language-plaintext highlighter-rouge">Unit_Tests</code> task stores <code class="language-plaintext highlighter-rouge">Pester</code> output, <strong>including code coverage information</strong>, in a variable accessible from other tasks. <code class="language-plaintext highlighter-rouge">Invoke-PSCodeHealth</code> will reuse it via its <code class="language-plaintext highlighter-rouge">TestsResult</code> parameter, instead of running the tests again.<br />
So the task to generate the <code class="language-plaintext highlighter-rouge">PSCodeHealth</code> report looks like this :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">task</span><span class="w"> </span><span class="nx">Generate_Quality_Report</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">Write-TaskBanner</span><span class="w"> </span><span class="nt">-TaskName</span><span class="w"> </span><span class="nv">$Task</span><span class="o">.</span><span class="nf">Name</span><span class="w">
</span><span class="nv">$CodeHealthSettings</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$Settings</span><span class="o">.</span><span class="nf">CodeHealthParams</span><span class="w">
</span><span class="nv">$</span><span class="nn">Script</span><span class="p">:</span><span class="nv">HealthReport</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">Invoke-PSCodeHealth</span><span class="w"> </span><span class="err">@</span><span class="nx">CodeHealthSettings</span><span class="w"> </span><span class="nt">-TestsResult</span><span class="w"> </span><span class="nv">$UnitTestsResult</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>Again, we store the output into a variable scoped at the script level because we’ll need to access it from a subsequent task.<br />
The arguments passed via <code class="language-plaintext highlighter-rouge">$Settings.CodeHealthParams</code> come from the BuildSettings file :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="w"> </span><span class="nf">CodeHealthParams</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">@{</span><span class="w">
</span><span class="nx">Path</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'.\PSGithubSearch\'</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>First, we’ll replicate what the task <code class="language-plaintext highlighter-rouge">Fail_If_Analyze_Findings</code> did. Basically, this task checked if <code class="language-plaintext highlighter-rouge">PSScriptAnalyzer</code> output was <code class="language-plaintext highlighter-rouge">Null</code>. If not, it ensured that the build failed.</p>
<p>To do that, we could read the <code class="language-plaintext highlighter-rouge">ScriptAnalyzerFindingsTotal</code> property of the code health report. But there is another (probably better) way : translate this requirement into a <strong>PSCodeHealth</strong> compliance rule and add it to our <code class="language-plaintext highlighter-rouge">ProjectRules.json</code> file.</p>
<p>So, we add a rule for the <strong>ScriptAnalyzerFindingsTotal</strong> metric, to consider any value greater than 0 a failure :</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
</span><span class="nl">"PerFunctionMetrics"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="w">
</span><span class="p">],</span><span class="w">
</span><span class="nl">"OverallMetrics"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"LinesOfCodeAverage"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"WarningThreshold"</span><span class="p">:</span><span class="w"> </span><span class="mi">115</span><span class="p">,</span><span class="w">
</span><span class="nl">"FailThreshold"</span><span class="p">:</span><span class="w"> </span><span class="mi">165</span><span class="p">,</span><span class="w">
</span><span class="nl">"HigherIsBetter"</span><span class="p">:</span><span class="w"> </span><span class="kc">false</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"TestCoverage"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"WarningThreshold"</span><span class="p">:</span><span class="w"> </span><span class="mi">75</span><span class="p">,</span><span class="w">
</span><span class="nl">"FailThreshold"</span><span class="p">:</span><span class="w"> </span><span class="mi">65</span><span class="p">,</span><span class="w">
</span><span class="nl">"HigherIsBetter"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"ScriptAnalyzerFindingsTotal"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"WarningThreshold"</span><span class="p">:</span><span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w">
</span><span class="nl">"FailThreshold"</span><span class="p">:</span><span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w">
</span><span class="nl">"HigherIsBetter"</span><span class="p">:</span><span class="w"> </span><span class="kc">false</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">Test-PSCodeHealthCompliance</code> always evaluates a metric against its <strong>FailThreshold</strong> first, so it is fine to have the same value for both thresholds. The compliance result will be <strong>Fail</strong> whenever the value is greater than 0 and the <strong>WarningThreshold</strong> will not be evaluated.</p>
<p>We now have a file containing custom metrics thresholds which are specific to the <code class="language-plaintext highlighter-rouge">PSGithubSearch</code> project and the metrics for the project’s code in <code class="language-plaintext highlighter-rouge">$HealthReport</code>. So we feed both of these into <code class="language-plaintext highlighter-rouge">Test-PSCodeHealthCompliance</code> and it will tells us if the code passes our quality bar.</p>
<p>Just before that, let’s list the metrics we care about :</p>
<ul>
<li><strong>LinesOfCodeAverage</strong> (custom thresholds)</li>
<li><strong>TestCoverage</strong> (custom thresholds)</li>
<li><strong>ScriptAnalyzerFindingsTotal</strong> (custom thresholds)</li>
<li><strong>ComplexityAverage</strong> (default thresholds)</li>
</ul>
<p>So the splatted arguments for <code class="language-plaintext highlighter-rouge">Test-PSCodeHealthCompliance</code> that we add into <code class="language-plaintext highlighter-rouge">PSGithubSearch.BuildSettings.ps1</code> are :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="w"> </span><span class="nf">QualityGateParams</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">@{</span><span class="w">
</span><span class="nx">CustomSettingsPath</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'.\ProjectRules.json'</span><span class="w">
</span><span class="nx">SettingsGroup</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'OverallMetrics'</span><span class="w">
</span><span class="nx">MetricName</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="err">@(</span><span class="s1">'LinesOfCodeAverage'</span><span class="p">,</span><span class="w">
</span><span class="s1">'TestCoverage'</span><span class="p">,</span><span class="w">
</span><span class="s1">'ScriptAnalyzerFindingsTotal'</span><span class="w">
</span><span class="s1">'ComplexityAverage'</span><span class="w">
</span><span class="err">)</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>Then, we add a task like this in the build script :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">task</span><span class="w"> </span><span class="nx">Fail_If_Quality_Goal_Not_Met</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">Write-TaskBanner</span><span class="w"> </span><span class="nt">-TaskName</span><span class="w"> </span><span class="nv">$Task</span><span class="o">.</span><span class="nf">Name</span><span class="w">
</span><span class="nx">Add-AppveyorTest</span><span class="w"> </span><span class="nt">-Name</span><span class="w"> </span><span class="s1">'Quality Gate'</span><span class="w"> </span><span class="nt">-Outcome</span><span class="w"> </span><span class="nx">Running</span><span class="w">
</span><span class="nv">$QualityGateSettings</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$Settings</span><span class="o">.</span><span class="nf">QualityGateParams</span><span class="w">
</span><span class="nv">$Compliance</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">Test-PSCodeHealthCompliance</span><span class="w"> </span><span class="err">@</span><span class="nx">QualityGateSettings</span><span class="w"> </span><span class="nt">-HealthReport</span><span class="w"> </span><span class="nv">$HealthReport</span><span class="w">
</span><span class="nv">$Compliance</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Select-Object</span><span class="w"> </span><span class="s1">'MetricName'</span><span class="p">,</span><span class="s1">'Value'</span><span class="p">,</span><span class="s1">'Result'</span><span class="w">
</span><span class="nv">$FailedRules</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$Compliance</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Where-Object</span><span class="w"> </span><span class="nx">Result</span><span class="w"> </span><span class="o">-eq</span><span class="w"> </span><span class="s1">'Fail'</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$FailedRules</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$FailedString</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$FailedRules</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Select-Object</span><span class="w"> </span><span class="s1">'MetricName'</span><span class="p">,</span><span class="s1">'Value'</span><span class="p">,</span><span class="s1">'Result'</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Out-String</span><span class="w">
</span><span class="nv">$WarningMessage</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"Failed compliance rules : </span><span class="se">`n</span><span class="s2">"</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nv">$FailedString</span><span class="w">
</span><span class="nf">Write-Warning</span><span class="w"> </span><span class="nv">$WarningMessage</span><span class="w">
</span><span class="bp">$Error</span><span class="nf">Message</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"Project's code didn't pass the quality gate. Aborting build"</span><span class="w">
</span><span class="nf">Update-AppveyorTest</span><span class="w"> </span><span class="nt">-Name</span><span class="w"> </span><span class="s1">'Quality Gate'</span><span class="w"> </span><span class="nt">-Outcome</span><span class="w"> </span><span class="nx">Failed</span><span class="w"> </span><span class="nt">-ErrorMessage</span><span class="w"> </span><span class="bp">$Error</span><span class="nx">Message</span><span class="w">
</span><span class="kr">Throw</span><span class="w"> </span><span class="bp">$Error</span><span class="nf">Message</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">Else</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">Update-AppveyorTest</span><span class="w"> </span><span class="nt">-Name</span><span class="w"> </span><span class="s1">'Quality Gate'</span><span class="w"> </span><span class="nt">-Outcome</span><span class="w"> </span><span class="nx">Passed</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>The key here is <code class="language-plaintext highlighter-rouge">$FailedRules</code>, if it’s not <code class="language-plaintext highlighter-rouge">$Null</code>, additional stuff needs to happen :</p>
<ul>
<li>Convert the failed rules to a string and build a warning message out of it</li>
<li>Write that to the warning stream to make it stand out in the build console</li>
<li>Build a brief but descriptive error message</li>
<li><code class="language-plaintext highlighter-rouge">Throw</code> this error message to make the build fail</li>
</ul>
<p>We could just use the <code class="language-plaintext highlighter-rouge">Summary</code> parameter to get the overall compliance result, but then we wouldn’t know which metric got a <strong>Fail</strong>.</p>
<p>Now, we need to tell <code class="language-plaintext highlighter-rouge">Invoke-Build</code> to run these new tasks. All it takes is to reference them in the default task (<code class="language-plaintext highlighter-rouge">.</code>) within the build script :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Default task :</span><span class="w">
</span><span class="nf">task</span><span class="w"> </span><span class="o">.</span><span class="w"> </span><span class="nx">Clean</span><span class="p">,</span><span class="w">
</span><span class="nf">Install_Dependencies</span><span class="p">,</span><span class="w">
</span><span class="nf">Test</span><span class="p">,</span><span class="w">
</span><span class="nf">Generate_Quality_Report</span><span class="p">,</span><span class="w">
</span><span class="nf">Fail_If_Quality_Goal_Not_Met</span><span class="p">,</span><span class="w">
</span><span class="nf">Set_Module_Version</span><span class="p">,</span><span class="w">
</span><span class="nf">Push_Build_Changes_To_Repo</span><span class="p">,</span><span class="w">
</span><span class="nf">Copy_Source_To_Build_Output</span><span class="w">
</span></code></pre></div></div>
<p>Here is the resulting build console output in AppVeyor :</p>
<p><img src="https://mathieubuisson.github.io/images/2017-10-24-pscodehealth-release-pipeline-quality-gate.png" alt="AppVeyor console passing gate" /></p>
<p>Cool, the happy path is working.</p>
<p>Now, I want to see an error, I need <span style="color:red"><strong>some red</strong></span>. I feed from consoles bleeding error messages. I’m a vampire…<br />
Sorry, got carried away for a second.</p>
<p>A simple way to get a failed compliance result is to remove the <strong>LinesOfCodeAverage</strong> entry from our <code class="language-plaintext highlighter-rouge">ProjectRules.json</code> file. This makes <code class="language-plaintext highlighter-rouge">Test-PSCodeHealthCompliance</code> use the default rule, which has a <strong>Fail</strong> threshold of 60. The current value for the project’s code is 110.4, so it should fail.</p>
<p><img src="https://mathieubuisson.github.io/images/2017-10-24-pscodehealth-release-pipeline-failure.png" alt="AppVeyor console failing gate" /></p>
<p>Indeed, we get some yellow and some red. Also, the build fails as expected.</p>
<h2 id="conclusion">Conclusion</h2>
<p><code class="language-plaintext highlighter-rouge">PSCodeHealth</code> has been designed to be flexible. It can be used to check only the metrics <strong>you</strong> care about and the rules can be easily customized to meet <strong>your</strong> goals or requirements.</p>
<p>Also, it can be leveraged in a release pipeline to create a quality gate covering various aspects of code quality and maintainability.</p>
<p>As a reminder, you can get <code class="language-plaintext highlighter-rouge">PSCodeHealth</code> from the <a href="https://www.powershellgallery.com/packages/PSCodeHealth">PowerShell Gallery</a> and of course, the code is <a href="https://github.com/MathieuBuisson/PSCodeHealth">on GitHub</a>.</p>TheShellNutIn this post, we'll look at PSCodeHealth's default code quality metrics rules and how to customize them to our requirements. Then, we'll use custom metrics rules in a release pipeline to decide if the build should pass or fail.How To Assess and Improve PowerShell Code Quality Using PSCodeHealth2017-10-04T00:00:00+01:002017-10-04T00:00:00+01:00https://mathieubuisson.github.io/powershell-code-quality-pscodehealth<aside class="sidebar__right">
<nav class="toc">
<header><h4 class="nav__title"><i class="fa fa-file-text"></i> In This Article</h4></header>
<ul class="toc__menu" id="markdown-toc">
<li><a href="#what-is-powershell-code-quality-" id="markdown-toc-what-is-powershell-code-quality-">What Is PowerShell Code Quality ?</a></li>
<li><a href="#generating-a-pscodehealth-report-for-a-powershell-project" id="markdown-toc-generating-a-pscodehealth-report-for-a-powershell-project">Generating a PSCodeHealth Report For a PowerShell Project</a></li>
<li><a href="#interpreting-a-pscodehealth-report" id="markdown-toc-interpreting-a-pscodehealth-report">Interpreting a PSCodeHealth Report</a></li>
<li><a href="#using-pscodehealth-to-improve-code-quality-and-track-our-progress" id="markdown-toc-using-pscodehealth-to-improve-code-quality-and-track-our-progress">Using PSCodeHealth To Improve Code Quality and Track Our Progress</a></li>
<li><a href="#conclusion" id="markdown-toc-conclusion">Conclusion</a></li>
</ul>
</nav>
</aside>
<h2 id="what-is-powershell-code-quality-">What Is PowerShell Code Quality ?</h2>
<p>Code quality is a vast and somewhat subjective notion, so what we cover here is <strong>my</strong> take on code quality <strong>for PowerShell</strong> and more specifically, the parts which underpin <code class="language-plaintext highlighter-rouge">PSCodeHealth</code>.</p>
<p>Instead of engaging in an abstract debate on what is quality, we’ll focus on attributes which can be observed and reasonably quantified by analyzing the code. This list of attributes is intentionally leaning to where <code class="language-plaintext highlighter-rouge">PSCodeHealth</code> can help.</p>
<p><img src="https://mathieubuisson.github.io/images/2017-10-04-powershell-code-quality-pscodehealth-wtf.png" alt="WTF is quality ?" /></p>
<p>High-quality PowerShell code tends to have the following characteristics :</p>
<h3 id="it-follows-general-best-practices-language-guidelines-and-conventions">It follows general best practices, language guidelines and conventions</h3>
<p>For any given language/framework, communities build consensuses about what are considered <em>good</em> practices. For PowerShell, these include :</p>
<ul>
<li>Avoid using aliases in scripts/modules</li>
<li>Avoid hard-coding credentials (especially in plain text) in scripts/modules</li>
<li>Avoid using <code class="language-plaintext highlighter-rouge">Write-Host</code></li>
</ul>
<p>Of course, there are exceptions to these rules but it is accepted that they should <strong>generally</strong> be followed. These rules exist for a reason, they prevent gotchas. For example, the 3 rules mentioned above help prevent the following (respectively) :</p>
<ul>
<li>Aliases tend to be less readable and lesser known than the full command name.</li>
<li>Credentials in plain text can be read by anyone who can read the code and reused for malicious purposes.</li>
<li>There are multiple reasons <strong>not</strong> to use <code class="language-plaintext highlighter-rouge">Write-Host</code>, <a href="http://www.jsnover.com/blog/2013/12/07/write-host-considered-harmful/">Jeffrey Snover</a>, <a href="http://windowsitpro.com/blog/what-do-not-do-powershell-part-1">Don Jones</a> and others have explained these reasons.</li>
</ul>
<p><strong>Conventions are immensely helpful</strong> : they lower the “<em>barrier to understanding</em>” for reviewers/contributors and they make developers more productive. The more developers follow them, the more useful they are. So do yourself and the community a favor : use established conventions, unless you have a documented reason not to.</p>
<p>To evaluate whether a piece of code follows sound practices and general language guidelines, this is where linting tools come in. In the PowerShell realm, we have <a href="https://github.com/PowerShell/PSScriptAnalyzer">PSScriptanalyzer</a>.</p>
<h3 id="it-follows-a-consistent-style">It follows a consistent style</h3>
<p>There are other practices which are more a matter of personal (or team) preference, like :</p>
<ul>
<li>Indentation</li>
<li>Opening braces on the same line or a new line</li>
<li>Capitalization</li>
<li>Naming conventions (especially for variables)</li>
</ul>
<p>The main rule for these type of practices is more akin to :</p>
<blockquote>
<p>Pick a style and stick to it.</p>
</blockquote>
<p>What matters here is <strong>consistency</strong> across a given file, module, project or team.<br />
Consistency improves readability because anyone reviewing the code can train one’s eye to a given style aspects and use that to read the code more quickly.</p>
<p>Any exception to the consistent use of a given style practice will confuse, or at best, slow down the reader.</p>
<p>When working as part of a team, style consistency reduces the number of <em>not-so-obvious</em> and <em>not-so-useful</em> changes in source control, for example, commits consisting only in :</p>
<ul>
<li>Changing a tab into 4 spaces</li>
<li>Adding or removing whitespace</li>
<li>Changing a pair of double quotes into single quotes</li>
</ul>
<p>You don’t want these silly little things to delay the merging of your pull requests, do you ?</p>
<p>Again, a linting tool, possibly with the addition of custom rules to match a specific style guide, can help greatly.</p>
<h3 id="single-purpose-short-simple-functions">Single-purpose, short, simple functions</h3>
<h4 id="single-purpose">Single-purpose</h4>
<p>A function should do 1 thing and do it well.</p>
<p>This makes the code modular and understandable, because it organizes it into logical, single-responsibility chunks. A function is basically a named code block and to contribute to this organization of the code, the name of a function should tell its purpose.</p>
<p>Functions should generally follow PowerShell cmdlets <code class="language-plaintext highlighter-rouge">Verb-Noun</code> naming convention and this helps keep them single-purpose. If multiple verbs could be used for a function name, this function is doing more than 1 type of operations and it should be split up. If we need an “And” or many words to compose a descriptive noun for a function, it is most likely doing too many things.</p>
<h4 id="short">Short</h4>
<p>The most obvious argument goes back to readability.</p>
<p>There is another argument in favor of short functions : it forces the developers to extract a lot of code into other functions, which are logically scoped, tightly focused and … short. Also, a function containing many calls to other functions with intent-revealing names can read like a narrative. We all love a good story, right ?</p>
<h4 id="simple">Simple</h4>
<blockquote>
<p>What is a simple function ?<br />
What is a complex function ?<br />
Why does it matter ?</p>
</blockquote>
<p>Well, I already wrote a <a href="https://mathieubuisson.github.io/measuring-powershell-code-complexity/">detailed PowerShell-specific answer to these questions</a>, so I invite you to take a look at it.</p>
<h3 id="documented">Documented</h3>
<p>There is a reason I use the term “<em>documented</em>” and not “<em>commented</em>”. Too much inline comments can clutter the code and can actually make it less readable.<br />
Inline comments should :</p>
<ul>
<li>Be used only when necessary</li>
<li>Not state the obvious</li>
<li>Used to explain the logic, not the syntax</li>
<li>Not be needed if the code is clear, idiomatic and uses intent-revealing names</li>
</ul>
<p>You don’t have to take my word for it :</p>
<blockquote>
<p>The proper use of comments is to compensate for our failure to express ourself in code.</p>
</blockquote>
<p class="small"><cite>Robert C. Martin</cite> — Clean Code : A Handbook of Agile Software Craftsmanship</p>
<p>On the other hand, there is a place where comments belong : <strong>comment-based help</strong>.<br />
Public functions should contains comment-based to provide the user a quick access to documentation (via <code class="language-plaintext highlighter-rouge">Get-Help</code>). Even private functions benefit from comment-based help, the only difference is the audience : code reviewers, maintainers or contributors instead of users.</p>
<h3 id="testable-and-tested">Testable and tested</h3>
<p>Again, we are opening a Pandora’s box because testing is a huge topic and has diverse, far-reaching implications. In the context of PowerShell, testing is pretty much synonymous with <a href="https://github.com/pester/Pester">Pester</a>, so that’s what we focus on here.</p>
<h4 id="tested">Tested</h4>
<p>To summarize, effective tests provide :</p>
<ul>
<li>Proof that the code works as intended/expected</li>
<li><strong>Early</strong> detection of code defects (the later a defect is detected, the harder it is to fix)</li>
<li>Fast feedback on whether code changes are breaking existing functionality</li>
</ul>
<p>There many other benefits, but essentially, they boil down to <strong>less defects</strong> and <strong>more confidence</strong>.</p>
<p>Anyone making code changes can run the tests to know immediately if the changes are breaking existing functionality or not. This makes <strong>changing code easier and safer</strong>, which is ultimately what maintainability is all about.</p>
<h4 id="testable">Testable</h4>
<p>PowerShell functions which are easy to test (particularly <strong>unit</strong> test) have the following attributes :</p>
<ul>
<li>Short</li>
<li>Simple</li>
<li>Tightly focused (they have a single, well-scoped responsibility)</li>
<li>Loosely coupled to their dependencies</li>
</ul>
<p>Does this ring a bell ? It should, because : these attributes are also <strong>attributes of high quality code</strong>.</p>
<p>This is why thinking about the tests during code design (or before, for those into <abbr title="Test-driven development">TDD</abbr>) tends to lead to higher quality code.</p>
<p>Now that we are on the same page regarding the characteristics which make up PowerShell code quality and maintainability, let’s look at how <code class="language-plaintext highlighter-rouge">PSCodeHealth</code> can help us measure these characteristics.</p>
<h2 id="generating-a-pscodehealth-report-for-a-powershell-project">Generating a PSCodeHealth Report For a PowerShell Project</h2>
<p><a href="https://github.com/MathieuBuisson/PSGithubSearch">PSGithubSearch</a> is a cute little PowerShell module I wrote a while back, but I’m concerned about its quality and maintainability, so we are going to use it as an example.</p>
<h3 id="pscodehealth-report-as-a-powershell-object">PSCodeHealth report as a PowerShell object</h3>
<p>Let’s start by creating a <code class="language-plaintext highlighter-rouge">PSCodeHealth</code> report for all the PowerShell files (<code class="language-plaintext highlighter-rouge">'*.ps*1'</code>) in the project and store it into a variable for later use :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\PSGithubSearch</span><span class="err">></span><span class="w"> </span><span class="nv">$Params</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">@{</span><span class="nx">Path</span><span class="o">=</span><span class="s1">'.\PSGithubSearch\'</span><span class="err">;</span><span class="w"> </span><span class="nx">TestsPath</span><span class="o">=</span><span class="s1">'.\Tests\Unit\'</span><span class="err">;</span><span class="w"> </span><span class="nx">Recurse</span><span class="o">=</span><span class="nv">$True</span><span class="p">}</span><span class="w">
</span><span class="nf">C:\PSGithubSearch</span><span class="err">></span><span class="w"> </span><span class="nv">$HealthReport</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">Invoke-PSCodeHealth</span><span class="w"> </span><span class="err">@</span><span class="nx">Params</span><span class="w">
</span><span class="nf">C:\PSGithubSearch</span><span class="err">></span><span class="w"> </span><span class="nv">$HealthReport</span><span class="w">
</span><span class="nf">Files</span><span class="w"> </span><span class="nx">Functions</span><span class="w"> </span><span class="nx">LOC</span><span class="w"> </span><span class="p">(</span><span class="nf">Average</span><span class="p">)</span><span class="w"> </span><span class="nf">Findings</span><span class="w"> </span><span class="p">(</span><span class="nf">Total</span><span class="p">)</span><span class="w"> </span><span class="nf">Findings</span><span class="w"> </span><span class="nx">Complexity</span><span class="w"> </span><span class="nx">Test</span><span class="w"> </span><span class="nx">Coverage</span><span class="w">
</span><span class="p">(</span><span class="nf">Average</span><span class="p">)</span><span class="w"> </span><span class="p">(</span><span class="nf">Average</span><span class="p">)</span><span class="w">
</span><span class="nf">-----</span><span class="w"> </span><span class="nf">---------</span><span class="w"> </span><span class="nf">-------------</span><span class="w"> </span><span class="nf">----------------</span><span class="w"> </span><span class="nf">----------------</span><span class="w"> </span><span class="nf">----------------</span><span class="w"> </span><span class="nf">-------------</span><span class="w">
</span><span class="nx">2</span><span class="w"> </span><span class="nx">5</span><span class="w"> </span><span class="nx">128.2</span><span class="w"> </span><span class="nx">0</span><span class="w"> </span><span class="nx">0</span><span class="w"> </span><span class="nx">15.2</span><span class="w"> </span><span class="nx">71.29</span><span class="w"> </span><span class="o">%</span><span class="w">
</span></code></pre></div></div>
<p>The default formatting view provides basic information for the overall project, but there is more :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\PSGithubSearch</span><span class="err">></span><span class="w"> </span><span class="nv">$HealthReport</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Select-Object</span><span class="w"> </span><span class="nt">-Property</span><span class="w"> </span><span class="s1">'*'</span><span class="w"> </span><span class="nt">-Exclude</span><span class="w"> </span><span class="s1">'FunctionHealthRecords'</span><span class="w">
</span><span class="nf">ReportTitle</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">PSGithubSearch</span><span class="w">
</span><span class="nf">ReportDate</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">2017-09-29</span><span class="w"> </span><span class="nx">13:23:30Z</span><span class="w">
</span><span class="nf">AnalyzedPath</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">C:\PSGithubSearch\PSGithubSearch\</span><span class="w">
</span><span class="nf">Files</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">2</span><span class="w">
</span><span class="nf">Functions</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">5</span><span class="w">
</span><span class="nf">LinesOfCodeTotal</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">641</span><span class="w">
</span><span class="nf">LinesOfCodeAverage</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">128.2</span><span class="w">
</span><span class="nf">ScriptAnalyzerFindingsTotal</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">1</span><span class="w">
</span><span class="nf">ScriptAnalyzerErrors</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">0</span><span class="w">
</span><span class="nf">ScriptAnalyzerWarnings</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">1</span><span class="w">
</span><span class="nf">ScriptAnalyzerInformation</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">0</span><span class="w">
</span><span class="nf">ScriptAnalyzerFindingsAverage</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">0.2</span><span class="w">
</span><span class="nf">FunctionsWithoutHelp</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">0</span><span class="w">
</span><span class="nf">NumberOfTests</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">27</span><span class="w">
</span><span class="nf">NumberOfFailedTests</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">0</span><span class="w">
</span><span class="nf">FailedTestsDetails</span><span class="w"> </span><span class="p">:</span><span class="w">
</span><span class="nf">NumberOfPassedTests</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">27</span><span class="w">
</span><span class="nf">TestsPassRate</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">100</span><span class="w">
</span><span class="nf">TestCoverage</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">71.29</span><span class="w">
</span><span class="nf">CommandsMissedTotal</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">89</span><span class="w">
</span><span class="nf">ComplexityAverage</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">15.2</span><span class="w">
</span><span class="nf">ComplexityHighest</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">30</span><span class="w">
</span><span class="nf">NestingDepthAverage</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">2.4</span><span class="w">
</span><span class="nf">NestingDepthHighest</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">3</span><span class="w">
</span></code></pre></div></div>
<p>For more information on these metrics and which aspects of quality they attempt to quantify, please refer to <a href="http://pscodehealth.readthedocs.io/en/latest/Metrics/">this documentation page</a>.</p>
<h3 id="pscodehealth-html-report">PSCodeHealth HTML report</h3>
<p>This raw data is nice but a <em>dashboard-like</em> HTML report would probably be a nicer way of viewing where this project is doing well and where it needs improvement. We can do that using the <code class="language-plaintext highlighter-rouge">HtmlReportPath</code> parameter :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\PSGithubSearch</span><span class="err">></span><span class="w"> </span><span class="nx">Invoke-PSCodeHealth</span><span class="w"> </span><span class="err">@</span><span class="nx">Params</span><span class="w"> </span><span class="nt">-HtmlReportPath</span><span class="w"> </span><span class="s1">'C:\HealthReport.html'</span><span class="w">
</span></code></pre></div></div>
<p>Here is what the <strong>Summary</strong> tab of the report looks like :</p>
<p><img src="https://mathieubuisson.github.io/images/2017-10-04-powershell-code-quality-pscodehealth-summary.png" alt="HTML report - Summary tab" /></p>
<p>The <strong>Summary</strong> tab is just an overview, the sidebar provides access to more specific sections of the report :</p>
<p><img src="https://raw.githubusercontent.com/MathieuBuisson/PSCodeHealth/master/Examples/SidebarScreenshot.png" alt="HTML report - Sidebar" /></p>
<p>To see it in action, you can play with <a href="https://mathieubuisson.github.io/assets/html/healthreport.html">a live version of this report</a>.</p>
<h2 id="interpreting-a-pscodehealth-report">Interpreting a PSCodeHealth Report</h2>
<p>The report’s color-coding is straightforward :</p>
<ul>
<li>Green means good</li>
<li>Yellow means warning</li>
<li>Red means danger</li>
</ul>
<p>It is designed to provide at-a-glance information about which aspects and sections of the code need attention or improvement.</p>
<h3 id="style--best-practices-tab">Style & Best Practices tab</h3>
<p>The section of the report focuses on <code class="language-plaintext highlighter-rouge">PSScriptAnalyzer</code> findings and comment-based help. There is only 1 finding in the whole project so this is fine.</p>
<h3 id="maintainability-tab">Maintainability tab</h3>
<h4 id="functions-length">Functions length</h4>
<p>The average number of lines of code per function (128.2) shows up in red, so it must be bad. How bad ?<br />
The compliance rule for this metric gives us a good idea :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\PSGithubSearch</span><span class="err">></span><span class="w"> </span><span class="nx">Get-PSCodeHealthComplianceRule</span><span class="w"> </span><span class="nt">-MetricName</span><span class="w"> </span><span class="s1">'LinesOfCodeAverage'</span><span class="w">
</span><span class="nf">Metric</span><span class="w"> </span><span class="nx">Name</span><span class="w"> </span><span class="nx">Metric</span><span class="w"> </span><span class="nx">Group</span><span class="w"> </span><span class="nx">Warning</span><span class="w"> </span><span class="nx">Fail</span><span class="w"> </span><span class="nx">Threshold</span><span class="w"> </span><span class="nx">Higher</span><span class="w"> </span><span class="nx">Is</span><span class="w">
</span><span class="nf">Threshold</span><span class="w"> </span><span class="nx">Better</span><span class="w">
</span><span class="nf">-----------</span><span class="w"> </span><span class="nf">------------</span><span class="w"> </span><span class="nf">----------------</span><span class="w"> </span><span class="nf">--------------</span><span class="w"> </span><span class="nf">---------------</span><span class="w">
</span><span class="nx">LinesOfCodeAverage</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">30</span><span class="w"> </span><span class="nx">60</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span></code></pre></div></div>
<p>So this means that this metric starts to show up in yellow from 30 (lines of code per function) and in red from 60. This metric is over twice the “<em>danger</em>” threshold ! This needs improvement, a lot of it.</p>
<p>The <em>Per Function Information</em> table tells us which particular function(s) we should focus on to improve the project’s overall maintainability.
We can see that the most serious offender is <code class="language-plaintext highlighter-rouge">Find-GitHubIssue</code> with <strong>219 lines of code</strong>. Ouch !</p>
<h4 id="complexity">Complexity</h4>
<p>We can see that the maximum cyclomatic complexity is green, which is good, but the average is yellow. Why ?</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\PSGithubSearch</span><span class="err">></span><span class="w"> </span><span class="nx">Get-PSCodeHealthComplianceRule</span><span class="w"> </span><span class="nt">-MetricName</span><span class="w"> </span><span class="s1">'ComplexityHighest'</span><span class="p">,</span><span class="s1">'ComplexityAverage'</span><span class="w">
</span><span class="nf">Metric</span><span class="w"> </span><span class="nx">Name</span><span class="w"> </span><span class="nx">Metric</span><span class="w"> </span><span class="nx">Group</span><span class="w"> </span><span class="nx">Warning</span><span class="w"> </span><span class="nx">Fail</span><span class="w"> </span><span class="nx">Threshold</span><span class="w"> </span><span class="nx">Higher</span><span class="w"> </span><span class="nx">Is</span><span class="w">
</span><span class="nf">Threshold</span><span class="w"> </span><span class="nx">Better</span><span class="w">
</span><span class="nf">-----------</span><span class="w"> </span><span class="nf">------------</span><span class="w"> </span><span class="nf">----------------</span><span class="w"> </span><span class="nf">--------------</span><span class="w"> </span><span class="nf">---------------</span><span class="w">
</span><span class="nx">ComplexityHighest</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">30</span><span class="w"> </span><span class="nx">60</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span><span class="nx">ComplexityAverage</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">15</span><span class="w"> </span><span class="nx">30</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span></code></pre></div></div>
<p>At 15.2, the average complexity is slightly above the <em>warning</em> threshold, but still, this is worth looking into. Once again, the <em>Per Function Information</em> table points at <code class="language-plaintext highlighter-rouge">Find-GitHubIssue</code> as the main offender, so we definitely need to take a hard look at this function.</p>
<h4 id="nesting-depth">Nesting depth</h4>
<p>Regarding nesting depth, all functions in the project are green, so we are good.</p>
<h3 id="tests-tab">Tests tab</h3>
<h4 id="tests-failures">Tests failures</h4>
<p>All 27 unit tests in this project have passed, so let’s move along.</p>
<h4 id="tests-code-coverage">Tests code coverage</h4>
<p>These tests exercise 71.29 % of the project’s code. The panel containing the overall “<em>Test Coverage</em>” chart is yellow which means this metric is at <em>warning</em> level. For more information, we can look at what the compliance rule has to say about that :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\PSGithubSearch</span><span class="err">></span><span class="w"> </span><span class="nx">Get-PSCodeHealthComplianceRule</span><span class="w"> </span><span class="nt">-MetricName</span><span class="w"> </span><span class="s1">'TestCoverage'</span><span class="w"> </span><span class="nt">-SettingsGroup</span><span class="w"> </span><span class="s1">'OverallMetrics'</span><span class="w">
</span><span class="nf">Metric</span><span class="w"> </span><span class="nx">Name</span><span class="w"> </span><span class="nx">Metric</span><span class="w"> </span><span class="nx">Group</span><span class="w"> </span><span class="nx">Warning</span><span class="w"> </span><span class="nx">Fail</span><span class="w"> </span><span class="nx">Threshold</span><span class="w"> </span><span class="nx">Higher</span><span class="w"> </span><span class="nx">Is</span><span class="w">
</span><span class="nf">Threshold</span><span class="w"> </span><span class="nx">Better</span><span class="w">
</span><span class="nf">-----------</span><span class="w"> </span><span class="nf">------------</span><span class="w"> </span><span class="nf">----------------</span><span class="w"> </span><span class="nf">--------------</span><span class="w"> </span><span class="nf">---------------</span><span class="w">
</span><span class="nx">TestCoverage</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">80</span><span class="w"> </span><span class="nx">70</span><span class="w"> </span><span class="nx">True</span><span class="w">
</span></code></pre></div></div>
<p>This yellow is actually dark orangish because the tests code coverage is close to the <em>danger</em> zone.</p>
<p>Coverage can vary widely from 1 function to another so it is probably a good idea to look at the <em>Per Function Information</em> table to see if there are low-hanging fruits (functions for which we could easily and significantly increase coverage).</p>
<p><code class="language-plaintext highlighter-rouge">Get-NumberOfPage</code> has only 41 % of its code exercised by unit tests. This is low, but I’m not too worried about it because it is just a private helper function and it is short and simple.</p>
<p><code class="language-plaintext highlighter-rouge">Find-GitHubIssue</code> on the other hand, is a public function. 65 % test coverage is fairly low, but is that a low-hanging fruit ? I’m afraid not because, as we have seen, this function is huge and has a high cyclomatic complexity. This metric has an <strong>inverse</strong> correlation with testability, because it tells the number of code paths that tests need to cover.</p>
<h4 id="missed-commands">Missed commands</h4>
<p>This measures the code which is <strong>not covered</strong> by tests, but in absolute number rather than in percentage.</p>
<p>Also, because <code class="language-plaintext highlighter-rouge">Pester</code> code coverage feature uses <em>commands</em> as the unit of code (instead of <em>lines</em>), that’s what <code class="language-plaintext highlighter-rouge">PSCodeHealth</code> uses.</p>
<p>89 may look like a lot, but it is OK, as far as <code class="language-plaintext highlighter-rouge">PSCodeHealth</code> is concerned :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\PSGithubSearch</span><span class="err">></span><span class="w"> </span><span class="nx">Get-PSCodeHealthComplianceRule</span><span class="w"> </span><span class="nt">-MetricName</span><span class="w"> </span><span class="nx">CommandsMissedTotal</span><span class="w">
</span><span class="nf">Metric</span><span class="w"> </span><span class="nx">Name</span><span class="w"> </span><span class="nx">Metric</span><span class="w"> </span><span class="nx">Group</span><span class="w"> </span><span class="nx">Warning</span><span class="w"> </span><span class="nx">Threshold</span><span class="w"> </span><span class="nx">Fail</span><span class="w"> </span><span class="nx">Threshold</span><span class="w"> </span><span class="nx">Higher</span><span class="w"> </span><span class="nx">Is</span><span class="w"> </span><span class="nx">Better</span><span class="w">
</span><span class="nf">-----------</span><span class="w"> </span><span class="nf">------------</span><span class="w"> </span><span class="nf">-----------------</span><span class="w"> </span><span class="nf">--------------</span><span class="w"> </span><span class="nf">----------------</span><span class="w">
</span><span class="nx">CommandsMissedTotal</span><span class="w"> </span><span class="nx">OverallMetrics</span><span class="w"> </span><span class="nx">200</span><span class="w"> </span><span class="nx">400</span><span class="w"> </span><span class="nx">False</span><span class="w">
</span></code></pre></div></div>
<p>This is because the compliance rule uses a 1000 <em>commands</em> project as its baseline and our current project is smaller than that. This shows that relative metrics (in percentage) are generally more meaningful.</p>
<p>Now that we know which aspects and sections of code affect the project’s quality and maintainability, we can use this information as a guide when improving its code.</p>
<h2 id="using-pscodehealth-to-improve-code-quality-and-track-our-progress">Using PSCodeHealth To Improve Code Quality and Track Our Progress</h2>
<p>The HTML report made it very clear that the functions in this project are too long. It also made the function <code class="language-plaintext highlighter-rouge">Find-GitHubIssue</code> stick out like a sore thumb, so this is where we’ll start our refactoring endeavor.</p>
<h3 id="a-refactoring-example">A refactoring example</h3>
<p>At the beginning of the function, we see this :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="w"> </span><span class="p">[</span><span class="kt">string</span><span class="p">]</span><span class="nv">$QueryString</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'q='</span><span class="w">
</span><span class="nv">$EmptyQueryString</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$True</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$Keywords</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$QueryString</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="p">(</span><span class="nv">$Keywords</span><span class="w"> </span><span class="o">-join</span><span class="w"> </span><span class="s1">'+'</span><span class="p">)</span><span class="w">
</span><span class="nv">$EmptyQueryString</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$False</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$Type</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$EmptyQueryString</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$QueryString</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="s1">'type:'</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nv">$Type</span><span class="w">
</span><span class="nv">$EmptyQueryString</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$False</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">Else</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$QueryString</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="s1">'+type:'</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nv">$Type</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$In</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$EmptyQueryString</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$QueryString</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="s1">'in:'</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nv">$In</span><span class="w">
</span><span class="nv">$EmptyQueryString</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$False</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">Else</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$QueryString</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="s1">'+in:'</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nv">$In</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$Author</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$EmptyQueryString</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$QueryString</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="s1">'author:'</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nv">$Author</span><span class="w">
</span><span class="nv">$EmptyQueryString</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$False</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">Else</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$QueryString</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="s1">'+author:'</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nv">$Author</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$Assignee</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$EmptyQueryString</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$QueryString</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="s1">'assignee:'</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nv">$Assignee</span><span class="w">
</span><span class="nv">$EmptyQueryString</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$False</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">Else</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$QueryString</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="s1">'+assignee:'</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nv">$Assignee</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="o">...</span><span class="w">
</span><span class="o">...</span><span class="w">
</span></code></pre></div></div>
<p>This doesn’t even show everything, this list of <code class="language-plaintext highlighter-rouge">If</code> statements goes on and on…<br />
What a convoluted way of building a query string ! This section of code alone, represent 121 lines of code and a cyclomatic complexity of 26.</p>
<p>On top of that, to build this query string it is using a concatenation operator (<code class="language-plaintext highlighter-rouge">+=</code>). Every. Single. Time. Which is inefficient.</p>
<p>Each variable tested in these <code class="language-plaintext highlighter-rouge">If</code> statements is used to build a search qualifier string. Almost all of these are built using the same pattern :<br />
variable name in lower case + ‘:’ + variable value.<br />
So we should be able to leverage this pattern against a collection of variables, instead of one at a time, to reduce repetition.</p>
<p>Also, all these variables are coming from the function parameters, so we can use <code class="language-plaintext highlighter-rouge">$PSBoundParameters</code>. And because they don’t have a default value, we know that if they are not in <code class="language-plaintext highlighter-rouge">$PSBoundParameters</code>, they are <code class="language-plaintext highlighter-rouge">Null</code>. This means we don’t need to check for <code class="language-plaintext highlighter-rouge">Null</code> and can get rid of all these <code class="language-plaintext highlighter-rouge">If</code> :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$Variable</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="o">...</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>First, we remove the parameters used by search qualifiers not following the pattern identified above (<code class="language-plaintext highlighter-rouge">$Keywords</code>, <code class="language-plaintext highlighter-rouge">$Labels</code> and <code class="language-plaintext highlighter-rouge">$SortBy</code>) :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="bp">$PSBoundParameters</span><span class="o">.</span><span class="nf">ContainsKey</span><span class="p">(</span><span class="s1">'Keywords'</span><span class="p">)</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$Null</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="bp">$PSBoundParameters</span><span class="o">.</span><span class="nf">Remove</span><span class="p">(</span><span class="s1">'Keywords'</span><span class="p">)</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="bp">$PSBoundParameters</span><span class="o">.</span><span class="nf">ContainsKey</span><span class="p">(</span><span class="s1">'Labels'</span><span class="p">)</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$Null</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="bp">$PSBoundParameters</span><span class="o">.</span><span class="nf">Remove</span><span class="p">(</span><span class="s1">'Labels'</span><span class="p">)</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="bp">$PSBoundParameters</span><span class="o">.</span><span class="nf">ContainsKey</span><span class="p">(</span><span class="s1">'SortBy'</span><span class="p">)</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$Null</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="bp">$PSBoundParameters</span><span class="o">.</span><span class="nf">Remove</span><span class="p">(</span><span class="s1">'SortBy'</span><span class="p">)</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>Then, we use the pattern against all the remaining parameters to build search qualifier strings :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span><span class="no">System.Collections.</span><span class="kt">ArrayList</span><span class="p">]</span><span class="nv">$JoinableFilterStrings</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">@()</span><span class="w">
</span><span class="kr">Foreach</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$ParamName</span><span class="w"> </span><span class="kr">in</span><span class="w"> </span><span class="bp">$PSBoundParameters</span><span class="o">.</span><span class="nf">Keys</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$JoinableFilterString</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'{0}:{1}'</span><span class="w"> </span><span class="nt">-f</span><span class="w"> </span><span class="nv">$ParamName</span><span class="o">.</span><span class="nf">ToLower</span><span class="p">(),</span><span class="w"> </span><span class="bp">$PSBoundParameters</span><span class="p">[</span><span class="nv">$ParamName</span><span class="p">]</span><span class="w">
</span><span class="nv">$Null</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$JoinableFilterStrings</span><span class="o">.</span><span class="nf">Add</span><span class="p">(</span><span class="nv">$JoinableFilterString</span><span class="p">)</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>Also, each if the original <code class="language-plaintext highlighter-rouge">If</code> statements have a nested <code class="language-plaintext highlighter-rouge">If/Else</code> :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$EmptyQueryString</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$QueryString</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="s1">'variable:'</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nv">$Variable</span><span class="w">
</span><span class="nv">$EmptyQueryString</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$False</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">Else</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$QueryString</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="s1">'+variable:'</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nv">$Variable</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>This checks whether the query string is empty to determine if we need to prefix the search qualifier with a <code class="language-plaintext highlighter-rouge">'+'</code>. Every. Single. Time.</p>
<p>To get rid of these repetitive checks, we can use the <code class="language-plaintext highlighter-rouge">join</code> operator with the <code class="language-plaintext highlighter-rouge">'+'</code> separator :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$JoinedFilterStrings</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="nv">$JoinableFilterStrings</span><span class="o">.</span><span class="nf">Count</span><span class="w"> </span><span class="o">-gt</span><span class="w"> </span><span class="nx">0</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="nv">$JoinableFilterStrings</span><span class="w"> </span><span class="o">-join</span><span class="w"> </span><span class="s1">'+'</span><span class="p">}</span><span class="w">
</span><span class="nv">$JoinedFilter</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="nv">$KeywordFilter</span><span class="p">,</span><span class="w"> </span><span class="nv">$JoinedFilterStrings</span><span class="p">,</span><span class="w"> </span><span class="nv">$LabelsFilter</span><span class="w"> </span><span class="o">-join</span><span class="w"> </span><span class="s1">'+'</span><span class="p">)</span><span class="o">.</span><span class="nf">Trim</span><span class="p">(</span><span class="s1">'+'</span><span class="p">)</span><span class="w">
</span></code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">$KeywordFilter</code> and <code class="language-plaintext highlighter-rouge">$LabelsFilter</code> are built prior to the <code class="language-plaintext highlighter-rouge">$JoinableFilterStrings</code> because they are built from a different pattern.</p>
<p>We join them with <code class="language-plaintext highlighter-rouge">$JoinableFilterStrings</code> even if they are <code class="language-plaintext highlighter-rouge">Null</code> or empty. In that case, the joined string would have a leading and/or trailing <code class="language-plaintext highlighter-rouge">'+'</code>. This is easily solved with the <code class="language-plaintext highlighter-rouge">Trim</code> method. No need for <code class="language-plaintext highlighter-rouge">Null</code> checks and these numerous <code class="language-plaintext highlighter-rouge">If/Else</code> anymore.</p>
<h3 id="quantifying-the-improvement">Quantifying the improvement</h3>
<p>Now, we can generate a new <code class="language-plaintext highlighter-rouge">PSCodeHealth</code> report and verify the progress we’ve made :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\PSGithubSearch</span><span class="err">></span><span class="w"> </span><span class="nv">$NewReport</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">Invoke-PSCodeHealth</span><span class="w"> </span><span class="err">@</span><span class="nx">Params</span><span class="w">
</span><span class="nf">C:\PSGithubSearch</span><span class="err">></span><span class="w"> </span><span class="nv">$IssueFunction</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$NewReport</span><span class="o">.</span><span class="nf">FunctionHealthRecords</span><span class="o">.</span><span class="nf">Where</span><span class="p">({</span><span class="bp">$_</span><span class="o">.</span><span class="nf">FunctionName</span><span class="w"> </span><span class="o">-eq</span><span class="w"> </span><span class="s1">'Find-GithubIssue'</span><span class="p">})</span><span class="w">
</span><span class="nf">C:\PSGithubSearch</span><span class="err">></span><span class="w"> </span><span class="nv">$IssueFunction</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">fl</span><span class="w"> </span><span class="o">*</span><span class="w">
</span><span class="nf">FunctionName</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">Find-GitHubIssue</span><span class="w">
</span><span class="nf">FilePath</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">C:\PSGithubSearch\PSGithubSearch\PSGithubSearch.psm1</span><span class="w">
</span><span class="nf">LinesOfCode</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">130</span><span class="w">
</span><span class="nf">ScriptAnalyzerFindings</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">0</span><span class="w">
</span><span class="nf">ScriptAnalyzerResultDetails</span><span class="w"> </span><span class="p">:</span><span class="w">
</span><span class="nf">ContainsHelp</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">True</span><span class="w">
</span><span class="nf">TestCoverage</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">87.5</span><span class="w">
</span><span class="nf">CommandsMissed</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">7</span><span class="w">
</span><span class="nf">Complexity</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">12</span><span class="w">
</span><span class="nf">MaximumNestingDepth</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="nx">2</span><span class="w">
</span></code></pre></div></div>
<p>We have ruthlessly axed almost 100 lines of code from this function, it feels good.</p>
<p>Initially, the cyclomatic complexity of this function was 30. By removing a large number of repetitive <code class="language-plaintext highlighter-rouge">If</code> and <code class="language-plaintext highlighter-rouge">Else</code> statements, we reduced it to 12.</p>
<p>Interestingly, this led to a substantial boost in test coverage (87.5 %). The exact same unit tests are now covering more code in that function. The main reason is that we have greatly reduced the number of code paths, so the tests have fewer paths to cover. This is a great example of the relationships between different types of metrics, in this case : <strong>cyclomatic complexity</strong> and <strong>tests code coverage</strong>.</p>
<p>Of course, there is a lot more refactoring work to be done, for example :</p>
<ul>
<li>Extract this whole query string building logic into a separate function</li>
<li>Reuse the same query string building logic in the other functions</li>
<li>Extract the handling of the API requests throttling into a separate function</li>
<li>Reuse the function handling the API requests throttling from the other functions</li>
</ul>
<p>These refactoring decisions and the work required to make them happen are still the developer’s responsibility. <code class="language-plaintext highlighter-rouge">PSCodeHealth</code> is just a tool to provide some guidance about where to focus and to help quantify improvements of the code.</p>
<div class="notice--warning">
<h4>Metrics have limitations :</h4>
<p>There is no perfect metric. Any metric can be gamed one way or another.<br />
So the focus should not be on improving metrics for the sake of improving metrics, but improving what they attempt to measure.</p>
</div>
<h2 id="conclusion">Conclusion</h2>
<p><code class="language-plaintext highlighter-rouge">PSCodeHealth</code> is considered stable, but not necessarily feature-complete. If there are additional metrics and features you would like to see, the preferred feedback channel is <a href="https://github.com/MathieuBuisson/PSCodeHealth/issues">GitHub issues</a>, but any type of feedback is welcome to make this tool more useful to more people.</p>
<p>Also, you might disagree with the tool’s default <em>warning</em> and/or <em>danger</em> thresholds for some metrics. This is perfectly fine, an upcoming article will show how to customize metrics rules to your project’s goals or requirements, and use this as a <strong>quality gate</strong> in a PowerShell release pipeline.</p>
<p>The easiest way to install <code class="language-plaintext highlighter-rouge">PSCodeHealth</code> is <a href="https://www.powershellgallery.com/packages/PSCodeHealth/">the PowerShell Gallery</a> and you can get <a href="https://github.com/MathieuBuisson/PSCodeHealth">the code from GitHub</a>. For more information on how to use this tool and its features, check out the <a href="http://pscodehealth.readthedocs.io/en/latest/">fairly extensive documentation</a>.</p>TheShellNutIn this article, we are going to start with what we mean by 'code quality' and why it matters. Then, we'll see how PSCodeHealth can help assess the quality and maintainability of a PowerShell project.Reducing whitespace in the HTML generated by Jekyll2017-09-08T00:00:00+01:002017-09-08T00:00:00+01:00https://mathieubuisson.github.io/reducing-whitespace-jekyll-html<p>The output HTML of Jekyll-based sites may contain quite a lot a whitespace. This is especially noticeable with pages relying heavily on <a href="https://shopify.github.io/liquid/">Liquid</a> tags with conditional logic or looping.</p>
<p>For example, here is a section of the <code class="language-plaintext highlighter-rouge">author-profile.html</code> template from the theme <a href="https://github.com/mmistakes/minimal-mistakes">Minimal Mistakes</a> :</p>
<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nt"><ul</span> <span class="na">class=</span><span class="s">"author__urls social-icons"</span><span class="nt">></span>
{% if author.location %}
<span class="nt"><li</span> <span class="na">itemprop=</span><span class="s">"homeLocation"</span> <span class="na">itemscope</span> <span class="na">itemtype=</span><span class="s">"http://schema.org/Place"</span><span class="nt">></span>
<span class="nt"><i</span> <span class="na">class=</span><span class="s">"fa fa-fw fa-map-marker"</span> <span class="na">aria-hidden=</span><span class="s">"true"</span><span class="nt">></i></span> <span class="nt"><span</span> <span class="na">itemprop=</span><span class="s">"name"</span><span class="nt">></span>{{ author.location }}<span class="nt"></span></span>
<span class="nt"></li></span>
{% endif %}
{% if author.uri %}
<span class="nt"><li></span>
<span class="nt"><a</span> <span class="na">href=</span><span class="s">"{{ author.uri }}"</span> <span class="na">itemprop=</span><span class="s">"url"</span><span class="nt">></span>
<span class="nt"><i</span> <span class="na">class=</span><span class="s">"fa fa-fw fa-chain"</span> <span class="na">aria-hidden=</span><span class="s">"true"</span><span class="nt">></i></span> {{ site.data.ui-text[site.locale].website_label | default: "Website" }}
<span class="nt"></a></span>
<span class="nt"></li></span>
{% endif %}
{% if author.email %}
<span class="nt"><li></span>
<span class="nt"><a</span> <span class="na">href=</span><span class="s">"mailto:{{ author.email }}"</span><span class="nt">></span>
<span class="nt"><meta</span> <span class="na">itemprop=</span><span class="s">"email"</span> <span class="na">content=</span><span class="s">"{{ author.email }}"</span> <span class="nt">/></span>
<span class="nt"><i</span> <span class="na">class=</span><span class="s">"fa fa-fw fa-envelope-square"</span> <span class="na">aria-hidden=</span><span class="s">"true"</span><span class="nt">></i></span> {{ site.data.ui-text[site.locale].email_label | default: "Email" }}
<span class="nt"></a></span>
<span class="nt"></li></span>
{% endif %}
{% if author.keybase %}
<span class="nt"><li></span>
<span class="nt"><a</span> <span class="na">href=</span><span class="s">"https://keybase.io/{{ author.keybase }}"</span> <span class="na">itemprop=</span><span class="s">"sameAs"</span><span class="nt">></span>
<span class="nt"><i</span> <span class="na">class=</span><span class="s">"fa fa-fw fa-key"</span> <span class="na">aria-hidden=</span><span class="s">"true"</span><span class="nt">></i></span> Keybase
<span class="nt"></a></span>
<span class="nt"></li></span>
{% endif %}
{% if author.twitter %}
<span class="nt"><li></span>
<span class="nt"><a</span> <span class="na">href=</span><span class="s">"https://twitter.com/{{ author.twitter }}"</span> <span class="na">itemprop=</span><span class="s">"sameAs"</span><span class="nt">></span>
<span class="nt"><i</span> <span class="na">class=</span><span class="s">"fa fa-fw fa-twitter-square"</span> <span class="na">aria-hidden=</span><span class="s">"true"</span><span class="nt">></i></span> Twitter
<span class="nt"></a></span>
<span class="nt"></li></span>
{% endif %}
</code></pre></div></div>
<p>Here is the HTML output :</p>
<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nt"><ul</span> <span class="na">class=</span><span class="s">"author__urls social-icons"</span><span class="nt">></span>
<span class="nt"><li</span> <span class="na">itemprop=</span><span class="s">"homeLocation"</span> <span class="na">itemscope</span> <span class="na">itemtype=</span><span class="s">"http://schema.org/Place"</span><span class="nt">></span>
<span class="nt"><i</span> <span class="na">class=</span><span class="s">"fa fa-fw fa-map-marker"</span> <span class="na">aria-hidden=</span><span class="s">"true"</span><span class="nt">></i></span> <span class="nt"><span</span> <span class="na">itemprop=</span><span class="s">"name"</span><span class="nt">></span>Ireland<span class="nt"></span></span>
<span class="nt"></li></span>
<span class="nt"><li></span>
<span class="nt"><a</span> <span class="na">href=</span><span class="s">"https://twitter.com/TheShellNut"</span> <span class="na">itemprop=</span><span class="s">"sameAs"</span><span class="nt">></span>
<span class="nt"><i</span> <span class="na">class=</span><span class="s">"fa fa-fw fa-twitter-square"</span> <span class="na">aria-hidden=</span><span class="s">"true"</span><span class="nt">></i></span> Twitter
<span class="nt"></a></span>
<span class="nt"></li></span>
</code></pre></div></div>
<p>The reason why there is so much whitespace between the 2 list items is that <code class="language-plaintext highlighter-rouge">Liquid</code> tags (like <code class="language-plaintext highlighter-rouge">{% if ... %}</code>) always render a blank line, even when they evaluate to <code class="language-plaintext highlighter-rouge">false</code>.<br />
And this is not the worst case : imagine if we were looping through the above section <code class="language-plaintext highlighter-rouge">N</code> times, then it would generate <code class="language-plaintext highlighter-rouge">N</code> times as much whitespace.</p>
<p>I’m not sure how this could possibly be useful and it makes the HTML output less readable. Plus, this might impact performance since the HTML file is bigger.</p>
<p>So, here is a simple trick to prevent <code class="language-plaintext highlighter-rouge">Liquid</code> tags from generating whitespace : replace this <code class="language-plaintext highlighter-rouge">{% ... %}</code> by this <code class="language-plaintext highlighter-rouge">{%- ... -%}</code> .</p>
<p>This is a fairly new <code class="language-plaintext highlighter-rouge">Liquid</code> feature (introduced in 4.0.0) and it is documented here : <a href="https://shopify.github.io/liquid/basics/whitespace/">Whitespace control</a>. This can safely be used with any Jekyll site hosted on GitHub Pages because the GitHub Pages gem enforces this version of <code class="language-plaintext highlighter-rouge">Liquid</code> <a href="https://pages.github.com/versions/">as a dependency</a>.</p>
<p>Using this <code class="language-plaintext highlighter-rouge">Liquid</code> tag syntax in the above template section outputs the following HTML :</p>
<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nt"><ul</span> <span class="na">class=</span><span class="s">"author__urls social-icons"</span><span class="nt">><li</span> <span class="na">itemprop=</span><span class="s">"homeLocation"</span> <span class="na">itemscope</span> <span class="na">itemtype=</span><span class="s">"http://schema.org/Place"</span><span class="nt">></span>
<span class="nt"><i</span> <span class="na">class=</span><span class="s">"fa fa-fw fa-map-marker"</span> <span class="na">aria-hidden=</span><span class="s">"true"</span><span class="nt">></i></span> <span class="nt"><span</span> <span class="na">itemprop=</span><span class="s">"name"</span><span class="nt">></span>Ireland<span class="nt"></span></span>
<span class="nt"></li><li></span>
<span class="nt"><a</span> <span class="na">href=</span><span class="s">"https://twitter.com/TheShellNut"</span> <span class="na">itemprop=</span><span class="s">"sameAs"</span><span class="nt">></span>
<span class="nt"><i</span> <span class="na">class=</span><span class="s">"fa fa-fw fa-twitter-square"</span> <span class="na">aria-hidden=</span><span class="s">"true"</span><span class="nt">></i></span> Twitter
<span class="nt"></a></span>
<span class="nt"></li></span>
</code></pre></div></div>
<p>Much better, right ?</p>
<h2 id="generalizing-the-hyphen-tag-syntax-across-the-site">Generalizing the hyphen tag syntax across the site</h2>
<p>An easy way to apply this tip to all <code class="language-plaintext highlighter-rouge">Liquid</code> tags across all files, is to use the “Search and replace” feature of your editor of choice.<br />
For example, in Visual Studio Code, I right-clicked on the <code class="language-plaintext highlighter-rouge">_layouts</code> folder in the site’s repo, chose “Find in Folder…” and did that :</p>
<p><img src="https://mathieubuisson.github.io/images/2017-09-08-reducing-whitespace-jekyll-html-opening-tag.jpg" alt="Opening tags" /> <img src="https://mathieubuisson.github.io/images/2017-09-08-reducing-whitespace-jekyll-html-closing-tag.png" alt="Closing tags" /></p>
<p>And then, I did the same for the <code class="language-plaintext highlighter-rouge">_includes</code> folder, since this is where there is the most <code class="language-plaintext highlighter-rouge">Liquid</code> stuff in a typical Jekyll theme.</p>
<p>The performance impact won’t be noticeable in most cases, because the file size reduction is not huge. For example, the <code class="language-plaintext highlighter-rouge">index.html</code> of this site went from 22 KB to 17.4 KB. In the unlikely case where you <em>do</em> notice a performance boost, then you would be better off using proper minification.</p>
<p>But still, more readable and less wasteful HTML is nice to have.</p>TheShellNutThe output HTML of Jekyll-based sites may contain quite a lot a whitespace. This is especially noticeable with pages relying heavily on Liquid tags with conditional logic or looping.Measuring PowerShell code complexity : Why and How2017-04-18T00:00:00+01:002017-04-18T00:00:00+01:00https://mathieubuisson.github.io/measuring-powershell-code-complexity<h2 id="why-does-code-complexity-matter-">Why does code complexity matter ?</h2>
<p>As Administrators or engineers, we deal with complexity all the time.<br />
We build, document, support and deploy complex systems on a pretty-much-daily basis.</p>
<blockquote>
<p>The expertise required to work with these systems is part of what makes us valuable.<br />
So why should we strive to limit our code complexity ?</p>
</blockquote>
<p>Because complex code tends to have the following properties :</p>
<h3 id="difficult-to-readunderstand">Difficult to read/understand</h3>
<p>Some believe that being the only one capable of using/maintaining their tool(s) makes them valuable to the team or business. Don’t be that guy.<br />
If we write code which can be used and understood by others, it doesn’t decrease our value, <strong>it multiplies it</strong>.<br />
Code (even automation scripts) tend to be read more often than it is written. So we should write our code to make it easy to understand for the next person who is going to read it.</p>
<h3 id="difficult-to-test">Difficult to test</h3>
<p>Generally, complex code means long, monolithic, non-modular code.<br />
Which means it is difficult to unit test, because many of the “units” are bundled together in a single big chunk of code. So testing any one of these units in isolation becomes almost impossible, even with mocking.<br />
On top of that, a complex piece of code will require many more tests than a simple one, to achieve the same level of code coverage (more on that later).</p>
<h3 id="more-prone-to-bugs">More prone to bugs</h3>
<p>Complex code tends to contain a lot of logic, which leads to <strong>many different code paths</strong>.<br />
It is fairly easy to picture a correlation between complexity and the number of defects. A maze-like piece of code has <strong>consequences that are difficult to understand and predict</strong>. Some these unexpected consequences may very well be defects.</p>
<h3 id="make-bugs-more-difficult-to-identify-troubleshoot">Make bugs more difficult to identify, troubleshoot</h3>
<p>When we do encounter issues with complex code, the size of the code base and its complexity makes it more difficult and time-consuming to pinpoint where the problem is.</p>
<h3 id="difficult-to-refactorchangemaintain">Difficult to refactor/change/maintain</h3>
<p>To make changes to an existing code base, we first need to understand it. If the code is complex, it will take time and effort to understand.<br />
Also, making changes to a complex system tends to be risky because any change may have unintended/unexpected consequences.<br />
In other words, we may end up being scared of touching this code because <strong>we might break other parts of the system</strong> without even knowing it.</p>
<p>Now that we are on the same page regarding complexity, we’ll want to keep it under control. But to do that, we first need to be able to measure it.</p>
<p>In this post, we are going to take a look at 2 different ways of measuring complexity and apply them to PowerShell code :</p>
<ul>
<li>Cyclomatic complexity</li>
<li>Nesting depth</li>
</ul>
<h2 id="cyclomatic-complexity">Cyclomatic complexity</h2>
<p><a href="https://en.wikipedia.org/wiki/Cyclomatic_complexity">Cyclomatic complexity</a> is the most widely used way of measuring code complexity. It has been used for decades, against many different programming languages, but I couldn’t find any example of it being applied to PowerShell.</p>
<p>It boils down to counting <strong>the number of possible paths through a given section of code</strong>. The section of code for which we measure complexity should represent the smallest unit of functionality.<br />
In object-oriented languages, this would generally be a method. In PowerShell, it would typically be a function.</p>
<p>Cyclomatic complexity has a nice benefit which makes it relevant in modern software delivery practices :<br />
<strong>it has a strong inverse correlation with how testable the code is</strong>.<br />
Indeed, if we want to achieve a test coverage of 100%, the tests need to exercise every possible code paths. This may be difficult or cumbersome to achieve for a piece of code which has dozens (or more) of possible code paths.</p>
<p>The Cyclomatic complexity of a piece of code depends on the number of logic constructs it contains, because logic constructs are where <strong>the flow of execution can branch out to 1 or more path(s)</strong>.</p>
<p>For example, a piece of code with no conditional logic has only 1 possible code path. If we add a <code class="language-plaintext highlighter-rouge">If</code> statement, we now have 2 possible paths : 1 for when the condition in the <code class="language-plaintext highlighter-rouge">If</code>statement is true and 1 for when it is false. So any <code class="language-plaintext highlighter-rouge">If</code> statement increases the Cyclomatic complexity by 1.</p>
<p>So, let’s look at all the PowerShell language constructs that can create branches (decisions) in the code flow, and their impact on Cyclomatic complexity :</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">If</code> : Creates 1 additional code path</li>
<li><code class="language-plaintext highlighter-rouge">If/Else</code> : 1 additional code path (the <code class="language-plaintext highlighter-rouge">Else</code> clause does not cause a new decision)</li>
<li><code class="language-plaintext highlighter-rouge">If/ElseIf/Else</code> : 1 additional code path + 1 per <code class="language-plaintext highlighter-rouge">ElseIf</code> statement</li>
<li><code class="language-plaintext highlighter-rouge">Switch</code> (with a <code class="language-plaintext highlighter-rouge">Break</code> statement in each clause) : 1 additional path per clause, excluding the <code class="language-plaintext highlighter-rouge">Default</code> clause</li>
<li><code class="language-plaintext highlighter-rouge">Switch</code> (with a <code class="language-plaintext highlighter-rouge">Break</code> statement in the clauses) : the number of possible paths explodes (my brain too : I couldn’t find a formula to calculate the number of possible paths based of the number of clauses)</li>
<li>Logical operators <code class="language-plaintext highlighter-rouge">-and</code>, <code class="language-plaintext highlighter-rouge">-or</code>, <code class="language-plaintext highlighter-rouge">-xor</code> : 1 additional path per operator</li>
<li><code class="language-plaintext highlighter-rouge">Try/Catch</code> : 1 additional path per <code class="language-plaintext highlighter-rouge">Catch</code> block. A <code class="language-plaintext highlighter-rouge">Finally</code> block doesn’t create an additional path because it is always executed</li>
<li><code class="language-plaintext highlighter-rouge">Trap</code> statement : 1 additional path</li>
</ul>
<p>By the way, I am not making this up out of thin air, this is heavily based on Cyclomatic complexity calculation examples applied to other languages, like <a href="http://radon.readthedocs.io/en/latest/intro.html">this one</a>.</p>
<p>Looping constructs are a little bit special because their main purpose is not really to determine <strong>if</strong> a code path is to be run or not, but rather <strong>how many times</strong> the code path is to be run. But still, some of the looping constructs in PowerShell allow to decide whether or not the code path in the loop is entered at all, based on a condition.</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">Foreach</code>statement : No additional code path. The body of the <code class="language-plaintext highlighter-rouge">Foreach</code> loop may not be entered in the case where the iterated collection is null, but this is an implied behaviour, not a decision</li>
<li><code class="language-plaintext highlighter-rouge">For</code> loop : the second placeholder in the <code class="language-plaintext highlighter-rouge">For</code> statement is a condition which is evaluated to decide whether or not the body of the loop is entered. So it creates 1 additional code path, unless the second placeholder is empty</li>
<li><code class="language-plaintext highlighter-rouge">While</code> loop : 1 additional path because the body of the loop may not be entered depending on the condition in the <code class="language-plaintext highlighter-rouge">While</code> statement</li>
<li><code class="language-plaintext highlighter-rouge">Do/While</code> or <code class="language-plaintext highlighter-rouge">Do/Until</code> : No additional code path because the body of these loops is always run at least once.</li>
</ul>
<blockquote>
<p>How can we apply this to measure the complexity of a PowerShell function ?</p>
</blockquote>
<p>We need to parse the function definition to detect the above-mentioned language constructs. To do that, we are going to rely on the <strong>AST-style parser</strong> introduced in PowerShell 3.0.</p>
<p>Let’s see a simple example on how to measure the number of code paths due to <code class="language-plaintext highlighter-rouge">If</code> statements in the following dummy function :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kr">Function</span><span class="w"> </span><span class="nf">Test-Conditional</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="p">[</span><span class="kt">CmdletBinding</span><span class="p">()]</span><span class="w">
</span><span class="kr">Param</span><span class="p">(</span><span class="w">
</span><span class="p">[</span><span class="kt">int</span><span class="p">]</span><span class="nv">$IfElseif</span><span class="w">
</span><span class="p">)</span><span class="w">
</span><span class="c"># Testing nested If statement</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$IfElseif</span><span class="w"> </span><span class="o">-gt</span><span class="w"> </span><span class="mi">20</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$IfElseif</span><span class="w"> </span><span class="o">-gt</span><span class="w"> </span><span class="mi">40</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">Write-Host</span><span class="w"> </span><span class="s1">'IfElseif is between 20 and 40'</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">Else</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">Write-Host</span><span class="w"> </span><span class="s1">'IfElseif is greater than 40'</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">Else</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$IfElseif</span><span class="w"> </span><span class="o">-ge</span><span class="w"> </span><span class="mi">10</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">Write-Host</span><span class="w"> </span><span class="s1">'IfElseif is a 2 digit number'</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">Else</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="c">#Testing For statements</span><span class="w">
</span><span class="kr">For</span><span class="w"> </span><span class="p">(</span><span class="nv">$i</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">1</span><span class="p">;</span><span class="w"> </span><span class="nv">$i</span><span class="w"> </span><span class="o">-lt</span><span class="w"> </span><span class="mi">99</span><span class="p">;</span><span class="w"> </span><span class="nv">$i</span><span class="o">++</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">Write-Host</span><span class="w"> </span><span class="s2">"</span><span class="si">$(</span><span class="nv">$IfElseif</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nv">$i</span><span class="si">)</span><span class="s2">"</span><span class="w">
</span><span class="kr">For</span><span class="w"> </span><span class="p">(</span><span class="nv">$j</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span><span class="w"> </span><span class="nv">$j</span><span class="w"> </span><span class="o">-lt</span><span class="w"> </span><span class="mi">10</span><span class="p">;</span><span class="w"> </span><span class="nv">$j</span><span class="o">++</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">Write-Host</span><span class="w"> </span><span class="s2">"</span><span class="si">$(</span><span class="nv">$IfElseif</span><span class="w"> </span><span class="nf">-</span><span class="w"> </span><span class="nv">$j</span><span class="p">)</span><span class="s2">"
}
}
For (</span><span class="nv">$k</span><span class="s2"> = 1;;</span><span class="nv">$k</span><span class="s2">++) {
Write-Host 'No Condition for this loop'
}
}
}
}
</span></code></pre></div></div>
<p>First, we take the file containing this function and extract the function as a <code class="language-plaintext highlighter-rouge">[FunctionDefinitionAst]</code> object, like so :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\</span><span class="err">></span><span class="w"> </span><span class="nv">$File</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'C:\Test-Complexity.ps1'</span><span class="w">
</span><span class="nf">C:\</span><span class="err">></span><span class="w"> </span><span class="nv">$FileAst</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">[</span><span class="no">System.Management.Automation.Language.</span><span class="kt">Parser</span><span class="p">]::</span><span class="nf">ParseFile</span><span class="p">(</span><span class="nv">$File</span><span class="p">,</span><span class="w"> </span><span class="p">[</span><span class="kt">ref</span><span class="p">]</span><span class="nv">$Null</span><span class="p">,</span><span class="w"> </span><span class="p">[</span><span class="kt">ref</span><span class="p">]</span><span class="nv">$Null</span><span class="p">)</span><span class="w">
</span><span class="nf">C:\</span><span class="err">></span><span class="w"> </span><span class="nv">$FileFunctions</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$FileAst</span><span class="o">.</span><span class="nf">FindAll</span><span class="p">({</span><span class="w"> </span><span class="bp">$args</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="w"> </span><span class="o">-is</span><span class="w"> </span><span class="p">[</span><span class="no">System.Management.Automation.Language.</span><span class="kt">FunctionDefinitionAst</span><span class="p">]</span><span class="w"> </span><span class="p">},</span><span class="w"> </span><span class="nv">$False</span><span class="p">)</span><span class="w">
</span><span class="nf">C:\</span><span class="err">></span><span class="w"> </span><span class="nv">$DummyFunction</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$FileFunctions</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="w">
</span><span class="nf">C:\</span><span class="err">></span><span class="w"> </span><span class="nv">$DummyFunction</span><span class="o">.</span><span class="nf">GetType</span><span class="p">()</span><span class="w">
</span><span class="nf">IsPublic</span><span class="w"> </span><span class="nx">IsSerial</span><span class="w"> </span><span class="nx">Name</span><span class="w"> </span><span class="nx">BaseType</span><span class="w">
</span><span class="nf">--------</span><span class="w"> </span><span class="nf">--------</span><span class="w"> </span><span class="nf">----</span><span class="w"> </span><span class="nf">--------</span><span class="w">
</span><span class="nx">True</span><span class="w"> </span><span class="nx">False</span><span class="w"> </span><span class="nx">FunctionDefinitionAst</span><span class="w"> </span><span class="nx">System.Management.Automation.Language.Stat...</span><span class="w">
</span><span class="nx">C:\</span><span class="err">></span><span class="w"> </span><span class="nv">$DummyFunction</span><span class="o">.</span><span class="nf">Name</span><span class="w">
</span><span class="nx">Test-Conditional</span><span class="w">
</span></code></pre></div></div>
<p>Now that we have our function represented as a <code class="language-plaintext highlighter-rouge">[FunctionDefinitionAST]</code> object, we can leverage the AST parser to count the <code class="language-plaintext highlighter-rouge">If</code> statements it contains, using a function like this :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kr">Function</span><span class="w"> </span><span class="nf">Measure-FunctionIfCodePath</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="p">[</span><span class="kt">CmdletBinding</span><span class="p">()]</span><span class="w">
</span><span class="p">[</span><span class="kt">OutputType</span><span class="p">([</span><span class="no">System.</span><span class="kt">Int32</span><span class="p">])]</span><span class="w">
</span><span class="kr">Param</span><span class="w"> </span><span class="p">(</span><span class="w">
</span><span class="p">[</span><span class="kt">Parameter</span><span class="p">(</span><span class="kt">Position</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="kt">Mandatory</span><span class="p">)]</span><span class="w">
</span><span class="p">[</span><span class="no">System.Management.Automation.Language.</span><span class="kt">FunctionDefinitionAst</span><span class="p">]</span><span class="nv">$FunctionDefinition</span><span class="w">
</span><span class="p">)</span><span class="w">
</span><span class="nv">$FunctionText</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$FunctionDefinition</span><span class="o">.</span><span class="nf">Extent</span><span class="o">.</span><span class="nf">Text</span><span class="w">
</span><span class="c"># Converting the function definition to a generic ScriptBlockAst because the FindAll method of FunctionDefinitionAst object work strangely</span><span class="w">
</span><span class="nv">$FunctionAst</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">[</span><span class="no">System.Management.Automation.Language.</span><span class="kt">Parser</span><span class="p">]::</span><span class="nf">ParseInput</span><span class="p">(</span><span class="nv">$FunctionText</span><span class="p">,</span><span class="w"> </span><span class="p">[</span><span class="kt">ref</span><span class="p">]</span><span class="bp">$null</span><span class="p">,</span><span class="w"> </span><span class="p">[</span><span class="kt">ref</span><span class="p">]</span><span class="bp">$null</span><span class="p">)</span><span class="w">
</span><span class="nv">$IfStatements</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$FunctionAst</span><span class="o">.</span><span class="nf">FindAll</span><span class="p">({</span><span class="w"> </span><span class="bp">$args</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="w"> </span><span class="o">-is</span><span class="w"> </span><span class="p">[</span><span class="no">System.Management.Automation.Language.</span><span class="kt">IfStatementAst</span><span class="p">]</span><span class="w"> </span><span class="p">},</span><span class="w"> </span><span class="nv">$True</span><span class="p">)</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="o">-not</span><span class="p">(</span><span class="nv">$IfStatements</span><span class="p">)</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="kr">return</span><span class="w"> </span><span class="p">[</span><span class="kt">int</span><span class="p">]</span><span class="mi">0</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="c"># If and ElseIf clauses are creating an additional path, not Else clauses</span><span class="w">
</span><span class="kr">return</span><span class="w"> </span><span class="nv">$IfStatements</span><span class="o">.</span><span class="nf">Clauses</span><span class="o">.</span><span class="nf">Count</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>To get all the <code class="language-plaintext highlighter-rouge">If</code> statements, we use the <code class="language-plaintext highlighter-rouge">FindAll</code> method and we tell it to search for objects of the type <code class="language-plaintext highlighter-rouge">[IfStatementAst]</code>. No regex needed when we have a proper parsing API !</p>
<p class="notice--info">The <code class="language-plaintext highlighter-rouge">$True</code> argument for the <code class="language-plaintext highlighter-rouge">FindAll</code> method is to include nested objects.</p>
<p>Also, the <code class="language-plaintext highlighter-rouge">Clauses</code> property of the <code class="language-plaintext highlighter-rouge">[IfStatementAst]</code> object represents the <code class="language-plaintext highlighter-rouge">ElseIf</code> clauses tied to that <code class="language-plaintext highlighter-rouge">If</code> statement.<br />
<code class="language-plaintext highlighter-rouge">Else</code> clauses are considered by AST as a separate statement, I’m not sure why, but this is very convenient for our purpose. We just need to count the number of clauses to get the number of code paths due to <code class="language-plaintext highlighter-rouge">If</code> and <code class="language-plaintext highlighter-rouge">ElseIf</code>.</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\</span><span class="err">></span><span class="w"> </span><span class="nx">Measure-FunctionIfCodePath</span><span class="w"> </span><span class="nt">-FunctionDefinition</span><span class="w"> </span><span class="nv">$DummyFunction</span><span class="w">
</span><span class="mi">3</span><span class="w">
</span></code></pre></div></div>
<p>This is correct because our dummy function contains 3 <code class="language-plaintext highlighter-rouge">If</code> statements with 0 <code class="language-plaintext highlighter-rouge">ElseIf</code> clause.</p>
<p>This dummy function contains <code class="language-plaintext highlighter-rouge">For</code> loops as well, so let’s see how to get the number of additional code paths due to <code class="language-plaintext highlighter-rouge">For</code> loops :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kr">Function</span><span class="w"> </span><span class="nf">Measure-FunctionForCodePath</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="p">[</span><span class="kt">CmdletBinding</span><span class="p">()]</span><span class="w">
</span><span class="p">[</span><span class="kt">OutputType</span><span class="p">([</span><span class="no">System.</span><span class="kt">Int32</span><span class="p">])]</span><span class="w">
</span><span class="kr">Param</span><span class="w"> </span><span class="p">(</span><span class="w">
</span><span class="p">[</span><span class="kt">Parameter</span><span class="p">(</span><span class="kt">Position</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="kt">Mandatory</span><span class="o">=</span><span class="nv">$True</span><span class="p">)]</span><span class="w">
</span><span class="p">[</span><span class="no">System.Management.Automation.Language.</span><span class="kt">FunctionDefinitionAst</span><span class="p">]</span><span class="nv">$FunctionDefinition</span><span class="w">
</span><span class="p">)</span><span class="w">
</span><span class="nv">$FunctionText</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$FunctionDefinition</span><span class="o">.</span><span class="nf">Extent</span><span class="o">.</span><span class="nf">Text</span><span class="w">
</span><span class="c"># Converting the function definition to a generic ScriptBlockAst because the FindAll method of FunctionDefinitionAst object work strangely</span><span class="w">
</span><span class="nv">$FunctionAst</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">[</span><span class="no">System.Management.Automation.Language.</span><span class="kt">Parser</span><span class="p">]::</span><span class="nf">ParseInput</span><span class="p">(</span><span class="nv">$FunctionText</span><span class="p">,</span><span class="w"> </span><span class="p">[</span><span class="kt">ref</span><span class="p">]</span><span class="bp">$null</span><span class="p">,</span><span class="w"> </span><span class="p">[</span><span class="kt">ref</span><span class="p">]</span><span class="bp">$null</span><span class="p">)</span><span class="w">
</span><span class="nv">$ForStatements</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$FunctionAst</span><span class="o">.</span><span class="nf">FindAll</span><span class="p">({</span><span class="w"> </span><span class="bp">$args</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="w"> </span><span class="o">-is</span><span class="w"> </span><span class="p">[</span><span class="no">System.Management.Automation.Language.</span><span class="kt">ForStatementAst</span><span class="p">]</span><span class="w"> </span><span class="p">},</span><span class="w"> </span><span class="nv">$True</span><span class="p">)</span><span class="w">
</span><span class="c"># Taking into account the rare cases where For statements don't contain a condition</span><span class="w">
</span><span class="nv">$ConditionalForStatements</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$ForStatements</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Where-Object</span><span class="w"> </span><span class="nx">Condition</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="o">-not</span><span class="p">(</span><span class="nv">$ConditionalForStatements</span><span class="p">)</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="kr">return</span><span class="w"> </span><span class="p">[</span><span class="kt">int</span><span class="p">]</span><span class="mi">0</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">return</span><span class="w"> </span><span class="nv">$ConditionalForStatements</span><span class="o">.</span><span class="nf">Count</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>We are still using the same <code class="language-plaintext highlighter-rouge">FindAll</code> method but this time, we tell it to search for <code class="language-plaintext highlighter-rouge">[ForStatementAst]</code> objects (including nested ones).</p>
<p>We need to count only the <code class="language-plaintext highlighter-rouge">For</code> loops where the second placeholder (the condition) is not empty. This is easy to do because this placeholder is represented by AST as the <code class="language-plaintext highlighter-rouge">Condition</code> property of <code class="language-plaintext highlighter-rouge">[ForStatementAst]</code> objects. That’s why we filter on the <code class="language-plaintext highlighter-rouge">Condition</code> property.</p>
<p>What is the result for our dummy function ?</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\</span><span class="err">></span><span class="w"> </span><span class="nx">Measure-FunctionForCodePath</span><span class="w"> </span><span class="nv">$DummyFunction</span><span class="w">
</span><span class="mi">2</span><span class="w">
</span></code></pre></div></div>
<p>This is correct because it counted all the <code class="language-plaintext highlighter-rouge">For</code> loops, including the nested one, but excluding the one which has no condition.</p>
<p>Now that we know how to count additional code paths due to <code class="language-plaintext highlighter-rouge">If</code>, <code class="language-plaintext highlighter-rouge">ElseIf</code> and <code class="language-plaintext highlighter-rouge">For</code>, we can use similar techniques for the other PowerShell language constructs mentioned earlier.<br />
I’m not going to show examples for every one of them here, but you can have a look at the functions <a href="https://github.com/MathieuBuisson/PSCodeHealth/tree/master/PSCodeHealth/Private/Metrics">in there</a>.</p>
<p>We can aggregate the number of the paths found by all these construct-specific functions in a separate function which takes a <code class="language-plaintext highlighter-rouge">FunctionDefinitionAST</code> object as input and spits out the total cyclomatic complexity of that function.<br />
It looks like this :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kr">Function</span><span class="w"> </span><span class="nf">Measure-FunctionComplexity</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="p">[</span><span class="kt">CmdletBinding</span><span class="p">()]</span><span class="w">
</span><span class="p">[</span><span class="kt">OutputType</span><span class="p">([</span><span class="no">System.</span><span class="kt">Int32</span><span class="p">])]</span><span class="w">
</span><span class="kr">Param</span><span class="w"> </span><span class="p">(</span><span class="w">
</span><span class="p">[</span><span class="kt">Parameter</span><span class="p">(</span><span class="kt">Position</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="kt">Mandatory</span><span class="o">=</span><span class="nv">$True</span><span class="p">)]</span><span class="w">
</span><span class="p">[</span><span class="no">System.Management.Automation.Language.</span><span class="kt">FunctionDefinitionAst</span><span class="p">]</span><span class="nv">$FunctionDefinition</span><span class="w">
</span><span class="p">)</span><span class="w">
</span><span class="c"># Default complexity value for code which contains no branching statement (1 code path)</span><span class="w">
</span><span class="p">[</span><span class="kt">int</span><span class="p">]</span><span class="nv">$DefaultComplexity</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">1</span><span class="w">
</span><span class="nv">$ForPaths</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">Measure-FunctionForCodePath</span><span class="w"> </span><span class="nv">$FunctionDefinition</span><span class="w">
</span><span class="nv">$IfPaths</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">Measure-FunctionIfCodePath</span><span class="w"> </span><span class="nv">$FunctionDefinition</span><span class="w">
</span><span class="nv">$LogicalOpPaths</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">Measure-FunctionLogicalOpCodePath</span><span class="w"> </span><span class="nv">$FunctionDefinition</span><span class="w">
</span><span class="nv">$SwitchPaths</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">Measure-FunctionSwitchCodePath</span><span class="w"> </span><span class="nv">$FunctionDefinition</span><span class="w">
</span><span class="nv">$TrapCatchPaths</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">Measure-FunctionTrapCatchCodePath</span><span class="w"> </span><span class="nv">$FunctionDefinition</span><span class="w">
</span><span class="nv">$WhilePaths</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">Measure-FunctionWhileCodePath</span><span class="w"> </span><span class="nv">$FunctionDefinition</span><span class="w">
</span><span class="p">[</span><span class="kt">int</span><span class="p">]</span><span class="nv">$TotalComplexity</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$DefaultComplexity</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nv">$ForPaths</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nv">$IfPaths</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nv">$LogicalOpPaths</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nv">$SwitchPaths</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nv">$TrapCatchPaths</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nv">$WhilePaths</span><span class="w">
</span><span class="kr">return</span><span class="w"> </span><span class="nv">$TotalComplexity</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>We start with an initial value of 1 because a piece of code which doesn’t contain any logic/branching has exactly 1 code path. And the total cyclomatic complexity of our function is :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\</span><span class="err">></span><span class="w"> </span><span class="nx">Measure-FunctionComplexity</span><span class="w"> </span><span class="nv">$DummyFunction</span><span class="w">
</span><span class="mi">6</span><span class="w">
</span></code></pre></div></div>
<blockquote>
<p>That’s great, but what is this telling us ?<br />
Is our function too complex or not ?</p>
</blockquote>
<p>Well, the purpose of metrics is to help our brain, <strong>not to replace it</strong>.<br />
Like any metric, we should take this number with a grain of salt and adapt it to our context. That said, the most commonly used thresholds are 10 and 15.<br />
I have a feeling that higher numbers might be fine for PowerShell code because it tends to be more readable than other languages, but that’s very much arguable.</p>
<p>Besides, this cyclomatic measurement is <strong>only one side of the complexity coin</strong>. To illustrate this, let’s look at this piece of code :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$Now</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">Get-Date</span><span class="w">
</span><span class="kr">Switch</span><span class="w"> </span><span class="p">(</span><span class="nv">$Now</span><span class="o">.</span><span class="nf">Month</span><span class="p">)</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="mi">1</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nv">$Month</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'January'</span><span class="p">;</span><span class="w"> </span><span class="kr">break</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="mi">2</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nv">$Month</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'February'</span><span class="p">;</span><span class="w"> </span><span class="kr">break</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="mi">3</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nv">$Month</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'March'</span><span class="p">;</span><span class="w"> </span><span class="kr">break</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="mi">4</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nv">$Month</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'April'</span><span class="p">;</span><span class="w"> </span><span class="kr">break</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="mi">5</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nv">$Month</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'May'</span><span class="p">;</span><span class="w"> </span><span class="kr">break</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="mi">6</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nv">$Month</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'June'</span><span class="p">;</span><span class="w"> </span><span class="kr">break</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="mi">7</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nv">$Month</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'July'</span><span class="p">;</span><span class="w"> </span><span class="kr">break</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="mi">8</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nv">$Month</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'August'</span><span class="p">;</span><span class="w"> </span><span class="kr">break</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="mi">9</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nv">$Month</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'September'</span><span class="p">;</span><span class="w"> </span><span class="kr">break</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="mi">10</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nv">$Month</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'October'</span><span class="p">;</span><span class="w"> </span><span class="kr">break</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="mi">11</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nv">$Month</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'November'</span><span class="p">;</span><span class="w"> </span><span class="kr">break</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="mi">12</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nv">$Month</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'December'</span><span class="p">;</span><span class="w"> </span><span class="kr">break</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>The cyclomatic complexity of this piece of code is 13. So the cyclomatic complexity is pretty high, even though most coders would consider this piece of code as easy to understand and maintain.</p>
<p>So it can be useful to look at another metric, which measures a different aspect of complexity.</p>
<h2 id="maximum-nesting-depth-">Maximum nesting depth :</h2>
<blockquote>
<p>What is the maximum <a href="https://help.semmle.com/wiki/display/CSHARP/Nesting+depth">nesting depth</a> ?</p>
</blockquote>
<p>It is the depth of the most deeply nested code in a given piece of code (a function, here).<br />
For example, if we have an <code class="language-plaintext highlighter-rouge">If</code> statement nested in a loop, which is itself nested in a <code class="language-plaintext highlighter-rouge">Catch</code> block, the most deeply nested section is the body of the <code class="language-plaintext highlighter-rouge">If</code> statement and its nesting depth is 3.</p>
<p>Let’s take a new look at our dummy function, but this time <strong>under the nesting depth lens</strong> (it’s the same function but I show it here again so you don’t have scroll up) :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kr">Function</span><span class="w"> </span><span class="nf">Test-Conditional</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="p">[</span><span class="kt">CmdletBinding</span><span class="p">()]</span><span class="w">
</span><span class="kr">Param</span><span class="p">(</span><span class="w">
</span><span class="p">[</span><span class="kt">int</span><span class="p">]</span><span class="nv">$IfElseif</span><span class="w">
</span><span class="p">)</span><span class="w">
</span><span class="c"># Testing nested If statement</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$IfElseif</span><span class="w"> </span><span class="o">-gt</span><span class="w"> </span><span class="mi">20</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$IfElseif</span><span class="w"> </span><span class="o">-gt</span><span class="w"> </span><span class="mi">40</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">Write-Host</span><span class="w"> </span><span class="s1">'IfElseif is between 20 and 40'</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">Else</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">Write-Host</span><span class="w"> </span><span class="s1">'IfElseif is greater than 40'</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">Else</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$IfElseif</span><span class="w"> </span><span class="o">-ge</span><span class="w"> </span><span class="mi">10</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">Write-Host</span><span class="w"> </span><span class="s1">'IfElseif is a 2 digit number'</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">Else</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="c">#Testing For statements</span><span class="w">
</span><span class="kr">For</span><span class="w"> </span><span class="p">(</span><span class="nv">$i</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">1</span><span class="p">;</span><span class="w"> </span><span class="nv">$i</span><span class="w"> </span><span class="o">-lt</span><span class="w"> </span><span class="mi">99</span><span class="p">;</span><span class="w"> </span><span class="nv">$i</span><span class="o">++</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">Write-Host</span><span class="w"> </span><span class="s2">"</span><span class="si">$(</span><span class="nv">$IfElseif</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nv">$i</span><span class="si">)</span><span class="s2">"</span><span class="w">
</span><span class="kr">For</span><span class="w"> </span><span class="p">(</span><span class="nv">$j</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span><span class="w"> </span><span class="nv">$j</span><span class="w"> </span><span class="o">-lt</span><span class="w"> </span><span class="mi">10</span><span class="p">;</span><span class="w"> </span><span class="nv">$j</span><span class="o">++</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nf">Write-Host</span><span class="w"> </span><span class="s2">"</span><span class="si">$(</span><span class="nv">$IfElseif</span><span class="w"> </span><span class="nf">-</span><span class="w"> </span><span class="nv">$j</span><span class="p">)</span><span class="s2">"
}
}
For (</span><span class="nv">$k</span><span class="s2"> = 1;;</span><span class="nv">$k</span><span class="s2">++) {
Write-Host 'No Condition for this loop'
}
}
}
}
</span></code></pre></div></div>
<p>The code at line 11 is inside an <code class="language-plaintext highlighter-rouge">If</code> Statement, which is itself in another <code class="language-plaintext highlighter-rouge">If</code> statement, so its nesting depth is 2.</p>
<p>The code at line 27 is inside a <code class="language-plaintext highlighter-rouge">For</code> loop, which is itself in a <code class="language-plaintext highlighter-rouge">For</code> loop, which is inside an <code class="language-plaintext highlighter-rouge">Else</code> statement, which is nested in another <code class="language-plaintext highlighter-rouge">Else</code> statement. So the nesting depth of line 27 is 4, and it is the most deeply nested section in our dummy function.<br />
So the maximum nesting depth of our function is 4.</p>
<blockquote>
<p>How does this matter ?<br />
What aspect of complexity is this measuring ?</p>
</blockquote>
<p>Well, this line is very simple in itself but, to understand completely what it does and under which condition, we need to understand its context.</p>
<p>We need to understand the value of <code class="language-plaintext highlighter-rouge">$j</code> (which changes through the inner loop) and the value of <code class="language-plaintext highlighter-rouge">$i</code> (which changes through the outer loop). We also need to understand the <code class="language-plaintext highlighter-rouge">If</code> statement at line 18 and even all the way up to the first <code class="language-plaintext highlighter-rouge">If</code> because we need to know in which set of conditions this section of code is run.</p>
<p>So basically, the nesting depth of a piece of code measures <strong>the complexity of its context</strong>.</p>
<p>Visually, this is easy to follow by looking at the indentation level, but to determine programmatically the nesting depth of a piece of code, we cannot assume that code is always properly indented.<br />
I could have used a recursive function or built a custom <a href="https://msdn.microsoft.com/en-us/library/system.management.automation.language.astvisitor(v=vs.85).aspx">AstVisitor</a>, but I found these methods too … complex (pun intended).</p>
<p>Fortunately, all the logic, control, and looping constructs where nesting occurs in PowerShell have something in common : <strong>curly braces</strong>.<br />
So we can extract all the curly braces from the code and count them to determine the nesting level, like so :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kr">Function</span><span class="w"> </span><span class="nf">Measure-FunctionMaxNestingDepth</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="p">[</span><span class="kt">CmdletBinding</span><span class="p">()]</span><span class="w">
</span><span class="p">[</span><span class="kt">OutputType</span><span class="p">([</span><span class="kt">Int32</span><span class="p">])]</span><span class="w">
</span><span class="kr">Param</span><span class="w"> </span><span class="p">(</span><span class="w">
</span><span class="p">[</span><span class="kt">Parameter</span><span class="p">(</span><span class="kt">Position</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="kt">Mandatory</span><span class="p">)]</span><span class="w">
</span><span class="p">[</span><span class="no">System.Management.Automation.Language.</span><span class="kt">FunctionDefinitionAst</span><span class="p">]</span><span class="nv">$FunctionDefinition</span><span class="w">
</span><span class="p">)</span><span class="w">
</span><span class="nv">$FunctionText</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$FunctionDefinition</span><span class="o">.</span><span class="nf">Extent</span><span class="o">.</span><span class="nf">Text</span><span class="w">
</span><span class="nv">$Tokens</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$Null</span><span class="w">
</span><span class="nv">$Null</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">[</span><span class="no">System.Management.Automation.Language.</span><span class="kt">Parser</span><span class="p">]::</span><span class="nf">ParseInput</span><span class="p">(</span><span class="nv">$FunctionText</span><span class="p">,</span><span class="w"> </span><span class="p">[</span><span class="kt">ref</span><span class="p">]</span><span class="nv">$Tokens</span><span class="p">,</span><span class="w"> </span><span class="p">[</span><span class="kt">ref</span><span class="p">]</span><span class="nv">$Null</span><span class="p">)</span><span class="w">
</span><span class="p">[</span><span class="no">System.Collections.</span><span class="kt">ArrayList</span><span class="p">]</span><span class="nv">$NestingDepthValues</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">@()</span><span class="w">
</span><span class="p">[</span><span class="kt">Int32</span><span class="p">]</span><span class="nv">$NestingDepth</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">0</span><span class="w">
</span><span class="p">[</span><span class="no">System.Collections.</span><span class="kt">ArrayList</span><span class="p">]</span><span class="nv">$CurlyBrackets</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$Tokens</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nf">Where-Object</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="bp">$_</span><span class="o">.</span><span class="nf">Kind</span><span class="w"> </span><span class="nt">-in</span><span class="w"> </span><span class="s1">'AtCurly'</span><span class="p">,</span><span class="s1">'LCurly'</span><span class="p">,</span><span class="s1">'RCurly'</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="c"># Removing the first opening curly and the last closing curly because they belong to the function itself</span><span class="w">
</span><span class="nv">$CurlyBrackets</span><span class="o">.</span><span class="nf">RemoveAt</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span><span class="w">
</span><span class="nv">$CurlyBrackets</span><span class="o">.</span><span class="nf">RemoveAt</span><span class="p">((</span><span class="nv">$CurlyBrackets</span><span class="o">.</span><span class="nf">Count</span><span class="w"> </span><span class="nf">-</span><span class="w"> </span><span class="nx">1</span><span class="p">))</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="o">-not</span><span class="w"> </span><span class="nv">$CurlyBrackets</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="kr">return</span><span class="w"> </span><span class="nv">$NestingDepth</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">Foreach</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$CurlyBracket</span><span class="w"> </span><span class="kr">in</span><span class="w"> </span><span class="nv">$CurlyBrackets</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="kr">If</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$CurlyBracket</span><span class="o">.</span><span class="nf">Kind</span><span class="w"> </span><span class="nt">-in</span><span class="w"> </span><span class="s1">'AtCurly'</span><span class="p">,</span><span class="s1">'LCurly'</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$NestingDepth</span><span class="o">++</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">ElseIf</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="nv">$CurlyBracket</span><span class="o">.</span><span class="nf">Kind</span><span class="w"> </span><span class="o">-eq</span><span class="w"> </span><span class="s1">'RCurly'</span><span class="w"> </span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$NestingDepth</span><span class="nf">--</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nv">$NestingDepthValues</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="nv">$NestingDepth</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="nf">Write-Verbose</span><span class="w"> </span><span class="s2">"Number of nesting depth values : </span><span class="si">$(</span><span class="nv">$NestingDepthValues</span><span class="o">.</span><span class="nf">Count</span><span class="p">)</span><span class="s2">"
</span><span class="nv">$MaxDepthValue</span><span class="s2"> = (</span><span class="nv">$NestingDepthValues</span><span class="s2"> | Measure-Object -Maximum).Maximum -as [Int32]
return </span><span class="nv">$MaxDepthValue</span><span class="s2">
}
</span></code></pre></div></div>
<p>First, we parse the text of the function definition into tokens.</p>
<p><code class="language-plaintext highlighter-rouge">$NestingDepth</code> represents the nesting level at any given point in time. Its different values at different points in time are stored in <code class="language-plaintext highlighter-rouge">$NestingDepthValues</code>.</p>
<p>Then, we filter the tokens corresponding to curly braces.<br />
<code class="language-plaintext highlighter-rouge">AtCurly</code> is a special case, these are the tokens representing opening braces for hashtables. I chose to include the curly braces for hashtables in the nesting calculation because there can be scriptblocks and expressions inside hashtables.</p>
<p>The core of this function is the <code class="language-plaintext highlighter-rouge">Foreach</code> loop. It loops through all the curly brace tokens and it increments by 1 the nesting depth if it is an opening curly brace and decrements it by 1 if it is a closing brace.<br />
Then, the new nesting depth value is added to <code class="language-plaintext highlighter-rouge">$NestingDepthValues</code>, which keeps track of all the different values of nesting depth.</p>
<p>When we are done looping through the curly brace tokens, we take all the values in <code class="language-plaintext highlighter-rouge">$NestingDepthValues</code> and we keep the highest one.</p>
<p>Let’s run that against our dummy function :</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">C:\</span><span class="err">></span><span class="w"> </span><span class="nx">Measure-FunctionMaxNestingDepth</span><span class="w"> </span><span class="nv">$DummyFunction</span><span class="w">
</span><span class="mi">4</span><span class="w">
</span></code></pre></div></div>
<p>As expected, the maximum nesting depth of the dummy function is 4.</p>
<blockquote>
<p>Great, but is 4 a good number or a bad number ?</p>
</blockquote>
<p>As you probably guessed, the answer is : <em>it depends</em>.<br />
But there seems to be a consensus, even across different languages (<a href="http://www.codergears.com/xclarify/Metrics#MetricsOnMethods">Java on this page</a> and <a href="http://www.ndepend.com/docs/code-metrics#ILNestingDepth">C# here</a> : a nesting depth of 4 or higher is complex, a nesting depth of 8 or higher is extremely complex.</p>
<p>With these 2 complementary metrics, we have a basis to make decisions on whether or not we should split our code into smaller, simpler functions.<br />
We can also use these metrics to track our progress when refactoring a PowerShell project, to ensure we are making it more testable and maintainable. <strong>Technical excellence is not a destination, it’s a journey</strong>. So how the numbers are changing over time is more important than the numbers themselves.</p>TheShellNutWhy does code complexity matter ? As Administrators or engineers, we deal with complexity all the time. We build, document, support and deploy complex systems on a pretty-much-daily basis. The expertise required to work with these systems is part of what makes us valuable. So why should we strive to limit our code complexity ? Because complex code tends to have the following properties : Difficult to read/understand Some believe that being the only one capable of using/maintaining their tool(s) makes them valuable to the team or business. Don’t be that guy. If we write code which can be used and understood by others, it doesn’t decrease our value, it multiplies it. Code (even automation scripts) tend to be read more often than it is written. So we should write our code to make it easy to understand for the next person who is going to read it. Difficult to test Generally, complex code means long, monolithic, non-modular code. Which means it is difficult to unit test, because many of the “units” are bundled together in a single big chunk of code. So testing any one of these units in isolation becomes almost impossible, even with mocking. On top of that, a complex piece of code will require many more tests than a simple one, to achieve the same level of code coverage (more on that later). More prone to bugs Complex code tends to contain a lot of logic, which leads to many different code paths. It is fairly easy to picture a correlation between complexity and the number of defects. A maze-like piece of code has consequences that are difficult to understand and predict. Some these unexpected consequences may very well be defects. Make bugs more difficult to identify, troubleshoot When we do encounter issues with complex code, the size of the code base and its complexity makes it more difficult and time-consuming to pinpoint where the problem is. Difficult to refactor/change/maintain To make changes to an existing code base, we first need to understand it. If the code is complex, it will take time and effort to understand. Also, making changes to a complex system tends to be risky because any change may have unintended/unexpected consequences. In other words, we may end up being scared of touching this code because we might break other parts of the system without even knowing it. Now that we are on the same page regarding complexity, we’ll want to keep it under control. But to do that, we first need to be able to measure it. In this post, we are going to take a look at 2 different ways of measuring complexity and apply them to PowerShell code : Cyclomatic complexity Nesting depth Cyclomatic complexity Cyclomatic complexity is the most widely used way of measuring code complexity. It has been used for decades, against many different programming languages, but I couldn’t find any example of it being applied to PowerShell. It boils down to counting the number of possible paths through a given section of code. The section of code for which we measure complexity should represent the smallest unit of functionality. In object-oriented languages, this would generally be a method. In PowerShell, it would typically be a function. Cyclomatic complexity has a nice benefit which makes it relevant in modern software delivery practices : it has a strong inverse correlation with how testable the code is. Indeed, if we want to achieve a test coverage of 100%, the tests need to exercise every possible code paths. This may be difficult or cumbersome to achieve for a piece of code which has dozens (or more) of possible code paths. The Cyclomatic complexity of a piece of code depends on the number of logic constructs it contains, because logic constructs are where the flow of execution can branch out to 1 or more path(s). For example, a piece of code with no conditional logic has only 1 possible code path. If we add a If statement, we now have 2 possible paths : 1 for when the condition in the Ifstatement is true and 1 for when it is false. So any If statement increases the Cyclomatic complexity by 1. So, let’s look at all the PowerShell language constructs that can create branches (decisions) in the code flow, and their impact on Cyclomatic complexity : If : Creates 1 additional code path If/Else : 1 additional code path (the Else clause does not cause a new decision) If/ElseIf/Else : 1 additional code path + 1 per ElseIf statement Switch (with a Break statement in each clause) : 1 additional path per clause, excluding the Default clause Switch (with a Break statement in the clauses) : the number of possible paths explodes (my brain too : I couldn’t find a formula to calculate the number of possible paths based of the number of clauses) Logical operators -and, -or, -xor : 1 additional path per operator Try/Catch : 1 additional path per Catch block. A Finally block doesn’t create an additional path because it is always executed Trap statement : 1 additional path By the way, I am not making this up out of thin air, this is heavily based on Cyclomatic complexity calculation examples applied to other languages, like this one. Looping constructs are a little bit special because their main purpose is not really to determine if a code path is to be run or not, but rather how many times the code path is to be run. But still, some of the looping constructs in PowerShell allow to decide whether or not the code path in the loop is entered at all, based on a condition. Foreachstatement : No additional code path. The body of the Foreach loop may not be entered in the case where the iterated collection is null, but this is an implied behaviour, not a decision For loop : the second placeholder in the For statement is a condition which is evaluated to decide whether or not the body of the loop is entered. So it creates 1 additional code path, unless the second placeholder is empty While loop : 1 additional path because the body of the loop may not be entered depending on the condition in the While statement Do/While or Do/Until : No additional code path because the body of these loops is always run at least once. How can we apply this to measure the complexity of a PowerShell function ? We need to parse the function definition to detect the above-mentioned language constructs. To do that, we are going to rely on the AST-style parser introduced in PowerShell 3.0. Let’s see a simple example on how to measure the number of code paths due to If statements in the following dummy function : Function Test-Conditional { [CmdletBinding()] Param( [int]$IfElseif ) # Testing nested If statement If ( $IfElseif -gt 20 ) { If ( $IfElseif -gt 40 ) { Write-Host 'IfElseif is between 20 and 40' } Else { Write-Host 'IfElseif is greater than 40' } } Else { If ( $IfElseif -ge 10 ) { Write-Host 'IfElseif is a 2 digit number' } Else { #Testing For statements For ($i = 1; $i -lt 99; $i++) { Write-Host "$($IfElseif + $i)" For ($j = 0; $j -lt 10; $j++) { Write-Host "$($IfElseif - $j)" } } For ($k = 1;;$k++) { Write-Host 'No Condition for this loop' } } } } First, we take the file containing this function and extract the function as a [FunctionDefinitionAst] object, like so : C:\> $File = 'C:\Test-Complexity.ps1' C:\> $FileAst = [System.Management.Automation.Language.Parser]::ParseFile($File, [ref]$Null, [ref]$Null) C:\> $FileFunctions = $FileAst.FindAll({ $args[0] -is [System.Management.Automation.Language.FunctionDefinitionAst] }, $False) C:\> $DummyFunction = $FileFunctions[0] C:\> $DummyFunction.GetType() IsPublic IsSerial Name BaseType -------- -------- ---- -------- True False FunctionDefinitionAst System.Management.Automation.Language.Stat... C:\> $DummyFunction.Name Test-Conditional Now that we have our function represented as a [FunctionDefinitionAST] object, we can leverage the AST parser to count the If statements it contains, using a function like this : Function Measure-FunctionIfCodePath { [CmdletBinding()] [OutputType([System.Int32])] Param ( [Parameter(Position=0, Mandatory)] [System.Management.Automation.Language.FunctionDefinitionAst]$FunctionDefinition ) $FunctionText = $FunctionDefinition.Extent.Text # Converting the function definition to a generic ScriptBlockAst because the FindAll method of FunctionDefinitionAst object work strangely $FunctionAst = [System.Management.Automation.Language.Parser]::ParseInput($FunctionText, [ref]$null, [ref]$null) $IfStatements = $FunctionAst.FindAll({ $args[0] -is [System.Management.Automation.Language.IfStatementAst] }, $True) If ( -not($IfStatements) ) { return [int]0 } # If and ElseIf clauses are creating an additional path, not Else clauses return $IfStatements.Clauses.Count } To get all the If statements, we use the FindAll method and we tell it to search for objects of the type [IfStatementAst]. No regex needed when we have a proper parsing API ! The $True argument for the FindAll method is to include nested objects. Also, the Clauses property of the [IfStatementAst] object represents the ElseIf clauses tied to that If statement. Else clauses are considered by AST as a separate statement, I’m not sure why, but this is very convenient for our purpose. We just need to count the number of clauses to get the number of code paths due to If and ElseIf. C:\> Measure-FunctionIfCodePath -FunctionDefinition $DummyFunction 3 This is correct because our dummy function contains 3 If statements with 0 ElseIf clause. This dummy function contains For loops as well, so let’s see how to get the number of additional code paths due to For loops : Function Measure-FunctionForCodePath { [CmdletBinding()] [OutputType([System.Int32])] Param ( [Parameter(Position=0, Mandatory=$True)] [System.Management.Automation.Language.FunctionDefinitionAst]$FunctionDefinition ) $FunctionText = $FunctionDefinition.Extent.Text # Converting the function definition to a generic ScriptBlockAst because the FindAll method of FunctionDefinitionAst object work strangely $FunctionAst = [System.Management.Automation.Language.Parser]::ParseInput($FunctionText, [ref]$null, [ref]$null) $ForStatements = $FunctionAst.FindAll({ $args[0] -is [System.Management.Automation.Language.ForStatementAst] }, $True) # Taking into account the rare cases where For statements don't contain a condition $ConditionalForStatements = $ForStatements | Where-Object Condition If ( -not($ConditionalForStatements) ) { return [int]0 } return $ConditionalForStatements.Count } We are still using the same FindAll method but this time, we tell it to search for [ForStatementAst] objects (including nested ones). We need to count only the For loops where the second placeholder (the condition) is not empty. This is easy to do because this placeholder is represented by AST as the Condition property of [ForStatementAst] objects. That’s why we filter on the Condition property. What is the result for our dummy function ? C:\> Measure-FunctionForCodePath $DummyFunction 2 This is correct because it counted all the For loops, including the nested one, but excluding the one which has no condition. Now that we know how to count additional code paths due to If, ElseIf and For, we can use similar techniques for the other PowerShell language constructs mentioned earlier. I’m not going to show examples for every one of them here, but you can have a look at the functions in there. We can aggregate the number of the paths found by all these construct-specific functions in a separate function which takes a FunctionDefinitionAST object as input and spits out the total cyclomatic complexity of that function. It looks like this : Function Measure-FunctionComplexity { [CmdletBinding()] [OutputType([System.Int32])] Param ( [Parameter(Position=0, Mandatory=$True)] [System.Management.Automation.Language.FunctionDefinitionAst]$FunctionDefinition ) # Default complexity value for code which contains no branching statement (1 code path) [int]$DefaultComplexity = 1 $ForPaths = Measure-FunctionForCodePath $FunctionDefinition $IfPaths = Measure-FunctionIfCodePath $FunctionDefinition $LogicalOpPaths = Measure-FunctionLogicalOpCodePath $FunctionDefinition $SwitchPaths = Measure-FunctionSwitchCodePath $FunctionDefinition $TrapCatchPaths = Measure-FunctionTrapCatchCodePath $FunctionDefinition $WhilePaths = Measure-FunctionWhileCodePath $FunctionDefinition [int]$TotalComplexity = $DefaultComplexity + $ForPaths + $IfPaths + $LogicalOpPaths + $SwitchPaths + $TrapCatchPaths + $WhilePaths return $TotalComplexity } We start with an initial value of 1 because a piece of code which doesn’t contain any logic/branching has exactly 1 code path. And the total cyclomatic complexity of our function is : C:\> Measure-FunctionComplexity $DummyFunction 6 That’s great, but what is this telling us ? Is our function too complex or not ? Well, the purpose of metrics is to help our brain, not to replace it. Like any metric, we should take this number with a grain of salt and adapt it to our context. That said, the most commonly used thresholds are 10 and 15. I have a feeling that higher numbers might be fine for PowerShell code because it tends to be more readable than other languages, but that’s very much arguable. Besides, this cyclomatic measurement is only one side of the complexity coin. To illustrate this, let’s look at this piece of code : $Now = Get-Date Switch ($Now.Month) { 1 { $Month = 'January'; break } 2 { $Month = 'February'; break } 3 { $Month = 'March'; break } 4 { $Month = 'April'; break } 5 { $Month = 'May'; break } 6 { $Month = 'June'; break } 7 { $Month = 'July'; break } 8 { $Month = 'August'; break } 9 { $Month = 'September'; break } 10 { $Month = 'October'; break } 11 { $Month = 'November'; break } 12 { $Month = 'December'; break } } The cyclomatic complexity of this piece of code is 13. So the cyclomatic complexity is pretty high, even though most coders would consider this piece of code as easy to understand and maintain. So it can be useful to look at another metric, which measures a different aspect of complexity. Maximum nesting depth : What is the maximum nesting depth ? It is the depth of the most deeply nested code in a given piece of code (a function, here). For example, if we have an If statement nested in a loop, which is itself nested in a Catch block, the most deeply nested section is the body of the If statement and its nesting depth is 3. Let’s take a new look at our dummy function, but this time under the nesting depth lens (it’s the same function but I show it here again so you don’t have scroll up) : Function Test-Conditional { [CmdletBinding()] Param( [int]$IfElseif ) # Testing nested If statement If ( $IfElseif -gt 20 ) { If ( $IfElseif -gt 40 ) { Write-Host 'IfElseif is between 20 and 40' } Else { Write-Host 'IfElseif is greater than 40' } } Else { If ( $IfElseif -ge 10 ) { Write-Host 'IfElseif is a 2 digit number' } Else { #Testing For statements For ($i = 1; $i -lt 99; $i++) { Write-Host "$($IfElseif + $i)" For ($j = 0; $j -lt 10; $j++) { Write-Host "$($IfElseif - $j)" } } For ($k = 1;;$k++) { Write-Host 'No Condition for this loop' } } } } The code at line 11 is inside an If Statement, which is itself in another If statement, so its nesting depth is 2. The code at line 27 is inside a For loop, which is itself in a For loop, which is inside an Else statement, which is nested in another Else statement. So the nesting depth of line 27 is 4, and it is the most deeply nested section in our dummy function. So the maximum nesting depth of our function is 4. How does this matter ? What aspect of complexity is this measuring ? Well, this line is very simple in itself but, to understand completely what it does and under which condition, we need to understand its context. We need to understand the value of $j (which changes through the inner loop) and the value of $i (which changes through the outer loop). We also need to understand the If statement at line 18 and even all the way up to the first If because we need to know in which set of conditions this section of code is run. So basically, the nesting depth of a piece of code measures the complexity of its context. Visually, this is easy to follow by looking at the indentation level, but to determine programmatically the nesting depth of a piece of code, we cannot assume that code is always properly indented. I could have used a recursive function or built a custom AstVisitor, but I found these methods too … complex (pun intended). Fortunately, all the logic, control, and looping constructs where nesting occurs in PowerShell have something in common : curly braces. So we can extract all the curly braces from the code and count them to determine the nesting level, like so : Function Measure-FunctionMaxNestingDepth { [CmdletBinding()] [OutputType([Int32])] Param ( [Parameter(Position=0, Mandatory)] [System.Management.Automation.Language.FunctionDefinitionAst]$FunctionDefinition ) $FunctionText = $FunctionDefinition.Extent.Text $Tokens = $Null $Null = [System.Management.Automation.Language.Parser]::ParseInput($FunctionText, [ref]$Tokens, [ref]$Null) [System.Collections.ArrayList]$NestingDepthValues = @() [Int32]$NestingDepth = 0 [System.Collections.ArrayList]$CurlyBrackets = $Tokens | Where-Object { $_.Kind -in 'AtCurly','LCurly','RCurly' } # Removing the first opening curly and the last closing curly because they belong to the function itself $CurlyBrackets.RemoveAt(0) $CurlyBrackets.RemoveAt(($CurlyBrackets.Count - 1)) If ( -not $CurlyBrackets ) { return $NestingDepth } Foreach ( $CurlyBracket in $CurlyBrackets ) { If ( $CurlyBracket.Kind -in 'AtCurly','LCurly' ) { $NestingDepth++ } ElseIf ( $CurlyBracket.Kind -eq 'RCurly' ) { $NestingDepth-- } $NestingDepthValues += $NestingDepth } Write-Verbose "Number of nesting depth values : $($NestingDepthValues.Count)" $MaxDepthValue = ($NestingDepthValues | Measure-Object -Maximum).Maximum -as [Int32] return $MaxDepthValue } First, we parse the text of the function definition into tokens. $NestingDepth represents the nesting level at any given point in time. Its different values at different points in time are stored in $NestingDepthValues. Then, we filter the tokens corresponding to curly braces. AtCurly is a special case, these are the tokens representing opening braces for hashtables. I chose to include the curly braces for hashtables in the nesting calculation because there can be scriptblocks and expressions inside hashtables. The core of this function is the Foreach loop. It loops through all the curly brace tokens and it increments by 1 the nesting depth if it is an opening curly brace and decrements it by 1 if it is a closing brace. Then, the new nesting depth value is added to $NestingDepthValues, which keeps track of all the different values of nesting depth. When we are done looping through the curly brace tokens, we take all the values in $NestingDepthValues and we keep the highest one. Let’s run that against our dummy function : C:\> Measure-FunctionMaxNestingDepth $DummyFunction 4 As expected, the maximum nesting depth of the dummy function is 4. Great, but is 4 a good number or a bad number ? As you probably guessed, the answer is : it depends. But there seems to be a consensus, even across different languages (Java on this page and C# here : a nesting depth of 4 or higher is complex, a nesting depth of 8 or higher is extremely complex. With these 2 complementary metrics, we have a basis to make decisions on whether or not we should split our code into smaller, simpler functions. We can also use these metrics to track our progress when refactoring a PowerShell project, to ensure we are making it more testable and maintainable. Technical excellence is not a destination, it’s a journey. So how the numbers are changing over time is more important than the numbers themselves.