Build a Scalable PowerShell Pester Testing Framework

If you find yourself spending more time maintaining your Pester tests than actually creating new ones, this post is for you. In this post, I will share a project I’ve been working on with Devolutions.

Backstory

We had a problem at Devolutions. The flagship PowerShell module Devolutions.PowerShell lacked Pester tests. I know, I know. They did have C# tests but that’s another story.

As a consultant for Devolutions focused on PowerShell, I was asked to create a suite of Pester tests to be used in the CI/CD pipeline and run before production deployment. No problem, I thought. I’ve used the module a few times, and it couldn’t be that hard. I was wrong.

2025-01-29_09-27-59.png

Without any specific request, just “build some tests,” I set out to build tests for the whole module, only to find that it had nearly 500 commands! This was going to take a while.

Since I always wanted to build solutions not just for today but for the long term, I didn’t want to throw a bunch of tests into a single script and call it a day. That script would be huge!

Instead, before writing a single test, I wanted to design a framework I could use that would:

1. Be scalable – Myself or others could easily add tests to the framework without much thought.
2. To prevent code duplication, this project needs to be DRY (don’t repeat yourself). I didn’t want to duplicate any code, so I instead used shared scripts, functions, etc.
3. To use data-driven tests: Data-driven tests promote code reusability by defining tests in a data structure and then having tests reference that structure instead of duplicating testing code for every test.
4. Be modular – All tests must be split up into some structure that would prevent having to scroll through unnecessary code to find what you’re looking for.
5. To support environment setup and teardown scenarios – These tests are end-to-end (E2E)/integration/acceptance tests, whatever you want. They’re testing the end-user usage; there are no unit tests, so they must have various things in place in the environment to run.
6. To support running groups of tests without running them all at once.

These requirements lead to the Pester testing framework for Devolutions.

Prerequisites

If you’d like to use this framework, I can’t guarantee it’ll work 100% unless you’re working in the same environment I am. To use this framework, I used:

• PowerShell v7.4.5 on Windows
• Pester 5.6.1

This will still work on Windows PowerShell and earlier versions of Pester v5, but there are no guarantees!

Framework Overview

This framework consists of four components: a caller script, various test definition scripts, an optional script to store helper functions, and optional before all / before each scripts. Each of these components is structured like this in the file system:

📁 root/
   📄 caller.tests.ps1
   📄 helper_functions.ps1
   📁 test_definitions/
      📁 group1/
         📄 beforeall.tests.ps1
         📄 beforeeach.tests.ps1
         📄 core.tests.ps1
         📄 subgroup.tests.ps1

When you invoke the caller script:

Invoke-Pester -Path root/caller.tests.ps1

The caller script:

1. Finds all of the test definition scripts in each group.
2. Finds all beforeeach and beforeall scripts for each group.
3. Creates Pester contexts for each group and subgroup.

   context 'group' {
       context 'subgroup' {
   
       }
   }

4. Invokes the beforeall script once before any test runs in a group.
5. Invokes the beforeeach script before every test in the group.
6. Finally, it invokes the test assertions defined in each test definition.

The Caller Test Script

The caller script is Pester’s invocation point. It’s the script that Invoke-Pester calls when you need to invoke any tests within the framework.

The caller script is broken out into X areas:

The BeforeDiscovery Block

One task of the caller script is to find all the test definitions and what the Pester’s discovery phase is for. Inside the script’s BeforeDiscovery block, test definitions are gathered, as well as any before each or before all scripts.

BeforeDiscovery {

    # Initialize hashtable to store all test definitions
    $tests = @{}

    # Iterate through test group directories (e.g. datasources, entries)
    Get-ChildItem -Path "$PSScriptRoot\test_definitions" -Directory -PipelineVariable testGroupDir | ForEach-Object {
        $testGroup = $testGroupDir.BaseName
        $tests[$testGroup] = @{}

        # Load beforeall script for test group if it exists
        # This script runs once before all tests in the group
        $testGroupBeforeAllScriptPath = Join-Path -Path $testGroupDir.FullName -ChildPath 'beforeall.ps1'
        if (Test-Path -Path $testGroupBeforeAllScriptPath) {
            $tests[$testGroup]['beforeall'] += . $testGroupBeforeAllScriptPath
        }

        # Load beforeeach script for test group if it exists
        # This script runs before each individual test in the group
        $testGroupBeforeEachScriptPath = Join-Path -Path $testGroupDir.FullName -ChildPath 'beforeeach.ps1'
        if (Test-Path -Path $testGroupBeforeEachScriptPath) {
            $tests[$testGroup]['beforeeach'] += . $testGroupBeforeEachScriptPath
        }

        # Load all test definition files in the group directory
        Get-ChildItem -Path $testGroupDir.FullName -Filter '*.ps1' -PipelineVariable testDefinitionFile | ForEach-Object {
            $tests[$testGroup][$testDefinitionFile.BaseName] = @()
            $tests[$testGroup][$testDefinitionFile.BaseName] += . $testDefinitionFile.FullName
        }
    }
}

The BeforeAll Block

The BeforeAll block runs to make any helper functions available to the caller script or test definitions. In Pester v5, this task cannot be in BeforeDiscovery; otherwise, it wouldn’t be available to the tests.

# Load helper functions used across all tests
BeforeAll {
    . (Join-Path $PSScriptRoot -ChildPath "_helper_functions.ps1")
}

The Describe Block and Contexts

Each environment configuration you need to run tests against is split into contexts with test groups underneath each of those.

For the Devolutions PowerShell module, since we need to test cmdlets against many different types of data sources, I created contexts by data source, but you can use anything here. Then, using Pester’s ForEach parameter, each test definition folder is a context, as is each subgroup. Then, each test is defined underneath as it blocks.

---

Notice the where() method on the it block. Where({ $_.environments -contains 'xxx' -or $_.environments.count -eq 0}). This is where the structure of each definition comes into play. This part is where we designate which tests run in which environment.

---

# Main test container for all tests
Describe 'RDM' {

    # Tests that run against an environment
    Context 'Environment1' -Tag 'Environment1' {

        BeforeEach {
            ## Do stuff to execute before each test in this content. For example, this is used to set up a specific data source for Devolutions Remote Desktop Manager
        }

        # Clean up
        AfterAll {

        }

        # Run each test group against the environment
        Context '<_>' -ForEach $tests.Keys -Tag ($_) {
            $testGroup = $_

            # Run each test subgroup (core, properties, etc)
            Context '<_>' -ForEach $tests[$testGroup].Keys -Tag ($_) {
                $testSubGroup = $_

                # Run tests marked for this environment or all
                It '<name>' -ForEach ($tests[$testGroup][$testSubGroup]).Where({ $_.environments -contains 'xxx' -or $_.environments.count -eq 0}) {
                    & $_.assertion
                }
            }
        }
    }

    ## Other environments. If you have specific configurations some tests must have, you can define them here by creating more context blocks.
}

Test Definitions

Next, you have the most essential part: the tests! The tests are created in subgroups (subgroup.tests.ps1) inside of each group folder and must be created as an array of hashtables with the following structure:

@(
    @{
        'name' = 'creates a thing'
        'environments' = @() ## Nothing means all environments
        'assertion' = {
            ## Teating something
            $true | should -Be $true
        }
    }
)

Here, you can define the environments in the caller script to execute the scripts. For example, if you have an environment for each data source I used for Devolutions RDM, my environments are xml, sqllite, etc.

Helper Functions

Finally, we have the helper functions script. This script contains functions we dot source in the BeforeAll block in the caller script. This is where you can put any functions you’ll be reusing. In my example, I have functions to set up data sources and remove all of them.

# Helper function to remove all entries from the current data source
# Used for cleanup in test scenarios
function Remove-AllEntries {
    try {
        # Get all entries and their IDs
        # Using ErrorAction SilentlyContinue to handle case where no entries exist
        $entries = @(Get-RDMEntry -ErrorAction SilentlyContinue)

        # If no entries found, just return silently
        # No cleanup needed in this case
        if ($entries.Count -eq 0) {
            return
        }

        # Delete entries one at a time
        # Using foreach loop to handle errors for individual entries
        foreach ($entry in $entries) {
            try {
                # Remove entry and refresh to ensure UI is updated
                Remove-RDMEntry -ID $entry.ID -Refresh -ErrorAction Stop
            } catch [System.Management.Automation.ItemNotFoundException] {
                # Silently ignore if entry is already gone
                # This can happen if entry was deleted by another process
                continue
            }
        }
    } catch {
        # Only warn about unexpected errors
        # Ignore "Connection not found" as this is expected in some cases
        if ($_.Exception.Message -ne 'Connection not found.') {
            Write-Warning "Error during cleanup: $_"
        }
    }
}

# Helper function to remove all data sources and their files
# Used for cleanup in test scenarios
function Remove-AllDatasources {
    # Remove any data sources currently loaded in memory
    # This ensures clean state for tests
    Get-RDMDataSource | ForEach-Object { Remove-RDMDataSource -DataSource $_ }

    # Delete any existing data source folders on disk
    # These are identified by GUIDs in the RDM application data folder
    Get-ChildItem $env:LOCALAPPDATA\Devolutions\RemoteDesktopManager | 
        Where-Object { $_.Name -match '^[0-9a-fA-F\-]{36}
}

How to Build Your Framework

Does this framework look useful? Do you think it’d be beneficial to your organization? If so, here are a few tips and questions to think about.

1. How many tests do you anticipate needing? Is your use case enough to support this framework instead of just a couple of test scripts?
2. Before writing any PowerShell, describe how you anticipate defining groups and subgroups. Make them a logical abstraction of whatever elements you are testing.
3. Think about environments and the configuration you’d need for each set of tests; create those as contexts in the caller script.
4. As you build test definitions, if you find yourself repeating code, create a function for it and put it in helper_functions.ps1.

---

Building a scalable Pester testing framework requires careful planning and organization, but the benefits are worth the effort. By following the structure outlined in this article – with its modular design, reusable components, and flexible environment handling – you can create a robust testing solution that grows with your needs. Whether you’re testing PowerShell modules, infrastructure configurations, or complex applications, this framework provides a solid foundation for maintaining code quality and reliability across your organization.