Losing My Mind – Weird Behaviour When Loading .NET 8 Custom Types in PowerShell

Hi folks, so I was tinkering with a script one of my colleagues wrote the other week which takes Azure Sentinel Alert Rules YAML definitions, and turns them into ARM Templates. This script was being repurposed as part of a CI/CD build I was working on. This little problem touched on a lot of neat subject areas, since I like telling a story we’ll begin with the Problem Definition.

Json Serialization – Producing 100% correct output, <60% of The Time

This script was using PowerShell’s native JSON serialization cmdlets and….had some problems in consistency. Essentially the element/attribute order of the generated JSON was non-deterministic past a certain depth in the resulting file.

e.g.

{Fpower
"sampleObject": {
"Attribute1": "Foo",
"Attribute2": "Bar"
}
}

Could just as easily have been written out as.

{
"sampleObject": {
"Attribute2": "Bar"
"Attribute1": "Foo",
}
}

This was somewhat problematic for me, because these files had to be deterministically generated so that the Git repo which they were contained in would detect whether they’d changed from a previous run of the script! I was getting hundreds of effectively “false positives” for detected changes in the repository each time the script was run. This changed depending on where the script ran? On my server, one set of results, on the Build Agent a different set of outputs, and even then I’m pretty sure the position of the Moon, and possibly even what I had for breakfast factored into the outputs somehow.

The Script in it’s current form just wasn’t up to scratch for what I needed it to do, my best guesses as to the myriad nature of the outputs was the following train of thought:

• The behavioural differences were partly down to PowerShell vs PWSH (classic PowerShell 5.1 or PowerShell 7).
• Running locally I was running the script in PowerShell 7 (aka PWSH)
• The Build Agent was running it as PowerShell 5.1 (initially).
• PowerShell’s native JSON cmdlets have different under-the-hood implementations depending on whether it’s old PowerShell or new PWSH. In either case they just weren’t going to cut the mustard in terms of fine tuning the output, they were a hammer and I needed a scalpel.

The solution. Enter custom .NET Code.

PowerShell and .NET – The Best of Friends

I <3 PowerShell. Ever since I learned to use it back 2013 to automate SharePoint solution deployments I’ve loved it. In my eyes it belongs to a family of skills which synergize with other skills you have. Everyone could do with being a bit more script savvy (imo). One of it’s oldest and best features is its ability to load .NET types to extend its functionality.

I realized what I needed was a custom serializer, which in .NET gave me 2 options. The old tried and tested NewtonSoft.Json, or the relatively new System.Text.Json. In the end I opted to go with the System.Text.Json implementation because it satisfied the requirement of my script being cross platform (.NET, formerly known as .NET Core) and it kept the external dependencies to a minimum.

I defined a class which represented my ARM Template Resource and using a decorator on the relevant properties for each class, I’m able to specify an order for serialization which is respected by my serializer.

[JsonPropertyName("$schema")]
[JsonPropertyOrder(1)]
public string schema { get; set; }
[JsonPropertyOrder(2)]
public string contentVersion { get; set; }
[JsonPropertyOrder(3)]
public Parameters parameters { get; set; }
[JsonPropertyOrder(4)]
public List resources { get; set; }

I put these classes in a .NET 8 class library which and unit tested it. I could deserialize and serialize my ARM Template till my heart was content and get consistent sequencing. So hard part done, now I just needed to call it from my PWSH script. I then loaded the necessary types, instantiated a JsonSerializerOptions object just like in my Unit Tests.

#Load my serialization contracts dll
Add-Type -Path $SerializationContractDllPath

#Lets Get Ready to Write Some JSON
$serializationOptions = $null
#region - Serialization Options Initialization
[System.Reflection.Assembly]::Load("System.Text.Json")
[System.Reflection.Assembly]::Load("System.Text.Encodings.Web")
$serializationOptions = New-Object -TypeName System.Text.Json.JsonSerializerOptions
$serializationOptions.AllowTrailingCommas = $false
$serializationOptions.DefaultBufferSize = 16384
$serializationOptions.DefaultIgnoreCondition = [System.Text.Json.Serialization.JsonIgnoreCondition]::Never
$serializationOptions.NumberHandling = [System.Text.Json.Serialization.JsonNumberHandling]::AllowReadingFromString
$serializationOptions.IgnoreReadOnlyProperties = $false
$serializationOptions.IgnoreReadOnlyFields = $false
$serializationOptions.IncludeFields = $false
$serializationOptions.MaxDepth = 64
$serializationOptions.PropertyNamingPolicy = [System.Text.Json.JsonNamingPolicy]::CamelCase
$serializationOptions.PropertyNameCaseInsensitive = $false
$serializationOptions.ReadCommentHandling = [System.Text.Json.JsonCommentHandling]::Disallow
$serializationOptions.UnknownTypeHandling = [System.Text.Json.Serialization.JsonUnknownTypeHandling]::JsonElement
$serializationOptions.WriteIndented = $true
$serializationOptions.ReferenceHandler = [System.Text.Json.Serialization.ReferenceHandler]::IgnoreCycles
$serializationOptions.Encoder = [System.Text.Encodings.Web.JavaScriptEncoder]::UnsafeRelaxedJsonEscaping

Wrote some more of the script…ran it locally and it worked great! Committed, pushed to my feature branch and ran the pipeline….and you guessed it. Bang! Got an exception.

The first exception was because I forgot to set the pwsh property on our pipeline task to true. No surprise there you can’t load .NET 8.0 libraries into PowerShell 5.1 scripts (a .NET Framework runtime). Oops.

- task: PowerShell@2
inputs:
filePath: '$(Build.SourcesDirectory)\my-project\Scripts\myscript.ps1'
arguments: '-blah blah blah some args'
pwsh: true

So I fixed that and ran it again.

Bang! Second Exception. Words to the effect of could not resolve the type CloudGuard.Serialization.Contracts.AlertRuleResource. It was occurring when I was invoking the serializer, specifically in conjunction with the class in my serialization contract library.

[CloudGuard.Serialization.Contracts.AlertRuleResource]$StronglyTypedAlert = [System.Text.Json.JsonSerializer]::Deserialize($fullAlert,[CloudGuard.Serialization.Contracts.AlertRuleResource], $serializationOptions)
[System.String]$alertText = [System.Text.Json.JsonSerializer]::Serialize($StronglyTypedAlert,[CloudGuard.Serialization.Contracts.AlertRuleResource],$serializationOptions)

I’d debugged the script locally before pushing it to my remote for the build agent to run it, so I knew it worked locally, but not the build agent. Well that is a Tale as old as time (aka “But It works on my PC!”, which is also btw the blog of my old mentor and co-worker Richard Fennel. Go check him out 🙂 ).

Debugging

I’m a war hardened veteran of BizTalk Server development…so I’m used to dealing with some weirdness about Assemblies.

So I thought to myself the Add-Type command isn’t giving me an error….but when I come to use the loaded Type it’s not resolving which must mean its not loaded.

I had checked and rechecked the command, checked my compilation target in my .NET project (.NET 8.0). That it worked locally told me it was likely an environmental difference between my dev machine and my agent. The trick was I had to figure out what that difference was. Just to prove to myself that the dll wasn’t loading (independently of the exception) I used a bit of .NET in my PWSH script to inspect what was loaded into memory.

Code a little to the effect of the following.
$isLoaded = [System.AppDomain]::CurrentDomain.GetAssemblies() | Where-Object { $_.Location -match $dllInstance.FullName }

Permutations of that command gave me a list of all assemblies currently loaded into the memory space my script was running in. On my machine everything was working as expected. When I remoted onto the build agent (privately hosted) and ran the same commands my Serialization Contracts dll was not listed, even after executing the Add-Type cmdlet which gave no error. So we were now doubly sure. Add-Types was not doing it’s job but why?

The Culprit

Since this was now clearly an environment fault, I started looking at the differences between my local machine and my private agent, I had the same patch level for the .NET 8 runtime, but not the same version of PWSH. My local version of PowerShell 7 was 0.1 minor versions ahead of our build agent.

Difference that minor couldn’t have made any differences…right?
Well here are the release notes.

And I’ll save you a read. Right there in the 1st paragraph of the release notes.

PowerShell 7.4 includes the following features, updates, and breaking changes. PowerShell 7.4 is built on .NET 8.0.0.

It makes sense when you think about it. PWSH 7.3  was built on .NET 7.0 and can load assemblies consequently which are compiled for .NET 7.0 (now EOL), it is incapable of loading a .NET 8.0 dll. The same issue that bit me with forgetting to set my task to use PWSH vs PowerShell had bitten me again just with less red text in the output stream.

It seems the devil in the detail difference in this instance was that Add-Type is a horrid little command that swallows exception when 2 different versions of .NET are in play (vs massive differences like .NET Framework vs .NET). ! I updated our build agent’s PWSH install and re-ran my build. I was pleased that everything then worked as expected.

Summary

So, in summary what was this little adventure of mine other than a slightly stale build agent. Hopefully it was a few things for you the reader.

1. A quick peek into tackling deterministic serialization with System.Text.Json.
2. How to leverage that in PowerShell 7, loading custom types/assemblies.
3. Some debugging commands and gotchas that can arise when using custom .NET types in PowerShell 7
4. A reminder to keep your self-hosted build agents patched!