The Surly Admin

Dynamic Properties in Objects…. and performance!

So last post I talked about dynamic properties in objects, and it was pretty cool.  Then a crazy thing happened, that same colleague who was playing around with a faster Get-ChildItem project?  He decided to get some real work done and began working on a Office365 script but the Get-MailboxStatistics cmdlet is a little bit different with O365 in the TotalItemSize property is deserialized and pretty much only has a lousy string output.  So how to get the raw number without the ToMB() method?  A little Googling and we found an Exchange blog post using RegEx to strip out all the extra crud, and a dynamic property!  In the years I’ve been using PowerShell I’ve never once seen a dynamic property and the very day I decided to learn it we actually found a real life usage!  What are the odds?  Anyway, the twist was the Exchange team was applying the Add-Member to an array of objects.  I didn’t realize you could do that.  And it worked!

But what about performance?

Typically we’d use a calculated field to strip out the crud and calculate the number, right?  It’s the PowerShell way.  This method seemed really cool and we wanted to use it, but the fear is it’d be slower than a Select-Object with calculated field.  So I tried it with some sample data and ran them both through Measure-Command.  And my jaw dropped.  The dynamic property and Add-Member technique was about 15x faster than the Select-Object with calculated field.  What?!  A few more test proved it out.

Prove it

Remember this last “wish list’ item I posted yesterday?

Let’s actually do it in real life, this time using both the Select-Object and Add-Member technique’s and see who wins.  Here’s the test code:

This is against my test folder where I keep a bunch of files (mostly scraps from other scripts!).  There are about 234 files in these folders.  The results?

Not the 15x speed difference we saw when doing string manipulation, but still 3x faster.  So next I decided to run it against my entire C: drive.  There are 216,177 files on my PC!

This time we got a little over FOUR time faster.

Limitations

If you find yourself working with really big data sets and you need a calculated field, there’s no question this is a great technique and much faster than Select-Object with calculated fields.  But there are some limitations.  With Select you can actually draw information from other sources.  On Friday I needed to produce a report of all of our VM’s that were not on hardware version 10 and had a 2TB virtual disk (we actually have a few).  Turns out VMware does not support VMDK’s over 2TB on hardware version 9.  So I setup a loop on our vCenter servers, then a loop on our VM’s and used Get-HardDisk on each VM.  But the problem is the pipeline from Get-HardDisk doesn’t include the vCenter information or the VM name, so I used calculated fields in a Select statement to refer to that information from my loop.  Since the object that can be created from Get-HardDisk does not contain that information I wouldn’t be able to use the Add-Member method to get information from the loop.

OK, technically that’s not true.  There is some reference information in Get-HardDisk, so in the value parameter I could use Get-VM and from that data use Get-Host to get that information, but it would be evaluated every time you display the object which would be painfully slow!

Conclusion

I love this.  I love its speed, and I love its simplicity.  It’s a narrow use case, admittedly, but another powerful weapon in my arsenal.  But Holy Crap!  I’ve been trying to get people off of Add-Member (at least for normal object creation) for years and now suddenly I’m going to embrace like a champ!  Fun stuff!

January 27, 2015 - Posted by Martin9700 | Powershell - Performance | Add-Member, ScriptProperty