Why is ”Add-Content” bad in PowerShell 5.1?

I have seen a lot of people using Add-Content in their scripts to write to a log file (and I confess I used to be one of them). But why is it bad and what can you do instead?

The problem with Add-Content is that in PowerShell 5.1 (and earlier) below the surface it is using the Enum FileShare.Write rather than FileShare.ReadWrite. MS documentation states the following, FileShare Enum (System.IO) | Microsoft Learn:

This means that Add-Content needs a read lock on the file before writing to it. This is BAD because if you are using a tool like CMTrace or some other tool that tails the file while the script is writing data to it will fail if the timing is wrong. The error is:
Add-Content : The process cannot access the file '<filepath>' because it is being used by another process.

Here you see that the simple script above failed and the log file is missing entry 10 and 11.

If you are using PS Core it has been fixed since 2018:

Why does Add-Content require a read lock when appending to files? · Issue #5924 · PowerShell/PowerShell (github.com)

But if you are writing a script that you don’t know on what system it will end up running you should avoid Add-Content for any log functions you might have.

What can I do instead?

What are the alternatives? Here are 2!

Out-File

Out-file does not suffer from the same issue as Add-Content, there are some caveats though:

• Default encoding is Little Endian (UTF16 LE), default for Add-Content is UTF8 no Bom
• You can specify -Encoding UTF8 BUT this will create a file with “UTF8 with BOM”. This is not recommended by Unicode standard:
  2.6 Encoding Schemes
  ... Use of a BOM is neither required nor recommended for UTF-8, but may be encountered in contexts where UTF-8 data is converted from other encoding forms that use a BOM or where the BOM is used as a UTF-8 signature. See the “Byte Order Mark” subsection in Section 16.8, Specials, for more information.
• In PowerShell Core this is fixed and there is an option called UTF8noBom. The default for -Encoding UTF8 in PS Core is without bom which means that you will get a different result/format depending if you are running in a PS 5.1 or a PS Core session.
  
  If you specify utf8NoBOM on a system with PS 5.1 it will fail.

.Net Method

Use native .net code in your script to write to the file. For example the .Net StreamWriter method can be used (this is what Out-File is using). So what do you need to know if using .Net:

• A bit more complex script but you will have full control.
• Faster!
• It will not work on a system running in PowerShell Constrained Language mode.

Speed

They also differ on how quick they are, comparing writing 10000 rows it looks like this:

PS 5.1:
Add-content = 119.1 Seconds
Out-File = 7.6 Seconds
.Net StreamWriter = 3.2 Seconds

PS 7.4 Core
Add-content = 68.7 Seconds
Out-File = 2.2 Seconds
.Net StreamWriter = 1.3 Seconds

The PS 5.1 test ran on a VM and the PS Core test was run on a physical machine and likely the reason the overall times were quicker. I am a little surprised seeing that Add-Content still is so much slower, even in the PS Core version. So why is that? Since we are seeing the same behavior in both 5.1 and in Core we can assume it uses the same logic behind the scene. Even if 5.1 is not open source, PS Core is! So lets dig a little deeper to see what is going on...

It turns out that the "Add-Content" command is using the .Net method Filestream and that "Out-File" is using the method StreamWriter. This means that Add-Content is "manually" seeking the end of the file for each operation by doing a System.IO.SeekOrigin.End call every time. Normally the Filestream and "Seek" method gives you much more control if you need to insert data at a specific location in a file, but in this case the Add-Content command only appends to the end of the file. The StreamWriter method is using native functions to append data to the end of the file rendering it much faster! And if you are referencing the streamwriter yourself, you get rid of some additional overhead, and it is even faster!

Conclusion

So, depending on the use case I would say go with a .Net StreamWriter if speed and UTF8 without BOM is important, this is also the only option to achieve consistency across PS versions. If you don't care that the file is UTF8 with BOM in PS5.1, UTF without BOM in PS Core and want to keep it simple, go with or Out-File!

But whatever you do, stay away from Add-Content. Atleast in PowerShell 5.1! The only feature I have found that Add-Content can do that Out-File can't is that Add-Content support transactions. If this is something you need and work with you might be stuck with Add-Content but beside that I have not find any good reason to use Add-Content. If you want to read more about transactions you will find it here. Please let me know on X/Twitter if you know any more use cases for Add-Content!

Here is the code used for speed testing above:

Measure-Command {
    $i = 0
    for ($i ; $i -le 10000 ; $i++ ) {
        $string = "$($i -f "{d:03}") : $(Get-date -format "yyyyMMdd:HHmmss") : Test Content"
        $string | Add-Content -Path "C:\temp\addcontent.log"
        #Start-Sleep -Milliseconds 100
    }
}
Measure-Command {
    $i = 0
    for ($i ; $i -le 10000 ; $i++ ) {
        $string = "$($i -f "{d:03}") : $(Get-date -format "yyyyMMdd:HHmmss") : Test Content"
        $string | out-file -filePath "C:\temp\outfile.log" -Append -Encoding utf8
        #Start-Sleep -Milliseconds 100
    }
}
Measure-Command {
    $path = "C:\temp\netwrite.log"
    $i = 0
    for ($i ; $i -le 10000 ; $i++ ) {
        $stream = [System.IO.StreamWriter]::new($path, $true, ([System.Text.Utf8Encoding]::new()))
        $string = "$($i -f "{d:03}") : $(Get-date -format "yyyyMMdd:HHmmss") : Test Content"
        $stream.WriteLine("$string")
        $stream.close()
        #Start-Sleep -Milliseconds 100
    }
}

And here is a PowerShell script template with a logging function (.Net StreamWriter) that logs in the CMTrace format that I wrote and use myself:
Template-Logging.ps1

/Mattias Benninge
Principal Engineer @ 2Pint Software
@matbg X / Twitter