Improve Array BinaryAdd Method for object[]

[jborean93]

PR Summary

Use Array.Resize when doing += on an array type. This is more efficient than the default BinaryAdd done on other enumerable types. As adding to an array is a common operation done by people new to PowerShell this should reduce some of the issues which come with that type of code.

An example of dramatic difference this can make is reflected by this code

$s = [gc]::GetTotalAllocatedBytes($true)

Measure-Command {
    & {
        $val = @()
        for ($i = 0; $i -lt 20000; $i++) {
            $val += $i
        }
    }
}

$e = [gc]::GetTotalAllocatedBytes($true);

Write-Host "Allocated Bytes $($e-$s)"

When run with the new code (TEST=true) you can see it is 8 seconds faster and requires less memory to allocate. As the arrays get larger the differences are even more profound.

jborean:~/dev/PowerShell$ TEST=true debug/pwsh -File jordan.ps1

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 795
Ticks             : 7950790
TotalDays         : 9.20230324074074E-06
TotalHours        : 0.000220855277777778
TotalMinutes      : 0.0132513166666667
TotalSeconds      : 0.795079
TotalMilliseconds : 795.079

Allocated Bytes 1604712232
jborean:~/dev/PowerShell$ debug/pwsh -File jordan.ps1

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 9
Milliseconds      : 145
Ticks             : 91453429
TotalDays         : 0.000105848876157407
TotalHours        : 0.00254037302777778
TotalMinutes      : 0.152422381666667
TotalSeconds      : 9.1453429
TotalMilliseconds : 9145.3429

Allocated Bytes 6372367600

This doesn't negate the existing performance impacts of adding to an array, it just removes extra work that wasn't needed in the first place (which was pretty inefficient) making it slower than it has to. People should still use an alternative like capturing the output from the pipeline or use List<T>.

PR Context

Partially Fixes: #23899

PR Checklist

• PR has a meaningful title
  • Use the present tense and imperative mood when describing your changes
• Summarized changes
• Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
• This PR is ready to merge and is not Work in Progress.
  • If the PR is work in progress, please add the prefix WIP: or [ WIP ] to the beginning of the title (the WIP bot will keep its status check at Pending while the prefix is present) and remove the prefix when the PR is ready.
• Breaking changes
  • None
  • OR
  • Experimental feature(s) needed
    • Experimental feature name(s):
• User-facing changes
  • Not Applicable
  • OR
  • Documentation needed
    - [ ] Issue filed:
• Testing - New and feature
  • N/A or can only be tested interactively
  • OR
  • Make sure you've added a new test if existing tests do not effectively test the code changed
• Tooling
  • I have considered the user experience from a tooling perspective and don't believe tooling will be impacted.
  • OR
  • I have considered the user experience from a tooling perspective and opened an issue in the relevant tool repository. This may include:
    • Impact on PowerShell Editor Services which is used in the PowerShell extension for VSCode
      (which runs in a different PS Host).
      • Issue filed:
    • Impact on Completions (both in the console and in editors) - one of PowerShell's most powerful features.
      • Issue filed:
    • Impact on PSScriptAnalyzer (which provides linting & formatting in the editor extensions).
      • Issue filed:
    • Impact on EditorSyntax (which provides syntax highlighting with in VSCode, GitHub, and many other editors).
      • Issue filed:

[fflaten]

People should still use an alternative like capturing the output from the pipeline or use List<T>.

In scripts/functions yes, but if this really reduces time spent on += by 90% I'll likely never bother using lists in interactive sessions anymore. Wow 🙌

[daxian-dbw]

The Engine WG discussed this and agree with the change, and it is ready to be reviewed.

[iSazonov]

nit: interesting, have we such issues with Collection, List (and others IEnumarable-s?

[jborean93]

nit: interesting, have we such issues with Collection, List (and others IEnumarable-s?

Somewhat yes, at least the known size types (not IEnumerable) could be improved by providing the capacity on the newly generated list but that's somewhat unrelated to fixing arrays. I didn't touch the code for other IList types because I was unsure whether it was safe to do IList.Count to get the capacity needed.

[SeeminglyScience]

I didn't touch the code for other IList types because I was unsure whether it was safe to do IList.Count to get the capacity needed.

I would lean toward "maybe, but out of scope for this PR" just to get the most common use case settled quickly.

[SeeminglyScience]

People should still use an alternative like capturing the output from the pipeline or use List<T>.

In scripts/functions yes, but if this really reduces time spent on += by 90% I'll likely never bother using lists in interactive sessions anymore. Wow 🙌

To be fair, interactively it's really hard to beat statement capture (e.g. $a = foreach ($b in $c) { $b }). Super easy, and still pretty performant

[fflaten]

To be fair, interactively it's really hard to beat statement capture (e.g. $a = foreach ($b in $c) { $b }). Super easy, and still pretty performant

True. I'm usually a few foreach-loops deep or splitting into multiple arrays when I need += or List in which case it's easier to manage imo.

Either way, looking forward to this.

[daxian-dbw]

LGTM

[iRon7]

This is a great improvement to PowerShell 🎉.

Btw. a similar issue exists for using the increase assignment operator (+=) to build a string. (see: https://stackoverflow.com/a/70093215/1701026)
I wonder if it be possible to do something similar there by first translating the (fixed) string into a StringBuilder object, add the strings, and rebuild the string (.ToString()) at the moment the variable is retrieved?

Stack Overflow

Is there a string concatenation shortcut in PowerShell?

Like with numerics?

For example,

$string = "Hello"
$string =+ " there"
In Perl you can do

my $string = "hello"
$string .= " there"
It seems a bit verbose to have to do

$string = $string + " t...