vswhere should support finding specific paths under installation roots

[heaths]

We have had a number of requests for vswhere to return paths to specific tools - most often MSBuild.exe. Because vswhere is a general tool that provides information not only for Visual Studio but Build Tools and other SKUs we install, and because any of those products have unknown numbers of executables or other useful files, locating specific components is not a capability we can accommodate.

At the same time, we realize that making every caller script algorithms like those in our examples puts a burden on all developers when a consolidated algorithm would be easier to use.

We propose a new switch -find (like the UNIX find program) that takes a glob pattern as an argument to find executables under the install root. Supported patterns include,

Token	Description
*	Find 0 or more of any character.
?	Find exactly one of any character.
**	In place of a directory token, recursively search for the remaining path.

This is similar to find and existing DOS-based globbing support in Windows, with the addition of the globstar (**) that is popular in minimatch patterns.

Used in conjunction with -latest, for example, one could easily find MSBuild with whatever directory name they used (it changed between Visual Studio 2017 and 2019):

$path = vswhere -latest -requires Microsoft.Component.MSBuild -find "MSBuild\**\Bin\MSBuild.exe"

Multiple results are returned in the selected format (-format value is the default, like with -property) if the pattern matches multiple results, so the more specific you can be the better. A new switch -sort will also be added to sort instances before returning results - an alternative to -latest if you want to find all candidates from newest instance to oldest.

Cc: @refack @KindDragon @3F @uli-weltersbach @IanKemp @zivkan

[terrajobst]

Very nice!

[3F]

It really can help to quickly resolve any numbered directories like:

VC\Tools\MSVC\14.15.26726\bin\Hostx64\x86\cvtres.exe

even to avoid manual checking of arch suffix x86/x32/Win32/ ... you know

For MSBuild, it allows to use vswhere simply from command line because MSBuild\**\ does not require extracting the version anymore.

But, the one note:

• MSBuild tools may require specific workload components for build with it (here's example of errors: Add filter or priorities for searching MSBuild together with specific workload components 3F/hMSBuild#8)

So, what about to add also some logical OR, for example, through the same multiple keys to vswhere.

-find "foo" -find "bar" 

also for -requires key:

-requires "Microsoft.Component.MSBuild Microsoft.VisualStudio.Component.VC.Tools.x86.x64" -requires "Microsoft.Component.MSBuild Microsoft.VisualStudio.Component.VC.ATL" 

By the way, can we specify multiple -property when -format value ?

Although, if the MSBuild\**\ feature will be finally implemented, it does not matter :) Mainly because of 'installationVersion':

if /I "%%~a"=="installationPath" set vspath=%%~b
if /I "%%~a"=="installationVersion" set vsver=%%~b

[heaths]

To make sure we find an instance that you want, the existing switches - like -requires and possibly -requiresAny (a recent addition) will filter those instances such that a single -find should suffice, right?

As for multiple properties, it's something I've been considering but not really sold on. With your last snippet, you can do that with all properties already. Everything is local, so its not like REST APIs that give you what you need plus a whole lot more.

The main purpose of this feature is to help simplify scripts. This doesn't provide anything that developers can't do today, but it does mean duplication of effort code. Drawing from my recently updated sample (for Visual Studio 2019), you could go from this:

$path = vswhere -latest -prerelease -products * -requires Microsoft.Component.MSBuild -property installationPath
if ($path) {
  $tool = join-path $path 'MSBuild\Current\Bin\MSBuild.exe'
  if (-not (test-path $tool)) {
    $tool = join-path $path 'MSBuild\15.0\Bin\MSBuild.exe'
    if (-not (test-path $tool)) {
      throw 'Failed to find MSBuild'
    }
  }
  & $tool $args
}

To this:

$path = vswhere -latest -prerelease -products * -requires Microsoft.Component.MSBuild -find 'MSBuild\**\Bin\MSBuild.exe'
if ($path) {
  & $path $args
}

The VC toolset still presents an interesting challenge if you have multiple versions installed. You may still want to sort the output lexigraphically to find the latest version (though I cannot guarantee their version directories will remain lexigraphically sortable in a significant way):

$path = vswhere -latest -products * `
  -requires "Microsoft.Component.MSBuild Microsoft.VisualStudio.Component.VC.Tools.x86.x64" `
  -requires "Microsoft.Component.MSBuild Microsoft.VisualStudio.Component.VC.ATL" `
  -find 'VC\Tools\MSVC\**\bin\Hostx64\x86\cvtres.exe' | sort | select -first 1

For the VC toolset, using the VC\Auxiliary\Build\Microsoft.VCToolsVersion.default.txt is recommended.

[3F]

@heaths

switches - like -requires and possibly -requiresAny (a recent addition) will filter those instances such that a single -find should suffice, right?

More probably it will be enough to have multiple -requires and just a single -find key (for the last phase). I'm not sure how does implemented -requires in vswhere today, but as the general solution, why not to filter this in order of addition.

This doesn't provide anything that developers can't do today, but it does mean duplication of effort code

Yes, I noticed :)

the mentioned hMSBuild uses similar tricks and additional -vsw-priority & -stable key features.

btw, "MSBuild\ Current \Bin" ?

Did I miss something?
3F/hMSBuild#3 (comment)

The VC toolset still presents an interesting challenge if you have multiple versions installed.

exactly :(

But I think, as I said above, we just need to use priority in order of mention for -requires keys.
that is,

-requires "Components set #1 we want" -or-> -requires "Components set #2 we want" -or->

For the VC toolset

It was just example, but for some cases (for example, our DllExport tool requires some tool from VC, lib.exe etc) so it may be really uncomfortable to use vsdevcmd.bat and so on.

The other way from wiki VC\Tools\MSVC\%Version%\bin\HostX64\x64\cl.exe looks not so pretty for batch scripts because of extracting %Version% (especially set Version=!Version: =!)

so \**\ really a lot :)

[heaths]

-requires accepts one or more arguments, and technically you should be able to repeat it though 1) that wasn't intentional and may change in the future, and 2) not necessary since you give it multiple arguments. By default, all IDs passed to -requires are required (and AND operation). The recently added -requiresAny makes it an OR because for a few workloads and components, they have different IDs for the Build Tools SKU.

You definitely should use these prior to relying on -find because you first need to find an instance that might possibly have what you're looking for. Imagine if you just did something like this:

vswhere -latest -find MSBuild\**\MSBuild.exe

The latest instance of VS may not have MSBuild installed, but because -latest already limited the results to search, this query would emit no output.

Also, yes, for VS2019 the MSBuild team is changing "{major}.0" to "Current" to make VS2019 and newer easier to automate in devops via vswhere.

[refack]

👍

Just so I'm clear, the VSInstaller will update vswhere independently of the actual VS version? So theoretically an MSVS2017 with a updated installer should include the new vswhere exe?

[3F]

@heaths

both keys cannot reproduce the following logic below:

today's:
-requires "Id1 and Id2 and Id3 ..."
-requiresAny "Id1 or Id2 or Id3 ..."

-requires "Id1 and Id2" or -requires "Id2 and Id3"

i.e.

-requires "MSBuild and Id1"
or 
-requires "MSBuild and Id2"

it may help at least for MSBuild component searching, like for 3F/hMSBuild#8

Because we need to filter instances with an Microsoft.Component.MSBuild component and its various possibilities to build C++ projects (for example), ATL, WPF, etc.

However, without this we must to check it in a separate request to vswhere.

I'm not saying that's not enough for happy use today. This was just my thought that MSBuild component for this case is definitely required in request, while others are optional.

same for -find

vswhere -latest -find MSBuild**\MSBuild.exe

I agree, that -requires should be in first phase of filtering.

The -find key/s should just return its found result from that result.

Also, yes, for VS2019 the MSBuild team is changing "{major}.0" to "Current" to make VS2019 and newer easier to automate in devops via vswhere.

nice,

Thanks for the information!

[heaths]

@refack Yes, installing the VS2019 version of the VS Installer will update vswhere, but if you have a hard dependency on newer features I recommend using the NuGet package regardless of target platform, since .nupkg files are just .zip files you can fetch from a stable URI, unzip, and copy as necessary. This is what most build scripts (including many tasks in Azure Pipelines) do so far as I've discovered or with which I've assisted.