Fix filename sort order edge cases in explorer.

[leilapearson]

This PR partially addresses issue #27759

This PR adds compareFileNamesNumeric() and compareFileExtensionsNumeric()
functions, and uses the new functions in explorer view in place of
compareFileNames() and compareFileExtensions().

The PR addresses the following edge cases:

1. Sorts the same name with different case in locale order (a < A) instead of
   unicode order (A < a).
2. Sorts the same name with different accents or diacritics by locale
   (á < à) instead of in unicode order (à < á) .
3. Sorts names that are equivalent in a numeric locale comparison
   as shortest numeric representation first (1 < 01 < 001) instead of
   unicode order (001 < 01 < 1).
4. For filenames with a name plus an extension, compares names first
   fand then compares extensions (aggregate.go < aggregate_repo.go)
   instead of comparing the whole name at once
   (aggregate_repo.go < aggregate.go)
5. Treats dotfiles (filenames that start with a dot) as filenames
   without extensions, instead of either as an empty filename plus
   an extension (single-dot dotfile), or a filename that starts with a
   dot plus an extension (multi-dot dotfile). This means all dotfiles
   group together in their own section when sorting by file type,
   (.eslintignore < .eslintrc.json < file.cs < file.js < file.json) instead
   of being interspersed between files grouped by type
   (file.cs < .eslintignore < file.js < .eslintrc.json < file.json).

Besides adding unit tests, to make it easier to see the effects of these changes, I created this repo:

• https://github.com/leilapearson/filename-sort-identifier

You can see the results comparing the existing behavior with the new behavior here:

• https://docs.google.com/spreadsheets/d/1p61ugzy9E9GYbvx5BQLssbU_P5L2WHWTCAcsH3ZiRnw/edit?usp=sharing

Note that this PR does not address issue #27759 requests to:

• sort in unicode order
• sort in locale order but with numeric sort turned off
• sort in locale order but grouped by case

A separate follow-up PR will provide new options that address these requests

Also note that other areas of workbench may benefit from adopting compareFileNamesNumeric
in place of compareFileNames, or compareFileExtensionsNumeric in place of
compareFileExtensions, but this PR only changes explorer view.

cc @isidorn

[leilapearson]

@isidorn the PR is failing CI, but I'm seeing the same problem when I run yarn run gulp editor-distro on my local master so I don't think it can be caused by my changes...

I also see odd behavior when I first build in a new branch or in master. The build reports ~33 errors - but if I go and click on those problems to open the files with the problems, then the problems go away? Did I miss a step somewhere that would prevent this?

[isidorn]

@leilapearson thank you very much for this great PR. Overall looks very good! Also thanks a lot for tests.
I have left some minor comments in the code so it would be great if you could tackle that.
I like how we only now use it in the explorer and later other parts of the workbench can adopt it.
Also note that these changes might affect the Search View, since the Search View uses compareFileExtensions.
Also do not worry about those failures, I just checked it locally and all is good, including running tests.

fyi @bpasero if you have time to review and to be aware to potentially adopt this in the quick open in the future.
I plan to merge this in once my comments have been adressed since I like the approach and how this is done. Also merging in to master explorer will give use feedback from VS Code insiders in case users do not like this so we can react on time.
fyi @roblourens for search view potential sort change

[bpasero]

LGTM, no plans currently to adopt this for quick open where imho the sorting is mainly driven by the score of the results and not by any alphabetic order.

[leilapearson]

Thanks for the quick review @isidorn. I'll look at these right away.

Regarding Search View, it still uses the original compareFileExtensions where the only change is the internal name of the collator - so it should not be impacted I think?

[leilapearson]

All done I think @isidorn. Let me know if I missed anything.

[isidorn]

Thanks a lot! Merging in ☀️
As for the other PR, if you plan to do it feel free to first create an empty one so we discuss the approach. I am not a big fan of introducing a new sort option, so we could discuss that among other things.

[leilapearson]

Thanks again @isidorn!

Wanting the ability to group files by case is the reason I got involved in this issue - so for sure I'd like to provide that functionality. These edge cases weren't on my radar at all until I ran into them when working on that other code.

I'll go ahead and create an empty PR as you suggest so that we can discuss the approach.