Support a better Windows update flow

[deepak1556]

The updater client on windows https://github.com/microsoft/inno-updater has been working reliably for > 10yrs now, however there are some edge cases (OS force shutdown) that the client cannot handle in its atomic replace operations that leads to broken updates. This issue is to see if we can come up with an alternative update flow that relies on versioned package paths with inspiration from Chromium's update client https://source.chromium.org/chromium/chromium/src/+/main:chrome/updater/

[iwangbowen]

The update issue is annoying

[deepak1556]

Breakdown of background update flow in VSCode today for User installation:

• Scheduled updates initialized by updateService.win32.ts
• Runs the new inno setup file in non gui mode to perfom file extraction with rules defined by

  vscode/build/win32/code.iss

  Lines 93 to 99 in 28af0e9

  	[Files]
  	Source: "*"; Excludes: "\CodeSignSummary*.md,\tools,\tools\*,\appx,\appx\*,\resources\app\product.json"; DestDir: "{code:GetDestDir}"; Flags: ignoreversion recursesubdirs createallsubdirs
  	Source: "tools\*"; DestDir: "{app}\tools"; Flags: ignoreversion
  	Source: "{#ProductJsonPath}"; DestDir: "{code:GetDestDir}\resources\app"; Flags: ignoreversion
  	#ifdef AppxPackageFullname
  	Source: "appx\*"; DestDir: "{app}\appx"; BeforeInstall: RemoveAppxPackage; AfterInstall: AddAppxPackage; Flags: ignoreversion; Check: IsWindows11OrLater and QualityIsInsiders
  	#endif

  . This will place the extracted files under <AppInstallDir>/_
• Post install step of the inno setup will run the inno_updater.exe from the new version once the running application exists defined by the application mutex

  vscode/build/win32/code.iss

  Lines 1506 to 1520 in 28af0e9

  	if IsBackgroundUpdate() then
  	begin
  	CreateMutex('{#AppMutex}-ready');
  	Log('Checking whether application is still running...');
  	while (CheckForMutexes('{#AppMutex}')) do
  	begin
  	Sleep(1000)
  	end;
  	Log('Application appears not to be running.');
  	StopTunnelServiceIfNeeded();
  	Exec(ExpandConstant('{app}\tools\inno_updater.exe'), ExpandConstant('"{app}\{#ExeBasename}.exe" ' + BoolToStr(LockFileExists()) + ' "{cm:UpdatingVisualStudioCode}"'), '', SW_SHOW, ewWaitUntilTerminated, UpdateResultCode);
  	end;

• Inno updater performs the following tasks,
  • Kills any process that have open handles to the current installed files
  • Delete files from current installation
  • Move files from <AppInstallDir>/_ to <AppInstallDir> for the new version
  • Patches unins000.dat to un-include files that are no longer part of the new version

We have seen repeated issues of update being broken if file move operation is corrupted either due to system or user interruption, a failure in inno updater etc. An exploration to understand how a well defined update system in a product like Chrome performs today and improve our update flow from this learning.

Comparison of directory structure:

**Before update**

%LOCALAPPDATA%/Programs/Microsoft/VS Code
  |
  |_ Code.exe
  |_ tools
    |_ inno_updater.exe

**Pending update**

%LOCALAPPDATA%/Programs/Microsoft/VS Code
  |
  |_ Code.exe
  |_ _
     |
     |_ Code.exe (new version)


**Before update**

%LOCALAPPDATA%/Google/Chrome/Application
  |
  |_ chrome.exe
  |_ <Version Folder>
    |_ Rest of the files...

**Pending update**

%LOCALAPPDATA%/Google/Chrome/Application
  |
  |_ chrome.exe
  |_ <Version Folder>
    |_ Rest of the files...
  |_ Temp
     |_ chrome.exe (new version)
     |_ <Version Folder> (new version)

Following is the breakdown of the install/update flow in Chrome for the user installation, there are 3 components involved in this process,

1. The application main process that lives in //chrome/browser
2. The setup or installer in //chrome/installer
3. The cross platform update client in //chrome/updater

The application main process

• browser_util::IsBrowserAlreadyRunning
  • Gets Directory of EXE for current running process via GetModuleFileName
    that relies on the PEB structure which will stay intact even when the
    EXE file got moved to a different location or renamed while running.
  • Creates a //GLOBAL event to check if any other process has handle to
    the event.
  • Util is used on shutdown via ShutdownPostThreadsStop to perform rename
    and swap when there is a pending update available (and is not an ENDSESSION)
    on disk via upgrade_util::SwapNewChromeExeIfPresent
  • On startup via DoUpgradeTasks
    • To run upgrade_util::SwapNewChromeExeIfPresent
    • Check if we are running old_chrome.exe via GetMappedFileName
    • If either of the above, then attempt to relaunch ensuring the relaunched
      child process starts from the non versioned directory to avoid any handles
      to the version directory in the child process.
  • upgrade_util::SwapNewChromeExeIfPresent
    • Triggered from chrome.exe running process and checks for the presence of new_chrome.exe
      file on disk which would be created by an installer when a new version is available,
      chrome.exe would then be renamed to old_chrome.exe and placed into a temporary
      directory and the current process would keep running as old_chrome.exe until it exits.
    • Rename is done via a sub process whose command is obtained from a registry key that was
      created by the installer. For system installs the renaming task is delegated to the
      updater client to perform an elevated action.

Setup/Installer

• SetupMain is the entry point
• Acquires a singleton to only allow a single instance at any time
• Handles various command line options
  • InstallOrUpdateProduct
    • InstallNewVersion
      • Extract archive to a temp dir <AppInstallDir>/Temp
      • Create VisualElementsManifest in temp dir
      • Delete any existing new_chrome.exe in <AppInstallDir>
      • Copy chrome.exe from temp dir to <AppInstallDir>, if file in use then copy as new_chrome.exe
        • Open a file handle with write | delete, if handle is invalid then file is in use
      • Move VisualElementsManifest from temp dir to <AppInstallDir> if it exists, otherwise
        delete any old VisualElementsManifest from <AppInstallDir> to avoid pointing to old installation
      • Move version directory from temp dir to <AppInstallDir>
        • check for duplicates if current version is in use and same as new version (repair installation flow)
        • TBD how this works
      • Delete any old_chrome.exe from <AppInstallDir> ignoring failures
      • If this is an in-use update, then register a rename command to the registry key
        that will be picked up by the application main process when launching the setup with --rename flag
    • Result of the above moves will yield any of the following results,
      • Move operation failed
        • If current version exists and new_chrome.exe exists and new version == current version
          • Same version repair failed
          • else, install failed
      • Move operation success
        • Current version does not exist
          • Fist install success
        • New version == current version
          • Install repaired
          • TBD, how does this work when running instance ??
        • New version > current version
          • new_chrome.exe exists then in-use updated
          • else, new version updated
            • TBD, How is version computed when current instance is not running ??
    • TBD, downgrade flow ??
    • If first install or install repaired then create shortcut else replace existing shortcut via a
      child process on the setup executable that will call CreateOrUpdateShortcuts.
    • If not an in-use update, attempt to delete old version in current installer via RepeatDeleteOldVersions
    • For in-use update, the renaming task will schedule deletion of the older version
  • RenameChromeExecutables performs the rename task (--rename)
    • On successful rename, launches a breakaway child process of setup exectuable
      that performs the deletion of old version directory and executables via LaunchDeleteOldVersionsProcess.
    • On failure, rollback file moves.
  • RepeatDeleteOldVersions (--delete-old-version)
    • Waits 15s before first attempt and 5min after each failed attempt for a maximum
      of 3 reties.
    • DeleteOldVersions
      • Get directory of old_chrome.exe by reading version information from file via
        GetFileVersionInfo, and exclude directories of new_chrome.exe, chrome.exe via strict version check
      • Get old_chrome.exe and related old executable files
      • Steps for deletion
        • Get locks to exe and dll inside the version directory
        • Get lock to the old exe in the install directory
        • Only if above succeeds continue to delete the executables
        • If above succeeds then proceed to delete rest of the files and folders inside the versioned directory

cross platform update client

https://source.chromium.org/chromium/chromium/src/+/main:docs/updater/design_doc.md

Not going into the details of the update client for this particular exploration but useful in the case of system installs to perform elevated actions.

[deepak1556]

Next Steps

• Run electron in which exe is isolated from rest of the components to emulate version folder
• New env mode for Electron to start with cli entry point from package.json
• Inno setup
  • Extracts setup to Temp directory
  • Moves version directory and files as new_**.exe (as needed)
  • Returns control to the application to signal update is ready to be installed
• Inno updater
  • Launched with --rename flag to perform final rename of exe files in the root install dir
  • If success, returns control to the application which will then relaunch with the exe from updated version
• VSCode
  • On startup and shutdown checks for the need to swap files and calls inno-updater with --rename
  • On startup in the non-critical path, trigger a gc to cleanup old version installations
    • Calls inno updater with --delete-old-versions

[NickCraver]

I want to make sure we're testing multiple instances open while updating here - that was at least my issue in #232817 (comment) and how I'm manually working around it today (going down to 1 instance, forcing update to run, then closing that last instance).