Docker issues with pypiwin32 on Windows Server machines

[MattGal]

As seen in recent helix-machines pipelines,

2022-11-07T23:42:20.932Z	INFO   	interval(67)	set_abort_after_timer	Set the timer's abort_after to 1260 seconds from now: abort_after = 2022-11-08 00:03:20.932918
2022-11-07T23:42:21.051Z	ERROR  	executor(729)	_execute_command_in_container	Exception: Install pypiwin32 package to enable npipe:// support
2022-11-07T23:42:21.052Z	INFO   	interval(67)	set_abort_after_timer	Set the timer's abort_after to 180 seconds from now: abort_after = 2022-11-07 23:45:21.052565

This has been consistently occurring since 11/5 with no obvious changes between when it repros and doesn't; I am investigating.

EDIT: This is occurring everywhere in production too so I've added the critical tag and known build error labels

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

Release Note Description

Fix issue with Windows Helix Docker scenarios failing with no logging and '999' exit code (caused by updated pywin32 package)

[MattGal]

I think this might be addressed via https://dnceng.visualstudio.com/internal/_git/dotnet-helix-machines/pullrequest/27346?_a=files. Will know in several hours. If it solves the problem, it should be safe to hotfix (just changes the installer to use the --force flag)

[MattGal]

The fix worked. As all Windows Docker scenarios are impacted currently I've set up a hotfix deployment here. ETA for fix availability is around noon PST.

[MattGal]

No more 999 exit code work items since at least 2022-11-10 20:40:07.1360000 UTC, closing.