Windows buildbots broken by the migration to GitHub (meta issue)

[vstinner]

BPO	29530
Nosy	@pfmoore, @vstinner, @tjguk, @jkloth, @bitdancer, @zware, @serhiy-storchaka, @zooba

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-06-28.00:57:41.501>
created_at = <Date 2017-02-11.01:58:36.192>
labels = ['3.7', 'tests', 'OS-windows']
title = 'Windows buildbots broken by the migration to GitHub (meta issue)'
updated_at = <Date 2017-06-28.00:57:41.498>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2017-06-28.00:57:41.498>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2017-06-28.00:57:41.501>
closer = 'vstinner'
components = ['Tests', 'Windows']
creation = <Date 2017-02-11.01:58:36.192>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 29530
keywords = ['buildbot']
message_count = 9.0
messages = ['287587', '287603', '287608', '287610', '287613', '287621', '287623', '287625', '297091']
nosy_count = 8.0
nosy_names = ['paul.moore', 'vstinner', 'tim.golden', 'jkloth', 'r.david.murray', 'zach.ware', 'serhiy.storchaka', 'steve.dower']
pr_nums = []
priority = 'critical'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue29530'
versions = ['Python 2.7', 'Python 3.5', 'Python 3.6', 'Python 3.7']

[vstinner]

(*) I noticed test_sax and test_random failures on Windows since CPython moved to GitHub. These failures seem to be caused by newlines: see issue bpo-27425.

(*) "AMD64 Windows7 SP1 3.x" fails on the "git" step with:

'git' is not recognized as an internal or external command

I contacted the slave owner, Jeremy, to ask him to install git.

[jkloth]

After ensuring that Git is available to the buildbot service, the initial update fails due to the existing build directory containing the Mercurial checkout.

After manually removing the contents of the build directories, the update task successfully completes.

[jkloth]

Tests are now failing due to end-of-line (?) differences.

I did a quick reading of the devguide, but didn't see any mention of the settings to be used for Git on Windows wrt core.autocrlf.

[zware]

We really need to be able to function regardless of the core.autocrlf setting. See bpo-27425 for more about the EOL issues.

[zooba]

My vote would be to make the tests not presume anything about line endings, to the point of converting test files on the fly to be sure.

I'm mostly free today (in a different city with nothing to do), so if my hotel internet will let me SSH to github then I can work on this today. If people are around, I'll be on IRC too.

[bitdancer]

Some of the tests (notably email tests) are *testing* platform line endings. In tests that I rewrote I did my best to make the tests independent of the line endings of checked in files (that is, constructing the files-to-test in the test itself instead of checking them in), but I wouldn't be surprised if there were a few tests that weren't converted to that paradigm. (I haven't looked to see if there are any test.test_email.test_email failures, which is where they'd be.)

[serhiy-storchaka]

Does Git support per-file settings like the svn:eol-style property in Subversion or .hgeol in Mercurial?

[zooba]

Does Git support per-file settings like the svn:eol-style property in Subversion or .hgeol in Mercurial?

The .gitattributes file seems to be full of glob patterns, so I assume a full file path will work.

However, that doesn't help with tarballs. Fixing the tests is the best way.

[vstinner]

Windows buildbots are green since at least 2 months, so I close this issue.

There is one remaining issue, but specific to the Windows *installer*: see bpo-30716 and bpo-27425.