Python implementation of datetime module is not being tested correctly.

[musically-ut]

BPO	30822
Nosy	@abalkin, @vstinner, @ned-deily, @serhiy-storchaka, @musically-ut
PRs	bpo-30822: Fix testing of datetime module. #2530 bpo-30822: Deduplicate ZoneInfoTest classes in test_datetime. #2534 [3.5] bpo-30822: Fix testing of datetime module. (GH-2530) #2550 [3.6] bpo-30822: Fix testing of datetime module. (GH-2530) #2551 Revert "bpo-30822: Fix testing of datetime module." #2588 bpo-30822: Exclude tzdata from regrtest --all #2775 [3.6] bpo-30822: Exclude tzdata from regrtest --all (#2775) #2777 [3.5] bpo-30822: Exclude tzdata from regrtest --all (#2775) #2781 [3.5] bpo-30822: regrtest: remove tzdata #2782 bpo-30822: Fix testing of datetime module. #2783 [2.7] bpo-30822: regrtest: fix -u extralargefile #2788 [3.5] bpo-30822: Fix testing of datetime module. (GH-2530) (GH-2783) #2815 [3.6] bpo-30822: Fix testing of datetime module. (GH-2530) (GH-2783) #2816 [3.6] bpo-30822: Deduplicate ZoneInfoTest classes in test_datetime. (GH-2534) #3405

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2017-07-26.11:52:55.129>
created_at = <Date 2017-06-30.23:47:48.837>
labels = ['3.7', 'type-bug', 'tests']
title = 'Python implementation of datetime module is not being tested correctly.'
updated_at = <Date 2017-09-08.22:44:35.784>
user = 'https://github.com/musically-ut'

bugs.python.org fields:

activity = <Date 2017-09-08.22:44:35.784>
actor = 'vstinner'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2017-07-26.11:52:55.129>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2017-06-30.23:47:48.837>
creator = 'musically_ut'
dependencies = []
files = []
hgrepos = []
issue_num = 30822
keywords = []
message_count = 48.0
messages = ['297455', '297470', '297515', '297517', '297518', '297519', '297520', '297522', '297523', '297524', '297525', '297530', '297546', '297548', '297554', '297555', '297557', '297559', '297562', '297571', '297573', '297612', '297734', '297736', '297743', '297752', '297753', '297760', '297762', '297763', '298032', '298033', '298059', '298664', '298717', '298721', '298725', '298726', '298729', '298732', '298755', '298756', '298786', '299221', '299222', '299223', '299224', '301749']
nosy_count = 5.0
nosy_names = ['belopolsky', 'vstinner', 'ned.deily', 'serhiy.storchaka', 'musically_ut']
pr_nums = ['2530', '2534', '2550', '2551', '2588', '2775', '2777', '2781', '2782', '2783', '2788', '2815', '2816', '3405']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue30822'
versions = ['Python 3.6', 'Python 3.7']

[musically-ut]

While investigating http://bugs.python.org/issue30302 it was discovered that the Python implementation of the datetime module was not being tested correctly.

The datetimetester.py was supposed to execute tests twice, once for the _Fast implementation (i.e. C extension code in _datetimemodule.c) and once for the _Pure implementation (i.e the Python code).

The fix is contained in the following two commits:

• 08e7548
• 94d5a4e

The bug does not effect Python 2.7 as the C version of the datetime module had not been introduced back then. However, as fas as I can tell, the fix needs to be applied to all Python 3.x branches.

[serhiy-storchaka]

Please provide a separate PR for this. After merging it with the master branch it should be backported to 3.6 and 3.5 (3.4 and 3.3 are for security only fixes). Add your name in Misc/ACKS. Changes in Misc/NEWS.d/ are not required.

[serhiy-storchaka]

New changeset 98b6bc3 by Serhiy Storchaka (Utkarsh Upadhyay) in branch 'master':
bpo-30822: Fix testing of datetime module. (bpo-2530)
98b6bc3

[musically-ut]

Are there any guides re: backporting commits?

Just to confirm verify that I'm going about it right; I’m planning on cherry-picking relevant commits back to branch 3.5, 3.6, resolving the merge conflicts, and then opening two separate PRs for them with bpo-30822 in their titles.

Sounds about right?

~
ut

[ned-deily]

Serhiy, it appears that the added tests are taking a long time to run and thus causing many buildbot failures, for example:

http://buildbot.python.org/all/builders/ARMv7%20Ubuntu%203.x/builds/1013

[serhiy-storchaka]

Oh, test.datetimetester.ZoneInfoTest[Europe/Andorra]_Pure_Pure_Pure -- something looks wrong.

[musically-ut]

I can reproduce the names with _Pure_Pure_Pure if I run the tests with -utzdata. Investigating.

~
ut

[serhiy-storchaka]

The quick solution -- just deduplicate test classes.

But the code needs rewriting. Current code is tricky and fragile. It doesn't work with --list-cases (and I suppose it doesn't work well with other advanced unittest features). This would be not easy.

[serhiy-storchaka]

Are there any guides re: backporting commits?

Just to confirm verify that I'm going about it right; I’m planning on cherry-picking relevant commits back to branch 3.5, 3.6, resolving the merge conflicts, and then opening two separate PRs for them with bpo-30822 in their titles.

All right. See https://docs.python.org/devguide/committing.html#backporting-changes-to-an-older-version . You can cherry-pick manually or using cherry_picker.py which adds correct prefix to commit message and helps to create a pull request.