tizinfo.fromutc changed for tzinfo wih StdOffset=0, DstOffset=1

[PeterJCLaw]

BPO	23600
Nosy	@tim-one, @abalkin, @PeterJCLaw, @serhiy-storchaka
Files	time_issues.py: Shorter demo of the issue issue23600.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/abalkin'
closed_at = <Date 2015-09-28.17:48:47.544>
created_at = <Date 2015-03-07.00:43:03.247>
labels = ['extension-modules', 'type-bug']
title = 'tizinfo.fromutc changed for tzinfo wih StdOffset=0, DstOffset=1'
updated_at = <Date 2015-09-28.19:30:39.956>
user = 'https://bugs.python.org/peterjclaw'

bugs.python.org fields:

activity = <Date 2015-09-28.19:30:39.956>
actor = 'tim.peters'
assignee = 'belopolsky'
closed = True
closed_date = <Date 2015-09-28.17:48:47.544>
closer = 'belopolsky'
components = ['Extension Modules']
creation = <Date 2015-03-07.00:43:03.247>
creator = 'peterjclaw'
dependencies = []
files = ['38372', '40603']
hgrepos = []
issue_num = 23600
keywords = ['patch']
message_count = 15.0
messages = ['237407', '237414', '237440', '251672', '251675', '251717', '251725', '251734', '251735', '251737', '251741', '251743', '251744', '251785', '251793']
nosy_count = 7.0
nosy_names = ['tim.peters', 'belopolsky', 'python-dev', 'PeterJCLaw', 'serhiy.storchaka', 'alynn', 'peterjclaw']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue23600'
versions = ['Python 3.4', 'Python 3.5', 'Python 3.6']

[PeterJCLaw]

There's a difference in behaviour between the fromutc method on a tzinfo between Python 2 and Python 3, though only under the specific case of Summer Time in regions whose usual offset is 0.

From what I can tell, it's the Python 3 one which is wrong, based on it no longer matching the sample implementation provided in the docs and on it appearing to report the wrong times for datetime.now(tz) when on a machine configured for BST during June 2015.
Similar results can also be achieved using a manually constructed datetime for dates during summer 2014.

I've put the python script (and sample outputs) I used to investigate in a gist at https://gist.github.com/PeterJCLaw/d8bcc168d68acf066811#file-time_issues-py. The outputs there are for pythons 2.7.9 and 3.4.0.

[abalkin]

Peter,

Can you attach your demo script to the issue? Better yet, is it possible to explain the issue without referring to 100 lines of code?

[PeterJCLaw]

Hi,

Sorry for the overkill demo. I've attached a much shorter version, the key portion of which seems to be that, for the case of UK summer time the timezone, the tzinfo's dst() and utcoffset() methods return the same value.

This results in the delta between the two (which my understanding suggests equates to the non-dst offset of the timezone) is zero (which is right).

The python implementations (both in datetime.py and in the docs) cope with this by checking the DST difference and applying this after the timezone adjustments have happened.

From a look through the CPython implementation, it looks to me like it's checking the wrong value before applying the DST difference. Specifically, on line 3033 in Modules/_datetimemodule.c, it checks delta before applying the DST.

For the majority of timezones this happens to work since the standard offset is not 0 and it ends up doing the addition anyway (and no functionality is lost if the DST value is 0).

Having tested changing the checked value to being dst rather than delta this does appear to fix this and all the existing tests still pass.

Peter

[abalkin]

I am afraid you misunderstand how fromutc() method works. Note that you rarely need to call it directly: use astimezone() method to convert between timezones.

[tim-one]

I expect Peter is correct: the C fromutc() doesn't match the logic of the Python fromutc(), and there are no comments explaining why the C version changed the logic.

The last 4 lines of his time_issues.py show the difference. The simplified UKSummerTime tzinfo always says the total UTC offset and the DST offset are +1:00:00. The Python .fromutc() adds that hour to the datetime passed in, but the C .fromutc() does not. That's because they implement different functions, not because Peter is using .fromutc() ;-)

[abalkin]

It looks like I introduced this bug ~ 5 years ago when I made a switch from integer minutes offset to an arbitrary timedelta. It's rather amazing that it took so long to discover it.

[abalkin]

Attaching a patch that should fix the issue.

[tim-one]

Patch looks good to me! Thanks :-)

[abalkin]

Thanks, Tim. Now I need to figure out how to commit to multiple branches. This goes to 3.4 through 3.6, right? or just to 3.5 and 3.6.

[tim-one]

Afraid that's a question for python-dev - I lost track of the active branches over year ago :-(