Different behavior between datetime.py and its C accelerator

[abalkin]

BPO	33812
Nosy	@tim-one, @abalkin
PRs	bpo-33812: Corrected astimezone for naive datetimes. #7578 [3.7] bpo-33812: Corrected astimezone for naive datetimes. (GH-7578) #7600 [3.6] bpo-33812: Corrected astimezone for naive datetimes. (GH-7578) #7601

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/abalkin'
closed_at = <Date 2018-06-10.22:06:00.260>
created_at = <Date 2018-06-09.00:02:26.755>
labels = ['extension-modules', '3.8', 'type-bug', 'library', '3.7']
title = 'Different behavior between datetime.py and its C accelerator'
updated_at = <Date 2018-06-10.22:06:00.259>
user = 'https://github.com/abalkin'

bugs.python.org fields:

activity = <Date 2018-06-10.22:06:00.259>
actor = 'belopolsky'
assignee = 'belopolsky'
closed = True
closed_date = <Date 2018-06-10.22:06:00.260>
closer = 'belopolsky'
components = ['Extension Modules', 'Library (Lib)']
creation = <Date 2018-06-09.00:02:26.755>
creator = 'belopolsky'
dependencies = []
files = []
hgrepos = []
issue_num = 33812
keywords = ['patch']
message_count = 9.0
messages = ['319121', '319122', '319131', '319134', '319195', '319196', '319245', '319253', '319254']
nosy_count = 2.0
nosy_names = ['tim.peters', 'belopolsky']
pr_nums = ['7578', '7600', '7601']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue33812'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']

[abalkin]

Consider the following code:

$ cat bug.py
import sys
if len(sys.argv) > 1:
    sys.modules['_datetime'] = None
from datetime import tzinfo, datetime, timezone

class TZ(tzinfo):
    def utcoffset(self, t):
        pass

print(datetime(2000,1,1,tzinfo=TZ()).astimezone(timezone.utc))

When running with no arguments (with C acceleration), I get

$ ./python.exe bug.py
2000-01-01 00:00:00+00:00

but the pure python code produces an error

$ ./python.exe bug.py pure
Traceback (most recent call last):
  File "bug.py", line 10, in <module>
    print(datetime(2000,1,1,tzinfo=TZ()).astimezone(timezone.utc))
  File ".../Lib/datetime.py", line 1783, in astimezone
    raise ValueError("astimezone() requires an aware datetime")
ValueError: astimezone() requires an aware datetime

Note that some kind of error is expected because TZ.utcoffset() returns None instead of a timedelta, but the error message produced by pure python code is confusing.

[tim-one]

The message isn't confusing - the definition of "aware" is confusing ;-)

"""
A datetime object d is aware if d.tzinfo is not None and d.tzinfo.utcoffset(d) does not return None. If d.tzinfo is None, or if d.tzinfo is not None but d.tzinfo.utcoffset(d) returns None, d is naive.
"""

[abalkin]

So you are suggesting that my datetime(2000,1,1,tzinfo=TZ()) should behave as a naive instance, right? Well, this would be a third behavior different from both current C and Python implementations:

>>> print(datetime(2000,1,1).astimezone(timezone.utc))
2000-01-01 05:00:00+00:00

(I am in US/Eastern timezone.)

[tim-one]

I copy/pasted the definitions of "aware" and "naive" from the docs. Your TZ's .utcoffset() returns None, so, yes, any datetime using an instance of that for its tzinfo is naive.

In

print(datetime(2000,1,1).astimezone(timezone.utc))

the docs for astimezone say, in part,

"""
If self is naive (self.tzinfo is None), it is presumed to represent time in the system timezone.
"""

So it converted your naive time (viewed as being in your system - EDT - time zone) to UTC.

That appears to be using a different definition of "naive" (looking only at whether self.tzinfo is None).

The original datetime.py certainly didn't do that ...

"""
def astimezone(self, tz):
if not isinstance(tz, tzinfo):
raise TypeError("tz argument must be an instance of tzinfo")
mytz = self.tzinfo
if mytz is None:
raise ValueError("astimezone() requires an aware datetime")

    if tz is mytz:
        return self

    # Convert self to UTC, and attach the new time zone object.
    myoffset = self.utcoffset()
    if myoffset is None:
        raise ValueError("astimezone() requires an aware datetime")
"""

So it originally used the definition I quoted first. The "sometimes pretend it's local time anyway" twist appeared to get added here:

fdc860f

[abalkin]

Tim, given that I've updated the documentation, should we treat this as a bug fix or a feature? Note that the type check is definitely a bug-fix (if not a security issue), but I clearly had a wrong definition of "naive" in mind when I was modifying astimezone to support naive instances.

In any case, it looks like a news entry is in order.

[tim-one]

I'd call it a bug fix, but I'm really not anal about what people call things ;-)

[abalkin]

New changeset 877b232 by Alexander Belopolsky in branch 'master':
bpo-33812: Corrected astimezone for naive datetimes. (GH-7578)
877b232

[abalkin]

New changeset 037e912 by Alexander Belopolsky (Miss Islington (bot)) in branch '3.7':
bpo-33812: Corrected astimezone for naive datetimes. (GH-7578) (GH-7600)
037e912