Windows: datetime.datetime.now() raises an OverflowError for date after year 2038

[vmurashev]

BPO	25155
Nosy	@pfmoore, @abalkin, @vstinner, @tjguk, @zware, @zooba, @vmurashev, @srinivasreddy, @MojoVampire, @PetterS

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-09-18.12:00:51.578>
created_at = <Date 2015-09-17.19:25:39.982>
labels = ['interpreter-core', '3.8', 'type-bug', '3.7', 'OS-windows']
title = 'Windows: datetime.datetime.now() raises an OverflowError for date after year 2038'
updated_at = <Date 2018-11-22.07:53:26.796>
user = 'https://github.com/vmurashev'

bugs.python.org fields:

activity = <Date 2018-11-22.07:53:26.796>
actor = 'thatiparthy'
assignee = 'none'
closed = True
closed_date = <Date 2015-09-18.12:00:51.578>
closer = 'vstinner'
components = ['Interpreter Core', 'Windows']
creation = <Date 2015-09-17.19:25:39.982>
creator = 'vmurashev'
dependencies = []
files = []
hgrepos = []
issue_num = 25155
keywords = ['3.5regression']
message_count = 16.0
messages = ['250914', '250925', '250963', '250967', '250968', '250969', '250971', '320431', '320432', '320441', '320447', '320473', '320476', '330231', '330232', '330234']
nosy_count = 11.0
nosy_names = ['paul.moore', 'belopolsky', 'vstinner', 'tim.golden', 'python-dev', 'zach.ware', 'steve.dower', 'vmurashev', 'thatiparthy', 'josh.r', 'Petter S']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue25155'
versions = ['Python 3.5', 'Python 3.7', 'Python 3.8']

[vmurashev]

Current time on my machine with Windows7x64 is set to year 2045 for test purposes.
Since Python3.5(amd64) I have an OverflowError when I am trying to call datetime.datetime.now()

It looks like a regress since there was no such error on Python3.4.3

Could anyone please give me a note, whether it would be reasonable for me to wait for a patch in Python3.5.x, or such behavior is common since 3.5 and should not use it in my 'strange' case at all ?

A bit of details:

Python 3.5.0 (v3.5.0:374f501f4567, Sep 13 2015, 02:27:37) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import datetime
>>> datetime.datetime.now()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: timestamp too large to convert to C _PyTime_t
>>>

Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:44:40) [MSC v.1600 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import datetime
>>> datetime.datetime.now()
datetime.datetime(2045, 4, 2, 2, 42, 8, 359375)
>>>

[MojoVampire]

It looks like between 3.4.3 and 3.5, datetime_best_possible changed from using _PyTime_gettimeofday with a _PyTime_timeval to using _PyTime_AsTimeval with struct timeval. The difference is that _PyTime_timeval appears to have been defined with a proper time_t for secs, where struct timeval is defined (on Windows) with a long which is still 32 bits on Windows, so we reintroduced the Y2K38 problem.

The bug was introduced while trying to support bpo-22117: "Rewrite pytime.h to work on nanoseconds" in reve93eeadef0c3: https://hg.python.org/cpython/rev/e93eeadef0c3

[python-dev]

New changeset 203134592edf by Victor Stinner in branch 'default':
Issue bpo-25155: Add _PyTime_AsTimevalTime_t() function
https://hg.python.org/cpython/rev/203134592edf

[python-dev]

New changeset 4ca99a0a18e4 by Victor Stinner in branch '3.5':
Issue bpo-25155: Add _PyTime_AsTimevalTime_t() function
https://hg.python.org/cpython/rev/4ca99a0a18e4

[vstinner]

Oh wow, I didn't expect such much headaches when I worked on unifiying the code to handle timestamps in the C part of CPython...

The bug was introduced while trying to support bpo-22117: "Rewrite pytime.h to work on nanoseconds" in reve93eeadef0c3: https://hg.python.org/cpython/rev/e93eeadef0c3

Yeah, the changeset e93eeadef0c3 was the last change to drop the last function of the old PyTime.

Well... in fact, I also had to keep _PyTime_ObjectToTime_t() and _PyTime_ObjectToTimeval() for the exact same reason: support timestamp after the year 2038 on Windows.

Right, the tv_sec field of a timeval structure on Windows has the C type long, whereas it has the type time_t on all other platforms...

I fixed the bug in Python 3.5 and 3.6. Thanks for your bug report. I didn't expect to test year 2038 bug on the latest release, thanks time traveler :-)

[python-dev]

New changeset 5bfcccf229c4 by Victor Stinner in branch '3.5':
Issue bpo-25155: document the bugfix in Misc/NEWS
https://hg.python.org/cpython/rev/5bfcccf229c4

[python-dev]

New changeset f1cc1f379b00 by Victor Stinner in branch '3.5':
Issue bpo-25155: Fix _PyTime_Divide() rounding
https://hg.python.org/cpython/rev/f1cc1f379b00

[PetterS]

I get this error when starting the interpreter in Windows subsystem for Linux (WSL).

I am using Python 2.7.15rc1

$ python --version
Python 2.7.15rc1
$ python
Fatal Python error: _Py_InitializeMainInterpreter: can't initialize time                                                                                                                                          OverflowError: timestamp too large to convert to C _PyTime_t                                                                                                                                                                                                                                                                                                                                                                        Current thread 0x00007fe547231080 (most recent call first):                                                                                                                                                       Aborted (core dumped)