test_time error on AIX

[sable]

BPO	11188
Nosy	@abalkin, @vstinner, @kadler
Files	strftime_aix.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2011-02-11.14:51:10.125>
labels = ['extension-modules', 'type-bug']
title = 'test_time error on AIX'
updated_at = <Date 2020-11-17.20:46:27.438>
user = 'https://bugs.python.org/sable'

bugs.python.org fields:

activity = <Date 2020-11-17.20:46:27.438>
actor = 'kadler'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Extension Modules']
creation = <Date 2011-02-11.14:51:10.125>
creator = 'sable'
dependencies = []
files = ['20762']
hgrepos = []
issue_num = 11188
keywords = ['patch']
message_count = 15.0
messages = ['128396', '128541', '128556', '128561', '128562', '128565', '128577', '128601', '128604', '128605', '128609', '128610', '128637', '131592', '211886']
nosy_count = 7.0
nosy_names = ['belopolsky', 'vstinner', 'sable', 'python-dev', 'alef', 'David.Edelsohn', 'kadler']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue11188'
versions = ['Python 3.2']

[sable]

I have the following errors on test_time on AIX:

======================================================================
ERROR: test_mktime (test.test_time.TestAsctime4dyear)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/test_time.py", line 351, in test_mktime
    self.assertEqual(time.mktime(tt), t)
OverflowError: mktime argument out of range

======================================================================
ERROR: test_mktime (test.test_time.TestStrftime4dyear)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/test_time.py", line 351, in test_mktime
    self.assertEqual(time.mktime(tt), t)
OverflowError: mktime argument out of range

======================================================================
ERROR: test_mktime (test.test_time.Test4dyearBool)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/test_time.py", line 351, in test_mktime
    self.assertEqual(time.mktime(tt), t)
OverflowError: mktime argument out of range

======================================================================
FAIL: test_negative (test.test_time.TestStrftime4dyear)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/test_time.py", line 337, in test_negative
    self.assertIn(text, ('-1', '-001'))
AssertionError: '000/' not found in ('-1', '-001')

---

Ran 42 tests in 1.217s

FAILED (failures=1, errors=3)

Haven't investigated yet.

[vstinner]

File ".../Lib/test/test_time.py", line 351, in test_mktime
self.assertEqual(time.mktime(tt), t)
OverflowError: mktime argument out of range

I don't know which values are "out of range". But I guess that the test fails because of t=-1, which would mean that tm_wday field is not modified by mktime().

I don't know if mktime() does really not support t=-1, or if we should use another sentinel. The original patch to fix this issue (support t=-1, issue bpo-1726687) used tm_wday=42 instead of tm_wday=-1:

+	/* invalid value that will not be changed if there is an error. */
+	buf.tm_wday = 42;
 	tt = mktime(&buf);
-	if (tt == (time_t)(-1)) {
+	if ((tt == (time_t)(-1)) && (buf.tm_wday == 42)) {
 		PyErr_SetString(PyExc_OverflowError,
 				"mktime argument out of range");
 		return NULL;

The current code uses:

    buf.tm_wday = -1;  /* sentinel; original value ignored */                   
    tt = mktime(&buf);                                                          
    /* Return value of -1 does not necessarily mean an error, but tm_wday       
     * cannot remain set to -1 if mktime succedded. */                          
    if (tt == (time_t)(-1) && buf.tm_wday == -1) {                              
        PyErr_SetString(PyExc_OverflowError,                                    
                        "mktime argument out of range");                        
        return NULL;                                                            
    }                 

File ".../Lib/test/test_time.py", line 337, in test_negative
self.assertIn(text, ('-1', '-001'))
AssertionError: '000/' not found in ('-1', '-001')

AIX doesn't support negative tm_year value. It should be added to the timemodule blacklist (#ifdef):

#if defined(_MSC_VER) || defined(sun)
    if (buf.tm_year + 1900 < 1 || 9999 < buf.tm_year + 1900) {
        PyErr_Format(PyExc_ValueError,
                     "strftime() requires year in [1; 9999]",
                     buf.tm_year + 1900);
        return NULL;
    }
#endif

I don't know if there is a reliable C define to check if the current OS is AIX (something like #ifdef sun). Python configure script checks if "uname -s" output starts with "AIX". We should maybe add a Python define in pyconfig.h using the configure script.

[sable]

I tried the following patch (_AIX is defined on AIX platforms):

Index: Modules/timemodule.c
===================================================================

--- Modules/timemodule.c	(révision 88420)
+++ Modules/timemodule.c	(copie de travail)
@@ -474,7 +474,7 @@
     else if (!gettmarg(tup, &buf) || !checktm(&buf))
         return NULL;
 
-#if defined(_MSC_VER) || defined(sun)
+#if defined(_MSC_VER) || defined(sun) || defined(_AIX)
     if (buf.tm_year + 1900 < 1 || 9999 < buf.tm_year + 1900) {
         PyErr_Format(PyExc_ValueError,
                      "strftime() requires year in [1; 9999]",
@@ -694,11 +694,12 @@
     time_t tt;
     if (!gettmarg(tup, &buf))
         return NULL;
-    buf.tm_wday = -1;  /* sentinel; original value ignored */
+    /* invalid value that will not be changed if there is an error. */
+    buf.tm_wday = 42;
     tt = mktime(&buf);
     /* Return value of -1 does not necessarily mean an error, but tm_wday
      * cannot remain set to -1 if mktime succedded. */
-    if (tt == (time_t)(-1) && buf.tm_wday == -1) {
+    if ((tt == (time_t)(-1)) && (buf.tm_wday == 42)) {
         PyErr_SetString(PyExc_OverflowError,
                         "mktime argument out of range");
         return NULL;

This resulted in the following:
======================================================================
ERROR: test_mktime (main.TestAsctime4dyear)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "Lib/test/test_time.py", line 351, in test_mktime
    self.assertEqual(time.mktime(tt), t)
OverflowError: mktime argument out of range

======================================================================
ERROR: test_mktime (main.TestStrftime4dyear)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "Lib/test/test_time.py", line 351, in test_mktime
    self.assertEqual(time.mktime(tt), t)
OverflowError: mktime argument out of range

======================================================================
ERROR: test_mktime (main.Test4dyearBool)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "Lib/test/test_time.py", line 351, in test_mktime
    self.assertEqual(time.mktime(tt), t)
OverflowError: mktime argument out of range

---

Ran 42 tests in 1.395s

FAILED (errors=3)

So test_negative is now OK, but tm_wday = 42 did not solve the problem it seems.

[vstinner]

So test_negative is now OK

Ok, I converted your comment to a patch: strftime_aix.patch.

[vstinner]

but tm_wday = 42 did not solve the problem it seems.

Can you try with tm_yday=-1 or tm_isdst=-2?

[abalkin]

I have come across the following post:

http://code.google.com/p/y2038/wiki/AmazingDiscoveries

I think some of the findings there may be relevant as we push the
boundaries of supported time values.

[vstinner]

http://code.google.com/p/y2038/wiki/AmazingDiscoveries

<<< AIX again

Merijn informs me that before year 0 AIX gets very, very slow. >>>

[sable]

Here is what I could find in the "man mktime":
"""
Upon successful completion, the mktime subroutine sets the values of the tm_wday and tm_yday fields appropriately. Other fields are set to
represent the specified time since January 1, 1970. However, the values are forced to the ranges specified in the /usr/include/time.h file.
The final value of the tm_mday field is not set until the values of the tm_mon and tm_year fields are determined. Note: The mktime
subroutine cannot convert time values before 00:00:00 UTC, January 1, 1970 and after 03:14:07 UTC, January 19, 2038.
"""

I tried tm_yday=-1 then tm_isdst=-2 and both gave the same errors as before.

[abalkin]

Sébastien,

Can you tell us what time.localtime(t) produces for t in (-2, -1, 0, 1)? Apparently time.mktime() fails on values produced by time.localtime() and this sounds like a platform bug. It is OK to restrict time_t to positive values, but in this case time.localtime(t) should reject t < 0.

If there is a Python issue here, it is likely to be in error detection in time.localtime().

Also, what is your timezone?

[abalkin]

It looks like different standards have different requirements for
localtime() error handling compare:

http://pubs.opengroup.org/onlinepubs/009695399/functions/localtime.html

and

http://pubs.opengroup.org/onlinepubs/7990989775/xsh/localtime.html

The later does not require that localtime() returns NULL or sets
errno on error. AIX documentation seems to be silent on this issue as
well:

http://publib.boulder.ibm.com/infocenter/aix/v6r1/index.jsp?topic=/com.ibm.aix.basetechref/doc/basetrf1/ctime.htm

[sable]

Python 3.2rc3+ (py3k:88422M, Feb 15 2011, 16:57:29) [C] on aix6
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> for t in (-2, -1, 0, 1):
...     print(time.localtime(t))
... 
time.struct_time(tm_year=1970, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=59, tm_sec=58, tm_wday=3, tm_yday=1, tm_isdst=0)
time.struct_time(tm_year=1970, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=59, tm_sec=59, tm_wday=3, tm_yday=1, tm_isdst=0)
time.struct_time(tm_year=1970, tm_mon=1, tm_mday=1, tm_hour=1, tm_min=0, tm_sec=0, tm_wday=3, tm_yday=1, tm_isdst=0)
time.struct_time(tm_year=1970, tm_mon=1, tm_mday=1, tm_hour=1, tm_min=0, tm_sec=1, tm_wday=3, tm_yday=1, tm_isdst=0)

I am in Paris. On the system, I get:
$ date
Tue Feb 15 18:18:58 NFT 2011
$ env | grep TZ
TZ=NFT-1DFT,M3.5.0,M10.5.0

which is strange since NFT seems to be in Australia.

I will check that tomorrow with the sysadmin.

[abalkin]

2011/2/15 Sébastien Sablé <report@bugs.python.org>:
..
>>>> for t in (-2, -1, 0, 1):
> ...     print(time.localtime(t))
> ...
> time.struct_time(tm_year=1970, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=59, tm_sec=58, tm_wday=3, tm_yday=1, tm_isdst=0)
..

This looks right. (For time.timezone = -3600.) I actually suspected
that you were east of Greenwich. My hypothesis is that AIX localtime
implementation adjusts t -> t - timezone before computing broken down
time and thus fails to detect that localtime() is given negative
argument. If my hypothesis is correct, time.gmtime(-1) should fail on
your system.

..

TZ=NFT-1DFT,M3.5.0,M10.5.0

which is strange since NFT seems to be in Australia.

In your TZ setting, the UTC offset and DST rules are specified
explicitly, so it does not matter what names are given to the
timezones for most time calculations.

[sable]

gmtime(-1) worked fine :/

>>> import time
>>> time.gmtime(-1)
time.struct_time(tm_year=1969, tm_mon=12, tm_mday=31, tm_hour=23, tm_min=59, tm_sec=59, tm_wday=2, tm_yday=365, tm_isdst=0)

[vstinner]

strftime_aix.patch is a little bit too strict: it looks like strftime() supports large year values (year > 9999). We may only raise an error if the year is smaller than 1.