Use time.steady() to implement timeout

[tasslehoff]

BPO	14222
Nosy	@rhettinger, @pitrou, @vstinner, @anacrolix
Files	queue_monotonic.patch threading_monotonic.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/rhettinger'
closed_at = <Date 2012-04-08.01:06:21.786>
created_at = <Date 2012-03-07.19:38:39.928>
labels = ['type-bug', 'library']
title = 'Use time.steady() to implement timeout'
updated_at = <Date 2014-02-25.22:26:22.207>
user = 'https://bugs.python.org/tasslehoff'

bugs.python.org fields:

activity = <Date 2014-02-25.22:26:22.207>
actor = 'vstinner'
assignee = 'rhettinger'
closed = True
closed_date = <Date 2012-04-08.01:06:21.786>
closer = 'rhettinger'
components = ['Library (Lib)']
creation = <Date 2012-03-07.19:38:39.928>
creator = 'tasslehoff'
dependencies = []
files = ['24834', '24835']
hgrepos = []
issue_num = 14222
keywords = ['patch']
message_count = 19.0
messages = ['155106', '155129', '155139', '155701', '155703', '155770', '155828', '155833', '156330', '157521', '157522', '157532', '157564', '157565', '157766', '157767', '157782', '212225', '212227']
nosy_count = 8.0
nosy_names = ['rhettinger', 'pitrou', 'vstinner', 'anacrolix', 'neologix', 'aljungberg', 'python-dev', 'tasslehoff']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue14222'
versions = ['Python 3.3']

[tasslehoff]

Queue.get([block[, timeout]]) uses time.time() for calculations of the timeout. If someone changes the system time this breaks. If a timeout of 1 second is set and someone (on a unix system) uses date to set the time 1 hour back, Queue.get will use 1 hour and 1 second to time out.

I'm guessing the fix is just to use another library function to measure time, but I don't know if one exists that works on all platforms.

[pitrou]

Well, in 3.3 we could use clock_gettime(CLOCK_MONOTONIC) where available.
That said, this is not specific to Queue.get() and will probably happen with many similar functions taking a timeout parameter.

[vstinner]

Well, in 3.3 we could use clock_gettime(CLOCK_MONOTONIC) where available.

It's better to use time.monotonic().

That said, this is not specific to Queue.get() and will probably happen with many similar functions taking a timeout parameter.

Yep, it may be used for lock.acquire(timeout=timeout) for example. It
would help to have a portable monotonic clock API in C.

[vstinner]

queue_monotonic.patch: simple patch using time.monotonic(), with a fallback to time.time() if monotonic failed or is not available.

[vstinner]

threading_monotonic.patch: similar patch for the threading module.

Drawback of these patchs: they now call time.monotonic() when the module is loaded (another useless syscall?).

[neologix]

Random thoughts:

• as noted by Antoine, it probably affects a lot of code throughout
  the standard library
• sem_timedwait() (used by lock.acquire() on POSIX) is affected (the
  implementation using a condition variable could use
  pthread_condattr_setclock(), see issue bpo-12822)
• there's a side effect to that change: on Linux, when the system is
  suspended, CLOCK_MONOTONIC stops: so on resume, you'd have to wait for
  the complete timeout to expire, whereas with time.time() you'll break
  out early

That being said, it's probably a good idea.

As for the patches, I don't really like the idea of having to use this
idiom everywhere:

try:
from time import monotonic as _time
_time()
except (ImportError, OSError):
from time import _time

[python-dev]

New changeset d97ae725814c by Victor Stinner in branch 'default':
Issue bpo-14222: Use the new time.steady() function instead of time.time() for
http://hg.python.org/cpython/rev/d97ae725814c

[rhettinger]

Yo, I had marked this as something I was going to review once the python-dev discussion to complete. FWIW, I'm the maintainer of the Queue module.

[vstinner]

Yo, I had marked this as something I was going to review once
the python-dev discussion to complete. FWIW, I'm the maintainer
of the Queue module.

And so, do you agree with the change?

[vstinner]

Other modules that should use a monotonic clock instead of the system time:

• timeit: default_timer = time.time (but time.clock is still the best clock on Windows for this mmodule)
• subprocess for Popen.communicate() timeoout
• sched
• socket, ssl: timeout. need a C function which uses a monotonic clock if available, or fallback to the system clock
• trace

See also issues bpo-12338 and bpo-14309.

[rhettinger]

I don't think *trace* and *timeit* should be changed.