warnings.simplefilter("always") does not make warnings always show up

[exarkun]

BPO	4180
Nosy	@mdickinson, @pitrou, @benjaminp, @florentx, @gerritholl, @berkerpeksag
Files	warn_explicit_replace.patch issue4180-py_warnings.diff issue4180-test_janitorial.diff: janitorial patch for FilterTests issue4180-test_filter_after_default.diff: new tests for filters after default handling issue4180-py_warnings2.diff: Improved python-side handling of filters after default warnings warnings_issue4180.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2014-09-18.00:45:19.259>
created_at = <Date 2008-10-22.21:49:31.888>
labels = ['type-bug', 'library']
title = 'warnings.simplefilter("always") does not make warnings always show up'
updated_at = <Date 2017-04-18.14:36:48.144>
user = 'https://bugs.python.org/exarkun'

bugs.python.org fields:

activity = <Date 2017-04-18.14:36:48.144>
actor = 'Gerrit.Holl'
assignee = 'none'
closed = True
closed_date = <Date 2014-09-18.00:45:19.259>
closer = 'pitrou'
components = ['Library (Lib)']
creation = <Date 2008-10-22.21:49:31.888>
creator = 'exarkun'
dependencies = []
files = ['11866', '17141', '17144', '17145', '17146', '36429']
hgrepos = []
issue_num = 4180
keywords = ['patch', 'needs review']
message_count = 21.0
messages = ['75113', '75117', '75120', '75121', '75122', '75123', '75124', '75131', '89285', '89287', '100525', '100686', '104610', '104617', '104618', '104619', '225618', '226378', '227018', '227019', '291837']
nosy_count = 9.0
nosy_names = ['exarkun', 'tseaver', 'mark.dickinson', 'pitrou', 'benjamin.peterson', 'flox', 'Gerrit.Holl', 'python-dev', 'berker.peksag']
pr_nums = []
priority = 'high'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue4180'
versions = ['Python 3.4', 'Python 3.5']

[exarkun]

If a warning is emitted then a filter with the "always" rule is added
then the is emitted again, it will not be shown the second time:

  exarkun@charm:~$ python
  Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52) 
  [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
  Type "help", "copyright", "credits" or "license" for more information.
  >>> import warnings
  >>> def f():
  ...     warnings.warn("foo")
  ... 
  >>> f()
  /home/exarkun/.pythonstartup.py:2: UserWarning: foo
  >>> warnings.simplefilter("always")
  >>> f()
  >>> 

The "always" filter is documented as "always print matching warnings"
which is contrary to this behavior. Also, the string "always" strongly
implies that it will be shown the second time.

[benjaminp]

If you didn't raise the warning before using the simple filter, this
would have worked. The undesired behavior is because of
__warningsregistry__. It is set the first time the warning is emitted.
When the second warning comes through, the filter isn't even looked at.
I think the best way to fix this is to invalidate __warningsregistry__
when a filter is used. It would probably be best to store warnings data
in a global then instead of on the module, so it is easy to invalidate.

[brettcannon]

Obviously the big problem with that change, Benjamin, is you will be
changing the semantics. And that is a tough thing to change when it
involves the warning machinery itself so emitting a warning about a
warning might get messy.

[benjaminp]

Oh, do we not consider __warningsregistry__ an implementation detail?

[brettcannon]

On Wed, Oct 22, 2008 at 3:39 PM, Benjamin Peterson
<report@bugs.python.org> wrote:

Benjamin Peterson <musiccomposition@gmail.com> added the comment:

Oh, do we not consider __warningsregistry__ an implementation detail?

I guess you could view it that way. I just know I find it extremely
annoying for testing for the exact reasons JP mentions; it's
unexpected if you don't know about it.

If we made __warningsregistry__ about just recording what warnings
have been raised and not include filter information (which I what I
think you are suggesting) then it would still server its purpose,
albeit for probably a small performance penalty. Probably whomever
came up with it over-optimized and just didn't realize that it might
play havoc with testing.

[benjaminp]

A simple solution would be to allow warn_explicit to be overridden. Then
a custom warn_explicit function could simply ignore the registry argument.

[brettcannon]

On Wed, Oct 22, 2008 at 4:11 PM, Benjamin Peterson
<report@bugs.python.org> wrote:

Benjamin Peterson <musiccomposition@gmail.com> added the comment:

A simple solution would be to allow warn_explicit to be overridden. Then
a custom warn_explicit function could simply ignore the registry argument.

You say "simple" from a comprehension standpoint, I say "pain" in
terms of implementing it in C (although most of the hard stuff I
already have worked out for showwarning).

[benjaminp]

Don't worry, Brett; you made it really easy. :) Here's a patch.

[exarkun]

So how about it?

[brettcannon]

I am still on sabbatical so no code review from me.

But from the standpoint of making warn_explicit be overloadable, I'm on
the fence. I don't mind it happening, but it will be just one more thing
we have to support. Since it's Benjamin's code he can make the call to add
the feature.

[mdickinson]

+1 for Benjamin's patch, having just been bitten by this exact problem.

I'm trying to do unit testing, checking both that a piece of code produces a DeprecationWarning and that it gives the correct result, with something like:

    with warnings.catch_warnings():
        warnings.filterwarnings("ignore", category=DeprecationWarning)
        self.assertEqual(my_func(my_args), expected_result)

    with warnings.catch_warnings():
        warnings.filterwarnings("error", category=DeprecationWarning)
        self.assertRaises(DeprecationWarning, my_func, *my_args)

The first call still registers the warning, even though it's ignored, so the second assertRaises fails. Benjamin's patch would seem to provide a way to fix this.

Perhaps I'm missing an obvious better way to do this.

N.B. The above is a too simple version of the real problem: it actually works as intended, for fragile reasons: the "ignore"d warning is registered on __name__, while the "always"d warning ends up being registered on unittest.case, so there's no conflict.

[pitrou]

It sounds like it would be cleaner to apply Brett's idea in msg75122, rather than have the user override (and alternate implementations implement) another obscure hook. We have enough hooks that nobody knows about, IMHO.

[tseaver]

The attached patch fixes the OP's use case on the Python side by re-ordering the tests, such that "always" prevents the short-circuit from firing::

 $ ./python 
 Python 2.6.5+ (release26-maint, Apr 29 2010, 21:24:12) 
 [GCC 4.3.3] on linux2
 Type "help", "copyright", "credits" or "license" for more information.
 >>> import warnings as o_warnings
 >>> import sys
 >>> sys.modules['_warnings'] = 0
 >>> del sys.modules['warnings']
 >>> import warnings as py_warnings
 >>> def f():
 ...     py_warnings.warn('foo')
 ... 
 >>> f()
 __main__:2: UserWarning: foo
 >>> f()
 >>> py_warnings.simplefilter('always')
 >>> f()
 __main__:2: UserWarning: foo
 >>> f()
 __main__:2: UserWarning: foo