Regression in logging configuration

[warsaw]

BPO	33802
Nosy	@warsaw, @rhettinger, @vsajip, @ned-deily, @ambv, @miss-islington, @kevans91, @koubaa
PRs	bpo-33802 - Fix regression in logging configuration #7509 bpo-33802: Do not interpolate in ConfigParser while reading defaults #7524 [3.7] bpo-33802: Do not interpolate in ConfigParser while reading defaults (GH-7524) #7529 bpo-41513: Save unnecessary steps in the hypot() calculation #21994 bpo-1635741 port _curses_panel to multi-phase init (PEP 489) #21986 Update all Python Cookbook links after migration to GitHub #22205 bpo-40423: Optimization: use close_range(2) if available #22651
Files	badconfig.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/warsaw'
closed_at = <Date 2018-06-08.16:55:14.312>
created_at = <Date 2018-06-07.23:00:35.278>
labels = ['3.8', 'type-bug', '3.7', 'release-blocker']
title = 'Regression in logging configuration'
updated_at = <Date 2020-10-11.20:42:06.877>
user = 'https://github.com/warsaw'

bugs.python.org fields:

activity = <Date 2020-10-11.20:42:06.877>
actor = 'kevans'
assignee = 'barry'
closed = True
closed_date = <Date 2018-06-08.16:55:14.312>
closer = 'barry'
components = []
creation = <Date 2018-06-07.23:00:35.278>
creator = 'barry'
dependencies = []
files = ['47635']
hgrepos = []
issue_num = 33802
keywords = ['patch', '3.7regression']
message_count = 7.0
messages = ['318980', '318984', '319037', '319046', '319065', '323213', '323291']
nosy_count = 9.0
nosy_names = ['barry', 'rhettinger', 'vinay.sajip', 'ned.deily', 'lukasz.langa', 'Steap', 'miss-islington', 'kevans', 'koubaa']
pr_nums = ['7509', '7524', '7529', '21994', '21986', '22205', '22651']
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue33802'
versions = ['Python 3.7', 'Python 3.8']

[warsaw]

This looks like a serious regression in 3.7. @ned.deily - I'm marking this as a release blocker, but feel free of course to downgrade it.

Run the following as

$ python3.6 badconfig.py
Hey, it works!

$ python3.7 badconfig.py
Traceback (most recent call last):
  File "../badconfig.py", line 77, in <module>
    fileConfig(inifile.name, defaults=GUNICORN_DEFAULTS)
  File "/Users/barry/projects/python/cpython/Lib/logging/config.py", line 65, in fileConfig
    cp = configparser.ConfigParser(defaults)
  File "/Users/barry/projects/python/cpython/Lib/configparser.py", line 639, in __init__
    self._read_defaults(defaults)
  File "/Users/barry/projects/python/cpython/Lib/configparser.py", line 1212, in _read_defaults
    self.read_dict({self.default_section: defaults})
  File "/Users/barry/projects/python/cpython/Lib/configparser.py", line 754, in read_dict
    self.set(section, key, value)
  File "/Users/barry/projects/python/cpython/Lib/configparser.py", line 1200, in set
    super().set(section, option, value)
  File "/Users/barry/projects/python/cpython/Lib/configparser.py", line 895, in set
    value)
  File "/Users/barry/projects/python/cpython/Lib/configparser.py", line 403, in before_set
    "position %d" % (value, tmp_value.find('%')))
ValueError: invalid interpolation syntax in "{'generic': {'format': '%(asctime)s [%(process)d] [%(levelname)s] %(message)s', 'datefmt': '[%Y-%m-%d %H:%M:%S %z]', 'class': 'logging.Formatter'}}" at position 26

I'm still investigating, but wanted to get the bug filed asap.

[warsaw]

I think the regression is caused by the fix for bpo-23835

https://bugs.python.org/issue23835

[ambv]

I'm very sorry for the trouble! And impressed at Barry's quick diagnosis.

[ambv]

New changeset 214f18e by Łukasz Langa in branch 'master':
bpo-33802: Do not interpolate in ConfigParser while reading defaults (GH-7524)
214f18e

[miss-islington]

New changeset f44203d by Miss Islington (bot) in branch '3.7':
bpo-33802: Do not interpolate in ConfigParser while reading defaults (GH-7524)
f44203d

[Steap]

It seems like this regression has not completely been fixed: there are still issues with "None":

$ python3.6 -c 'import configparser; configparser.ConfigParser(defaults={"a": None})'

$ python3.7 -c 'import configparser; configparser.ConfigParser(defaults={"a": 1})'

$ python3.7 -c 'import configparser; configparser.ConfigParser(defaults={"a": None})'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.7/configparser.py", line 638, in __init__
    self._read_defaults(defaults)
  File "/usr/lib/python3.7/configparser.py", line 1216, in _read_defaults
    self.read_dict({self.default_section: defaults})
  File "/usr/lib/python3.7/configparser.py", line 753, in read_dict
    self.set(section, key, value)
  File "/usr/lib/python3.7/configparser.py", line 1197, in set
    self._validate_value_types(option=option, value=value)
  File "/usr/lib/python3.7/configparser.py", line 1182, in _validate_value_types
    raise TypeError("option values must be strings")
TypeError: option values must be strings

Should "None" not be used, or should this bug be reopened?

[ambv]

None is an invalid value in the configparser. It only accepts strings. See:

>>> cp = ConfigParser()
>>> cp['asd'] = {'a': None}
Traceback (most recent call last):
...
TypeError: option values must be strings

The DEFAULT section was an omission which is now fixed. You can use a RawConfigParser if you want to put invalid types as option values:

>> rcp = RawConfigParser()
>> rcp['asd'] = {'a': None}
>>