test.support has way too many imports

[vstinner]

BPO	40275
Nosy	@terryjreedy, @vstinner, @pablogsal, @miss-islington, @shihai1991
PRs	bpo-40275: Add asyncioutils in test.support #19592 [WIP] bpo-40275: Add loggingutils in test.support #19599 bpo-40275: Avoid importing asyncio in test.support #19600 bpo-40275: Avoid importing logging in test.support #19601 bpo-40275: Avoid importing socket in test.support #19603 bpo-40275: Move transient_internet from test.support to socket_helper #19711 bpo-40275: Move requires_hashdigest from test.support to hashlib_helper #19716 bpo-40275: Remove duplicate import of locale module #19761 bpo-40275: Fix name error in support.socket_helper #19825 bpo-40275: import locale lazily in gettext #19905 bpo-40275: lazy import modules in test.support #20128 bpo-40275: lazy import module in test.supports' decorators #20131 bpo-40275: reduce importing module nums by lazy import in libregrtest #20207 bpo-40275: Adding a threading_helper in test.support #20263 [WIP] bpo-40275: add filesystem_helper in test.support #20459 bpo-40275: test.support.check_impl_detail() uses sys.implementation #20468 bpo-40275: test.support imports subprocess lazily #20471 bpo-40275: Fix a calling bug in threading_helper #20479 bpo-40275: remove TESTFN_ENCODING in test.support #20482 bpo-40275: Fix test.support.threading_helper #20488 bpo-40275: Adding filesystem_helper submodule in test.support #20625 bpo-40275: Adding filesystem_helper submodule in test.support #20689 bpo-40275: Add os_helper submodule in test.support #20732 bpo-40275: Add os_helper submodule in test.support #20765 bpo-40275: Add import_helper submodule in test.support #20794 bpo-40275: Add warnings_helper submodule in test.support #20797 bpo-40275: test.supports imports lazily fnmatch, glob, struct #20810 bpo-40275: Move TransientResource to test_urllib2net #20812 bpo-40275: Use new test.support helper submodules in tests #20824 bpo-40275: Use test helpers to replace test.support [part 2] #20849 bpo-40275: Remove redundant import in libregrtest #20957 bpo-40275: Use new test.support helper submodules in tests #21151 bpo-40275: Remove all new modules imported by tests #21167 bpo-40275: Use new test.support helper submodules in tests #21169 bpo-40275: Use new test.support helper submodules in tests #21219 bpo-40275: Use new test.support helper submodules in tests #21314 bpo-40275: Use new test.support helper submodules in tests #21315 bpo-40275: Use new test.support helper submodules in tests #21317 bpo-40275: Use new test.support helper submodules in tests #21412 bpo-40275: Use new test.support helper submodules in tests #21448 bpo-40275: Use new test.support helper submodules in tests #21449 bpo-40275: Use new test.support helper submodules in tests #21450 bpo-40275: Use new test.support helper submodules in tests #21451 bpo-40275: Use new test.support helper submodules in tests #21452 bpo-40275: Use new test.support helper submodules in tests #21727 bpo-40275: Use new test.support helper submodules in tests #21743 bpo-40275: Use new test.support helper submodules in tests #21764 bpo-40275: Use new test.support helper submodules in tests #21772 bpo-40275: Remove test helpers import in test.support #21771 bpo-40275: Use new test.support helper submodules in tests #21785 bpo-40275: Fix failed test cases by using test helpers #21811
Files	count_imports.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-09-04.15:37:27.746>
created_at = <Date 2020-04-13.22:24:51.262>
labels = ['tests', '3.10']
title = 'test.support has way too many imports'
updated_at = <Date 2020-09-04.15:37:27.744>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2020-09-04.15:37:27.744>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2020-09-04.15:37:27.746>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2020-04-13.22:24:51.262>
creator = 'vstinner'
dependencies = []
files = ['49197']
hgrepos = []
issue_num = 40275
keywords = ['patch']
message_count = 80.0
messages = ['366337', '366765', '366766', '366767', '366768', '366776', '366780', '366786', '366787', '366789', '366810', '366815', '366823', '367261', '367262', '367265', '367603', '367629', '367818', '368073', '368814', '368816', '368840', '369029', '369030', '369032', '369062', '369214', '369296', '370142', '370146', '370150', '370152', '370154', '370217', '370221', '371188', '371253', '371295', '371296', '371297', '371298', '371299', '371303', '371751', '372339', '372361', '372381', '372690', '372691', '373081', '373082', '373109', '373403', '374738', '374739', '374740', '374828', '374829', '374931', '375002', '375021', '375039', '375042', '375043', '375051', '375059', '375060', '375067', '375068', '375069', '375122', '375124', '375150', '375151', '375269', '375315', '375367', '375368', '376375']
nosy_count = 5.0
nosy_names = ['terry.reedy', 'vstinner', 'pablogsal', 'miss-islington', 'shihai1991']
pr_nums = ['19592', '19599', '19600', '19601', '19603', '19711', '19716', '19761', '19825', '19905', '20128', '20131', '20207', '20263', '20459', '20468', '20471', '20479', '20482', '20488', '20625', '20689', '20732', '20765', '20794', '20797', '20810', '20812', '20824', '20849', '20957', '21151', '21167', '21169', '21219', '21314', '21315', '21317', '21412', '21448', '21449', '21450', '21451', '21452', '21727', '21743', '21764', '21772', '21771', '21785', '21811']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue40275'
versions = ['Python 3.10']

[vstinner]

test_threading.ThreadJoinOnShutdown.test_reinit_tls_after_fork() does crash on AIX: bpo-40068. One of the issue is that logging does crash at fork: bpo-40091. The strange thing is that test_threading doesn't use logging. I'm quite sure that logging is imported through test.support.

"import test.support" imports not less than 171... That's quite "heavy". It includes "heavy" modules like concurrent.futures, asyncio or multiprocessing.

It's maybe time to slice again test.support until submodules to reduce the number of imports to the bare minimum. For example, "import test.support" imports asyncio, whereas it's only used by very few tests. Maybe we should add "test.support.asyncioutils".

$ ./python
Python 3.9.0a5+ (heads/pycore_interp:a1ff2c5cf3, Apr 13 2020, 12:27:13) 
>>> import sys
>>> import test

>> before=set(sys.modules)
>> import test.support
>> after=set(sys.modules)

>>> len(after - before)
171

>>> import pprint
>>> pprint.pprint(sorted(after - before))
['__mp_main__',
 '_asyncio',
 '_bisect',
 '_blake2',
 '_bz2',
 '_collections',
 '_compat_pickle',
 '_compression',
 '_contextvars',
 '_datetime',
 '_elementtree',
 '_functools',
 '_hashlib',
 '_heapq',
 '_lzma',
 '_opcode',
 '_operator',
 '_pickle',
 '_posixsubprocess',
 '_queue',
 '_random',
 '_sha3',
 '_sha512',
 '_socket',
 '_sre',
 '_ssl',
 '_string',
 '_struct',
 '_sysconfigdata_d_linux_x86_64-linux-gnu',
 '_weakrefset',
 'argparse',
 'array',
 'asyncio',
 'asyncio.base_events',
 'asyncio.base_futures',
 'asyncio.base_subprocess',
 'asyncio.base_tasks',
 'asyncio.constants',
 'asyncio.coroutines',
 'asyncio.events',
 'asyncio.exceptions',
 'asyncio.format_helpers',
 'asyncio.futures',
 'asyncio.locks',
 'asyncio.log',
 'asyncio.protocols',
 'asyncio.queues',
 'asyncio.runners',
 'asyncio.selector_events',
 'asyncio.sslproto',
 'asyncio.staggered',
 'asyncio.streams',
 'asyncio.subprocess',
 'asyncio.tasks',
 'asyncio.transports',
 'asyncio.trsock',
 'asyncio.unix_events',
 'base64',
 'binascii',
 'bisect',
 'bz2',
 'collections',
 'collections.abc',
 'concurrent',
 'concurrent.futures',
 'concurrent.futures._base',
 'contextlib',
 'contextvars',
 'copy',
 'copyreg',
 'datetime',
 'difflib',
 'dis',
 'email',
 'email.base64mime',
 'email.charset',
 'email.encoders',
 'email.errors',
 'email.header',
 'email.quoprimime',
 'enum',
 'errno',
 'faulthandler',
 'fnmatch',
 'functools',
 'gc',
 'gettext',
 'glob',
 'grp',
 'gzip',
 'hashlib',
 'heapq',
 'importlib',
 'importlib._bootstrap',
 'importlib._bootstrap_external',
 'importlib.abc',
 'importlib.machinery',
 'importlib.util',
 'inspect',
 'itertools',
 'keyword',
 'linecache',
 'locale',
 'logging',
 'logging.handlers',
 'lzma',
 'math',
 'multiprocessing',
 'multiprocessing.context',
 'multiprocessing.process',
 'multiprocessing.reduction',
 'nntplib',
 'opcode',
 'operator',
 'pickle',
 'platform',
 'pprint',
 'pwd',
 'pyexpat',
 'pyexpat.errors',
 'pyexpat.model',
 'queue',
 'quopri',
 'random',
 're',
 'reprlib',
 'resource',
 'select',
 'selectors',
 'shutil',
 'signal',
 'socket',
 'sre_compile',
 'sre_constants',
 'sre_parse',
 'ssl',
 'string',
 'struct',
 'subprocess',
 'sysconfig',
 'tempfile',
 'test.support',
 'test.support.testresult',
 'threading',
 'token',
 'tokenize',
 'traceback',
 'types',
 'typing',
 'typing.io',
 'typing.re',
 'unittest',
 'unittest.async_case',
 'unittest.case',
 'unittest.loader',
 'unittest.main',
 'unittest.result',
 'unittest.runner',
 'unittest.signals',
 'unittest.suite',
 'unittest.util',
 'urllib',
 'urllib.error',
 'urllib.response',
 'warnings',
 'weakref',
 'xml',
 'xml.etree',
 'xml.etree.ElementPath',
 'xml.etree.ElementTree',
 'zlib']

[shihai1991]

"import test.support" imports not less than 171... That's quite "heavy".

If we split some "heavy" modules to xxxutils, what benefits will we make in fact(more exact module importing behavior)?

[serhiy-storchaka]

+1 for splitting test.support on several submodules!

Some imports can also be lazy.

[serhiy-storchaka]

asyncio is now imported in unittest. Removing direct import from test.support does not help.

[shihai1991]

asyncio is now imported in unittest. Removing direct import from test.support does not help.

Oh, thanks, you are right. Looks like we need check which submodules should be splitted.

[serhiy-storchaka]

logging is also imported in unittest and indirectly in asyncio.

[serhiy-storchaka]

These three PRs combined reduce the number of imported modules from 179 to 105 (not including 24 modules imported by the bare interpreter) and reduce the hot import time from 160 ms to 80 ms.

[shihai1991]

These three PRs combined reduce the number of imported modules from 179 to 105 (not including 24 modules imported by the bare interpreter) and reduce the hot import time from 160 ms to 80 ms.

Good job, serhiy. I tested your patches in my vm, the number of importing module have been reduced. Could you paste your performance benchmark test? I got a no siginificant change :( when I run:
$ ./python -m pyperf timeit --compare-to python3.9 "from test import support" -p 100
python3.9: ..................................................................................................... 820 ns +- 206 ns

python: ..................................................................................................... 809 ns +- 201 ns

Mean +- std dev: [python3.9] 820 ns +- 206 ns -> [python] 809 ns +- 201 ns: 1.01x faster (-1%)
Not significant!

[serhiy-storchaka]

I used -X importtime.

Your benchmark measures the performance of look up in sys.modules.

[shihai1991]

I used -X importtime.
copy that, thanks.

In master branch:
import time: 5011 | 133562 | test.support

After apply serhiy's patches:
import time: 4863 | 66940 | test.support

[vsajip]

What is the practical impact on the time taken for test runs? How much does the total time for a test run reduce as a result of refactoring test.support to minimise imports?