Code coverage report not entirely accurate

[lrjball]

BPO	40474
Nosy	@brettcannon, @terryjreedy, @lrjball
PRs	bpo-40474: Updated coverage.yml to better report coverage stats #19851

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-05-28.18:52:33.568>
created_at = <Date 2020-05-02.01:11:22.744>
labels = ['type-feature', 'tests', '3.9']
title = 'Code coverage report not entirely accurate'
updated_at = <Date 2020-05-28.18:52:33.567>
user = 'https://github.com/lrjball'

bugs.python.org fields:

activity = <Date 2020-05-28.18:52:33.567>
actor = 'brett.cannon'
assignee = 'none'
closed = True
closed_date = <Date 2020-05-28.18:52:33.568>
closer = 'brett.cannon'
components = ['Tests']
creation = <Date 2020-05-02.01:11:22.744>
creator = 'Lewis Ball'
dependencies = []
files = []
hgrepos = []
issue_num = 40474
keywords = ['patch']
message_count = 5.0
messages = ['367904', '368204', '368460', '368802', '370260']
nosy_count = 3.0
nosy_names = ['brett.cannon', 'terry.reedy', 'Lewis Ball']
pr_nums = ['19851']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue40474'
versions = ['Python 3.9']

[lrjball]

The current reported code coverage stats on codecov.io miss some global statements of modules imported before coveragepy begins. This is documented in the dev guide, and the suggested fix is here: https://devguide.python.org/coverage/#coverage-results-for-modules-imported-early-on but that recommended fix doesn't seem to be included in the CI.

Using the recommended 'hack', the coverage report shows an extra 4k lines are coverage in Lib, increasing coverage percentage by around 3%.

Having a more accurate view of the coverage makes it much easier to see which parts of the code need help with tests, without having to go into every individual report to see if global statements are misreported.

I'll raise a PR for this shortly

[lrjball]

I've updated the travis.yml now to fix this issue and global statements are now showing as covered in the coverage report. This means an extra 4k lines show as covered which weren't previously showing. See the report for the PR here (https://codecov.io/gh/python/cpython/tree/64d521b5d34c25b83d0472608d1eab3a6334bf59/Lib).

[terryjreedy]

Use git to find out who previously edited the file and request reviews from at least a couple. The 2 recommendations on the PR may be based on that, but good to check.

[lrjball]

I don't have triage permissions so I don't think I can request a review. Although @aeros has requested one.

[brettcannon]

New changeset d9c1f19 by lrjball in branch 'master':
bpo-40474: Updated coverage.yml to better report coverage stats (bpo-19851)
d9c1f19