Add missing dependencies between AST/parser header files

[vstinner]

BPO	35177
Nosy	@vstinner
PRs	bpo-35177: Add dependencies between header files #10361 bpo-35177, Python-ast.h: Fix compiler warning on Yield #10664

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-11-22.17:39:41.969>
created_at = <Date 2018-11-06.14:08:41.266>
labels = ['interpreter-core', '3.8']
title = 'Add missing dependencies between AST/parser header files'
updated_at = <Date 2018-11-22.17:39:41.968>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2018-11-22.17:39:41.968>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2018-11-22.17:39:41.969>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2018-11-06.14:08:41.266>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 35177
keywords = ['patch']
message_count = 6.0
messages = ['329359', '329696', '329699', '329738', '330272', '330273']
nosy_count = 1.0
nosy_names = ['vstinner']
pr_nums = ['10361', '10664']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue35177'
versions = ['Python 3.8']

[vstinner]

Currently, there are *implicit* dependencies between AST/parser header files. For example, ast.h uses node and mod_ty types but don't include node.h nor Python-ast.h. And parsetok.h uses node and grammer but don't include nor grammar.h.

Because of that, C files have to include header files in the "correct order" and need to include header files even if they don't use directly.

At the end, we get something like pythonrun.c:

#include "Python-ast.h"
#include "pycore_state.h"
#include "grammar.h"
#include "node.h"
#include "token.h"
#include "parsetok.h"
#include "errcode.h"
#include "code.h"
#include "symtable.h"
#include "ast.h"
#include "marshal.h"
#include "osdefs.h"
#include <locale.h>

whereas most header files are useless, pythonrun.c still compiles with:

#include "pycore_state.h"
#include "token.h"      /* INDENT in err_input() */
#include "parsetok.h"   /* PyParser_ParseFileObject() */
#include "errcode.h"    /* E_EOF */
#include "symtable.h"   /* PySymtable_BuildObject() */
#include "ast.h"        /* PyAST_FromNodeObject() */
#include "marshal.h"    /* PyMarshal_ReadLongFromFile */
#include <locale.h>

I propose to add explicit dependencies in header files directly, rather than using black magic in C files.

Attached PR fix this issue.

[vstinner]

New changeset 5f2df88 by Victor Stinner in branch 'master':
bpo-35177: Add dependencies between header files (GH-10361)
5f2df88

[vstinner]

This change doesn't try to fix "all dependencies issues", but it fix a few of them :-) See bpo-35197 for another example of funny header issue.

Please open a new issues if you see other header dependencies issues. I close this one since the initial bug has been fixed.

[vstinner]

Oh, the Yield warning made its come back :-( Example:

import.c
c:\program files (x86)\windows kits\10\include\10.0.17134.0\um\winbase.h(102): warning C4005: 'Yield': macro redefinition [C:\projects\cpython\PCbuild\pythoncore.vcxproj]
c:\projects\cpython\include\python-ast.h(549): note: see previous definition of 'Yield'

[vstinner]

New changeset 3bb183d by Victor Stinner in branch 'master':
bpo-35177, Python-ast.h: Fix "Yield" compiler warning (GH-10664)
3bb183d

[vstinner]

Ok, I fixed the "Yield warning" again.