Exclude the private API from the stable API

[serhiy-storchaka]

BPO	26900
Nosy	@loewis, @vstinner, @zware, @serhiy-storchaka
Files	private_limited_api.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2016-09-13.06:29:27.779>
created_at = <Date 2016-05-01.10:17:53.238>
labels = ['interpreter-core', 'type-feature']
title = 'Exclude the private API from the stable API'
updated_at = <Date 2016-09-13.06:29:27.778>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2016-09-13.06:29:27.778>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2016-09-13.06:29:27.779>
closer = 'serhiy.storchaka'
components = ['Interpreter Core']
creation = <Date 2016-05-01.10:17:53.238>
creator = 'serhiy.storchaka'
dependencies = []
files = ['42671']
hgrepos = []
issue_num = 26900
keywords = ['patch']
message_count = 6.0
messages = ['264591', '264594', '264613', '275741', '275746', '275747']
nosy_count = 5.0
nosy_names = ['loewis', 'vstinner', 'python-dev', 'zach.ware', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue26900'
versions = ['Python 3.6']

[serhiy-storchaka]

Proposed patch excludes the private API from the stable API. This includes:

_Py_Identifier-related API (other _Py_Identifier-related functions are already excluded):

struct _Py_Identifier;
#define _Py_static_string_init(value)
#define _Py_static_string(varname, value)
#define _Py_IDENTIFIER(varname)
PyAPI_FUNC(PyObject*) _PyUnicode_FromId(_Py_Identifier*);
PyAPI_FUNC(PyObject *) _PyObject_GetAttrId(PyObject *, struct _Py_Identifier *);
PyAPI_FUNC(int) _PyObject_SetAttrId(PyObject *, struct _Py_Identifier *, PyObject *);
PyAPI_FUNC(int) _PyObject_HasAttrId(PyObject *, struct _Py_Identifier *);
PyAPI_FUNC(PyObject *) _PyObject_CallMethodId(PyObject *o, _Py_Identifier *method, const char *format, ...);
PyAPI_FUNC(PyObject *) _PyObject_CallMethodId_SizeT(PyObject *o, _Py_Identifier *name, const char *format, ...);
PyAPI_FUNC(PyObject *) _PyObject_CallMethodIdObjArgs(PyObject *o, struct _Py_Identifier *method, ...);
PyAPI_FUNC(PyObject *) _PyDict_GetItemIdWithError(PyObject *dp, struct _Py_Identifier *key);
PyAPI_FUNC(PyObject *) _PyDict_GetItemId(PyObject *dp, struct _Py_Identifier *key);
PyAPI_FUNC(int) _PyDict_SetItemId(PyObject *dp, struct _Py_Identifier *key, PyObject *item);
PyAPI_FUNC(PyObject *) _PySys_GetObjectId(_Py_Identifier *key);

File helpers that hides OS-specific details and handle EINTR:

PyAPI_FUNC(PyObject *) _Py_device_encoding(int);
PyAPI_FUNC(FILE *) _Py_wfopen(const wchar_t *path, const wchar_t *mode);
PyAPI_FUNC(FILE*) _Py_fopen(const char *pathname, const char *mode);
PyAPI_FUNC(FILE*) _Py_fopen_obj(PyObject *path, const char *mode);
PyAPI_FUNC(Py_ssize_t) _Py_read(int fd, void *buf, size_t count);
PyAPI_FUNC(Py_ssize_t) _Py_write(int fd, const void *buf, size_t count);
PyAPI_FUNC(Py_ssize_t) _Py_write_noraise(int fd, const void *buf, size_t count);
PyAPI_FUNC(int) _Py_wreadlink(const wchar_t *path, wchar_t *buf, size_t bufsiz);
PyAPI_FUNC(wchar_t*) _Py_wrealpath(const wchar_t *path, wchar_t *resolved_path, size_t resolved_path_size);
PyAPI_FUNC(wchar_t*) _Py_wgetcwd(wchar_t *buf, size_t size);

Other OS functions:

PyAPI_FUNC(int) _PyOS_IsMainThread(void);
PyAPI_FUNC(void*) _PyOS_SigintEvent(void);
PyAPI_FUNC(int) _PyOS_GetOpt(int argc, wchar_t **argv, wchar_t *optstring);
PyAPI_FUNC(int) _PyOS_URandom(void *buffer, Py_ssize_t size);

Old helpers kept for binary compatibility of extensions built before 3.2.4:

PyAPI_FUNC(void) _PyTrash_deposit_object(PyObject*);
PyAPI_FUNC(void) _PyTrash_destroy_chain(void);
PyAPI_DATA(int) _PyTrash_delete_nesting;
PyAPI_DATA(PyObject *) _PyTrash_delete_later;

And other functions and data:

PyAPI_DATA(PyTypeObject) _PyNone_Type;
PyAPI_DATA(PyTypeObject) _PyNotImplemented_Type;
PyAPI_DATA(PyTypeObject) _PyMethodWrapper_Type;
PyAPI_DATA(int) _Py_SwappedOp[];

PyAPI_FUNC(Py_ssize_t) _Py_GetAllocatedBlocks(void);
PyAPI_FUNC(PyObject *) _PyObject_GC_Malloc(size_t size);
PyAPI_FUNC(PyObject *) _PyObject_GC_Calloc(size_t size);
PyAPI_FUNC(int) _PyObject_IsAbstract(PyObject *);

PyAPI_FUNC(PyObject*) _Py_strhex(const char* argbuf, const Py_ssize_t arglen);
PyAPI_FUNC(PyObject*) _Py_strhex_bytes(const char* argbuf, const Py_ssize_t arglen);

PyAPI_FUNC(void) _Py_add_one_to_index_F(int nd, Py_ssize_t *index, const Py_ssize_t *shape);
PyAPI_FUNC(void) _Py_add_one_to_index_C(int nd, Py_ssize_t *index, const Py_ssize_t *shape);

PyAPI_DATA(PyTypeObject) _PyNamespace_Type;
PyAPI_FUNC(PyObject *) _PyNamespace_New(PyObject *kwds);

PyAPI_FUNC(int) _PyOS_IsMainThread(void);
PyAPI_FUNC(void*) _PyOS_SigintEvent(void);
PyAPI_FUNC(int) _PyOS_GetOpt(int argc, wchar_t **argv, wchar_t *optstring);
PyAPI_FUNC(int) _PyOS_URandom(void *buffer, Py_ssize_t size);

PyAPI_FUNC(int) _PyState_AddModule(PyObject*, struct PyModuleDef*);
PyAPI_FUNC(PyThreadState *) _PyThreadState_Prealloc(PyInterpreterState *);
PyAPI_FUNC(void) _PyThreadState_Init(PyThreadState *);
PyAPI_FUNC(void) _PyThreadState_DeleteExcept(PyThreadState *tstate);
PyAPI_FUNC(void) _PyGILState_Reinit(void);
PyAPI_FUNC(PyThreadState *) _PyThreadState_UncheckedGet(void);

PyAPI_FUNC(void) _PyUnicode_ClearStaticStrings(void);
PyAPI_FUNC(int) _PyUnicode_EQ(PyObject *, PyObject *);

PyAPI_FUNC(void) _Py_DumpTraceback(int fd, PyThreadState *tstate);
PyAPI_FUNC(const char*) _Py_DumpTracebackThreads(int fd, PyInterpreterState *interp, PyThreadState *current_tstate);

PyAPI_FUNC(void) _PyImportZip_Init(void);

PyAPI_FUNC(PyObject *) _PyLong_GCD(PyObject *, PyObject *);

All these names are not documented (except _PyObject_GC_Calloc accidentally mentioned in What's New in 3.5) and are not used implicitly in macros (except _PyObject_CallMethodId_SizeT used in _PyObject_CallMethodId).

Many of these functions were added after implementing PEP-384 and changed its signature during its lifetime. They are not explicitly exclude from the stable API only by accident.

[serhiy-storchaka]

Added

PyAPI_FUNC(PyObject *) _PyObject_GenericGetAttrWithDict(PyObject *, PyObject *, PyObject *);
PyAPI_FUNC(int) _PyObject_GenericSetAttrWithDict(PyObject *, PyObject *, PyObject *, PyObject *);
PyAPI_FUNC(int) _Py_fstat(int fd, struct _Py_stat_struct *status);
PyAPI_FUNC(void) _PyEval_FiniThreads(void);

[zware]

bpo-23903 is closely related, but I haven't made it back to that one in some time.

[python-dev]

New changeset d00f15af75ea by Serhiy Storchaka in branch 'default':
Issue bpo-26900: Excluded underscored names and other private API from limited API.
https://hg.python.org/cpython/rev/d00f15af75ea

[vstinner]

I didn't follow this discussion, but thanks for the fix Serhiy :-) By
the way, I started a thread on python-dev to discuss a new
organization of the C API for Python 3.7.

[serhiy-storchaka]

Maybe new private functions was added after writing the patch.

And many new public functions was added without guarding by specific Py_LIMITED_API value. This is separate issue.