gdb support fails with "Invalid cast."

[tilsche]

BPO	26799
Nosy	@vstinner
Files	gdb-python-invalid-cast.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2016-04-20.16:28:59.542>
created_at = <Date 2016-04-18.21:10:40.243>
labels = ['type-crash']
title = 'gdb support fails with "Invalid cast."'
updated_at = <Date 2016-04-21.09:00:28.246>
user = 'https://bugs.python.org/tilsche'

bugs.python.org fields:

activity = <Date 2016-04-21.09:00:28.246>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2016-04-20.16:28:59.542>
closer = 'vstinner'
components = ['Demos and Tools']
creation = <Date 2016-04-18.21:10:40.243>
creator = 'tilsche'
dependencies = []
files = ['42538']
hgrepos = []
issue_num = 26799
keywords = ['patch']
message_count = 10.0
messages = ['263690', '263747', '263752', '263758', '263845', '263847', '263848', '263849', '263890', '263901']
nosy_count = 3.0
nosy_names = ['vstinner', 'python-dev', 'tilsche']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue26799'
versions = ['Python 3.6']

[tilsche]

Trying to use any kind of python gdb integration results in the following error:

(gdb) py-bt
Traceback (most recent call first):
Python Exception <class 'gdb.error'> Invalid cast.: 
Error occurred in Python command: Invalid cast.

I have tracked it down to the _type_... globals, and I am able to fix it with the following commands:

(gdb) pi
>>> # Look up the gdb.Type for some standard types:
... _type_char_ptr = gdb.lookup_type('char').pointer() # char*
>>> _type_unsigned_char_ptr = gdb.lookup_type('unsigned char').pointer() # unsigned char*
>>> _type_void_ptr = gdb.lookup_type('void').pointer() # void*
>>> _type_unsigned_short_ptr = gdb.lookup_type('unsigned short').pointer()
>>> _type_unsigned_int_ptr = gdb.lookup_type('unsigned int').pointer()

After this, it works correctly. I was able to workaround it by making a fix_globals that resets the globals on each gdb.Command. I do not understand why the originally initialized types are not working properly. It feels like gdb-inception trying to debug python within a gdb that debugs cpython while executing python code.

I have tried this using hg/default cpython (--with-pydebug --without-pymalloc --with-valgrind --enable-shared)

1. System install of gdb 7.11, linked against system libpython 3.5.1.
2. Custom install of gdb 7.11.50.20160411-git, the debug cpython I am trying to debug

[vstinner]

I tried Python 3.6 with "./configure --with-pydebug --without-pymalloc --with-valgrind". The py-bt commands works as expect in gdb.

Can you try to modify python-gdb.py to get a traceback of your Invalid cast error?

I am able to fix it with the following commands:
(...) _type_char_ptr = gdb.lookup_type('char').pointer() # char*

These defines are already in Tools/gdb/libpython.py (python-gdb.py is just a copy). I don't understand why you have to redeclare these variables again.

[tilsche]

I have done a bit more digging, turns out it is actually no problem at all to debug python in gdb with gdb with python support (at least using a fixed python-gdb-py).

Turns out the type->length of the the globally initialized ptr types is wrong: It is 4 instead of 8, causing the cast to fail. I suspect the initialization is done before the executable is loaded and gdb is using some default. To verify, I have put two prints in the global initialization and in a command invocation:

GOBAL INITIALIZATION: gdb.lookup_type('char').pointer().sizeof == 4
COMMAND INVOKE: gdb.lookup_type('char').pointer().sizeof == 8

I guess to be fully portable those types need to be looked up at least whenever gdb changes it's binary, but I have no idea if there is a hook for that. But it seems reasonable to just replace those globals with on-demand lookup functions or properties.

If you are interested in the actual python/c stack traces for the error:

Thread 1 "gdb" hit Breakpoint 1, value_cast (type=type@entry=0x2ef91e0, arg2=arg2@entry=0x32b13f0) at ../../gdb/valops.c:571
571	      error (_("Invalid cast."));
(gdb) py-bt
Traceback (most recent call first):
  <built-in method cast of gdb.Value object at remote 0x3260de0>
  File "[...]/python-gdb.py", line 1151, in proxyval
    field_str = field_str.cast(_type_unsigned_char_ptr)
  File "[...]/python-gdb.py", line 945, in print_traceback
    % (self.co_filename.proxyval(visited),
  File "[...]/python-gdb.py", line 1578, in print_traceback
    pyop.print_traceback()
  File "[...]/python-gdb.py", line 1761, in invoke
    frame.print_traceback()
(gdb) bt
#0  value_cast (type=type@entry=0x2ef91e0, arg2=arg2@entry=0x32b13f0) at ../../gdb/valops.c:571
#1  0x000000000052261f in valpy_do_cast (self=<gdb.Value at remote 0x3260de0>, args=<optimized out>, op=UNOP_CAST) at ../../gdb/python/py-value.c:525
#2  0x00007fc7ce2141de in PyCFunction_Call (func=func@entry=<built-in method cast of gdb.Value object at remote 0x3260de0>, args=args@entry=(<gdb.Type at remote 0x2ed0fb0>,), kwds=kwds@entry=0x0)
    at ../../Python-3.5.1/Objects/methodobject.c:109
#3  0x00007fc7ce2c8887 in call_function (pp_stack=pp_stack@entry=0x7ffffc81cec8, oparg=oparg@entry=1) at ../../Python-3.5.1/Python/ceval.c:4655
#4  0x00007fc7ce2c57ac in PyEval_EvalFrameEx (

[vstinner]

Turns out the type->length of the the globally initialized ptr types is wrong: It is 4 instead of 8, causing the cast to fail. I suspect the initialization is done before the executable is loaded and gdb is using some default.

Hum. I see two options:

• try late initialization: initialize types at the first usage
• remove variables: always lookup types at runtime

Would you like to try to implement these options?

[tilsche]

The second option seems like the safest choice, attached is a patch that addresses just that.

[python-dev]

New changeset e1c6f8895fd8 by Victor Stinner in branch '3.5':
python-gdb.py: get C types at runtime
https://hg.python.org/cpython/rev/e1c6f8895fd8

New changeset 6425728d8dc6 by Victor Stinner in branch 'default':
Merge 3.5: Issue bpo-26799
https://hg.python.org/cpython/rev/6425728d8dc6

[python-dev]

New changeset e4561aad29e6 by Victor Stinner in branch '2.7':
Fix python-gdb.py: get C types on demand
https://hg.python.org/cpython/rev/e4561aad29e6

New changeset 952c89a10be6 by Victor Stinner in branch '3.5':
Issue bpo-26799: Fix typo in Misc/NEWS
https://hg.python.org/cpython/rev/952c89a10be6

[vstinner]

The second option seems like the safest choice, attached is a patch that addresses just that.

Thanks, I applied your patch (your a bugfix, you forgot one "return" ;-)).

I also inlined _type_void_ptr() into _sizeof_void_p(), since it was only called there.

Thanks for your bug report and your patch.

Can you please sign the PSF Contributor Agreement? https://www.python.org/psf/contrib/contrib-form/

It's required when you contribute to Python. In short, it warns you that Python is free software ;-)

[tilsche]

Thank you for the quick integration and fixing the return. I have signed the electronic form yesterday.

[vstinner]

Great!