Embedded python thread crashes

[ugodiggi]

BPO	1193099
Nosy	@amauryfa
Superseder	bpo-1193099: Embedded python thread crashes

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2008-02-12.12:54:56.370>
created_at = <Date 2005-04-30.19:03:33.000>
labels = ['OS-windows']
title = 'Embedded python thread crashes'
updated_at = <Date 2020-03-26.23:04:48.316>
user = 'https://bugs.python.org/ugodiggi'

bugs.python.org fields:

activity = <Date 2020-03-26.23:04:48.316>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2008-02-12.12:54:56.370>
closer = 'amaury.forgeotdarc'
components = ['Windows']
creation = <Date 2005-04-30.19:03:33.000>
creator = 'ugodiggi'
dependencies = []
files = []
hgrepos = []
issue_num = 1193099
keywords = []
message_count = 7.0
messages = ['25190', '25191', '25192', '25193', '25194', '25195', '62314']
nosy_count = 3.0
nosy_names = ['nnorwitz', 'amaury.forgeotdarc', 'ugodiggi']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = None
status = 'closed'
superseder = '1193099'
type = None
url = 'https://bugs.python.org/issue1193099'
versions = []

[ugodiggi]

The following code crashes about 1/3 of the times.

My platform is Python 2.4.1 on WXP (I tried the release
version from the msi and the debug version built by me).
I can reproduce the same behavior on another wxp
system, under python 2.4.

The crash happens (in the python thread) while the main
thread is in Py_Finalize.
I traced the crash to _Py_ForgetReference(op) in
object.c at line 1847, where I have op->_ob_prev == NULL.

The open file seems to be the issue, since if I remove
all the references to the file I cannot get the program
to crash.

Cheers and ciao

Ugo

////////////////////////// TestPyThreads.cpp
////////////////////////// 
#include <windows.h> // Sleep
#include "Python.h" 

int main() 
{ 
        PyEval_InitThreads(); 
        Py_Initialize(); 
        PyGILState_STATE main_restore_state =
PyGILState_UNLOCKED; 
        PyGILState_Release(main_restore_state); 

    // start the thread 
    { 
            PyGILState_STATE state =

PyGILState_Ensure();
int trash = PyRun_SimpleString(
"import thread\n"
"import time\n"
"def foo():\n"
" f =
open('pippo.out', 'w', 0)\n"
" i = 0;\n"
" while 1:\n"
" f.write('%d\\n'%i)\n"
" time.sleep(0.01)\n"
" i += 1\n"
"t =
thread.start_new_thread(foo, ())\n"
);
PyGILState_Release(state);
}

    // wait 300 ms 
    Sleep(300); 

        PyGILState_Ensure(); 
        Py_Finalize(); 
        return 0; 
}

[nnorwitz]

Logged In: YES
user_id=33168

I can't reproduce on gentoo linux (amd64) with a debug
build. I played with different values for sleep. BTW, it
would be better if you attached the code as a file, since
formatting is lost.

Can you try to debug this problem?
Can you attach info from the debugger?

[ugodiggi]

Logged In: YES
user_id=1269908

I would guess that the problem is going to be windows
specific?
I tested similar problem on both the debug and release build,
and I did try to get some comment on comp.lang.python
i.e.
http://mail.python.org/pipermail/python-list/2005-
April/278404.html

my comment on the debugging was:

The crash happens while the main thread is in Py_Finalize.
I traced the crash to _Py_ForgetReference(op) in object.c
at line 1847,
where I have op->_ob_prev == NULL.

I did not look at the python code long enough to follow it.
I will try to reproduce tonight and send out the stack - for
whatever reason I'm not finding it any longer.

[ugodiggi]

Logged In: YES
user_id=1269908

Something changed on the computer I'm testing on, and the
crash happens way less often (say 1/10 of the times in debug
mode, and very rarely in Release mode).

The compiler is MSVC7.1 on WXP, using Python 241

This is the stack of the main thread

 	7ffe0304()	
 	ntdll.dll!77f5b5d4() 	
 	kernel32.dll!77e7a683() 	
 	msvcr71d.dll!_close_lk(int fh=0x00000003)  Line 115 + 
0x4a	C
 	msvcr71d.dll!_close(int fh=0x00000003)  Line 60 + 0x9	
C
 	msvcr71d.dll!_fclose_lk(_iobuf * str=0x1027c898)  Line 
127 + 0xc	C
 	msvcr71d.dll!fclose(_iobuf * stream=0x1027c898)  Line 
58 + 0x9	C
>	python24_d.dll!file_dealloc(PyFileObject * 
f=0x00919ec8)  Line 308 + 0xd	C
 	python24_d.dll!_Py_Dealloc(_object * op=0x00919ec8)  
Line 1870 + 0x7	C
 	python24_d.dll!frame_dealloc(_frame * f=0x00972960)  
Line 394 + 0x67	C
 	python24_d.dll!_Py_Dealloc(_object * op=0x00972960)  
Line 1870 + 0x7	C
 	python24_d.dll!PyThreadState_Clear(_ts * 
tstate=0x00892c88)  Line 200 + 0x6c	C
 	python24_d.dll!PyInterpreterState_Clear(_is * 
interp=0x00894f48)  Line 93 + 0x9	C
 	python24_d.dll!Py_Finalize()  Line 401 + 0x9	C
 	TestPyThreads.exe!main()  Line 33 + 0x8	C++
 	TestPyThreads.exe!mainCRTStartup()  Line 259 + 0x19	
C
 	kernel32.dll!77e8141a()

and this is the stack of the other thread (_threadstart)
(the crash is here, where "if (frame->f_exc_type != NULL) "
gets called with frame == NULL)

python24_d.dll!reset_exc_info(_ts * tstate=0x00892c88)
Line 2861 + 0x3 C
python24_d.dll!PyEval_EvalFrame(_frame *
f=0x00972960) Line 2490 + 0x9 C
python24_d.dll!PyEval_EvalCodeEx(PyCodeObject *
co=0x00943ad0, _object * globals=0x008e3230, _object *
locals=0x00000000, _object * * args=0x008c104c, int
argcount=0x00000000, _object * * kws=0x00000000, int
kwcount=0x00000000, _object * * defs=0x00000000, int
defcount=0x00000000, _object * closure=0x00000000) Line
2730 + 0x9 C
python24_d.dll!function_call(_object * func=0x009431f0,
_object * arg=0x008c1038, _object * kw=0x00000000) Line
553 + 0x40 C
python24_d.dll!PyObject_Call(_object *
func=0x009431f0, _object * arg=0x008c1038, _object *
kw=0x00000000) Line 1751 + 0xf C
python24_d.dll!PyEval_CallObjectWithKeywords(_object

• func=0x009431f0, _object * arg=0x008c1038, _object *
  kw=0x00000000) Line 3419 + 0x11 C
  python24_d.dll!t_bootstrap(void * boot_raw=0x008c8148)
  Line 433 + 0x1a C
  python24_d.dll!bootstrap(void * call=0x0012f694) Line
  166 + 0x7 C
  msvcr71d.dll!_threadstart(void * ptd=0x009101a0) Line
  196 + 0xd C
  kernel32.dll!77e7d28e()

as you can see, the crash is also happening at a different
place than what I had observed before, but I cannot find the
stack that I had in that case.

[nnorwitz]

Logged In: YES
user_id=33168

Perhaps this is caused by different runtime DLLs. Here's
what I googled for: site:mail.python.org windows python
runtime dll crash

See http://python.org/sf/1003535

Can you try to research more and see if you can come up with
anything? (I don't run windows, so I can't test anything.)
Do you get this problem with the version built from
python.org or only with the version you built?

[ugodiggi]

Logged In: YES
user_id=1269908

I'm sorry I've been so late in answering.
I don't have the time to investigate on the short term, and I'm
very unfamiliar with python internals.

I did do a fair amount of testing when I originally posted the
item, including running with the 2.4.0 and 2.4.1 downloaded
from python.org and 2.4.1 built by me in release and debug
mode. I also tried on a different windows machine, running XP
and 2.4.?.
All were showing the same problem.

I've followed the google thread and I don't think it's the same
problem.
My best guess is that this is caused by a leaked open file
handle, that does not get closed correctly in the Py_Finalize
call, but I'm not quite sure on how to follow this in the
debugger.

[amauryfa]

This is a duplicate of bpo-1856: while a thread is sleeping,
Py_Finalize() deallocates the thread's frame. During deallocation the
thread wakes up and boom.