can't make IDLEX work with python._pth and python-3.6.0b2

[BigStone]

BPO	28522
Nosy	@terryjreedy, @pfmoore, @tjguk, @serwy, @zware, @eryksun, @zooba
PRs	[Do Not Merge] Convert Misc/NEWS so that it is managed by towncrier #552
Files	Report.wer

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/zooba'
closed_at = <Date 2016-10-28.18:28:03.384>
created_at = <Date 2016-10-24.20:45:13.189>
labels = ['3.7', 'OS-windows', 'type-crash']
title = "can't make IDLEX work with python._pth and python-3.6.0b2"
updated_at = <Date 2017-03-31.16:36:32.396>
user = 'https://bugs.python.org/BigStone'

bugs.python.org fields:

activity = <Date 2017-03-31.16:36:32.396>
actor = 'dstufft'
assignee = 'steve.dower'
closed = True
closed_date = <Date 2016-10-28.18:28:03.384>
closer = 'steve.dower'
components = ['Windows']
creation = <Date 2016-10-24.20:45:13.189>
creator = 'Big Stone'
dependencies = []
files = ['45303']
hgrepos = []
issue_num = 28522
keywords = []
message_count = 29.0
messages = ['279338', '279342', '279354', '279423', '279425', '279431', '279459', '279468', '279487', '279503', '279504', '279515', '279527', '279539', '279551', '279552', '279563', '279571', '279572', '279616', '279617', '279624', '279626', '279690', '279691', '279703', '279871', '279872', '279873']
nosy_count = 10.0
nosy_names = ['terry.reedy', 'paul.moore', 'tim.golden', 'roger.serwy', 'python-dev', 'zach.ware', 'eryksun', 'steve.dower', 'illagrenan', 'Big Stone']
pr_nums = ['552']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue28522'
versions = ['Python 3.6', 'Python 3.7']

[BigStone]

on WinPython-64bit-3.6.0.0Zerorc2.exe, python-3.6.0b2 based, I can't get IDLEX working with "python._pth".

If I put "Lib\site-packages\" in python._pth, python.exe dies.
If I put "#Lib\site-packages\", idlexlib is said "unable to located".

Could it be a python-3.6.0b2 bug, or just a wrong-doing from WinPython ?

"Python._pth"=
.
Lib
import site
DLLs
#Lib/site-packages
#python36.zip

[zooba]

Might have to use a backslash in the path - I don't remember whether I tried to handle forward slashes or not, but I suspect not. Definitely tried it with site-packages though.

[BigStone]

python.exe crashes when I try this python._pth:
.
Lib
import site
DLLs
Lib\site-packages
#python36.zip

all variation I try on Lib\site-packages do fail, when I not commenting # the line...
Nevertheless, jupyter/numpy/bokeh do work with original setting.

I'm lost in thoughts: how adding a line in this file can make Python crash ?

[zooba]

Can you tell me more about the crash? It doesn't cause a crash when I try it with my own install.

[BigStone]

I just see a windows screen poping up with (translated from french)

"Python has stopped to work"

a problem caused this program to stop working correctly. Windows is going to close this program and will inform you if a solution is available.

[zooba]

Check in the Event Log viewer to see if there is an "Application Error" entry for python.exe. Also, if you run python.exe from a command prompt there may be more information displayed in the output.

[BigStone]

possible particularities of my PC vs yours:

• I have no python entry at all in the regex
• I have no py.exe,
• I have no Visual Studio (but the compiler)

with Windows 10, I don't know where is the even viewer.

[zooba]

If you right-click the Start button, Event Viewer is near the top.

[eryksun]

I installed "WinPython-64bit-3.6.0.0Zerorc2.exe" on Windows 10. As you can see below, the included version of IDLEX depends on idlelib implementation details that have changed between 3.5 and 3.6:

    C:\WinPython36\python-3.6.0b2.amd64>.\python
    Python 3.6.0b2 (default, Oct 10 2016, 21:15:32) [MSC v.1900 64 bit (AMD64)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>>  import idlexlib
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "C:\WinPython36\python-3.6.0b2.amd64\Lib\site-packages\idlexlib\__init__.py", line 10, in <module>
        from .idlexMain import version as __version__
      File "C:\WinPython36\python-3.6.0b2.amd64\Lib\site-packages\idlexlib\idlexMain.py", line 46, in <module>
        from idlexlib.extensionManager import extensionManager
      File "C:\WinPython36\python-3.6.0b2.amd64\Lib\site-packages\idlexlib\extensionManager.py", line 60, in <module>
        from idlelib.configHandler import idleConf, IdleConfParser
    ModuleNotFoundError: No module named 'idlelib.configHandler'

The "Unable to located" [sic] error is from the idlex.py script due to the above import error.

I couldn't reproduce the crash due to python._pth. Your Windows application log should provide the DLL (module) and exception code for the crash, but a dump file would be even better. With the error reporting dialog still open, look for the crashed python.exe in the task manager details tab (the working set of the crashed process should be small -- about 100K). Right-click it and select the option to create a dump file. Zip the dump file and upload it here.

[BigStone]

Event Viewer says, when I put "Lub\site-packages" in python._pth:
""""
Nom de l’application défaillante python.exe, version : 3.6.112.1013, horodatage : 0x57fc0593
Nom du module défaillant : ucrtbase.dll, version : 10.0.14393.0, horodatage : 0x578997b5
Code d’exception : 0xc0000409
Décalage d’erreur : 0x000000000006d5b8
ID du processus défaillant : 0x190c
Heure de début de l’application défaillante : 0x01d22fafd622ed09
Chemin d’accès de l’application défaillante : C:\WinPython\basedir36\build\winpython-64bit-3.6.x.0\python-3.6.0b2.amd64\python.exe
Chemin d’accès du module défaillant: C:\WINDOWS\System32\ucrtbase.dll
ID de rapport : ec44b511-6196-48a0-95ec-dc997b4d0302
Nom complet du package défaillant :
ID de l’application relative au package défaillant :

[BigStone]

Thanks Eryk,

So the root cause is that IDLEX is no more compatible with IDLE in python3.6.

==> I can survive this loss... Now, I don't if the "python._pth" crash is a problem, as I can stay with "#Lib\site-packages" for now.

[terryjreedy]

I don't know anything about 'python._pth' and whether there is any bug with respect to that.

As far as IDLE goes, there is no bug and this issue should be closed.

In bpo-24225, before the release of 3.6.0a2, most file names within idlelib were changed to shorter or lowercased names in conformance with PEP-8 and as anticipated by PEP-434. In particular, 'configHandler.py' is now 'config.py'. API changes within and between files are and will be much more disruptive to external users. As Nick Coughlin said in msg266409, 3rd party idlelib users are free to bundle or depend on a frozen copy of a past version.

WinPython should test the third party modules it includes with the python it is releasing. Importing a module is as minimal as it gets. Consider reporting the incompatibility to them.

[eryksun]

You may have uncovered a bug in Python that's causing the invalid parameter handler to be invoked. It would help if you uploaded the zipped dump file for the crashed process.

The status code you're getting (i.e. STATUS_STACK_BUFFER_OVERRUN, 0xC0000409) is used by the __fastfail intrinsic 1. The exception occurred in ucrtbase.dll at offset 0x6d5b8. In a debugger you can see it's in the CRT's invoke_watson function:

0:000> u ucrtbase + 0x6d5b8 - 0x18 L7
ucrtbase!invoke_watson:
00007ffd`8984d5a0 4883ec28        sub     rsp,28h
00007ffd`8984d5a4 b917000000      mov     ecx,17h
00007ffd`8984d5a9 ff1531310400    call    qword ptr
    [ucrtbase!_imp_IsProcessorFeaturePresent (00007ffd`898906e0)]
00007ffd`8984d5af 85c0            test    eax,eax
00007ffd`8984d5b1 7407            je      ucrtbase!invoke_watson+0x1a
                                            (00007ffd`8984d5ba)
00007ffd`8984d5b3 b905000000      mov     ecx,5
00007ffd`8984d5b8 cd29            int     29h

A fast fail executes an int 29h software interrupt, which gets handled by the following system function:

lkd> !idt 0x29
Dumping IDT: fffff801819a8070
29:     fffff8017fd66680 nt!KiRaiseSecurityCheckFailure

In this case the CRT passes a value of 5 (FAST_FAIL_INVALID_ARG) in register rcx (ecx), so it's not a critical security failure.

[zooba]

I suspect there's a .pth file in site-packages that is importing something to trigger the failure. Without a crash dump (or debug build) it's going to be difficult to find it, but it is certainly an unwrapped invalid parameter termination.

To save me some time, where can I get that build of WinPython from?