Issue383

Title KeyError: 'six' in pkg_resources
Priority bug Status resolved
Superseder Nosy List kayhayen, tiangong
Assigned To Keywords standalone

Created on 2017-10-25.17:44:30 by tiangong, last changed by kayhayen.

Files
File name Uploaded Type Edit Remove
module.six.c tiangong, 2018-04-15.13:20:11 text/x-csrc
Messages
msg2382 (view) Author: tiangong Date: 2018-04-18.15:28:16
There was some version mismatch (my fault). But now it does compile. Thank you
very much for your time. I really appreciate your efforts.
msg2379 (view) Author: kayhayen Date: 2018-04-18.09:58:49
So I made a diff of the file created by me and you, and it shows:

- * created by Nuitka version 0.5.29.4
+ * created by Nuitka version 0.5.28.1

Also the --debug issue with "array" seems to have been resolved in the mean 
time, not reproducible for me here.

Please use 0.5.29.4 and not 0.5.28 and the issue will be gone.

Yours,
Kay
msg2378 (view) Author: tiangong Date: 2018-04-15.13:20:11
Using the --debug flag I get the following error on compilation:

    Nuitka:INFO:Total memory usage before running scons: 314.57 MB (329846784
bytes):
    scons: *** example.pyw.build\module.arrow.arrow.o] Error 1
    b"example.pyw.build\\module.arrow.arrow.c: In function
    'PyInit_arrow$arrow':\nexample.pyw.build\\module.arrow.arrow.c:37509:11:
    error: unused variable 'locals_dict_2' [-Werror=unused-variable]\n PyObject
    *locals_dict_2 = PyDict_New();\n           ^~~~~~~~~~~~~\ncc1.exe: all 
    warnings being treated as errors\n"

Please see the attached "module.six.c".
msg2376 (view) Author: kayhayen Date: 2018-04-14.11:01:25
I was doing this on 3.6.4, I really doubted it makes a difference, but I updated, and it didn't change. I think we 
need to go deeper.

Your traces suggest that this code has an issue:

    if ( isVerbose() )
    {
        PySys_WriteStderr( "Loaded %s\n", entry->name );
    }

    return LOOKUP_SUBSCRIPT( PyImport_GetModuleDict(), module_name );

It really shouldn't happen, but maybe that lookup is the one is giving the key error. I am going to add an assertion 
there, so it is knowable. Can you attach the "module.six.cpp" you get from your ".build" directory for me?

Also please run it with --debug just to be sure nothing strange is going on.

Yours,
Kay
msg2375 (view) Author: tiangong Date: 2018-04-10.14:39:24
I tried it again with Python 3.6.5 (amd64) on Windows 7 with Nuitka 0.5.30rc5,
but still get "KeyError: 'six'".
msg2374 (view) Author: kayhayen Date: 2018-04-10.11:22:10
So I cannot reproduce this on Windows as well. The reason is that for 0.5.29 
this got fixed, possibly even in a hotfix on the .28 series, I recall doing 
that.

Anyway, seems to have been fixed with 0.5.29 at least. So this is not the 
regression of this fix, but its absence with different symptoms. My bad.

Putting to resolved.
msg2373 (view) Author: kayhayen Date: 2018-04-09.22:48:41
My bad, seems my self compiled Python 3.6 didn't have six installed. Once 
installed it works the same. I will retry on Windows.
msg2372 (view) Author: kayhayen Date: 2018-04-09.22:02:40
Hello,

sorry for being stupid, but seems I oversaw your reproducer, and that should be good enough:

~/repos/Py2C$ python3.6 ./bin/nuitka-run --stand Mini.py 
Nuitka:WARNING:Mini.py:2: Cannot find 'six' as absolute import.
Traceback (most recent call last):
  File "/data/home/hayen/repos/Py2C/Mini.dist/Mini.py", line 2, in <module>
ModuleNotFoundError: No module named 'six'

So this is with Python 3.6 self compiled on a Debian stable box. It doesn't do this with the Python 3.5 that is part of Debian stable. I am assuming, 
there are things going on with "_vendor" for Debian vs. Vanilla Python.

Notice the more concise error, and the warning from Nuitka, seems much nicer. But the same issue remains.

Yours,
Kay
msg2370 (view) Author: tiangong Date: 2018-04-04.16:47:26
The executable compiled with --python-flag=-v shows the following error:

[...]
Loading six
Loaded six
Traceback (most recent call last):
[...]
KeyError: 'six'
[...]

So, it's "six" again ...
msg2345 (view) Author: kayhayen Date: 2018-03-25.16:08:59
Can you compile with --python-flag=-v and then post what it says last, max 10 
lines. This is strange.

Yours,
Kay
msg2334 (view) Author: tiangong Date: 2018-03-20.13:51:44
The Python project builds with Nuitka 0.5.29rc5, but the executable won’t start.
No error message appears.
msg2316 (view) Author: kayhayen Date: 2018-03-13.19:29:52
Anyway, I think this got addressed in another issue. The pkg_resources.extern 
injects meta path based loaders that then do mean trick. Nuitka now sees through 
those and should work.
msg2315 (view) Author: kayhayen Date: 2018-03-13.19:28:49
d) Can you can your Python directory there for all the six.py and list 
them.
Pardon?

I meant "Can you scan..."

Yours,
Kay
msg2280 (view) Author: tiangong Date: 2017-10-26.16:00:57
Hi Kay,

> a) This is import "six" that hbmqtt\plugins\manager.py is doing there.
I can't confirm this for sure, but it's likely.

> b) Your main program name ends in ".pyw" as opposed to ".py". I have 
seen it rarely used, 
> but it seems to need handling as well. That is only beauty issue though.
Yes, it ends with ".pyw", but the error occurs on a file with ending ".py" 
as well.

> c) The same happens on your install if you do "import 
pkg_resources.extern" and "import 
> six" in a simple file as well.
I compiled the following programme:

    import pkg_resources.extern
    import six

    print('test')

Nuitka shows the warning:

Nuitka:WARNING:Unresolved '__import__' call at 'C:\Program 
Files\Python36\Lib\site-packages\six.py:82' may require use of '--recurse-
directory'.

Running the compiled programme "test.exe" fails with "KeyError: 'six'".

d) Can you can your Python directory there for all the six.py and list 
them.
Pardon?

Best regards,
Philipp
msg2275 (view) Author: kayhayen Date: 2017-10-26.10:31:17
Hello there,

can you confirm the following assumptions:

a) This is import "six" that hbmqtt\plugins\manager.py is doing there.
b) Your main program name ends in ".pyw" as opposed to ".py". I have seen it rarely used, 
but it seems to need handling as well. That is only beauty issue though.
c) The same happens on your install if you do "import pkg_resources.extern" and "import 
six" in a simple file as well.
d) Can you can your Python directory there for all the six.py and list them.

In terms of workaround, this ought to be a regression of a recent change, where hidden 
dependencies of "pkg_resources.extern" were added, that should make it able to load 
embedded copies as opposed to external ones, as a fallback. But maybe the "spec" Nuitka 
provides for newer Python then is surprising to the module importer, as it might well say 
"pkg_resources.external.six" where "six" is expected or the other way around. I will need a 
reproducer that I hopefully get from your responses.

Yours,
Kay
msg2274 (view) Author: tiangong Date: 2017-10-25.17:44:30
The following error occurs on Windows 7 (x86-64) with Python 3.6.1 and Nuitka 0.5.28. Trying to run the exe file compiled with Nuitka leads 
to:

  [...]
  File "C:\Users\User\Python\<project>\dist\<project>.pyw.dist\hbmqtt\plugins\manager.py", line 7, in <module hbmqtt.plugins.manager>
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 646, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 616, in _load_backward_compatible
  File "C:\Users\User\Python\<project>\dist\<project>.pyw.dist\pkg_resources\__init__.py", line 51, in <module>
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 646, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 616, in _load_backward_compatible
  File "C:\Users\User\Python\<project>\dist\<project>.pyw.dist\pkg_resources\extern\__init__.py", line 43, in load_module
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 646, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 616, in _load_backward_compatible
KeyError: 'six'

Help is appreciated.
History
Date User Action Args
2018-04-18 15:46:03kayhayensetstatus: chatting -> resolved
2018-04-18 15:28:16tiangongsetstatus: resolved -> chatting
messages: + msg2382
2018-04-18 09:58:49kayhayensetstatus: chatting -> resolved
messages: + msg2379
2018-04-15 13:20:12tiangongsetfiles: + module.six.c
messages: + msg2378
2018-04-14 11:01:27kayhayensetnosy: + kayhayen
messages: + msg2376
2018-04-10 14:39:24tiangongsetstatus: resolved -> chatting
messages: + msg2375
2018-04-10 11:22:11kayhayensetstatus: chatting -> resolved
messages: + msg2374
keyword: + standalone
2018-04-09 22:48:41kayhayensetmessages: + msg2373
2018-04-09 22:02:41kayhayensetmessages: + msg2372
2018-04-04 16:47:27tiangongsetmessages: + msg2370
2018-03-25 16:09:00kayhayensetmessages: + msg2345
2018-03-20 13:51:45tiangongsetstatus: resolved -> chatting
messages: + msg2334
2018-03-13 19:29:52kayhayensetstatus: chatting -> resolved
messages: + msg2316
2018-03-13 19:28:49kayhayensetmessages: + msg2315
2017-10-26 16:00:57tiangongsetmessages: + msg2280
2017-10-26 10:31:17kayhayensetstatus: unread -> chatting
messages: + msg2275
2017-10-25 17:44:30tiangongcreate