Issue309

Title Standalone: Nuitka does not include stdlib that has no bytecode files
Priority bug Status chatting
Superseder Nosy List dmilinevskyi
Assigned To Keywords standalone

Created on 2016-05-26.09:51:29 by dmilinevskyi, last changed by kayhayen.

Files
File name Uploaded Type Edit Remove
Parent.zip bbusacker, 2016-06-23.23:10:37 application/x-zip-compressed
Messages
msg2377 (view) Author: kayhayen Date: 2018-04-14.11:14:10
At some point, I changed where the Dependency walker binary is stored, using 
"appdirs" module, I think, whereas before it might have been more manual.

So far dependency walker was been the only solution used  by Nuitka ever for 
Windows.

Yours,
Kay
msg2290 (view) Author: laraaj Date: 2017-11-06.21:12:58
After reverting back to version 0.5.7 here is what i got after running Nuitka to compile a 
python script: 

---------------------------------------------------------------------------
Nuitka will make use of Dependency Walker (http://dependencywalker.com) tool
to analyze the dependencies of Python extension modules. Is it OK to download
and put it in "C:\Users\Administrator\AppData\Local\Nuitka\Nuitka".
No installer needed, cached, one time question.

Proceed and download? [Yes]/No
yes
Nuitka:INFO:Downloading 'http://dependencywalker.com/depends22_x64.zip'
Nuitka:INFO:Extracting to 
'C:\Users\Administrator\AppData\Local\Nuitka\Nuitka\x86_64\depends.exe'

-----------------------------------------------------------------------------------

It seems that in the subsequent versions after 0.5.7 either Nuitka stopped checking for 
dependency walker or switched to some other alternative
msg2065 (view) Author: kayhayen Date: 2016-11-16.06:05:09
Yes, totally an edge case, but yet an interesting difference. I will try and 
look into this later. I think manually deleting one of "decimal.pyc" or similar 
should do the trick I suppose.

In theory, the user might lack the permissions to do the bytecode compilation 
and still should work. I don't see anything in the code insisting on ".pyc", the 
standard library scan checks out for ".py" files it seems. And early import 
detections should print ".pyc" files just as they do ".py" files, and both ought 
to work.

I think for me it's not visible, because I installed CPython with the same user 
than what the compilation is done. Only for Python3 are "sourcefile" annotations 
produced manually from "sys.modules". I wonder why that was necessary, and why 
we wouldn't do it for Python2 as well. That's probably the solution.

However, I recently updated my Python installations, all of them. I think some 
MSI installers I ran had an explicit compile all step, I remember that. Do you 
automate the MSI installs, maybe that doesn't compile?

Anyway, I am going to fix something about that. An extra step, where the 
"__file__" is output and that is used seems reasonable. Might take time though 
for me to get around to that.

Yours,
Kay
msg1941 (view) Author: zpin Date: 2016-07-02.09:19:28
I've found something under Windows that could also be related to this issue. Installing Python under Windows does not compile all 
.py files to .pyc automatically. When running nuitka to compile an application it seemingly ignores uncompiled modules. This can 
be seen during the compilation: "/D_NUITKA_FROZEN=25", after compiling all files (python -m compileall C:\Python27) and re-running 
nuitka: "/D_NUITKA_FROZEN=133".

The application produced during the first run will then throw ImportErrors for modules like "decimal" or "copy_reg" while the one 
produced during the second run will be fine. Even with --explain-imports the output of both runs will be exactly the same (except 
for the NUITKA_FROZEN and NUITKA_MODULE_COUNT values), i.e. the modules seem to be detected fine but they are not included in the 
final exe.

This is probably an edge case because usually the application to be frozen would have run before so these files would be compiled. 
I'm spawning a VM with a fresh Python installation so this caused the issue above for every compilation.
msg1939 (view) Author: bbusacker Date: 2016-06-23.23:10:37
I have tracked this issue down to this changeset I believe:

https://github.com/kayhayen/Nuitka/commit/7131cab4a1ef077f72df9e46323291bd1e9733
e4

I have confirmed it broke between 0.5.7 and 0.5.8.  0.5.7 works while 0.5.8 
does not and it has been broken ever since.  

I've include a simple example.  Basically Parent.py does an 
importlib.import_module of the Child Module and then creates the ChildClass 
object and calls the method Hello() on the ChildClass.

When you run nuitka --recurse-all --standalone Parent.py you will see that the 
resulting executable doesn't include the Child.pys code as c code but instead 
you will need to copy the Child.py file into the .dist folder to get the 
resulting executable to run properly.  Would love to see this fixed.  We'll 
hold back on 0.5.7 until it is.  Thanks.
msg1934 (view) Author: kayhayen Date: 2016-05-26.13:12:28
A minimal reproducer would be nice.

Yours,
Kay
msg1932 (view) Author: dmilinevskyi Date: 2016-05-26.09:51:29
Nuitka 0.5.21.2 does not recurse into the module neither with --recurse-
directory= nor with --recurse-all options while building a module.
Note, no issue with 0.5.3.3. If you want I can bisect the releases if it helps.
History
Date User Action Args
2018-04-14 11:14:10kayhayensetmessages: + msg2377
2017-11-06 21:12:59laraajsetmessages: + msg2290
2016-11-16 06:05:55kayhayensetkeyword: + standalone
title: Nuitka does not recurse -> Standalone: Nuitka does not include stdlib that has no bytecode files
2016-11-16 06:05:09kayhayensetmessages: + msg2065
2016-07-02 09:19:28zpinsetmessages: + msg1941
2016-06-23 23:10:37bbusackersetfiles: + Parent.zip
messages: + msg1939
2016-05-26 13:12:28kayhayensetstatus: unread -> chatting
messages: + msg1934
2016-05-26 09:51:29dmilinevskyicreate