Issue126

Title Standalone built in virtualenv dies with “No such file or directory”
Priority bug Status resolved
Superseder Nosy List brandon, kayhayen
Assigned To kayhayen Keywords standalone

Created on 2014-04-02.01:18:29 by brandon, last changed by kayhayen.

Messages
msg698 (view) Author: kayhayen Date: 2014-07-05.11:05:57
Puting this to resolved, as in the current pre-release the PYTHONPATH is indeed
forwarded when re-executing.
msg579 (view) Author: kayhayen Date: 2014-04-03.06:13:15
The fix is in hotfix Nuitka 0.5.2.1 which has just been released.

Testing virtual-env standalone fails currently, because some tests use "--
python-flag=-S" which does disable "site" module, but in a virtualenv, that 
means a whole lot more, e.g. "logging" module is not reachable without it, and 
so Nuitka fails.

This would only be fixed, if Nuitka (which is then re-executing itself with 
python -S), would retain the sys.path via smuggeling it into PYTHONPATH, so that 
the PYTHONPATH aspect of "site.py" is not lost.

This has so far been seen as only an optimization for standalone, but it's for 
virtualenv actually required, for this option to work.

I hope to include that in a coming development release. For as long as stdlib is 
included as bytecode, it won't matter though.

As for the fix, the data file is now copied to the "dist" directory as well, and 
used from there. A source level patch is applied to "site" module before byte 
compiling it, and that sets "__file__" to the run time directory as found at 
Nuitka start.
msg574 (view) Author: kayhayen Date: 2014-04-02.06:47:06
As a first test, I changed things, so that "__file__" gets set to the binary 
directory. Then copying the file to there helps. Current directory came from the 
fact that "__file__" was "<frozen>", giving an empty dirname.

What's left is merely to copy the file in a hard coded way, and to run the 
tests, I expect to release a hotfix later today.

Before "virtualenv" and "standalone" didn't mix. I suspect afterwards it will 
be.

Yours,
Kay
msg573 (view) Author: kayhayen Date: 2014-04-02.04:39:03
Hello Brandon,

copying data file along, is not yet supported by Nuitka. Once it becomes likely 
for it to optimize constructs as this to constants:

os.path.join(os.path.dirname(__file__), 'orig-prefix.txt')

Nuitka could start to pick up such files automatically. In this case, esp. 
because it's a stdlib library thing, a temporary workaround will be to detect 
virtualenv, and copy the file to the "dist" directory.

Users rightfully will expect to see this happen.

Currently Nuitka is a bit in a bind though. The standard library is now included 
as bytecode (i.e. invisible/untouchable to Nuitka), and the Cish branch intended 
to solve the performance problem of compilation, is not yet finished (although I 
am running large parts of the CPython test suite already).

So (hard coded) copying files is all we can do for now. I will check out how 
that fares.

It may e.g. well be that "__file__" attribute of included bytecode is the 
original one, and not set to something that suggests the "dist" directory.

Yours,
Kay
msg572 (view) Author: brandon Date: 2014-04-02.01:18:29
Given the following hello.py:

def main():
    print "Hello, world."
if __name__ == '__main__':
    main()

I tried compiling it standalone:

nuitka --standalone hello.py

This was within an activated virtualenv, where I had Nuitka installed.
The result is a binary that dies when the virtualenv's special site.py
is reached:

Traceback (most recent call last):
  File "../scripts/hello.py", line 1, in <module>
  File "/home/brandon/.v/exe-from-python/lib/python2.7/site.py", line 703, in 
<module>
    main()
  File "/home/brandon/.v/exe-from-python/lib/python2.7/site.py", line 670, in 
main
    virtual_install_main_packages()
  File "/home/brandon/.v/exe-from-python/lib/python2.7/site.py", line 553, in 
virtual_install_main_packages
    f = open(os.path.join(os.path.dirname(__file__), 'orig-prefix.txt'))
IOError: [Errno 2] No such file or directory: 'orig-prefix.txt'

I am going to try to figure out how to get Nuitka to ignore the site.py
and compile anyway. And, if I put a file orig-prefix.txt in the current
directory just for fun, then hello.exe executes successfully. :)
History
Date User Action Args
2014-07-05 11:05:57kayhayensetstatus: done-cbb -> resolved
messages: + msg698
2014-04-03 06:13:15kayhayensetstatus: in-progress -> done-cbb
messages: + msg579
2014-04-02 06:47:06kayhayensetmessages: + msg574
2014-04-02 04:39:03kayhayensetstatus: unread -> in-progress
assignedto: kayhayen
messages: + msg573
keyword: + standalone
nosy: + kayhayen
2014-04-02 01:18:29brandoncreate