Issue400

Title Mode between accelerated and standalone for local machine with source removed.
Priority feature Status chatting
Superseder Nosy List aluedo, kayhayen
Assigned To kayhayen Keywords

Created on 2018-05-17.10:24:26 by aluedo, last changed by kayhayen.

Messages
msg2402 (view) Author: kayhayen Date: 2018-05-21.12:36:18
I accept this use case of "--file-reference-choice=runtime". But what does happen is that 
"__path__" is then also going to point into near the binary, where nothing will be living 
for actual loading.

The main program, and package resources, maybe are different that way. To Nuitka of 
course, it's hard to tell, when __file__ will be good to be local, and when it should be 
the original location, because an extension module will only be there.

This is kind of a light form of standalone that you are asking for. It would add even more 
complexity to trying to understand what is going on.

Something, that came to my mind is this:

    # Inform about the extra path, format is NUITKA_PACKAGE_fullname where
    # dots become "_" and should point to the directory where external code
    # to be loaded will live under. Probably should be an absolute path, but
    # we avoid it here.
    os.environ["NUITKA_PACKAGE_package"] = "./package.ext/package"

That kind of environment variable is intended to make for compiled packages find their 
test path. And I think "__path__" indeed may be the only thing holding us back here. If 
you set that environment variable correctly, it will most likely work for you. From my 
check, the code may be there to use for accelerated binaries too. Then we just need a way 
to say that you want "__file__" to be program relative and "__path__" to be original. It 
could be a --path-reference-mode maybe.

Needs more though and experimentation. If you could make a minimal reproducer out of this, 
it clearly would help.

Thanks,
Kay
msg2399 (view) Author: aluedo Date: 2018-05-17.12:56:46
Hello,

We are using this option in order to be able to run the accelerated binary without 
the underlying code. For some reasons, we cannot keep the code on the machine we 
are compiling on, so we need to be able to lookup data from the same directory as 
the accelerated copy is in. 

Since this is on the same machine as we are compiling on, the standalone option 
doesn't make sense, since we already have all the depdencies installed in the 
python path.
msg2398 (view) Author: kayhayen Date: 2018-05-17.11:33:42
Hello there,

I will have to check if using that option makes sense outside of standalone 
mode, because the __file__ will be where the "_counter.so" is searched, and 
then of course it cannot be found. For modules, I think "__file__" is the 
actual load path.

Question to you: Why were you using option.

Yours,
Kay
msg2397 (view) Author: aluedo Date: 2018-05-17.10:24:23
Hello,

When we attempt to compile a program using the "--file-reference-choice=runtime" flag in 
non-standalone mode, we encounter the following error:

Traceback (most recent call last):
  File "/tmp/nuitka_test/test.py", line 1, in <module>
    from Crypto import Random
  File "/tmp/nuitka_test/Crypto/Random/__init__.py", line 29, in <module Crypto.Random>
  File "/tmp/nuitka_test/Crypto/Random/_UserFriendlyRNG.py", line 38, in <module 
Crypto.Random._UserFriendlyRNG>
  File "/tmp/nuitka_test/Crypto/Random/Fortuna/FortunaAccumulator.py", line 39, in <module 
Crypto.Random.Fortuna.FortunaAccumulator>
  File "/tmp/nuitka_test/Crypto/Random/Fortuna/FortunaGenerator.py", line 35, in <module 
Crypto.Random.Fortuna.FortunaGenerator>
  File "/tmp/nuitka_test/Crypto/Util/Counter.py", line 59, in <module Crypto.Util.Counter>
ImportError: cannot import name '_counter'

From what we have been able to find, "_counter" exists (_counter.cpython-36m-i386-linux-
gnu.so), and can be loaded just fine with regular cpython. It also works just fine if we 
exclude the "--file-reference-choice=runtime".

Any help is appreciated.

Setup:

nuitka: 0.5.31rc1
python: 3.6.3
OS: Ubuntu 17.10 32bit
History
Date User Action Args
2018-05-21 12:36:18kayhayensetpriority: bug -> feature
messages: + msg2402
title: ImportError: cannot import name '_counter' -> Mode between accelerated and standalone for local machine with source removed.
2018-05-17 12:56:46aluedosetmessages: + msg2399
2018-05-17 11:33:42kayhayensetstatus: unread -> chatting
messages: + msg2398
2018-05-17 10:24:27aluedocreate