Issue398

Title Correct way to package large application using Nuitka
Priority bug Status resolved
Superseder Nosy List kayhayen, masq
Assigned To Keywords

Created on 2018-04-22.12:07:48 by masq, last changed by kayhayen.

Messages
msg2536 (view) Author: kayhayen Date: 2018-09-26.07:39:48
Lack of feedback and moving to Github.
msg2388 (view) Author: kayhayen Date: 2018-04-25.04:19:12
I am not so sure right now, but --debugger only works for executables. However, 
there won't be an issue with you running gdb yourself on python -c "import 
mystuff" and typing "where" command.
msg2387 (view) Author: masq Date: 2018-04-24.21:04:12
Thanks, I'll try using those flags.  Its rather hard to reproduce the issue in 
a simple way because it involves a relatively large code base with a few 
packages and I'm trying to compile whole packages.  Some modules inside the 
package work while other cause seg faults.  So I will start working through it 
systematically using the debugging flags and will report back once I've been 
able to pinpoint an issue.
msg2385 (view) Author: kayhayen Date: 2018-04-24.16:36:02
Also, --lto really only changes what gcc does for code generation. It shifts 
the bug around, no more. It's fine if it reproduces with --lto however.

You can also use --debugger, which will throw it into a gdb.

Find the frame line number setting where the assertion triggers, and you can 
be relatively close to what source lines it is.

Thanks in advance,
Kay
msg2384 (view) Author: kayhayen Date: 2018-04-24.16:33:49
Please use --debug during compilation, it should lead to assertions failing, 
if so, this pinpoints the issue.

Then also consider --python-debug if that doesn't find anything. This enables 
more assertions for the object allocation.

Fortunately corruption issues are rare, and if you have a reproducer, that is 
really great.

Yours,
Kay
msg2383 (view) Author: masq Date: 2018-04-22.12:07:48
I am trying to find a strategy to compile a large application that consists 
of several of my own packages (by package here I mean a directory with a lot 
of python modules and subdirs, not a real pypi package).  My first attempt 
was to compile the packages one at a time using:

nuitka --module <module> --recurse-directory=<pkg-directory> --output-
dir=/opt/pybuild/  --lto

I added the '--lto' flag because because of issue 238 and the fact that these 
modules all import each other.  Ideally I would simply like a bunch of .so 
files (one for each package) that can be used in other modules (including 
each other).

Unfortunately, I'm seeing lots of problems:

- When compiling the package I get a WARNING that a submodule in the 
directory specified by resurce-directory is not being compiled.  Even so I'm 
sometimes able to access that module?

- I'm getting segfaults for lots operations. Sometimes this happens when I 
import a module, other times its when I e.g. instantiate an object in the 
module.  Unfortunately I have no idea how to figure out what's generating the 
seg fault.  Some modules inside the packages work but I haven't found a 
consistent pattern in the segfaults (though I suspect modules with less 
dependencies on other modules are behaving better).

I suspect I am doing something wrong so rather than log each individual issue 
I was wondering if someone could suggest the right argument to compile (just 
my own) modules so that they can import each other and work within the same 
application?

This is all being done inside a docker container running ubuntu 16.04 with 
python 2.7 and nuitka 0.5.29.4.  I am happy to provide more details but I 
suspect, given the extent of the difficulties, that I'm just donig this 
wrong.  I would appreciate any suggestions on how ot do this properly or at 
least debug the seg faults.
History
Date User Action Args
2018-09-26 07:39:48kayhayensetstatus: chatting -> resolved
messages: + msg2536
2018-04-25 04:19:12kayhayensetnosy: + kayhayen
messages: + msg2388
2018-04-24 21:04:12masqsetmessages: + msg2387
2018-04-24 16:36:02kayhayensetmessages: + msg2385
2018-04-24 16:33:49kayhayensetstatus: unread -> chatting
messages: + msg2384
2018-04-22 12:07:48masqcreate