Issue288

Title Module twisted.protocols.tls problem
Priority bug Status resolved
Superseder Nosy List kayhayen, marc.jofre
Assigned To kayhayen Keywords

Created on 2016-02-12.10:33:02 by marc.jofre, last changed by kayhayen.

Files
File name Uploaded Type Edit Remove
autobahnTestNuitka.zip marc.jofre, 2016-02-12.10:33:02 application/octet-stream
twistedTest.py marc.jofre, 2016-02-12.13:35:03 text/plain
twistedTest.py marc.jofre, 2016-02-12.17:22:14 text/plain
Messages
msg2251 (view) Author: kayhayen Date: 2017-10-17.13:45:14
The last release has significantly improved standalone mode scanning, the issue 
should be fixed now. Please re-open if necessary.
msg2107 (view) Author: ab1 Date: 2016-12-20.13:28:10
I've done some more troubleshooting and it would appear to be some dependency issue inside Python modules 
(e.g. requests/pyOpenSSL/cryptography/etc.).

I've dropped the --upgrade flags from requirements install command, relying on the OS provided packages and 
pip installed dependencies as part of upgrading pip itself (pip depends on requests module anyway).

The build succeeds under this environment (Debian 8 x64 Digital Ocean, Python 2.7.9, OpenSSL 1.0.1t  3 May 
2016, Nuitka 0.5.24.4).

Interestingly, "cryptography.hazmat.bindings._openssl" also does not exist in this environment.

-- ab1
msg2106 (view) Author: ab1 Date: 2016-12-17.11:38:52
Hi all, I've recently tried compiling my app with latest Nuitka (stable) under Debian 8 x64 (DigitalOcean) with Python 2.7.9, 
which resulted in a failure in:

import cryptography.hazmat.bindings._openssl

To replicate, build a Debian 8 x64 Droplet (or similar) and run:

# prepare build environment
apt-get update && \
  apt-get install -y chrpath build-essential git python python-dev python-pip python-setuptools libssl-dev libffi-dev && \
  pip install setuptools pip --upgrade

# generate Python requirements file
cat << EOF > requirements.txt
requests[security]
urllib3[secure]
pyopenssl
ndg-httpsclient
pyasn1
EOF

# install requirements and Nuitka
/usr/local/bin/pip install -r requirements.txt --upgrade && \
  /usr/local/bin/pip install nuitka --upgrade

# create test app
cat << EOF > test.py
from cryptography.hazmat.bindings._openssl import ffi, lib
EOF

# run test app under Python interpreter (v2.7.9) -- pass
python test.py

# compile with Nuitka
nuitka --standalone test.py

# run compiled version --fail
test.dist/test.exe

Traceback (most recent call last):
  File "test.py", line 1, in <module>
    from cryptography.hazmat.bindings._openssl import ffi, lib
SystemError: dynamic module not initialized properly


This is where my technical ability ends. If anyone has managed to figure this out, please let me know.

FWIW, under Python 2.7.12 on OS X with Nuitka 0.5.22 and OpenSSL 1.0.2j  26 Sep 2016 it works.

Cheers!
msg1900 (view) Author: marc.jofre Date: 2016-04-10.20:19:21
Hi Kay,

Sure, this sounds good.

We are in WynPython, windows 7, 64 bits, 12GB and python 2.7.

Marc

On 10/04/2016 19:34, Kay Hayen (via Issue Tracker) wrote:
> Kay Hayen <kay.hayen@gmail.com> added the comment:
>
> Hello there,
>
> the demotion of modules from compiled to uncompiled works. Unfortunately it only saves C compile
> time as of yet, but I am going to change that too. But that way it still takes huge amounts of
> memory to get through.
>
> I need to change it so that it creates code for complete modules immediately instead of waiting for
> everything to be optimized, and then to expunge it from the Nuitka process. In an ideal world, it
> would already compile it in the background, but that's not workable yet. That change is what I am
> heading for next. Then scalability work will already have yielded some results.
>
> However, my main desktop has 16G, and so I am currently running this, and pushed to factory git the
> latests findings, again some in crypto module, that might cause this. When it's through, I am going
> to make a pre-release out of it and port it to Windows (I don't think I ran that test yet on
> Windows), so you or anybody can run it too. However, it would be great to be much more easy on
> memory usage before that's an option.
>
> BTW: Are you still on WinPython or AnacondaPython, just curious. Because I would go first for what
> you use.
>
> Yours,
> Kay
>
> _______________________________________________
> Nuitka issue tracker <issue_tracker@nuitka.net>
> <http://bugs.nuitka.net/issue288>
> _______________________________________________

-- 
Marc Jofre
Nanophotonics-Optoelectronics
ICFO-The Institute of Photonic Sciences
Mediterranean Technology Park
Av. Carl Friedrich Gauss, 3
08860 Castelldefels (Barcelona), Spain
Telephone: +34 935542230
Fax: +34 935534000
E-mail: marc.jofre@icfo.es
Internet: www.icfo.es
msg1899 (view) Author: kayhayen Date: 2016-04-10.17:34:57
Hello there,

the demotion of modules from compiled to uncompiled works. Unfortunately it only saves C compile 
time as of yet, but I am going to change that too. But that way it still takes huge amounts of 
memory to get through. 

I need to change it so that it creates code for complete modules immediately instead of waiting for 
everything to be optimized, and then to expunge it from the Nuitka process. In an ideal world, it 
would already compile it in the background, but that's not workable yet. That change is what I am 
heading for next. Then scalability work will already have yielded some results.

However, my main desktop has 16G, and so I am currently running this, and pushed to factory git the 
latests findings, again some in crypto module, that might cause this. When it's through, I am going 
to make a pre-release out of it and port it to Windows (I don't think I ran that test yet on 
Windows), so you or anybody can run it too. However, it would be great to be much more easy on 
memory usage before that's an option.

BTW: Are you still on WinPython or AnacondaPython, just curious. Because I would go first for what 
you use.

Yours,
Kay
msg1888 (view) Author: marc.jofre Date: 2016-04-08.17:09:10
Hi all,

With nuitka 0.5.21pre3 in windows standalone compilation the windows command 
line is:

D:\SixSensoCytometerWebBased\include\bin>RPCconnectCYT.exe
Traceback (most recent call last):
  File "D:\SixSensoCytometerWebBased\include\bin\RPCconnectCYT.py", line 54, in
<module>
    from autobahn.wamp import exportRpc, WampServerProtocol # ImportError: canno
t import name exportRpc because autobahn is installed in the system (pip install
 autobahn)
  File "D:\SixSensoCytometerWebBased\include\bin\autobahn\__init__.py", line 45,
 in autobahn

  File "D:\SixSensoCytometerWebBased\include\bin\autobahn\websocket.py", line 30
, in websocket
    from twisted.internet import reactor, protocol
  File "D:\SixSensoCytometerWebBased\include\bin\twisted\internet\reactor.py", l
ine 38, in reactor
  File "D:\SixSensoCytometerWebBased\include\bin\twisted\internet\default.py", l
ine 56, in default
  File "D:\SixSensoCytometerWebBased\include\bin\twisted\internet\default.py", l
ine 50, in _getInstallFunction
  File "D:\SixSensoCytometerWebBased\include\bin\twisted\internet\selectreactor.
py", line 18, in selectreactor
  File "D:\SixSensoCytometerWebBased\include\bin\twisted\internet\posixbase.py",
 line 23, in posixbase
  File "D:\SixSensoCytometerWebBased\include\bin\twisted\internet\tcp.py", line
29, in tcp
  File "D:\SixSensoCytometerWebBased\include\bin\twisted\internet\_newtls.py", l
ine 21, in _newtls
SystemError: NULL result without error in CALL_FUNCTION
msg1871 (view) Author: marc.jofre Date: 2016-03-18.07:20:06
Hi Kay,

With nuitka version 0.5.20pre3 in standalone execution the command line dump is:

D:\SixSensoCytometerWebBased\include\bin>RPCconnectCYT.exe
Traceback (most recent call last):
  File "D:\SixSensoCytometerWebBased\include\bin\RPCconnectCYT.py", line 54, in
<module>
    from autobahn.wamp import exportRpc, WampServerProtocol # ImportError: canno
t import name exportRpc because autobahn is installed in the system (pip install
 autobahn)
  File "D:\SixSensoCytometerWebBased\include\bin\autobahn\__init__.py", line 45,
 in autobahn

  File "D:\SixSensoCytometerWebBased\include\bin\autobahn\websocket.py", line 30
, in websocket
    from twisted.internet import reactor, protocol
  File "D:\SixSensoCytometerWebBased\include\bin\twisted\internet\reactor.py", l
ine 38, in reactor
  File "D:\SixSensoCytometerWebBased\include\bin\twisted\internet\default.py", l
ine 56, in default
  File "D:\SixSensoCytometerWebBased\include\bin\twisted\internet\default.py", l
ine 50, in _getInstallFunction
  File "D:\SixSensoCytometerWebBased\include\bin\twisted\internet\selectreactor.
py", line 18, in selectreactor
  File "D:\SixSensoCytometerWebBased\include\bin\twisted\internet\posixbase.py",
 line 23, in posixbase
  File "D:\SixSensoCytometerWebBased\include\bin\twisted\internet\tcp.py", line
29, in tcp
  File "D:\SixSensoCytometerWebBased\include\bin\twisted\internet\_newtls.py", l
ine 21, in _newtls
SystemError: NULL result without error in CALL_FUNCTION
msg1870 (view) Author: kayhayen Date: 2016-03-18.06:59:34
The finding of DLLs will be needed for "uuid" module, and flask. I was confused. 
This may be more hidden dependencies. Try out the latest pre-release. I added 
more in the same region.

There is now a test that finds these things. Unfortunately, in some cases, it 
errors out, and doesn't give a result yet (lack of memory). So I turned towards 
making Nuitka capable of avoiding compilation of "setuptools", 
"numpy.distutils", "numpy.f2py" and other useless stuff, that would only be 
relevant for installation.

So this might not be complete yet, but it's a lot more already, and specifcally 
similar ones in other crypto modules.

Let me know if this is good enough. I intend to make a release, and it would be 
nice to see your new problem solved too.

Yours,
Kay
msg1869 (view) Author: marc.jofre Date: 2016-03-12.18:43:19
Sure Kay
msg1867 (view) Author: kayhayen Date: 2016-03-09.09:06:31
That is the same as before, and I think it's because I forgot to implement the 
finding of DLLs for Windows. So far this only works on Linux. I will add an 
implementation for Windows of course too.

Yours,
Kay
msg1866 (view) Author: marc.jofre Date: 2016-03-07.08:01:24
Hi Kay,

I installed nuitka version 0.5.20pre2. When executing the script compiled with 
standalone the execution fails with a NULL in CALL_Function. The command line 
dump is:

D:\SixSensoCytometerWebBased\include\bin>RPCconnectCYT.exe
Traceback (most recent call last):
  File "D:\SixSensoCytometerWebBased\include\bin\RPCconnectCYT.py", line 54, in
<module>
    from autobahn.wamp import exportRpc, WampServerProtocol # ImportError: canno
t import name exportRpc because autobahn is installed in the system (pip install
 autobahn)
  File "D:\SixSensoCytometerWebBased\include\bin\autobahn\__init__.py", line 45,
 in autobahn

  File "D:\SixSensoCytometerWebBased\include\bin\autobahn\websocket.py", line 30
, in websocket
    from twisted.internet import reactor, protocol
  File "D:\SixSensoCytometerWebBased\include\bin\twisted\internet\reactor.py", l
ine 38, in reactor
  File "D:\SixSensoCytometerWebBased\include\bin\twisted\internet\default.py", l
ine 56, in default
  File "D:\SixSensoCytometerWebBased\include\bin\twisted\internet\default.py", l
ine 50, in _getInstallFunction
  File "D:\SixSensoCytometerWebBased\include\bin\twisted\internet\selectreactor.
py", line 18, in selectreactor
  File "D:\SixSensoCytometerWebBased\include\bin\twisted\internet\posixbase.py",
 line 23, in posixbase
  File "D:\SixSensoCytometerWebBased\include\bin\twisted\internet\tcp.py", line
29, in tcp
  File "D:\SixSensoCytometerWebBased\include\bin\twisted\internet\_newtls.py", l
ine 21, in _newtls
SystemError: NULL result without error in CALL_FUNCTION
msg1864 (view) Author: kayhayen Date: 2016-03-06.10:34:18
The URL worked when I tried it, but I have never tried on Windows. Maybe git+http does not 
work for pip there. Can always use the git repo mentioned instead. I will check next time I 
am on Windows. Anyway, I made a pre-release just now with this included.

I am currently working on a library test to compile all ".so" modules for standalone mode 
with just an import on them, to find more hidden dependencies of the like. Hope is to find 
more issues like this on my own.

Yours,
Kay
msg1854 (view) Author: marc.jofre Date: 2016-03-05.10:33:42
Hi Kay,

When the suggested command line I get the below error:

d:\Program Files (x86)\Microsoft Visual Studio 10.0\VC>pip install 
git+http://nu
itka.net/gitweb/?p=Nuitka.git;a=snapshot;h=refs/heads/factory;sf=tgz
Downloading/unpacking 
git+http://nuitka.net/gitweb/?p=Nuitka.git;a=snapshot;h=re
fs/heads/factory;sf=tgz
   Cloning 
http://nuitka.net/gitweb/?p=Nuitka.git;a=snapshot;h=refs/heads/factory
;sf=tgz to c:\users\mjofre\appdata\local\temp\pip-kolmad-build
fatal: unable to access 
'http://nuitka.net/gitweb/?p=Nuitka.git;a=snapshot;h=ref
s/heads/factory;sf=tgz/': The requested URL returned error: 400
   Complete output from command "D:\Program Files (x86)\Git\cmd\git.exe" 
clone -q
  http://nuitka.net/gitweb/?p=Nuitka.git;a=snapshot;h=refs/heads/factory;sf=tgz c
:\users\mjofre\appdata\local\temp\pip-kolmad-build:

----------------------------------------
Cleaning up...
Command "D:\Program Files (x86)\Git\cmd\git.exe" clone -q 
http://nuitka.net/gitw
eb/?p=Nuitka.git;a=snapshot;h=refs/heads/factory;sf=tgz 
c:\users\mjofre\appdata\
local\temp\pip-kolmad-build failed with error code 128 in None
Storing debug log for failure in C:\Users\mjofre\pip\pip.log

On 05/03/2016 11:23, Kay Hayen (via Issue Tracker) wrote:
> Kay Hayen <kay.hayen@gmail.com> added the comment:
>
> http://nuitka.net/doc/factory.html
>
> ----------
> assignedto:  -> kayhayen
> nosy: +kayhayen
> status: chatting -> testing
>
> _______________________________________________
> Nuitka issue tracker <issue_tracker@nuitka.net>
> <http://bugs.nuitka.net/issue288>
> _______________________________________________

-- 
Marc Jofre
Nanophotonics-Optoelectronics
ICFO-The Institute of Photonic Sciences
Mediterranean Technology Park
Av. Carl Friedrich Gauss, 3
08860 Castelldefels (Barcelona), Spain
Telephone: +34 935542230
Fax: +34 935534000
E-mail: marc.jofre@icfo.es
Internet: www.icfo.es
msg1853 (view) Author: kayhayen Date: 2016-03-05.10:23:21
http://nuitka.net/doc/factory.html
msg1852 (view) Author: marc.jofre Date: 2016-03-05.10:17:45
Hi Kay.

Sounds good.

I usually pull from master as: git pull https://github.com/kayhayen/Nuitka.git

How do I pull from factory?
msg1851 (view) Author: kayhayen Date: 2016-03-05.09:50:32
Hello Marc,

I just pushed a fix to the factory branch, from which I hope to make a release 
these days. It has a bunch of other improvements too, let me know if it works 
for you. I expect that there can be other issues.

Also with "_cffi_backend", I wonder, if there is more stuff that can cause the 
problem, or similar. To the point, that I think it might be worth it, for make 
the compile library test variant, that only imports extension modules, and 
checks that works in standalone.

This might reveal many of these kinds of hidden dependency issues, while being 
relatively straightforward.

Therefore I think, "done-cbb" is adequate status of this issue. Open a new one 
for other findings of the same top level imports if you have them.

Yours,
Kay
msg1850 (view) Author: marc.jofre Date: 2016-03-05.09:48:44
Sure Kay

On 05/03/2016 10:25, Kay Hayen (via Issue Tracker) wrote:
> Kay Hayen <kay.hayen@gmail.com> added the comment:
>
> Only Linux, I got this:
>
> open("/data/home/hayen/repos/Py2C/Mini.dist/cryptography/hazmat/bindings/_openssl.so", O_RDONLY
> |O_CLOEXEC) = 4
> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\3206\3\0\0\0\0\0"..., 832) = 832
> fstat(4, {st_mode=S_IFREG|0644, st_size=882432, ...}) = 0
> mmap(NULL, 2977904, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x7f795997f000
> mprotect(0x7f7959a3b000, 2093056, PROT_NONE) = 0
> mmap(0x7f7959c3a000, 114688, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0xbb
> 000) = 0x7f7959c3a000
> mmap(0x7f7959c56000, 112, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0
> x7f7959c56000
> close(4)                                = 0
> mprotect(0x7f7959c3a000, 86016, PROT_READ) = 0
> stat("/data/home/hayen/repos/Py2C/Mini.dist/_cffi_backend", 0x7ffe7c2a91d0) = -1 ENOENT (No suc
> h file or directory)
> open("/data/home/hayen/repos/Py2C/Mini.dist/_cffi_backend.x86_64-linux-gnu.so", O_RDONLY) = -1
> ENOENT (No such file or directory)
> open("/data/home/hayen/repos/Py2C/Mini.dist/_cffi_backend.so", O_RDONLY) = -1 ENOENT (No such f
> ile or directory)
> open("/data/home/hayen/repos/Py2C/Mini.dist/_cffi_backendmodule.so", O_RDONLY) = -1 ENOENT (No
> such file or directory)
> open("/data/home/hayen/repos/Py2C/Mini.dist/_cffi_backend.py", O_RDONLY) = -1 ENOENT (No such f
> ile or directory)
> open("/data/home/hayen/repos/Py2C/Mini.dist/_cffi_backend.pyc", O_RDONLY) = -1 ENOENT (No such
> file or directory)
>
> So, maybe it's the openssl bindings of hazmat that is the offender there, I will try and
> make a test case reduced even further. Seems, a hidden dependency might be all that is
> needed here. The ".so" cannot be analyzed by Nuitka and therefore it's invisible.
>
> Gives me hope to have a fix for this soon.
>
> Yours,
> Kay
>
> _______________________________________________
> Nuitka issue tracker <issue_tracker@nuitka.net>
> <http://bugs.nuitka.net/issue288>
> _______________________________________________

-- 
Marc Jofre
Nanophotonics-Optoelectronics
ICFO-The Institute of Photonic Sciences
Mediterranean Technology Park
Av. Carl Friedrich Gauss, 3
08860 Castelldefels (Barcelona), Spain
Telephone: +34 935542230
Fax: +34 935534000
E-mail: marc.jofre@icfo.es
Internet: www.icfo.es
msg1849 (view) Author: kayhayen Date: 2016-03-05.09:39:07
Yes, all that is takes is this now:

import cryptography.hazmat.bindings._openssl

which gives with Nuitka standalone:

Traceback (most recent call last):
  File "/data/home/hayen/repos/Py2C/Mini.dist/Mini.py", line 9, in <module>
    import cryptography.hazmat.bindings._openssl
SystemError: dynamic module not initialized properly

So, this module at least gives a proper exception, which one of the others than looses, 
resulting in the NULL error, which is kind of sad, but maybe we don't have to care about that, 
this exception didn't tell which one is failing either.

Yours,
Kay
msg1848 (view) Author: kayhayen Date: 2016-03-05.09:25:18
Only Linux, I got this:

open("/data/home/hayen/repos/Py2C/Mini.dist/cryptography/hazmat/bindings/_openssl.so", O_RDONLY
|O_CLOEXEC) = 4
read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\3206\3\0\0\0\0\0"..., 832) = 832
fstat(4, {st_mode=S_IFREG|0644, st_size=882432, ...}) = 0
mmap(NULL, 2977904, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x7f795997f000
mprotect(0x7f7959a3b000, 2093056, PROT_NONE) = 0
mmap(0x7f7959c3a000, 114688, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0xbb
000) = 0x7f7959c3a000
mmap(0x7f7959c56000, 112, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0
x7f7959c56000
close(4)                                = 0
mprotect(0x7f7959c3a000, 86016, PROT_READ) = 0
stat("/data/home/hayen/repos/Py2C/Mini.dist/_cffi_backend", 0x7ffe7c2a91d0) = -1 ENOENT (No suc
h file or directory)
open("/data/home/hayen/repos/Py2C/Mini.dist/_cffi_backend.x86_64-linux-gnu.so", O_RDONLY) = -1 
ENOENT (No such file or directory)
open("/data/home/hayen/repos/Py2C/Mini.dist/_cffi_backend.so", O_RDONLY) = -1 ENOENT (No such f
ile or directory)
open("/data/home/hayen/repos/Py2C/Mini.dist/_cffi_backendmodule.so", O_RDONLY) = -1 ENOENT (No 
such file or directory)
open("/data/home/hayen/repos/Py2C/Mini.dist/_cffi_backend.py", O_RDONLY) = -1 ENOENT (No such f
ile or directory)
open("/data/home/hayen/repos/Py2C/Mini.dist/_cffi_backend.pyc", O_RDONLY) = -1 ENOENT (No such 
file or directory)

So, maybe it's the openssl bindings of hazmat that is the offender there, I will try and
make a test case reduced even further. Seems, a hidden dependency might be all that is
needed here. The ".so" cannot be analyzed by Nuitka and therefore it's invisible.

Gives me hope to have a fix for this soon.

Yours,
Kay
msg1846 (view) Author: kayhayen Date: 2016-03-03.18:24:32
I reproduced the issue on Linux too. That will make it much easier to find the 
solution for me.
msg1823 (view) Author: marc.jofre Date: 2016-02-12.17:22:14
Hi Kay,

I attach the twistedTest.py updated to identify the import that fails.

The import that fails is import twisted.protocols.tls. Because, the import line 
that fails with NULL is number 10.:

D:\SixSensoCompilation\cytometer\PythonWebBased64bits\twistedTest.dist>twistedTe
st.exe
Traceback (most recent call last):
  File "D:\SixSensoCompilation\cytometer\PythonWebBased64bits\twistedTest.dist\t
wistedTest.py", line 10, in <module>
SystemError: NULL result without error in CALL_FUNCTION

Marc
msg1820 (view) Author: kayhayen Date: 2016-02-12.16:43:37
Marc,

that's great seems we reduced the problem by a lot. The twisted code apparently 
fails the import without proper error being set. I doubt anything of the import 
logic is at fault. I suspect, it wants to find some file, likely certificates, 
that kind of thing.

What we would need is the equivalent of "strace" for Windows catching failed 
open calls done at import time. I will be able to do it next week.

Suspect must of course also be a hidden dependency of some sorts. The crypto 
layer that is behind TLS sports a few potentially, this can become tricky 
potentially.

If you can, please split up the import. To demonstrate the issue, do:

import twisted
import twisted.protocols
import twisted.protocols.tls 
import twisted.protocols.tls.TLSMemoryBIOFactory
import twisted.protocols.tls.TLSMemoryBIOProtocol

Depending on which line crashes, we know where to search.

Yours,
Kay
msg1819 (view) Author: marc.jofre Date: 2016-02-12.13:35:03
Hi Kay,

I attacth the twistedTest.py script. I check the correct execution in non 
standalone and NULL error in standalone:

Without standalone:
D:\SixSensoCompilation\cytometer\PythonWebBased64bits>twistedTest.exe
(<class twisted.protocols.tls.TLSMemoryBIOFactory at 0x00000000039853A8>, False)

(<class twisted.protocols.tls.TLSMemoryBIOProtocol at 0x0000000003985288>, False
)

With standalone:
D:\SixSensoCompilation\cytometer\PythonWebBased64bits\twistedTest.dist>twistedTe
st.exe
Traceback (most recent call last):
  File "D:\SixSensoCompilation\cytometer\PythonWebBased64bits\twistedTest.dist\t
wistedTest.py", line 9, in <module>
SystemError: NULL result without error in CALL_FUNCTION

Is this check what you were asking for? let me know any other check/test

Marc
msg1818 (view) Author: kayhayen Date: 2016-02-12.12:29:17
That "NULL result without error in CALL_FUNCTION" is totally some kind of severe 
bug, and *very* interesting. I have no Windows available here right now (on 
holiday, Linux laptop only), but I doubt it's too platform specific.

In my Linux, I am not getting that though. Can you check if this is true 
statement, line 21 of _newtls.py as mentioned in the call stack, is an import 
statement, in my case:

from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol

Since "tls" may well have some issue with some library unavaiable, which will 
then show under Windows, or your setup only, I am concluding that "__import__" 
is the culprit here. Of course, I will need to check. When calling "__import__" 
if a mere NULL result from it is acceptable to the Python core. I didn't think 
so, but you never know.

Yours,
Kay
msg1816 (view) Author: marc.jofre Date: 2016-02-12.12:12:11
Hi Kay,

It is correct the output that you get. This autobahn module is not in 
the system path. Therefore I added in nuitka compilation time as 
PYTHONPATH as:
vcvars64.bat

set 
PYTHONPATH=%PYTHONPATH%;D:/productgeneraldistributables/backend/Backend_GUI/mysite/mysite/autobahn
set PYTHONPATH=%PYTHONPATH%;../../backend/Backend_GUI/mysite/mysite/autobahn
set PYTHONPATH=%PYTHONPATH%;./autobahn

When I compile with non-standalone I get:
d:\SixSensoCompilation\cytometer\PythonWebBased64bits>autobahnTest.exe
(<compiled_function exportRpc at 0x00000000042264C8>, False)
(<class autobahn.wamp.WampServerProtocol at 0x000000000422A108>, False)

The issue I get when compiling and executing with standalone is:
d:\SixSensoCompilation\cytometer\PythonWebBased64bits\autobahnTest.dist>autobahn
Test.exe
Traceback (most recent call last):
   File 
"d:\SixSensoCompilation\cytometer\PythonWebBased64bits\autobahnTest.dist\
autobahnTest.py", line 23, in <module>
   File 
"d:\SixSensoCompilation\cytometer\PythonWebBased64bits\autobahnTest.dist\
autobahn\__init__.py", line 45, in autobahn
   File 
"d:\SixSensoCompilation\cytometer\PythonWebBased64bits\autobahnTest.dist\
autobahn\websocket.py", line 30, in websocket
   File 
"d:\SixSensoCompilation\cytometer\PythonWebBased64bits\autobahnTest.dist\
twisted\internet\reactor.py", line 38, in reactor
   File 
"d:\SixSensoCompilation\cytometer\PythonWebBased64bits\autobahnTest.dist\
twisted\internet\default.py", line 56, in default
   File 
"d:\SixSensoCompilation\cytometer\PythonWebBased64bits\autobahnTest.dist\
twisted\internet\default.py", line 50, in _getInstallFunction
   File 
"d:\SixSensoCompilation\cytometer\PythonWebBased64bits\autobahnTest.dist\
twisted\internet\selectreactor.py", line 18, in selectreactor
   File 
"d:\SixSensoCompilation\cytometer\PythonWebBased64bits\autobahnTest.dist\
twisted\internet\posixbase.py", line 23, in posixbase
   File 
"d:\SixSensoCompilation\cytometer\PythonWebBased64bits\autobahnTest.dist\
twisted\internet\tcp.py", line 29, in tcp
   File 
"d:\SixSensoCompilation\cytometer\PythonWebBased64bits\autobahnTest.dist\
twisted\internet\_newtls.py", line 21, in _newtls
SystemError: NULL result without error in CALL_FUNCTION

Can you Kay try with standalone? I am using windows.

Best,

Marc

On 12/02/2016 12:04, Kay Hayen (via Issue Tracker) wrote:
> Kay Hayen <kay.hayen@gmail.com> added the comment:
>
> Hello Marc,
>
> this says the following on my system:
>
> (<function exportRpc at 0x7f6074bc8cf8>, False)
> (<class autobahn.wamp.WampServerProtocol at 0x7f6074b57328>, False)
>
> The many sys.path.append() do not do anything on my system, except the one that
> deals with autombahn, that's OK, right?
>
> Also, they won't be effective at compile time, there a PYTHONPATH will be needed
> to setup. Maybe some day, somebody will create a plugin that detect sys.path
> changes adding compile time predictable strings like these, and add it to the
> search path, but that's a bit far ahead.
>
> So, please let me know the expected result.
>
> Yours,
> Kay
>
> Yours,
> Kay
>
> ----------
> status: unread -> chatting
>
> _______________________________________________
> Nuitka issue tracker <issue_tracker@nuitka.net>
> <http://bugs.nuitka.net/issue288>
> _______________________________________________

-- 
Marc Jofre
Nanophotonics-Optoelectronics
ICFO-The Institute of Photonic Sciences
Mediterranean Technology Park
Av. Carl Friedrich Gauss, 3
08860 Castelldefels (Barcelona), Spain
Telephone: +34 935542230
Fax: +34 935534000
E-mail: marc.jofre@icfo.es
Internet: www.icfo.es
msg1814 (view) Author: kayhayen Date: 2016-02-12.11:04:36
Hello Marc,

this says the following on my system:

(<function exportRpc at 0x7f6074bc8cf8>, False)
(<class autobahn.wamp.WampServerProtocol at 0x7f6074b57328>, False)

The many sys.path.append() do not do anything on my system, except the one that 
deals with autombahn, that's OK, right?

Also, they won't be effective at compile time, there a PYTHONPATH will be needed 
to setup. Maybe some day, somebody will create a plugin that detect sys.path 
changes adding compile time predictable strings like these, and add it to the 
search path, but that's a bit far ahead.

So, please let me know the expected result.

Yours,
Kay 

Yours,
Kay
msg1813 (view) Author: marc.jofre Date: 2016-02-12.10:33:02
Hi all,

Using nuitka version 0.5.20pre1, Autobahn in non standalone works. In 
standalone, does not work. I attach the autobahnTest.py script and the folder 
containing the specific autobahn module.

Best,

Marc
History
Date User Action Args
2017-10-17 13:45:14kayhayensetstatus: testing -> resolved
messages: + msg2251
2016-12-20 13:28:10ab1setmessages: + msg2107
2016-12-17 11:38:52ab1setmessages: + msg2106
2016-04-10 20:19:21marc.jofresetmessages: + msg1900
2016-04-10 17:34:57kayhayensetmessages: + msg1899
2016-04-08 17:09:10marc.jofresetmessages: + msg1888
2016-03-18 07:20:06marc.jofresetmessages: + msg1871
2016-03-18 06:59:34kayhayensetmessages: + msg1870
2016-03-12 18:43:19marc.jofresetmessages: + msg1869
2016-03-09 09:06:31kayhayensetmessages: + msg1867
2016-03-07 08:01:24marc.jofresetmessages: + msg1866
2016-03-06 10:34:18kayhayensetmessages: + msg1864
2016-03-05 10:33:42marc.jofresetmessages: + msg1854
2016-03-05 10:23:21kayhayensetstatus: chatting -> testing
assignedto: kayhayen
messages: + msg1853
nosy: + kayhayen
2016-03-05 10:17:45marc.jofresetmessages: + msg1852
2016-03-05 09:50:32kayhayensetmessages: + msg1851
2016-03-05 09:48:44marc.jofresetmessages: + msg1850
2016-03-05 09:39:07kayhayensetmessages: + msg1849
2016-03-05 09:25:18kayhayensetmessages: + msg1848
2016-03-03 18:24:32kayhayensetmessages: + msg1846
title: Windows: twisted.protocols.tls problem -> Module twisted.protocols.tls problem
2016-02-12 17:22:14marc.jofresetfiles: + twistedTest.py
messages: + msg1823
2016-02-12 16:43:37kayhayensetmessages: + msg1820
2016-02-12 16:24:50kayhayensettitle: autobahn standalone tester -> Windows: twisted.protocols.tls problem
2016-02-12 13:35:03marc.jofresetfiles: + twistedTest.py
messages: + msg1819
2016-02-12 12:29:17kayhayensetmessages: + msg1818
2016-02-12 12:12:11marc.jofresetmessages: + msg1816
2016-02-12 11:04:36kayhayensetstatus: unread -> chatting
messages: + msg1814
2016-02-12 10:33:02marc.jofrecreate