Issue219

Title multiprocessing.forking
Priority bug Status resolved
Superseder Nosy List kayhayen, marc.jofre, univerz
Assigned To kayhayen Keywords windows

Created on 2015-07-18.09:11:22 by marc.jofre, last changed by kayhayen.

Files
File name Uploaded Type Edit Remove
Mini_m.py kayhayen, 2015-09-24.02:17:41 text/x-python
nuitkaMpImportingTest.py univerz, 2015-10-08.09:34:48 text/x-java
nuitkaMultiprocessingTest.py marc.jofre, 2015-09-26.11:24:32 text/plain
nuitkaResponse.TXT marc.jofre, 2015-10-07.16:45:28 text/plain
Messages
msg1660 (view) Author: kayhayen Date: 2015-10-25.18:26:54
I am confused now, but's in a stable release if I am not mistaken. Or will be 
soon. Putting this to resolved. :)
msg1658 (view) Author: univerz Date: 2015-10-25.18:01:17
works for me too.
msg1657 (view) Author: marc.jofre Date: 2015-10-25.16:44:43
Hi Kay,

I compiled the whole python project successfully. Also, I ran the software 
package successfully. I will keep testing it and providing feedback.

Best,

Marc
msg1628 (view) Author: marc.jofre Date: 2015-10-10.17:11:13
Hi Kay,

Thanks for the detailed update.

wampserver was crashing both for non-standalone and standalone.

Marc

On 10/10/2015 13:47, univerz (via Issue Tracker) wrote:
> univerz <univerz@fu-solution.com> added the comment:
>
>> It's missing an "import os", and I have no idea, why sys._MEIPASS would
>> exist.
> hups, i was cleaning up imports too eagerly.
>
> now i checked why that weird code is there and it's a hack due to
> pyinstaller. so we can consider it unimportant and minimize the test to
> default multiprocessing.Process
>
> http://stackoverflow.com/questions/24944558/pyinstaller-built-windows-exe-fails-with-multiprocessing
>
> _______________________________________________
> Nuitka issue tracker <issue_tracker@nuitka.net>
> <http://bugs.nuitka.net/issue219>
> _______________________________________________

-- 
Marc Jofre
Nanophotonics-Optoelectronics
ICFO-The Institute of Photonic Sciences
Mediterranean Technology Park
Av. Carl Friedrich Gauss, 3
08860 Castelldefels (Barcelona), Spain
Telephone: +34 935542230
Fax: +34 935534000
E-mail: marc.jofre@icfo.es
Internet: www.icfo.es
msg1626 (view) Author: univerz Date: 2015-10-10.11:47:53
> It's missing an "import os", and I have no idea, why sys._MEIPASS would 
> exist.

hups, i was cleaning up imports too eagerly.

now i checked why that weird code is there and it's a hack due to 
pyinstaller. so we can consider it unimportant and minimize the test to 
default multiprocessing.Process

http://stackoverflow.com/questions/24944558/pyinstaller-built-windows-exe-fails-with-multiprocessing
msg1624 (view) Author: kayhayen Date: 2015-10-10.10:53:59
Marc,

was this crash of wampserver.exe in standalone mode?

Yours,
Kay
msg1623 (view) Author: kayhayen Date: 2015-10-10.10:51:29
So, univerz, the example you gave, actually kind of works on windows, just not on wine, 
lets continue in that bug then. It's missing an "import os", and I have no idea, why 
sys._MEIPASS would exist.

So far, I intend to not have sys.frozen in the final solution. That is mainly because at 
compile time, you are not frozen, but only at run time. Since Nuitka does compile time 
evaluations, it will never fit well. I hope we might make it without that.

Yours,
Kay
msg1616 (view) Author: univerz Date: 2015-10-08.09:34:48
hi,

i ran into predicted problem with importing mechanism of Nuitka. 
stripped code is attached.

there is some _Popen magic and weird target left to ensure that solution 
will work for me, but result is the same without using it.

univerz@localhost ~/.wine/drive_c $ wine Python27/python.exe 
c:/Nuitka-develop/bin/nuitka --standalone --show-progress --recurse-all 
--recurse-to=multiprocessing --plugin-enable=multiprocessing 
c:/nuitkaMpImportingTest.py
...
univerz@localhost ~/.wine/drive_c $ wine 
nuitkaMpImportingTest.dist/nuitkaMpImportingTest.exe
starting processes
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Python27\lib\multiprocessing\forking.py", line 380, in main
prepare(preparation_data)
File "C:\Python27\lib\multiprocessing\forking.py", line 489, in prepare
file, path_name, etc = imp.find_module(main_name, dirs)
ImportError: No module named nuitkaMpImportingTest
msg1614 (view) Author: marc.jofre Date: 2015-10-07.16:45:28
Hi Kay,

I installed the latest release and compiled the whole code. The execution fails 
with windows command line prompt dump:

The thread 'Win64 Thread' (0x180c) has exited with code 0 (0x0).
Unhandled exception at 0x777d8485 in wampserver.exe: 0x80000001: Not 
implemented.
The thread 'Win64 Thread' (0xff0) has exited with code 0 (0x0).
The thread 'Win64 Thread' (0x1778) has exited with code 0 (0x0).

I attach the whole dump file as nuitkaResponse.txt

Which information do you require or tests I have to perform?
msg1610 (view) Author: kayhayen Date: 2015-10-05.08:14:14
So looks perfect to me. Please check if it works for larger things of your too.

I am going to make cleanups, and add this is a form of test to Nuitka, so we 
notice when multiprocessing breaks. Not sure when I get to that though. But you 
shouldn't need that. The danger is "sys.frozen" as it can harm the importing 
mechanism of Nuitka potentially. I would like to get rid of that.
msg1609 (view) Author: marc.jofre Date: 2015-10-04.18:04:27
Hi Kay,

The latest pre-release compiles and executes with the following windows command 
prompt:

D:\SixSensoCompilation\cytometer\PythonWebBased64bits>nuitkaMultiprocessingTest.
exe
<type 'compiled_function'>
PROGRAM START: 5468 EXE nuitkaMultiprocessingTest.exe named __main__
IS FORKING 5468 ['nuitkaMultiprocessingTest.exe'] False
F is now compiled
Executing __main__ code:
<type 'compiled_function'>
PROGRAM START: 4816 EXE nuitkaMultiprocessingTest.exe named __parents_main__
IS FORKING 4816 ['nuitkaMultiprocessingTest.exe', '--multiprocessing-fork', '156
'] True
F is now compiled
5
('hello', 4816, '<class __parents_main__.foo at 0x0000000002653228>') D:\product
generaldistributables\backend\Backend_GUI\mysite\mysite\test\nuitkaMultiprocessi
ngTest.py

D:\SixSensoCompilation\cytometer\PythonWebBased64bits>
msg1607 (view) Author: kayhayen Date: 2015-10-03.06:00:35
So I added code for it in the latest pre-release, which is also on currently 
factory, that runs your example under Python2 just fine.

I need to make it a test, so it's run across versions, but I think that's really 
nice. What's a shame is that "compiled_method" is not a name that is currently 
accessible by any means except having a method object that is compiled.

Maybe I should sneak them into say the "types" module to have clean ways of 
accessing them. And maybe I should at least in "--debug" warn people when they 
use "type(method)" and the result is instancemethod or compiled method, because 
it's very much likely a bug to use that in "is". The only exception for "==" or 
"ininstance" which Nuitka patches up.

But anyway, enjoy for now. The adding of --recurse-to=multiprocessing is still 
necessary on top of the plugin-enable=multiprocessing, I am going to eliminate 
that before I declare it usable.

Yours,
Kay
msg1606 (view) Author: kayhayen Date: 2015-10-01.19:28:28
Ah yes, it does: This is from multiprocessing.forking:_

def _reduce_method(m):
    if m.im_self is None:
        return getattr, (m.im_class, m.im_func.func_name)
    else:
        return getattr, (m.im_self, m.im_func.func_name)
ForkingPickler.register(type(ForkingPickler.save), _reduce_method)

and more like this, some related to method descriptors:

So, pickle is wrapped with something that can do methods. But of course not
compiled methods.

The game plan will of course be to hack the post load of multiprocessing.forking
to do the same thing for compiled methods. I will have to check Python3 if it
is roughly similar there, but likely it is.

Also "type(ForkingPickler.save)" is really a bad choice, depending on how Nuitka
is used, it might be a bytecode method object, or it might be a compiled method
object, and so it's a pretty bad way to do the job. The "types.MethodType" would
have been way better. Other code is using "type(list.append)" to get at types
that are also there.

I am going to be able to straighten that out I figure. But really, somebody has
deeply caused a lot of weirdness with copy, pickle, and multiprocessing, each
doing stuff in a different way.

The next pre-release might not have it, but let's hope the release will see the
experimental proper support for this then.

Yours,
Kay
msg1605 (view) Author: kayhayen Date: 2015-10-01.19:16:11
So I managed to boot Windows and try it out, and unfortunately the same EOFError kicks in, 
this time by the host process getting:

TypeError: Can't pickle instancemethod objects

So this makes it a question, of how multiprocessing does what it does.

I was mistakenly assuming, this was about pickling fooAction, but of course, it's inside of 
that:

p = Process(target=self.fooActionThread,args={self.arguments})

This line transmits the arguments it is getting. And I wonder, how it would work if that's not 
pickle-able. Maybe it is doing something special there.
msg1604 (view) Author: kayhayen Date: 2015-10-01.13:22:15
You can try out factory, to which I pushed, which may behave better or not, but 
it seems the new approach to __deepcopy__ will need bugfixing.

Yours,
Kay
msg1603 (view) Author: marc.jofre Date: 2015-10-01.11:33:00
Thanks Kay for keeping us up to date

Best,

Marc

On 01/10/2015 10:51, Kay Hayen (via Issue Tracker) wrote:
> Kay Hayen <kay.hayen@gmail.com> added the comment:
>
> Turns out that the "copy" module has a dictionary with methods for built-in types, to use in deepcopy. Therefore
> nothing in the "instancemethod" code relates to it.
>
> So what I intend to do, is to add __deepcopy__ (which instance methods don't have) which does the right thing,
> and then to change "__reduce__" to rejection of pickling all the way.
>
> That might work. Needs to go through the CPython test suites, but I believe chances are for this to be good.
>
> Once that works, I am going to run your reproducer on Windows, and we will see if it works better now. There may
> be other built-in types that need explicit deepcopy and pickle rejection though, we will see.
>
> Yours,
> Kay
>
> _______________________________________________
> Nuitka issue tracker <issue_tracker@nuitka.net>
> <http://bugs.nuitka.net/issue219>
> _______________________________________________

-- 
Marc Jofre
Nanophotonics-Optoelectronics
ICFO-The Institute of Photonic Sciences
Mediterranean Technology Park
Av. Carl Friedrich Gauss, 3
08860 Castelldefels (Barcelona), Spain
Telephone: +34 935542230
Fax: +34 935534000
E-mail: marc.jofre@icfo.es
Internet: www.icfo.es
msg1602 (view) Author: kayhayen Date: 2015-10-01.08:51:55
Turns out that the "copy" module has a dictionary with methods for built-in types, to use in deepcopy. Therefore 
nothing in the "instancemethod" code relates to it.

So what I intend to do, is to add __deepcopy__ (which instance methods don't have) which does the right thing, 
and then to change "__reduce__" to rejection of pickling all the way.

That might work. Needs to go through the CPython test suites, but I believe chances are for this to be good.

Once that works, I am going to run your reproducer on Windows, and we will see if it works better now. There may 
be other built-in types that need explicit deepcopy and pickle rejection though, we will see.

Yours,
Kay
msg1601 (view) Author: kayhayen Date: 2015-10-01.08:31:32
Ah well, so I am really confused now.

Seems the __reduce__ method is used and needed by deepcopy. But refused towards
pickle. No idea yet how that works. I was thinking maybe __reduce_ex__ which is attempted first by 
pickle is the key, but turns out having one that refuses is not working for deepcopy anymore too.

So right now, I am at loss at how to have deepcopy work and pickle not. Need to check out how copy 
module's deepcopy works precisely. I think I understood that __reduce_ex__ for instancemethod is indeed 
correct to refuse to work. There must be someelse that makes deepcopy work.

Yours,
Kay
msg1600 (view) Author: kayhayen Date: 2015-09-30.19:10:55
This would be the minimal reproducer:

import pickle

class F():
    def m(self):
        pass

pickle.dumps(F().m)

gives:

pickle.PicklingError: Can't pickle <type 'compiled_method'>: it's not found as 
__builtin__.compiled_method

but for CPython gives:

TypeError: can't pickle instancemethod objects

That is a slight incompatibility, but I don't get if or how that could be the
cause. But to be safe, I will try and eliminate the difference just in case.

Yours,
Kay
msg1599 (view) Author: kayhayen Date: 2015-09-30.08:12:04
That's strange, since I didn't get that. It would be the pickle module doing 
that.

I thought classes are not pickled at all really, but I have the check. Maybe it 
does pickle the dictionary aside methods, and doesn't recognize the compiled 
method then.

This should be reducible to pickle only, and not multiprocessing. Technically 
multiprocessing might be good already, and we now just facing heavier usage of 
pickle which is not yet as perfect and used so much across the board. Which is 
good.

Yours,
Kay
msg1596 (view) Author: marc.jofre Date: 2015-09-29.17:10:45
Hi Kay,

I cloned the nuitka factory branch. The windows command line prompt when 
executing the nuitkaMultiprocessingTest compiled as non-standalone as 
nuitka --exe --explain-imports --file-reference-choice=original --output-
dir=D:/SixSensoCompilation/cytometer/PythonWebBased64bits/ --recurse-all --
recurse-to=multiprocessing --plugin-enable=multiprocessing --recurse-not-
to=PySide 
D:/productgeneraldistributables/backend/Backend_GUI/mysite/mysite/test/nuitkaMul
tiprocessingTest.py > D:\NuitkaOutput.txt

is:

D:\SixSensoCompilation\cytometer\PythonWebBased64bits>nuitkaMultiprocessingTest.
exe
PROGRAM START: 5312 EXE nuitkaMultiprocessingTest.exe named __main__
IS FORKING 5312 ['nuitkaMultiprocessingTest.exe'] False
F is now compiled
Executing __main__ code:
Traceback (most recent call last):
  File "D:\productgeneraldistributables\backend\Backend_GUI\mysite\mysite\test\n
uitkaMultiprocessingTest.py", line 36, in <module>
    fooClass1.fooAction()
  File "D:\productgeneraldistributables\backend\Backend_GUI\mysite\mysite\test\n
uitkaMultiprocessingTest.py", line 23, in fooAction
    p.start()
  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\site-packages\multipro
cessing\process.py", line 109, in start
    self._popen = Popen(self)
  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\site-packages\multipro
cessing\forking.py", line 244, in __init__
    dump(process_obj, to_child, HIGHEST_PROTOCOL)
  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\site-packages\multipro
cessing\forking.py", line 167, in dump
    ForkingPickler(file, protocol).dump(obj)
  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\pickle.py", line 224,
in dump
    self.save(obj)
  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\pickle.py", line 331,
in save
    self.save_reduce(obj=obj, *rv)
  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\pickle.py", line 419,
in save_reduce
    save(state)
  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\pickle.py", line 286,
in save
    f(self, obj) # Call unbound method with explicit self
  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\pickle.py", line 649,
in save_dict
    self._batch_setitems(obj.iteritems())
  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\pickle.py", line 681,
in _batch_setitems
    save(v)
  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\pickle.py", line 331,
in save
    self.save_reduce(obj=obj, *rv)
  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\pickle.py", line 400,
in save_reduce
    save(func)
  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\pickle.py", line 286,
in save
    f(self, obj) # Call unbound method with explicit self
  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\pickle.py", line 748,
in save_global
    (obj, module, name))
pickle.PicklingError: Can't pickle <type 'compiled_method'>: it's not found as _
_builtin__.compiled_method

D:\SixSensoCompilation\cytometer\PythonWebBased64bits>PROGRAM START: 10008 EXE n
uitkaMultiprocessingTest.exe named __parents_main__
IS FORKING 10008 ['nuitkaMultiprocessingTest.exe', '--multiprocessing-fork', '15
6'] True
F is now compiled
Traceback (most recent call last):
  File "D:\productgeneraldistributables\backend\Backend_GUI\mysite\mysite\test\n
uitkaMultiprocessingTest.py", line 39, in __parents_main__

  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\site-packages\multipro
cessing\forking.py", line 347, in main
    self = load(from_parent)
  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\pickle.py", line 1378,
 in load
    return Unpickler(file).load()
  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\pickle.py", line 858,
in load
    dispatch[key](self)
  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\pickle.py", line 880,
in load_eof
    raise EOFError
EOFError
msg1594 (view) Author: kayhayen Date: 2015-09-29.13:41:12
I created a page recently for this purpose:

http://nuitka.net/doc/factory.html

I guess it should also detail how to update an existing git repo.
msg1591 (view) Author: marc.jofre Date: 2015-09-29.08:02:31
Hi Kay,

thanks for the information. I alreade solved the issue with the end of line.

How do I pull from factory?

git pull https://github.com/kayhayen/Nuitka.git factory
msg1587 (view) Author: kayhayen Date: 2015-09-29.07:09:46
Hello Marc,

the "F is now pure" of the first case is already wrong. It should say "compiled" for both. If 
that is now happening, you have lacked to enable to plugin, or something is wrong. I get it 
though that this is probably the intention there to have one run without plugin.

Second thing is because Nuitka patches main code ending, and expects a new line
there, which is of course wrong. I am fixing that now.

The pickle I hope is a follow-up from the crash of the forked main. We shall see.

Can you try out current factory from git repository, the MSIs won't be updated yet. I will make a 
pre-release only later this week.

Yours,
Kay
msg1583 (view) Author: marc.jofre Date: 2015-09-26.11:24:32
Hi Kay,

I did three tests. Described below:

Script Mini_m.py. Non-standalone good compilation and execution:
D:\SixSensoCompilation\cytometer\PythonWebBased64bits>nuitkaMultiprocessingTest.
exe
PROGRAM START: 2172 EXE nuitkaMultiprocessingTest.exe named __main__
IS FORKING 2172 ['nuitkaMultiprocessingTest.exe'] False
F is now compiled
Executing __main__ code:
PROGRAM START: 6952 EXE nuitkaMultiprocessingTest.exe named __parents_main__
IS FORKING 6952 ['nuitkaMultiprocessingTest.exe'] False
F is now pure
('hello', 6952, '<function foo at 0x0000000002353CF8>') D:\productgeneraldistrib
utables\backend\Backend_GUI\mysite\mysite\test\nuitkaMultiprocessingTest.pyc

Script Mini_m.py. Standalone fails at compilation time:
D:\SixSensoCompilation\cytometer\PythonWebBased64bits>nuitka --exe --standalone
--explain-imports --file-reference-choice=original --output-dir=D:/SixSensoCompi
lation/cytometer/PythonWebBased64bits/ --recurse-all --recurse-to=multiprocessin
g --plugin-enable=multiprocessing --recurse-not-to=PySide D:/productgeneraldistr
ibutables/backend/Backend_GUI/mysite/mysite/test/nuitkaMultiprocessingTest.py >
D:\NuitkaOutput.txt
Nuitka:INFO:Injecting plug-in based pre load code for module 'multiprocessing.fo
rking':
Nuitka:INFO:    Monkey patching "multiprocess" load environment.
Problem with statement at D:/productgeneraldistributables/backend/Backend_GUI/my
site/mysite/test/nuitkaMultiprocessingTest.py:16:
  File "D:/productgeneraldistributables/backend/Backend_GUI/mysite/mysite/test/n
uitkaMultiprocessingTest.py", line 29
    p.start()__import__("sys").modules["__main__"] = __import__("sys").modules[_
_name__]
                      ^
SyntaxError: invalid syntax

Script nuitkaMultiprocessingTest.py. Non-standalone with Pickle (arguments 
passed from parent to child process). I reproduced the error explained in 
messages below.

D:\SixSensoCompilation\cytometer\PythonWebBased64bits>PROGRAM START: 5424 EXE nu
itkaMultiprocessingTest.exe named __parents_main__
IS FORKING 5424 ['nuitkaMultiprocessingTest.exe'] False
F is now pure
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\site-packages\multipro
cessing\forking.py", line 347, in main
    self = load(from_parent)
  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\lib\pickle.py", line 1378,
 in load
    return Unpickler(file).load()
  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\lib\pickle.py", line 858,
in load
    dispatch[key](self)
  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\lib\pickle.py", line 880,
in load_eof
    raise EOFError
EOFError
msg1577 (view) Author: kayhayen Date: 2015-09-26.10:38:33
Did you have a chance to try that minimal example I attached?
msg1570 (view) Author: kayhayen Date: 2015-09-24.02:17:11
No idea how to debug that, seems the initial pickle transfer from main to slave process fails there, 
but we obviously won't know why from this.

Our best bet probably would be if you give me a pointer to that program as an example, so I can try 
for myself. Maybe it's about pickling things that are compiled, although that is supposed to work 
well (there have been all kinds of issues solved long ago).

Also please try to make the example minimal to see if it works at all for you. I am attaching what I 
used, let me know if you can make that work for you.

Yours,
Kay
msg1568 (view) Author: marc.jofre Date: 2015-09-23.06:37:10
Hi Kay,

I nuitka non-standalone compiled a project containing multiprocessing. The 
improvement is clear since a small error is only raised up. I will also check 
with standalone.

In non-standalone nuitka compilation using multiprocessing, the windows command 
line prompt dump:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\lib\multiprocessing\forkin
g.py", line 381, in main
    self = load(from_parent)
  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\lib\pickle.py", line 1378,
 in load
    return Unpickler(file).load()
  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\lib\pickle.py", line 858,
in load
    dispatch[key](self)
  File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\lib\pickle.py", line 880,
in load_eof
    raise EOFError
EOFError
msg1565 (view) Author: kayhayen Date: 2015-09-22.11:06:56
Hello Marc,

 - for non-standalone and module: --recurse-to=multiprocessing
>  - for standalone: --recurse-to=multiprocessing --plugin-
> enable=multiprocessing
>

For modules you don't have to do anything, the main program has to. It's a
good
point though, another limitation of the plug-in. Nuitka plug-ins need to
indicate if
they are allowed or make sense. With module mode, it's probably going to do
crap
right now.

Yours,
Kay
msg1564 (view) Author: marc.jofre Date: 2015-09-22.10:54:55
Hi Kay,
Thanks for the information. I understand?:
 - for non-standalone and module: --recurse-to=multiprocessing
 - for standalone: --recurse-to=multiprocessing --plugin-
enable=multiprocessing

I am compiling with the latest release to return with feedback.

Best

Marc
msg1563 (view) Author: kayhayen Date: 2015-09-21.19:18:48
Hello Marc,

I just pushed it out as a pre-release. This was real tough stuff, need quite a 
bit new infrastructure, so that is why it took longer.

You need to use --recurse-to=multiprocessing to compile that part. With frozen 
code it won't work yet, a mere --standalone won't do I think. And then you also 
need to do "--plugin-enable=multiprocessing" to make it apply the necessary 
hacks.

This sets "sys.frozen" which I don't like doing, but for the time being it was 
easiest.

Let's see how this works for you.

There are a couple of things left to do. For one it should be limited to 
Windows, and it should get a "detector" plug-in, one that informs the user about 
it not being on by default.

Let me know how this fares for you. This is of course pushed to github, but 
improvements will continue on "factory" for a while.

Yours,
Kay
msg1560 (view) Author: marc.jofre Date: 2015-09-20.06:57:44
Hi Kay,

Can you push multiprocessing support to github?

Best,

Marc
msg1501 (view) Author: marc.jofre Date: 2015-09-06.05:49:01
Hi Kay,

This sounds great. I will spread the word that nuitka is almost about to support 
multiprocessing. All my colleagues are hesitating to have the multiprocessing 
module running for their python projects.

I attach the example of py2exe support for multiprocessing:

Support multiprocessing in py2exe services:

https://gist.github.com/conorbranagan/7138292
msg1496 (view) Author: kayhayen Date: 2015-09-03.06:54:25
So yes, pre-load does the trick:

Executing this:

def f():
    pass

import os, sys, multiprocessing.forking
print "PROGRAM START", os.getpid(), "EXE", sys.argv[0], __name__
print "IS FORKING", os.getpid(), sys.argv, multiprocessing.forking.is_forking(sys.argv)
print "F is now", "pure" if "compiled" not in repr(f) else "compiled"

from multiprocessing import Process, freeze_support

def foo():
    print('hello', os.getpid(), repr(foo)), __file__

if __name__ == '__main__':
    print "Executing __main__ code:"

    # Multiprocessing requires to do this ASAP.
    multiprocessing.freeze_support()

    # Start a process, and make us see what it did.
    p = Process(target=foo)
    p.start()


$ /c/Python27_64/python bin/nuitka --run --recurse-to=multiprocessing Mini_m.py
Nuitka:INFO:Injecting plug-in based pre load code for module 'multiprocessing.forking':
Nuitka:INFO:    Monkey patching "multiprocess" load environment.
PROGRAM START 3596 EXE Mini_m.exe __main__
IS FORKING 3596 ['Mini_m.exe'] False
F is now compiled
Executing __main__ code:
Mini_m.exe "Mini_m.exe" "--multiprocessing-fork" "492"
PROGRAM START 5976 EXE Mini_m.exe __parents_main__
IS FORKING 5976 ['Mini_m.exe', '--multiprocessing-fork', '492'] True
F is now compiled

A lot of things now work. The forking to the exe itself works, massaging its command line arguments 
works, the plugin is only triggered by the recursion, currently for frozen code, it's not possible 
to have implicit module dependencies, which the pre and post load modules are.

The compilation nature of the code is there. What I think is missing is the executing of the task 
after executing the "__main__" as "__parents_main__", but that should be possible to add. I most 
probably will just append the boot strap code to the main module code used for that copy.

I don't think I will manage this today, because I got stuff to do, but I think this is nearing 
completion finally. My main mistake was probably to try and fix up things after the loading of the 
module, whereas what's needed, is to make things more similar to what py2exe does.

The change of "sys.executable" is not a good idea, because in principle, for non standalone, it 
should still be possible to fork a new and other Python program. But I believe, that can be undone 
in postLoad code, and it will still work.

So, yeah, something to this end is going to hit factory later.

Yours,
Kay
msg1495 (view) Author: kayhayen Date: 2015-09-02.21:06:02
So, I am annoyed now.

Turns out, this heavily depends on sys.frozen, which OK, we can set. But it 
also uses this at module load time. The current "onLoad" mechanism loads after 
the fact, but "sys.executable" is supposed to be the binary name there, which 
is currently only done afterwards.

So I am added "preLoad" and "postLoad" module support, so modules can change 
things before they get loaded. This is a nasty puzzle. Might be a while until 
the last piece will be found....

Anyway, hope goes on. :)

Yours,
Kay
msg1494 (view) Author: marc.jofre Date: 2015-08-31.15:39:42
Hi Kay,

I will use "--recurse-not-to=PySide" and give feedback to all of you in 
the next compilation with the multiprocessing plugin in place.

I am very enthusiastic of the coming compatibility with multiprocessing, 
which many software packages end up using.

Marc

On 31/08/2015 12:04, Kay Hayen (via Issue Tracker) wrote:
> Kay Hayen <kay.hayen@gmail.com> added the comment:
>
> So I tried this, an ran into all kinds of issues. Part of which I resolved.
>
> So, by default it will use sys.executable, but you can change that, which it will use, if sys.frozen
> exists, which for Nuitka it doesn't. I was cheating it for tests, but generally, it will be a bad idea.
> More stuff to monkey patch then.
>
> Also, I realized that this is not a standalone issue. The accelerated binary is affected too, but for it,
> no compiled multiprocessing module will be present unless we force it to recurse into standard library.
> And then the "onLoad" mechanism doesn't trigger.
>
> And what I managed, is to patch the command line arguments on the fly. The multiprocessing expects that
> the executable, or python.exe have already remove the "-R" and "-c" commands it gives, and only the
> argument that indicates the port number to remain.
>
> So, what I thought I will do, is for the accelerated binary to either force the compilation of the
> multiprocessing module from the plugin, or to include it "frozen" even without standard library, which
> would mean, that parts of the code that currently assume "frozen modules == standalone", must be sorted
> out. As there are other issues that might need it, or might make it sensible to ultimately have an option
> to include modules or part of it as bytecode rather than compiling them.
>
> So this has become big. It didn't help that all weekend heat records were broken. I will try and make it
> possible for the multiprocessing plugin to force the recursion to multiprocessing and
> multiprocessing.forking, so we can control their loading. And to expand the onLoad mechanism to frozen
> bytecode loads, so we can not do it for standalone mode.
>
> This is pretty tricky, and I want to get rid of it, so that's all I care about now.
>
> (For your PySide note, there is an issue for that already, isn't there? But I was wondering if you could
> use "--recurse-not-to PySide" or PyQt, whatever your preference is. The issue is that PySide and PyQt
> plugins will not coexist I think. And if that's the case, the plugin should force you to decide I guess,
> but please follow up on that in that existing issue.)
>
>
> Yours,
> Kay
>
> ----------
> keyword: +windows -standalone
>
> _______________________________________________
> Nuitka issue tracker <issue_tracker@nuitka.net>
> <http://bugs.nuitka.net/issue219>
> _______________________________________________

-- 
Marc Jofre
Nanophotonics-Optoelectronics
ICFO-The Institute of Photonic Sciences
Mediterranean Technology Park
Av. Carl Friedrich Gauss, 3
08860 Castelldefels (Barcelona), Spain
Telephone: +34 935542230
Fax: +34 935534000
E-mail: marc.jofre@icfo.es
Internet: www.icfo.es
msg1492 (view) Author: kayhayen Date: 2015-08-31.10:04:34
So I tried this, an ran into all kinds of issues. Part of which I resolved.

So, by default it will use sys.executable, but you can change that, which it will use, if sys.frozen 
exists, which for Nuitka it doesn't. I was cheating it for tests, but generally, it will be a bad idea. 
More stuff to monkey patch then.

Also, I realized that this is not a standalone issue. The accelerated binary is affected too, but for it, 
no compiled multiprocessing module will be present unless we force it to recurse into standard library. 
And then the "onLoad" mechanism doesn't trigger.

And what I managed, is to patch the command line arguments on the fly. The multiprocessing expects that 
the executable, or python.exe have already remove the "-R" and "-c" commands it gives, and only the 
argument that indicates the port number to remain.

So, what I thought I will do, is for the accelerated binary to either force the compilation of the 
multiprocessing module from the plugin, or to include it "frozen" even without standard library, which 
would mean, that parts of the code that currently assume "frozen modules == standalone", must be sorted 
out. As there are other issues that might need it, or might make it sensible to ultimately have an option 
to include modules or part of it as bytecode rather than compiling them.

So this has become big. It didn't help that all weekend heat records were broken. I will try and make it 
possible for the multiprocessing plugin to force the recursion to multiprocessing and 
multiprocessing.forking, so we can control their loading. And to expand the onLoad mechanism to frozen 
bytecode loads, so we can not do it for standalone mode.

This is pretty tricky, and I want to get rid of it, so that's all I care about now.

(For your PySide note, there is an issue for that already, isn't there? But I was wondering if you could 
use "--recurse-not-to PySide" or PyQt, whatever your preference is. The issue is that PySide and PyQt 
plugins will not coexist I think. And if that's the case, the plugin should force you to decide I guess, 
but please follow up on that in that existing issue.)


Yours,
Kay
msg1491 (view) Author: marc.jofre Date: 2015-08-29.04:15:57
Sure Kay,

I understand that the plugin way is our best chance for multiprocessing in 
windows.

Furthermore, in windows, for plugin for PyQt the dependency dll retrieval stops 
with command line promp:

Windows command line dump:
Currently, with nuitka 0.5.14pre9 the compilation windows command prompt line 
is:
Nuitka:INFO:Copying all Qt plug-ins to 'D:\SixSensoCompilation\cytometer\ExeWebB
ased64bits\RPCconnectCYT.dist\PyQt4\qt-plugins'.
Error, conflicting DLLs for 'qtsvg4.dll'.
d:\winpython-64bit-2.7.6.3\python-2.7.6.amd64\lib\site-packages\pyqt4\qtsvg4.dll
 used by:
   D:\SixSensoCompilation\cytometer\ExeWebBased64bits\RPCconnectCYT.dist\PyQt4\Q
tSvg.pyd
   D:\SixSensoCompilation\cytometer\ExeWebBased64bits\RPCconnectCYT.dist\PyQt4\q
t-plugins\iconengines\qsvgicon4.dll
   D:\SixSensoCompilation\cytometer\ExeWebBased64bits\RPCconnectCYT.dist\PyQt4\q
t-plugins\imageformats\qsvg4.dll
different from
d:\winpython-64bit-2.7.6.3\python-2.7.6.amd64\lib\site-packages\pyside\qtsvg4.dl
l used by
   D:\SixSensoCompilation\cytometer\ExeWebBased64bits\RPCconnectCYT.dist\PySide\
QtSvg.pyd
msg1484 (view) Author: kayhayen Date: 2015-08-28.20:15:02
So, all there is will be a monkey patching need, to be done in a plugin, to make 
multiprocessing.forking launch things properly. It needs to launch the compiled 
binary instead of "sys.executable", which seems easy.

I will try and spin up a plugin that does this, for PyQt plugins there is 
something similar already. If all goes well, we could see a hotfix of it even.

Yours,
Kay
msg1483 (view) Author: kayhayen Date: 2015-08-28.09:00:32
Seems the "prepare" of multiprocessing does stuff only needed for Linux, 
mainly to refresh the main module, so it can be loaded again, just having the 
".py" file there seems sufficient indeed.

I am now not sure, what makes that work, but apparently it's python.exe which 
then runs the slaves, definitely not going to be portable to other machines:

MAIN ('hello', 528, '<compiled function foo at 0x03321300>')
('hello', 3904, '<function foo at 0x02A67270>')

The second line is from the slave. I will try and make it not find python.exe 
to see where it is called, and how to make it call the binary instead. Seems 
very easy to create a fork bomb it seems. I still don't know how 
multiprocessing code manages to run. 

I think that "freeze_support" gives the new process the chance to talk to the 
old process and never return. If that's true, we need to just patch it up and 
be good, provided sys.argv is good enough for it.

Yours,
Kay
msg1482 (view) Author: kayhayen Date: 2015-08-28.07:44:29
Ah, got it, that explains why everything not Windows just works.

For Windows I am expecting it see the main program enter again, with either special 
command line options to tell Python.exe to take not __main__ as entry, but rather 
multiprocessing, but thinking of it, how would it do the imports then?

So maybe the magic is happening in either import multiprocessing or freeze_support, 
where it then does magic stuff. I got an idea how to trace the command line options 
of the second execution that I will try now. I didn't find anything in "pythonrun.c" 
which is what Nuitka has to emulate.

The question will be, if were that transparent, why doesn't it work already. Maybe 
not having "python.exe program.py" but "program.exe" in argv does confuse it. But I 
never saw anything in CPython source that deals with it.

Yours,
Kay
msg1481 (view) Author: marc.jofre Date: 2015-08-28.06:27:24
Hi Kay,

Indeed, there is a difference between Windows and Linux multiprocessing 
behaviour:

http://rhodesmill.org/brandon/2010/python-multiprocessing-linux-windows/

There is one rather startling difference which the multiprocessing module does 
not hide: the fact that while every Windows process must spin up independently 
of the parent process that created it, Linux supports the fork(2) system call 
that creates a child processes already in possession of exactly the same 
resources as its parent: every data structure, open file, and database 
connection that existed in the parent process is still sitting there, open and 
ready to use, in the child.
msg1469 (view) Author: marc.jofre Date: 2015-08-23.11:07:48
Hi Kay,

Thanks a lot for the explanation and public discussion.

I add some extra observations:

When compiled without standalone, this issue does not appear for the time being 
(it might be resilent). While in standalone, appears just initiating the 
executable. In particular, the standalone nuitka compiles but fails to 
incorporate all relevant dlls (shown below); can it be a possible reason for 
failing in standalone?

Windows command line dump:
Currently, with nuitka 0.5.14pre9 the compilation windows command prompt line 
is:
Nuitka:INFO:Copying all Qt plug-ins to 'D:\SixSensoCompilation\cytometer\ExeWebB
ased64bits\RPCconnectCYT.dist\PyQt4\qt-plugins'.
Error, conflicting DLLs for 'qtsvg4.dll'.
d:\winpython-64bit-2.7.6.3\python-2.7.6.amd64\lib\site-packages\pyqt4\qtsvg4.dll
 used by:
   D:\SixSensoCompilation\cytometer\ExeWebBased64bits\RPCconnectCYT.dist\PyQt4\Q
tSvg.pyd
   D:\SixSensoCompilation\cytometer\ExeWebBased64bits\RPCconnectCYT.dist\PyQt4\q
t-plugins\iconengines\qsvgicon4.dll
   D:\SixSensoCompilation\cytometer\ExeWebBased64bits\RPCconnectCYT.dist\PyQt4\q
t-plugins\imageformats\qsvg4.dll
different from
d:\winpython-64bit-2.7.6.3\python-2.7.6.amd64\lib\site-packages\pyside\qtsvg4.dl
l used by
   D:\SixSensoCompilation\cytometer\ExeWebBased64bits\RPCconnectCYT.dist\PySide\
QtSvg.pyd
msg1465 (view) Author: kayhayen Date: 2015-08-22.15:20:45
So what multiprocess.forking does want to do, is to use imp.find_module, and then to manually load stuff:

            file, path_name, etc = imp.find_module(main_name, dirs)
            try:
                # We would like to do "imp.load_module('__main__', ...)"
                # here.  However, that would cause 'if __name__ ==
                # "__main__"' clauses to be executed.
                main_module = imp.load_module(
                    '__parents_main__', file, path_name, etc
                    )
            finally:
                if file:
                    file.close()

            sys.modules['__main__'] = main_module
            main_module.__name__ = '__main__'

Obviously, that main program code is then to be executed again, instead of this happening. I am now wondering, 
how to intercept this code:

a) Change its source before including via say a plugin, that modifies stuff, and makes also makes a copy of the 
main program, as "__parents_main__" module, and makes sure that's assumed as an entry point too.

b) Monkey patch imp.load_module, or "multiprocessing.imp" after it's loaded, but before it's used, if that is 
possible into something that just does returns the expected stuff. There I would need to find out, what causes 
the multiprocessing to do its magic in the first place.

c) Provide a copy of the source code to use for this purpose, and make it so that imp.find_module will actually 
find it. Which may mean it has to be top level, which would be bad. That would not compile the main program, 
which would look bad in benchmark kind of things.

Gotta think a bit...

Yours,
Kay
msg1464 (view) Author: kayhayen Date: 2015-08-22.15:08:43
The following fails under Windows (Linux is OK):

---------
from multiprocessing import Process, freeze_support

import os

def foo():
    print('hello', os.getpid())

if __name__ == '__main__':
    print "MAIN", os.getpid()
    freeze_support()
    p = Process(target=foo)
    p.start()
---------

It seems as thought it tries to open the original source file in order to run it and that fails, I am 
going to investigate further. But in principle it looks as if multiprocessing works with compiled code 
in forked instances, even though I don't know why, at least on Linux. Maybe it doesn't fork a new 
Python there, but only a thread. I shall see.
msg1378 (view) Author: marc.jofre Date: 2015-07-25.16:39:51
Hi Kay,

Definetely multiprocessing.forking is of interest in many applications dealing 
with user interfaces and more advanced features. Is your suggestion of using 
monkey patching a possibility for this multiprocessing.forking?

#Using Coroutines to Create Efficient, High-Concurrency Web Applications: 
several ways to deploy WSGI applications while being compatible with nuitka 
compilation in terms of multiprocessing forking. When threads are monkey patched 
in gevent, they behave as coroutines. This means that you have to explicitly 
yield control to make it possible for other coroutines to execute.
import gevent
from gevent import monkey
monkey.patch_all(thread=False, socket=False)
msg1366 (view) Author: marc.jofre Date: 2015-07-18.10:31:44
Hi Kay,

Thanks for the complete explanation of multiprocessing issue. I fully
understand your timmings which are ok for me.

I will check regularly on your comments, developments. Always I will give
back feedback.

Best

>
> Kay Hayen <kay.hayen@gmail.com> added the comment:
>
> Hello Marc,
>
> the multiprocessing issue is a bit more difficult than what I can handle
> right now before
> EuroPython (preparing slides now ;-), it involves the following:
>
> 1. Need to tell multiprocessing, it's frozen or standalone as we say.
> Otherwise it wants to
> start another Python instance.
> 2. At launch, multiprocessing needs to get a chance to intercept command
> line
>    arguments and not execute "main" module, but its own entry point, as
> said
>    on the command line.
>
> Nuitka doesn't have the second thing yet. It will take some modifications
> to the
> main code template, and a switch to enable it potentially, as we wouldn't
> want to
> load multiprocessing for each run.
>
> So in reality, that isn't proper multiprocessing usage you have there, as
> it will run
> uncompiled stuff most likely. I figure this is two bugs, multiprocessing
> not being done
> properly without python.exe, and then it not working well together. That
> however could well
> be something about multiprocessing not knowing it's frozen.
>
> I will turn to these issue after Europython as it's a relatively important
> gap, isn't it, the
> multiprocessing ought to just work for standalone too.
>
> Yours,
> Kay
>
> ----------
> assignedto:  -> kayhayen
> nosy: +kayhayen
> status: unread -> chatting
>
> _______________________________________________
> Nuitka issue tracker <issue_tracker@nuitka.net>
> <http://bugs.nuitka.net/issue219>
> _______________________________________________
>
msg1365 (view) Author: kayhayen Date: 2015-07-18.10:13:16
Hello Marc,

the multiprocessing issue is a bit more difficult than what I can handle right now before 
EuroPython (preparing slides now ;-), it involves the following:

1. Need to tell multiprocessing, it's frozen or standalone as we say. Otherwise it wants to 
start another Python instance.
2. At launch, multiprocessing needs to get a chance to intercept command line
   arguments and not execute "main" module, but its own entry point, as said
   on the command line.

Nuitka doesn't have the second thing yet. It will take some modifications to the
main code template, and a switch to enable it potentially, as we wouldn't want to
load multiprocessing for each run.

So in reality, that isn't proper multiprocessing usage you have there, as it will run 
uncompiled stuff most likely. I figure this is two bugs, multiprocessing not being done 
properly without python.exe, and then it not working well together. That however could well 
be something about multiprocessing not knowing it's frozen.

I will turn to these issue after Europython as it's a relatively important gap, isn't it, the 
multiprocessing ought to just work for standalone too.

Yours,
Kay
msg1364 (view) Author: marc.jofre Date: 2015-07-18.09:11:22
Hi all,

Nuitka standalone compilation works well. When executing the compiled exe the 
below windows command line prompt dump is on multiprocessing.forking:

Which advices and directions can I focus to advance in this issue??


D:\SixSensoCytometerWebBased\include\bin>RPCconnectCYT.exe
2015-07-18 11:01:30+0200 [-] Log opened.
2015-07-18 11:01:30+0200 [-] WampServerFactory starting on 9000
2015-07-18 11:01:30+0200 [-] Starting factory <autobahn.wamp.WampServerFactory i
nstance at 0x000000003E5C5FC8>
2015-07-18 11:01:30+0200 [-] WampServerFactory starting
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named multiprocessing.forking
2015-07-18 11:02:20+0200 [SimpleServerProtocol,0,127.0.0.1] Unhandled Error
        Traceback (most recent call last):
          File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\site-packages\
twisted\python\log.py", line 94, in callWithLogger
            lp = logger.logPrefix()
          File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\site-packages\
twisted\python\log.py", line 83, in callWithContext
            newCtx.update(ctx)
          File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\site-packages\
twisted\python\context.py", line 118, in callWithContext
            return self.currentContext().callWithContext(ctx, func, *args, **kw)

          File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\site-packages\
twisted\python\context.py", line 79, in callWithContext
            self.contexts.append(newContext)
        --- <exception caught here> ---
          File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\site-packages\
twisted\internet\selectreactor.py", line 149, in _doReadOrWrite
            why = getattr(selectable, method)()
          File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\site-packages\
twisted\internet\tcp.py", line 209, in doRead
            return self._dataReceived(data)
          File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\site-packages\
twisted\internet\tcp.py", line 215, in _dataReceived
            rval = self.protocol.dataReceived(data)
          File "D:\productgeneraldistributables\backend\Backend_GUI\mysite\mysit
e\autobahn\websocket.py", line 961, in dataReceived
            self.consumeData()
          File "D:\productgeneraldistributables\backend\Backend_GUI\mysite\mysit
e\autobahn\websocket.py", line 985, in consumeData
            self.processHandshake()
          File "D:\productgeneraldistributables\backend\Backend_GUI\mysite\mysit
e\autobahn\websocket.py", line 2157, in processHandshake
            self.onOpen()
          File "D:\productgeneraldistributables\backend\Backend_GUI\mysite\mysit
e\autobahn\wamp.py", line 208, in onOpen
            self.onSessionOpen()
          File "D:\productgeneraldistributables\backend\Backend_GUI\mysite\mysit
e\RPCconnectCYT.py", line 1202, in onSessionOpen
            self.connection = ConnectionRpc()
          File "D:\productgeneraldistributables\backend\Backend_GUI\mysite\mysit
e\RPCconnectCYT.py", line 113, in __init__
            self.queue=multiprocessing.Manager().Queue()
          File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\site-packages\
multiprocessing\__init__.py", line 104, in Manager
            m.start()
          File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\site-packages\
multiprocessing\managers.py", line 498, in start
            self._process.start()
          File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\site-packages\
multiprocessing\process.py", line 109, in start
            self._popen = Popen(self)
          File "D:\WinPython-64bit-2.7.6.3\python-2.7.6.amd64\Lib\site-packages\
multiprocessing\forking.py", line 247, in __init__
            to_child.close()
        exceptions.IOError: [Errno 22] Invalid argument

2015-07-18 11:02:20+0200 [SimpleServerProtocol,0,127.0.0.1] unsubscribed peer 12
7.0.0.1:65157 from all topics
History
Date User Action Args
2015-10-25 18:26:54kayhayensetstatus: testing -> resolved
messages: + msg1660
2015-10-25 18:01:17univerzsetmessages: + msg1658
2015-10-25 16:44:43marc.jofresetmessages: + msg1657
2015-10-10 17:11:13marc.jofresetmessages: + msg1628
2015-10-10 11:47:53univerzsetmessages: + msg1626
2015-10-10 10:53:59kayhayensetmessages: + msg1624
2015-10-10 10:51:29kayhayensetmessages: + msg1623
2015-10-08 09:34:48univerzsetfiles: + nuitkaMpImportingTest.py
messages: + msg1616
2015-10-08 05:43:24univerzsetnosy: + univerz
2015-10-07 16:45:28marc.jofresetfiles: + nuitkaResponse.TXT
messages: + msg1614
2015-10-05 08:14:14kayhayensetstatus: in-progress -> testing
messages: + msg1610
2015-10-04 18:04:27marc.jofresetmessages: + msg1609
2015-10-03 06:00:36kayhayensetmessages: + msg1607
2015-10-01 19:28:28kayhayensetmessages: + msg1606
2015-10-01 19:16:11kayhayensetmessages: + msg1605
2015-10-01 13:22:15kayhayensetmessages: + msg1604
2015-10-01 11:33:00marc.jofresetmessages: + msg1603
2015-10-01 08:52:10kayhayensetstatus: testing -> in-progress
2015-10-01 08:51:55kayhayensetmessages: + msg1602
2015-10-01 08:31:32kayhayensetmessages: + msg1601
2015-09-30 19:10:55kayhayensetmessages: + msg1600
2015-09-30 08:12:04kayhayensetmessages: + msg1599
2015-09-29 17:10:45marc.jofresetmessages: + msg1596
2015-09-29 13:41:12kayhayensetmessages: + msg1594
2015-09-29 08:02:31marc.jofresetmessages: + msg1591
2015-09-29 07:09:46kayhayensetmessages: + msg1587
2015-09-26 11:24:32marc.jofresetfiles: + nuitkaMultiprocessingTest.py
messages: + msg1583
2015-09-26 10:38:33kayhayensetmessages: + msg1577
2015-09-24 02:17:41kayhayensetfiles: + Mini_m.py
2015-09-24 02:17:11kayhayensetmessages: + msg1570
2015-09-23 06:37:10marc.jofresetmessages: + msg1568
2015-09-22 11:06:56kayhayensetmessages: + msg1565
2015-09-22 10:54:56marc.jofresetmessages: + msg1564
2015-09-21 19:18:48kayhayensetstatus: chatting -> testing
messages: + msg1563
2015-09-20 06:57:44marc.jofresetmessages: + msg1560
2015-09-06 05:49:01marc.jofresetmessages: + msg1501
2015-09-03 06:54:25kayhayensetmessages: + msg1496
2015-09-02 21:06:02kayhayensetmessages: + msg1495
2015-08-31 15:39:42marc.jofresetmessages: + msg1494
2015-08-31 10:04:34kayhayensetmessages: + msg1492
keyword: + windows, - standalone
2015-08-29 04:15:57marc.jofresetmessages: + msg1491
2015-08-28 20:15:02kayhayensetmessages: + msg1484
2015-08-28 09:00:32kayhayensetmessages: + msg1483
2015-08-28 07:44:29kayhayensetmessages: + msg1482
2015-08-28 06:27:24marc.jofresetmessages: + msg1481
2015-08-23 11:07:48marc.jofresetmessages: + msg1469
2015-08-22 15:20:45kayhayensetmessages: + msg1465
2015-08-22 15:08:43kayhayensetmessages: + msg1464
2015-07-25 16:39:51marc.jofresetmessages: + msg1378
2015-07-18 10:31:44marc.jofresetmessages: + msg1366
2015-07-18 10:13:32kayhayensetkeyword: + standalone
2015-07-18 10:13:16kayhayensetstatus: unread -> chatting
assignedto: kayhayen
messages: + msg1365
nosy: + kayhayen
2015-07-18 09:11:22marc.jofrecreate