Issue62

Title object delete timing different from CPython
Priority bug Status resolved
Superseder Nosy List kayhayen, nssh
Assigned To kayhayen Keywords wrong_execution

Created on 2013-03-11.14:40:48 by nssh, last changed by kayhayen.

Files
File name Uploaded Type Edit Remove
test_tarai.py nssh, 2013-03-11.20:42:40 application/octet-stream
Messages
msg686 (view) Author: kayhayen Date: 2014-06-23.07:39:00
Just released as part of 0.5.2
msg658 (view) Author: kayhayen Date: 2014-05-17.07:11:19
This should be fixed in 0.5.2pre5 that I just made. The new C-ish code
generation does this right automatically. I added the test you provided to
"OrderingChecks" (currently factory), as obviously previously the tests didn't
cover it.
msg364 (view) Author: kayhayen Date: 2013-06-12.21:54:33
The problem has become a lot easier to resolve. Now that the assignment to 
temporary variables is order enforced, just as well, could be the release order.

This is now just a matter of adding methods to indicate if something has a 
release need and that release is being order relevant potentially or not. I am 
likely going to add it in one of the next releases, once I thought this through 
well enough.
msg274 (view) Author: kayhayen Date: 2013-03-17.17:07:28
This reminds me of tail end recursion, something I wish to also do.
msg209 (view) Author: nssh Date: 2013-03-11.20:42:25
Add a script (test_tarai.py) for test recursion memory usage.
In this test nuitka always use more than 50% ~ 90% memory.
Run result:
[test@localhost Nuitka]$ python ../test_tarai.py 4 2 0
init count: 120
del count: 120
max init - del count: 31
VmPeak:	   54452 kB
done.
[test@localhost Nuitka]$ ./test_tarai.exe 4 2 0
init count: 120
del count: 120
max init - del count: 59
VmPeak:	  105036 kB
done.
[test@localhost Nuitka]$ python ../test_tarai.py 8 4 0
init count: 28362
del count: 28362
max init - del count: 111
VmPeak:	  175468 kB
done.
[test@localhost Nuitka]$ ./test_tarai.exe 8 4 0
init count: 28362
del count: 28362
max init - del count: 183
VmPeak:	  295296 kB
done.

Just for compare in future.
msg207 (view) Author: nssh Date: 2013-03-11.19:08:44
Yes I understand problem such like this is hard to resolve.
At now this is not a big problem,
I may make some different test base on thie problem in other time.
msg204 (view) Author: kayhayen Date: 2013-03-11.18:45:02
So, this is solvable, but only with big changes.

Unfortunately C++ loves temporaries so much, it won't immediately delete them 
after use, but only after full expression, which means ")" or ";", and my 
attempts with fake sequence points all failed.

But, there may be another way. Right now, "PyObject *" is the interface for 
about everything passed around. And "PyObjectTemporary" releases values that are 
needed by the interface to have a reference, and deleted afterwards.

But it could, and should be different. All helper functions should be template 
functions that accept "PyObjectRef1" and "PyObjectRef0", and know about the 
reference, and then manage "PyObjectRef1" instances to release their reference 
as soon as they are not needed. With "PyObjectRef1" that would be a no-op.

As you can imagine, this is a lot of work. The good news, is that it's work that 
will be needed, to support types other than "PyObject *" efficiently. Them being 
converted to "PyObject *" and releasing that reference, it would be transparent 
to all code.

Hence, I am deciding to defer the solution of this issue and to instead focus on 
optimization that leads to type inference. Once we have that, we can implement 
the proposed changes, and then massively benefit from it too.

Otherwise, I see the risk that this delays Nuitka getting useful in terms of 
performance. And the gravity of the issue is not very high, as Python as a 
language makes no promises on when to call "__del__".

Hope you agree. If you don't. I can try and implement it for a subset of things, 
so that e.g. CALL_FUNCTION uses it. That probably wouldn't hurt, but knowing me, 
I would then go all the way anyway.
msg203 (view) Author: kayhayen Date: 2013-03-11.15:51:08
When you do a(b(c())) in Python, it deletes the argument value, i.e. return 
value of c() immediately after calling b().

Currently we translate that to C++ roughtly like this: a(b(c())) as well. Only 
that in C++, b returns an object, that has a scope. It appears, the destructor 
is executed at the end of the statement. In C++ the ";" is a sequence point, 
i.e. things must be done by then. 

Before a sequence point, the compiler may delay all work and do it when it so 
pleases. Other sequence points also exist. I will have to experiment a bit, to 
try and find another way to say it.

In my head, sequences, could be used. Consider this:

( tmp_c = c(), tmp_b = b( tmp_c ), tmp_c.delete(), tmp_b )

It may be possible, to replace "b( c() )" as from above with this. I need to 
experiment a bit. Of course, it would be nice to avoid it when it's not 
necessary, i.e. to only do it, where the call is used inside something else.

Also other things may be affected too. a[ b ].c.d may hold intermediate 
references for too long as well.

Calling "__del__" at wrong times is a minor but possibly hard to detect 
difference. If nothing can be done about it, we may have to at least document 
it.

Or, give up, and manage temporary variables ourselves, which totally breaks 
using exceptions, and will make code generation a nightmare.
msg202 (view) Author: nssh Date: 2013-03-11.14:40:48
test code:
import time
class A:
        def __del__(self):
                print "del A()"
def check(obj):
        print "start check"
        time.sleep(1)
def p(obj):
        pass
check(p(A()))
print "wait exit"
time.sleep(1)

execute result:
[test@localhost Downloads]$ python 1.py
del A()
start check
wait exit
[test@localhost Downloads]$ nuitka-python 1.py
start check
del A()
wait exit
History
Date User Action Args
2014-06-23 07:39:00kayhayensetstatus: testing -> resolved
messages: + msg686
2014-05-17 07:11:19kayhayensetstatus: deferred -> testing
messages: + msg658
2013-06-12 21:54:33kayhayensetmessages: + msg364
2013-03-17 17:07:28kayhayensetmessages: + msg274
2013-03-11 20:42:40nsshsetfiles: + test_tarai.py
2013-03-11 20:42:25nsshsetmessages: + msg209
2013-03-11 19:08:44nsshsetmessages: + msg207
2013-03-11 18:45:02kayhayensetstatus: chatting -> deferred
assignedto: kayhayen
messages: + msg204
keyword: + wrong_execution
nosy: + kayhayen
2013-03-11 15:51:08kayhayensetstatus: unread -> chatting
messages: + msg203
2013-03-11 14:40:48nsshcreate