Issue123

Title nuitka 0.5.0.1 _swapFiber not found OS X 10.9.1 Mavericks
Priority bug Status resolved
Superseder Nosy List fpbhb, kayhayen
Assigned To kayhayen Keywords compiler_crash, macos

Created on 2014-01-30.18:00:54 by fpbhb, last changed by kayhayen.

Files
File name Uploaded Type Edit Remove
run_all.log fpbhb, 2014-02-03.09:08:37 application/octet-stream
Messages
msg692 (view) Author: kayhayen Date: 2014-06-23.07:44:58
Nuitka is working fine on MacOS now, even standalone is supported as of 0.5.2
msg635 (view) Author: kayhayen Date: 2014-04-25.18:46:03
That will be great, Frank. Looking forward to it.
msg634 (view) Author: fpbhb Date: 2014-04-25.07:18:26
Hello Kay,

no, I did not loose interest. It's just that my time is quite scarce,
and lost focus when I was at it. I'll get back to that issue soon.

Regards,
	- frank

Am 24.04.2014 um 22:04 schrieb Kay Hayen (via Issue Tracker) <issue_tracker@nuitka.net>:

> 
> Kay Hayen <kay.hayen@gmail.com> added the comment:
> 
> Hello,
> 
> any progress on this, I would welcome any work towards standalone mode on MacOS, 
> or did you loose interest. 
> 
> Yours,
> Kay
> 
> _______________________________________________
> Nuitka issue tracker <issue_tracker@nuitka.net>
> <http://bugs.nuitka.net/issue123>
> _______________________________________________
msg632 (view) Author: kayhayen Date: 2014-04-24.20:04:20
Hello,

any progress on this, I would welcome any work towards standalone mode on MacOS, 
or did you loose interest. 

Yours,
Kay
msg566 (view) Author: kayhayen Date: 2014-02-08.14:42:00
The email interface should be working now, the fix is pushed to master, but my 
release process kind of stalled, due to some rst2pdf trouble.
msg564 (view) Author: kayhayen Date: 2014-02-05.11:44:34
I am pushing the hotfix later today. I believe this is a regression from when I made a cleanup of 
how 32/64 bits and OS were then suddenly transported separately and both, so choosing MSVC properly 
to match Python would work. I believe I caused this as a regression then.

As for standalone, from the kind of paths I see used on MacOS, the -R $ORIGIN may simply be not 
necessary, try to make it conditional. Indeed, MacOS is not supported and you are very welcome to 
add it. I would hope it should be relatively straightforward. For the relevant run time code parts 
(binary to find itself, etc.) I have parts that should work on MacOS. 

I am going to put it as done-cbb, because the fiber implementation could surely be done for MacOS, 
and swapcontext may be slow on it (nor not). On x64 Linux, it was much slower, because it was making 
a useless syscall, where the whole idea would be to not do that. If you are interested, I can point 
you to a benchmark that will tell. Also, as Darwin and its libc are open source, we could easily 
check what the code does.
msg563 (view) Author: fpbhb Date: 2014-02-04.21:15:14
Have been trying the other tests. Ok so far: basics, syntax (obviously), programs, packages. 
The next failure is in "standalone". It starts with OS X ld not supporting -R. After fixing 
that, the code in "freezer" fails (no surprise, there is no code for OS X, which is 
sufficiently different to other Unices to require a separate implementation). I'll look into 
that and may be able to provide an implementation.

Perhaps the issue's title should be change to "A first look into OS X support" or similar ;-)
msg562 (view) Author: fpbhb Date: 2014-02-04.20:49:33
Am 04.02.2014 um 00:19 schrieb Kay Hayen <issue_tracker@nuitka.net>:
> In nuitka/build/SingleExe.scons change this:
> -   elif target_arch == "x86_64":
> +   elif target_arch == "x86_64" and "linux" in sys.platform:

Great, that works. Now everything in tests/basic/run_all.py passes!

> BTW: Can you give me what "sys.platform" is on your platform in MacOS, and maybe
> also what "objcopy -s" says in its last line "objcopy: supported targets:".

$ python
Python 2.7.5 (default, Aug 25 2013, 00:04:04)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.platform
'darwin'

On Mac OS there is no objcopy. There is an alternative OSS implementation called
objconv (of which I've never heard before digging today):

http://stackoverflow.com/questions/3475854/an-objcopy-equivalent-for-mac-iphone

What is the information you require/what are you intending to do with objcopy?

BTW, replying to your roundup instance via email doesn't seem to
work, saying "<issue_tracker@nuitka.net>: user unknown".
msg561 (view) Author: kayhayen Date: 2014-02-04.16:43:20
Bad news as in my FreeBSD was x86 and not x64, and currently my Debian won't 
execute FreeBSD virtual machines (temporarily). Which means, the diff from 
below, is likely what I will have to apply, assuming FreeBSD x64 would also be 
affected until I can prove otherwise.

I am aiming for this to be in a hotfix during the week.
msg559 (view) Author: kayhayen Date: 2014-02-03.23:19:32
The log doesn't look acceptable at all. Most of the failing tests, I know use 
generators, and fail completely.

In nuitka/build/SingleExe.scons change this:

-   elif target_arch == "x86_64":
+   elif target_arch == "x86_64" and "linux" in sys.platform:

and check again. The change in swapFiber.S should no longer be necessary. That 
will make that code "linux" only, and clearly not MacOS X. As I said, I will 
have to check FreeBSD to see if that limit applies. Didn't have the time for 
that yet.

BTW: Can you give me what "sys.platform" is on your platform in MacOS, and maybe 
also what "objcopy -s" says in its last line "objcopy: supported targets:".
msg558 (view) Author: fpbhb Date: 2014-02-03.09:08:37
Hello Kay, 

find attached the output of run_all.py; I've run the tests in a source
checkout from Github (this commit: 
https://github.com/kayhayen/Nuitka/commit/22a32ef0970c55891b345b581738dba596857e84)

It doesn't look like literally everything fails, but it is still a lot.
I'll try to find time to look into it (hints welcome!).
msg557 (view) Author: kayhayen Date: 2014-01-31.10:19:41
Hello,

I believe the file in question is designed for x64 Linux, and should not be used 
on x64 anything else. It may indeed work though, but it may also be just sheer 
luck, that you don't encounter problems. There would be generic code, that is 
only slower at worst.

But can I ask you to run ./tests/basics/run_all.py search in a source checkout 
with your fix, and tell me if all tests pass. Because that is going to excercise 
generators and let me know, if this is way to go.

You may well be the first to use Nuitka in x64 MacOS, but Nuitka itself it very 
portable code, and clang is a tested compiler, although I personally use it on 
Linux. The thing there is that I don't own a MacOS, and can't have a virtual one 
to my knowledge. The closest I get is supporting FreeBSD with clang I suppose, 
which I do.

So MacOS support is more user driven. But there isn't much to fear beyond 
technical stuff.

In this case, the check for usability of swapFiber.S may have to narrowed 
somewhat. I will check if my FreeBSD is x64, and if it works there.

The ABI of MacOS is different, and it may well be that a different set of 
registers may have to be saved and restored. Or it may be the same. We shall 
see.
msg556 (view) Author: fpbhb Date: 2014-01-30.18:00:54
I've tried to use nuitka on OS X 10.9.1 (meaning clang, x64 etc. -- is this even 
supported?). Compilation gave me:

clang -o hashtree2.exe -lstdc++ hashtree2.build/static/CompiledFunctionType.o 
hashtree2.build/static/CompiledGeneratorType.o 
hashtree2.build/static/CompiledMethodType.o 
hashtree2.build/static/CompiledFrameType.o 
hashtree2.build/static/CompiledCodeHelpers.o 
hashtree2.build/static/InspectPatcher.o 
hashtree2.build/static/x64_ucontext_src/fibers_x64.o 
hashtree2.build/static/x64_ucontext_src/swapfiber.o hashtree2.build/__constants.o 
hashtree2.build/__helpers.o hashtree2.build/module.__main__.o 
hashtree2.build/__constants_data.o -Lcython/lib -lpython2.7
Undefined symbols for architecture x86_64:
  "_swapFiber", referenced from:
      Nuitka_Generator_send(Nuitka_GeneratorObject*, _object*) in 
CompiledGeneratorType.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
scons: *** [hashtree2.exe] Error 1
scons: building terminated because of errors.

I fixed this by changing "swapFiber" to "_swapFiber" in swapfiber.S; after that it 
seems to work fine.

Thx for nuitka, BTW!
History
Date User Action Args
2014-06-23 07:44:58kayhayensetstatus: in-progress -> resolved
messages: + msg692
2014-04-25 18:46:03kayhayensetmessages: + msg635
2014-04-25 07:18:26fpbhbsetmessages: + msg634
2014-04-24 20:04:20kayhayensetmessages: + msg632
2014-02-08 14:42:00kayhayensetmessages: + msg566
2014-02-05 11:44:34kayhayensetmessages: + msg564
2014-02-04 21:15:14fpbhbsetmessages: + msg563
2014-02-04 20:49:33fpbhbsetmessages: + msg562
2014-02-04 16:43:20kayhayensetstatus: chatting -> in-progress
messages: + msg561
2014-02-03 23:19:32kayhayensetmessages: + msg559
2014-02-03 09:08:37fpbhbsetfiles: + run_all.log
messages: + msg558
2014-01-31 10:19:41kayhayensetstatus: unread -> chatting
assignedto: kayhayen
messages: + msg557
keyword: + compiler_crash, macos
nosy: + kayhayen
2014-01-30 18:00:54fpbhbcreate