Jorj X. McKie

Hello everyone,

I am Jorj X. McKie (this is not my real name!) and I am an enthusiastic supporter of Nuitka. For the time being, my focus area within this package is standalone compilation and maintenance of the plugin feature.

Apart from extending Nuitka's support to virtually all Python packages living out there, my driving motivation in this respect is keeping up and improving the usability of Nuitka.

You may want to look into the Nuitka Utilities repository, which contains many things I am working on.

images/jorjmckie-harald-lieder.jpg

Short Biography

  • Born 1951 as Harald Lieder near the city of Leipzig

  • Studied Mathematics and Physics at the Frankfurt/M Johann-Wolfgang-Goethe University from 10/1970 through 06/1976 (Diploma in Mathematics)

  • Professional career overview
    • insurance company (4 years): application and systems programming (IBM mainframe, Cobol, Assembler), data center lead

    • book wholesaler (5 years): systems programming and data center lead (IBM mainframe)

    • consulting company (until 2009): leading of or participating in numerous large and very large projects mostly around banking applications, banking mergers, data center optimization and quality assurance.

    • since 2009: retirement, living on Isla Margarita, Venezuela. Enjoying my beautiful house, pursuing my hobbies: mathematics, science theory, biological evolution theory, science fiction (my pseudonym comes from novels by one of my favorite authors Frank Herbert) and of course programming (maintainer of PyMuPDF, and Nuitka).

Read more…

Tommy's Google Summer of Code Intro

Hi all! :)

My name is Taofeng Li and I go by the nickname of Tommy, github handle tommyli3318. I'm a Computer Science major at the University of California, Irvine in the United States. I am very excited to announce that I'll be contributing to Nuitka this summer under the Google Summer of Code program!

My project is "Nuitka support for PyPI top 50", it ensures Nuitka’s compatibility with the 50 most popular PyPI packages by setting up automated testing for each package. These automated tests will serve to be very important tools for the development of Nuitka, as they will be used to assure Nuitka’s compatibility with the most used Python packages every time Nuitka receives a new update.

My mentors will be Vaibhav Tulsyan, Kamran Ghanaat, and Kay Hayen. My project abstract can be seen here and the full details of my project can be found here.

I am greatly looking forward to the many things that I'll learn this summer!

Batakrishna Sahu

Hello everyone!

My name is Batakrishna Sahu and I am from India. I'm very grateful to be one of the 2 selected students who will work with Nuitka during Google Summer of Code 2019. My project is on "Optimizing Python 2 and Python 3 built-ins for Nuitka" mentored by Vaibhav Tulsyan, Kamran Ghanaat, Kay Hayen.

So far, ​there are a total of 69 built-ins in Python 3 and ​76 built-in in Python 2 out of which 27 built-ins are yet to be optimized in Nuitka​. My aim is to optimize those missing built-ins in Nuitka. This will make Nuitka a lot faster while working with built-ins.

I'm really excited to gain new experience while working on this project this summer.

Yours,

Nuitka this week #13

Communication vs. Coding

Communication was a lot more, just not these postings. Nuitka has ventured into important realms.

First, active developers have joined Nuitka. Second, as a consequence of the later, Nuitka indeed was able to participate with a insanely powerful offering of 4 mentors.

This meant, that a lot of technical or project management debt hat to be dealt with, and there was no time to make this kind of posting. This is probably good news on all fronts, except that I feel they are missing, and am glad to be able to resume them.

Google Summer of Code 2019

My Experience

This was the most crazy ride and fun. Became the admin of an active sub-organisation under the PSF umbrella for GSoC 2019. So shortly after the announcement of the project lists, students flooded in, and starting working on things right away as early as February.

They were working hard to prove themselves and get to know Nuitka, asking many questions, with us having lots of email interviews, and even a lot of video calls, where I personally talked to people about ideas.

This happened simultaneously to more experienced developers joining the project as well, making very important contributions.

This basically meant, that many days 90%-100% of my Nuitka time was for communication or working off technical debts I knew there were, or for preparing things, I want to have with multiple people working on the code base. This will be detailed in other sections.

Accepted Students

All of this has paid of in that we now have 2 really good students to work on 2 Nuitka projects over the summer. Let's welcome Batakrishna and Tommy in separate posts. They will introduce themselves and what they are going to do shortly, in separate posts.

Lets just say, this is fantastic news. We had so many good applications and its a shame, but we knew right away, not everybody who deserved it could be picked. But this is going to be a good chance for us to get to be open and welcoming to new people.

Optimization Work

Core Stuff

So, the 0.6.3 release (btw. on Windows, be sure to use the 0.6.3.1 hotfix), which was made as a consolidation effort to get the good work of mostly other people out, didn't contain much optimization work for the core, as that is still my thing.

However, this changed a lot. An idea that came to my mind for how to do the massive amounts of specialized helpers needed beyond + and +=, with which I had started for prior releases. And that is to use Jinja2 based templates for C, to generate the code.

This was an important idea. Took a while, but soon the manual code for + was already replaced with generated code, fixing a few bugs by the way, and from there, the generation was expanded to cover * as well.

Currently, support for the 3 (!) different kinds of additions (TrueDiv and FloorDir as well as Python2 default division, dubbed OldDiv in Nuitka was added along with - .

The reason, + and * were done first, is that they have special treatment for sequences, using sq_concat and sq_repeat, where the other operations will be more straightforward, e.g. nb_subtract (-) has a lot types supporting it and that makes those the easy cases.

I am saving a deeper explanation of 3 things we will need for the next time. Basically we need optimization of these things at compile time, and that is getting there, and code to use in the backend, and that is getting there, and a third thing, that is to use optimization knowledge to apply the special code as much as possible, and that is not yet fully there.

Faster Windows Dependencies

This is going to excite Windows users. After Orsiris de Jong had done a replacement for dependency walker that is faster, this had remained in an experimental status, just due to lack of time.

Recently however, I felt there is more time, after GSoC student selection has happened, and that I could finally work a bit on open issues like this. And when I wrote a dedicated tool, to analyse dependences with either technology to compare the results, I found that dendency walker founds a lot more things.

That was a turn down, but turns out, nothing it finds is stuff that should not be on the white list. In fact, it's all core Windows things, and from the System32 folder. That made me question, why we take anything from there (except maybe PythonXY.dll) at all, and after that change the performance changed dramatically.

The dependency walker now finishes a file in milliseconds. Actually the pefile is now slow (surely it ought to be compiled), and takes some seconds, for a file. That is amazing, and has lead to me to remove the parallel usage, and since pefile allows for perfect caching, and is Free Software, we will probably keep it.

This will address a widespread complaint of many Windows users of the standalone mode. This is now a relatively unnoticable part of the overall experience.

Currently I need to finish off some remaining problems with it, before putting it out in the wild. Getting this into a release will solve many newcomer issues.

Nuitka Organisation

Esp. for Google Summer of Code, Nuitka has sought and found mentors, some of which are highly experienced for the task. I will let them decide and write their own introduction, but I feel really blessed by them helping me out in my desperate calls for help. Without them, neither could Nuitka participate, nor could it even overachieve as much as it does.

Therefore I welcomed Kamran and Vaibhav into the organisation and they are excited to work the the 2 accepted students, that are also added.

Plugin Work

On the jorj branch there is a lot of work from Jorj that aims at adding support for more of the beasty stuff with hidden dependency and plugin needs.

He is also working at run time tracing of your program to be translated to automatic imports of just that. I am going to highlight this later, once I manage to cherry-pick the release ready parts from it for the next release.

But this definitely awesome stuff, and going to make Nuitka very easy to use for some people, even with stranger software.

Opening Up

This is also a teaser. But we did so much work for the 0.6.3 release to make sure information is there, and things are accessible and changeable by everyone, or even the pre-commit hook that I am very proud of.

But I shall save this for next week, otherwise it will be too much new information.

Twitter

I continue to be active there, although often I fall prey to of not wanting to talk about unfinished things. On Twitter of all things.

Follow @kayhayen

And lets not forget, having followers make me happy. So do re-tweets.

Adding Twitter more prominently to the web site is something that is also going to happen.

Help Wanted

If you are interested, I am tagging issues help wanted and there is a bunch, and very likely at least one you can help with.

Nuitka definitely needs more people to work on it.

Donations

If you want to help, but cannot spend the time, please consider to donate to Nuitka, and go here:

Donate to Nuitka