pypy sprint

Been doing a bit of reading this morning on pypy things.

Trying to prepare myself a bit for the coming 'winter sprint'. One topic I'm interested in for the sprint is finishing off remaining issues for pygame running on pypy. Probably not the most common reason for people to go to the Swiss mountains I guess! So this means learning about the cpyext part of pypy... the one which allows it to use the python C API, and modules that use it.

Open source sprints are great. A bunch of people get together trying to make things. A very rare time when people get to meet other people-from-the-internet. It's a great place to learn, and to see how projects you're interested in are made.

I guess I'm a little bit too excited, and started playing with pypy early. Below are some notes from my investigations finding out how to do things.

Getting started, finding docs.

There's a getting started guide for pypy which has some basic notes on running tests, and compiling stuff.

Also, cython has a pypy section in their user guide - http://cython.readthedocs.io/en/latest/src/userguide/pypy.html

cpyext is the module which implements the python C API in pypy.

A wiki page called 'c-api' contains some documentation including a talk on cpyext. It's a little bit dated (from 7 years ago), but still interesting reading(and watching).

This wiki page was interesting:  "In which we document cpyext compatibility progress, this time with cython and pandas."
https://bitbucket.org/pypy/pypy/wiki/cpyext_2_-_cython%20and%20pandas

It's a blog of development of pypy cpyext getting numpy and pandas features working. From this I noticed a few interesting techniques... like running tests separated processes. Very useful when debugging. There is also an older document called 'Adventures in cpyext compatibility (ended)'. With even more notes on issues that came up, along with their work arounds.

Of course the Python C API docs are also useful when one wants to do Python C API things.

Use the source (Luke).

But the best source of information on cpyext seems to be the source code itself.
https://bitbucket.org/pypy/pypy/src/default/pypy/module/cpyext/

Lots and lots and lots of tests. And lots and lots of code. Bed time reading perhaps.

It seems to be a whole implementation of CPython, with bridges into pypy. But with tests. Lots and lots of tests for the python C API.

What are useful debugging techniques?

CPython has a thing called a debug build which is useful.
https://docs.python.org/3/c-api/intro.html#debugging-builds

I don't know of any special debug tools for pypy (gdb extensions or such like).

Wheels on pypy?

I found this page "pypy binary wheels for some popular packages", and a blog post about it.

It seems pypy doesn't compile on the ancient centos 5 that many linux uses. Perhaps the new manylinux which uses a newer linux will work.

It seems the piwheels project (compiles wheels for raspberry pi automatically) doesn't have pypy support yet. piwheels is really great, in that it makes lots of ARM builds available so people don't need to compile stuff themselves on the raspberrypi.

No idea if windows, or mac binary wheels work for pypy. Without binary wheels for these platforms, most pypy users won't be able to install a C based library like pygame on pypy.

Which platform?

It seems linux amd64 is the best supported, but I'm also interested in Mac, and Windows builds.
Especially setting them up on Appveyor, and Travis /Mac.

Translating on windows is the document which describes the windows builds. Not sure my cheap windows laptop has enough memory for it. It's only got 4GB of ram, and I expect it's going to take hours to compile.

The buildbot contains Nightly builds for windows as well as other platforms. So I might try using them to debug stuff, and I guess do extension development. Failing that, maybe I can copy a build off someones computer!

Hopefully I won't need to hire some remote servers for compiling stuff. It doesn't seem like I'd need to be recompiling all of pypy each time for extension development.