Sunday, January 10, 2010

pypy svn jit cheat sheet.

A quick cheat sheet for trying out pypy jit. This is for a ubuntu/debian 32bit machine translating the jit version of pypy-c.
# install dependencies for debian/ubuntu.
sudo apt-get install python-dev libz-dev libbz2-dev libncurses-dev libexpat1-dev libssl-dev libgc-dev libffi-dev

# download from svn.
svn co pypy-trunk

# Translate/compile the jit. This part can take a while.
# Don't worry. Relax have a home brew.
cd pypy-trunk/pypy/translator/goal
./ -Ojit

# Or for the low memory pypy-c use this translate instead...
#./ --gcremovetypeptr targetpypystandalone --objspace-std-withsharingdict
The pypy getting started documentation has more about it if you're interested.

pypy has most standard modules up to python2.5... and some from newer versions.

I didn't need any external compiled extension modules... just sqlite, and cherrypy. So for this project I could use pypy! *happy dance*

Didn't take long to port my code to it. I only had a couple of issues, which people in the #pypy channel on freenode helped me with.

There is an alpha version of the sqlite3 which uses ctypes included with pypy as the 'pysqlite2' module. Seemed to work well enough for me, and passed all my tests.
import sqlite3
# for pypy, since the sqlite3 package is empty... but they have a pysqlite2.
if not hasattr(sqlite3, "register_converter"):
from pysqlite2 import dbapi2 as sqlite3
Another issue I had, was I was using 'x is not y' to compare two integers in one function. In cpython they use a hack so that the first 100 numbers or so share the same identity. Since the numbers were always less than 100, this code was working fine in cpython. However, pypy doesn't have that problem/feature - so I just used the != instead.

I think those were the only two things I had to change. All my tests were passing, and it was the end of the day, so I went down the pub to see a friend visiting from Tokyo who was having a birthday.


holger krekel said...

nice write up! do you plan to do some benchmarks as well?

illume said...

No I wasn't planning on it... last post I did that, it didn't go down well with some people and made them angry. Less angry people in the world is better :) Also it takes quite a while to do.

But you can try:
pypy-c -m cherrypy.test.benchmark

To see how well different pypy translations do with a threaded webserver.

At some point I'll measure memory usage, since for the project I'm going to try pypy with, using as little memory as possible is a goal. So each separate web server does not use up much memory on the 512MB of ram server. I'm hoping pypy will do better than cpython in this area.