pypy svn jit cheat sheet.

-
A quick cheat sheet for trying out pypy jit. This is for a ubuntu/debian 32bit machine translating the jit version of pypy-c.
# install dependencies for debian/ubuntu.
sudo apt-get install python-dev libz-dev libbz2-dev libncurses-dev libexpat1-dev libssl-dev libgc-dev libffi-dev

# download from svn.
svn co http://codespeak.net/svn/pypy/trunk pypy-trunk

# Translate/compile the jit.  This part can take a while.
#   Don't worry. Relax have a home brew.
cd pypy-trunk/pypy/translator/goal
./translate.py -Ojit targetpypystandalone.py

# Or for the low memory pypy-c use this translate instead...
#./translate.py --gcremovetypeptr targetpypystandalone --objspace-std-withsharingdict
The pypy getting started documentation has more about it if you're interested.

pypy has most standard modules up to python2.5... and some from newer versions.

I didn't need any external compiled extension modules... just sqlite, and cherrypy. So for this project I could use pypy! *happy dance*

Didn't take long to port my code to it. I only had a couple of issues, which people in the #pypy channel on freenode helped me with.

There is an alpha version of the sqlite3 which uses ctypes included with pypy as the 'pysqlite2' module. Seemed to work well enough for me, and passed all my tests.
import sqlite3
# for pypy, since the sqlite3 package is empty... but they have a pysqlite2.
if not hasattr(sqlite3, "register_converter"):
from pysqlite2 import dbapi2 as sqlite3
Another issue I had, was I was using 'x is not y' to compare two integers in one function. In cpython they use a hack so that the first 100 numbers or so share the same identity. Since the numbers were always less than 100, this code was working fine in cpython. However, pypy doesn't have that problem/feature - so I just used the != instead.

I think those were the only two things I had to change. All my tests were passing, and it was the end of the day, so I went down the pub to see a friend visiting from Tokyo who was having a birthday.

Comments

holger krekel said…
nice write up! do you plan to do some benchmarks as well?
1:27 PM
René Dudfield said…
No I wasn't planning on it... last post I did that, it didn't go down well with some people and made them angry. Less angry people in the world is better :) Also it takes quite a while to do.

But you can try:
pypy-c -m cherrypy.test.benchmark

To see how well different pypy translations do with a threaded webserver.

At some point I'll measure memory usage, since for the project I'm going to try pypy with, using as little memory as possible is a goal. So each separate web server does not use up much memory on the 512MB of ram server. I'm hoping pypy will do better than cpython in this area.


cu.
1:40 PM
Post a Comment

Popular posts from this blog

Draft 3 of, ^Let's write a unit test!^

-
Image
So, I started writing this for people who want to 'contribute' to Community projects, and also Free Libre or Open source projects. Maybe you'd like to get involved, but are unsure of where to begin? Follow along with this tutorial, and peek at the end in the "what is a git for?" section for explanations of what some of the words mean. Draft 1, 2018/07/18 - initial draft. Draft 2, 2019/11/04 - two full unit test examples, assertions, making a pull request, use python 3 unittest substring search, "good first issue" is a thing now. Started "What is a git for? Jargon" section. Draft 3, 2020/12/12 - default branch name is main. What's first? A test is first. A unit test is a piece of code which tests one thing works well in isolation from other parts of software. In this guide, I'm going to explain how to write one using the standard python unittest module, for the pygame game library. You can apply this advice to most python pr...
Read more

Is PostgreSQL good enough?

-
Image
tldr; you can do jobs, queues, real time change feeds, time series, object store, document store, full text search with PostgreSQL. How to, pros/cons, rough performance and complexity levels are all discussed. Many sources and relevant documentation is linked to. Your database is first. But can PostgreSQL be second? Web/app projects these days often have many distributed parts. It's not uncommon for groups to use the right tool for the job. The right tools are often something like the choice below. Redis for queuing, and caching. Elastic Search for searching, and log stash. Influxdb or RRD for timeseries. S3 for an object store. PostgreSQL for relational data with constraints, and validation via schemas. Celery for job queues. Kafka for a buffer of queues or stream processing. Exception logging with PostgreSQL (perhaps using Sentry) KDB for low latency analytics on your column oriented data. Mongo/ZODB for storing documents JSON (or mangodb for /dev/null replacemen...
Read more

post modern C tooling - draft 6

-
Image
Contemporary C tooling for making higher quality C, faster or more safely. DRAFT 0 - 10/11/18,  DRAFT 1 - 9/16/19, 7:19 PM, I'm still working on this, but it's already useful and I'd like some feedback - so I decided to share it early. DRAFT 2 - 10/1/19, mostly additions to static analysis tools. DRAFT 3 - 10/4/19, updates on build systems, package management, and complexity analysis.  DRAFT 4 - 10/6/19, run time dynamic verification and instrumentation, sanitizers (asan/ubsan/etc), performance tools, static analyzers. DRAFT 5 - C interpreter(s).  DRAFT 6 - 11/6/19 , mention TermDebug vim,  more windows debugging tools, C drawing for intro. In 2001 or so people started using the phrase "Modern C++". So now that it's 2019, I guess we're in the post modern era? Anyway, this isn't a post about C++ code, but some of this information applies there too. No logo, but it's used everywhere. Welcome to the post modern era. S...
Read more