Posts

Showing posts from January, 2010

good bye Vivienne

will miss my little sister. wish we had more time together. so sad. shit

Jquery 1.4

Image
Another quality jquery release... jquery 1.4.

Some very handy changes, my favourite probably being able to use functions to set values. In short, some of the changes are:

setting values with functionshtml5 form supportbetter JSON/JSONP and ajax functionalityeasier element construction (just pass a dict of attributes)reverse indexing like in python, where -1 is the last item.better animation for multiple attributes at onceperformance improvements, code cleanups, and bug fixing

Full details in the jquery 1.4 release notes.

worker pool optimization - batching apis, for some tasks.

Image
What are worker pools anyway?Worker pools are an optimization to the problem of processing many things in parallel. Rather than have a worker for every item you want to process, you spread the load between the available workers. As an example, creating 1,000,000 processes to add 1,000,000 numbers together is a bit heavy weight. You probably want to divide it up between 8 processes for your 8 core machine. This is the optimization worker pools do.

With a usual worker pool there is a queue of jobs/data for the workers to work on. The workers are usually threads, processes or separate machines which do the work passed to them.

So for 1000 items on that queue, there are around 1000 accesses to that queue. Usually the queue has to be thread safe or process safe, so that pieces of data are not sent to many workers at once.

This can be an efficient method to use for some types of data. For example, if each job can take different amounts of time, like IO tasks over the internet... this is…

mini languages that non programmers can understand

Image
There are hopefully a number of mini text based programming languages that non-programmers can understand. But what are they?

One that I've used in the past is something like this:

Which would parse into a python/javascript data structure like this:
{name: 'Bob',
gender: 'male/female',
age:'22'
}
It's suprisingly common in things like search engines. Grandmas who occasionally check their email might not get it (but many do I'm sure!)... but I think a lot of others do. For things like search it is ok, if people know the magic 'terms'. If they do not know the terms, then they can just enter text to search normally. The mini language is used by advanced users.

This is quite good for single line free form data entry. Since people only need to know the concept that you have 'key:value'. It's slightly easier than using urls, since separators can be different things.

csv files
Next up are comma separated files - csv.
For example:
name,gender…

pypy svn jit cheat sheet.

Image
A quick cheat sheet for trying out pypy jit. This is for a ubuntu/debian 32bit machine translating the jit version of pypy-c.
# install dependencies for debian/ubuntu.
sudo apt-get install python-dev libz-dev libbz2-dev libncurses-dev libexpat1-dev libssl-dev libgc-dev libffi-dev

# download from svn.
svn co http://codespeak.net/svn/pypy/trunk pypy-trunk

# Translate/compile the jit. This part can take a while.
# Don't worry. Relax have a home brew.
cd pypy-trunk/pypy/translator/goal
./translate.py -Ojit targetpypystandalone.py

# Or for the low memory pypy-c use this translate instead...
#./translate.py --gcremovetypeptr targetpypystandalone --objspace-std-withsharingdict
The pypy getting started documentation has more about it if you're interested.

pypy has most standard modules up to python2.5... and some from newer versions.

I didn't need any external compiled extension modules... just sqlite, and cherrypy. So for this project I could use pypy! *happy dance*

Didn't take long …

Unladen swallow review. Still needs work.

Image
Tried out unladen swallow on two work loads today. After the announcement they are going to try and bring it into core. So I finally got around to trying it (last time the build failed). An hour or so later, the build finally finished and I could try it out. The C++ llvm takes ages to compile, which is what took most of the extra time. What follows is a review of unladen swallow - as it stands today.

The good part? Extensions work(mostly)! w00t. I could compile the C extension pygame, and run things with it.

Now to run code I care about, my work loads - to see if their numbers hold true for me.
cherrypy webserver benchmark: crash
pygame tests: some crashes, mostly work.
pygame.examples.testsprite : random pauses in the animation.


The crashes I've found so far seem to be thread related I guess. The cherrypy one, and some of the pygame ones both use threads, so I'm guessing that's it.

Random pauses for applications is a big FAIL. Animations fail to work, and user interact…

Using a html form as the model.

Image
I've mentioned this technique before, but I think it is worth repeating more clearly. Without any other code clouding the main message: 'All you need is love html.

A HTML form can describe a model of your data quite well. This lets you go from HTML form design to a working Create Read Update Delete (CRUD) system. A CRUD can be thought of as a nice admin interface to edit your data (or a CMS if you prefer that acronym).

For example: <form action='savepage' method='post'>
title:<input type='text' name='title'>
<br>textarea:
<textarea name='content'></textarea>
<input type='submit' name='submit'></form>
That's all you need to create a CRUD. Things like validation can be defined in the html easily enough then implemented in your code to be checked server side, and client side. Parse the form to get the model, and go from there.

The benefits are simplicity, and that designers can make…

Multiple file uploads with html 5, cherrypy 3.2 and firefox 3.6beta5.

Image
Here's an example for uploading multiple files with HTML 5 and newer browsers, like the firefox 3.6 beta5. You can shift/ctrl select multiple files, and even drag and drop them in. Makes for a much nicer user experience uploading files - rather than having to select one at a time, or having to load some java/flash thing.

It uses the unreleased cherrypy 3.2, with it's new request entity parsing tool hooks. See http://www.cherrypy.org/wiki/RequestBodies for details about the new control allowed over the whole process. It's a lot easier to make custom request entity parsing behaviour now, and in a much less hacky way than before.

http://rene.f0o.com/~rene/stuff/cherry_multi_fileupload.tar.gz

With the tool in there, files come in as a list of files instead.

Oldest python file in your home directory?

Image
Feeling just a little nostalgic this time of year.

Just made a little script to find the oldest python files on your hard drive.
http://rene.f0o.com/~rene/stuff/oldest_python.py

oldest_python.py [path]
oldest_python.py mystuff/python
oldest_python.py
oldest_python.py ~
Update: Lennart mentions a unixy way in the comments of finding oldest files with this: find . -name '*.py' -printf "%T+ %p \n" | sort | more
With that I found some really old python files of mine... woh! the oldest ones dated in 1998. There are older C, C++, haskel, javascript, java, pascal, prolog, asm, sql, perl, etc, etc and heaps of other old junk files, but those are the first ones I could find written in python.

I guess that means I've been programming python for around 12 years now. Python was at version 1.4 or so, and 1.5 was released not long after. New style objects did not exist, and it was not all too uncommon to be able to segfault the interpreter (ping could easily crash linux & windo…

Fifth London code dojo 6.30pm Thursday 7th of Jan.

Here's the london dojo google calendar for much of 2010.

More details, and signup here: http://ldnpydojo.eventwax.com/5th-london-python-code-dojo/. The idea is to practice, learn and teach python skills.