making apps, making webs.

Very often I'm sitting next to a HD teevee, and people say 'that looks weird'. Why? There's a number of reasons, but the two main ones are: when compared to SD digital teevee is: Poor colour depth compared to the number of pixels . For a person used to analog teevee (PAL, or NTSC) - and not digital teevee, a major weirdness is the digital compression used. Digital teevee is compressed Compared to SD digital teevee - No increase in colour richness. They increased the number of pixels shown per cm (or inch), but did not increase the richness of colour displayed. On PAL for example, you may have 422 colour depth - which is 8 bit colour. If you come from the computer world you may think that is 8 bits per channel colour (RGB 888). However in the broadcast world they use the YUV colour space, and this is usually only 8 or 10 bits per pixel. 4 bits Y, 2 bits U, 2 bits V == 8 bits per pixel. If the colour depth is the same, then why does it look weirder in HD? Imagine a

Dear reader, I joined the iOS web developers for adventure and a chance to see the world but instead I am working in a mud hole, freezing my arse off, with a constant fear of death. I am writing this note in hope that it makes it out of the trenches. In case I do not. It's bloody here in the iOS trenches and I feel my days are numbered. With these thoughts on my mind, I hope to share this with you. Some of this stuff is not documented in the standard issue manual, or disseminated via the standard propaganda channels. I feel it ought be of use to you. Unfortunately it has been very hectic here, so the words will likely be rushed and detail will be lacking. I apologise for this, but I still think it will be useful (no brain rockets, just some Damn Useful Information). In case I don't make it, Your friend from the trenches, René Dudfield. ps. if you find this note, please consider commenting on the back with any other useful information your fellow iOS trench mates might fi

I've begun 'tweeting' python packaging tips . I'm hoping to go up to 100 useful python packaging tips. If you're a twit too, please feel free to join in on the conversations . update : I wrote a little script to download all the python packaging related tweets from twitter. For some reason twitter is only giving me 32 of them... but I've written about 40 so far. Going to run through these at the london python dojo tonight in a mini talk. setup.py build_ext --inplace to compile your extensions inside the source directory. Good for developing inplace distribute 'setup.py develop' Installed pkg points to dev copy, quicker changes during dev. develop -u to remove For pyrex/cython/swig packages, include the generated C code so people do not need to install cython/pyrex/swig. For debugging info set DISTUTILS_DEBUG. os.environ['DISTUTILS_DEBUG'] = "on" OR export DISTUTILS_DEBUG=on Don't put code in package/__init__.py file. Makes debu

pokket mixer I went to the big market at the Berlin park today, and saw this little sound mixer there. This dude and his girlfriend who were at the stall make them! They are his design too. Really cool buying electronics from the people who make them. Did I mention I like buying small things that can fit in my pocket, and in my carry-on luggage? It's passive - so it does not need to be powered. It's also very small. Seems to work quite well. The eq seems to work ok. It has the normal Hi, Mid, Lo for each of the two channels. The sound quality seems pretty good (even when going out to high quality studio speakers). I've bought more expensive mixers with worse audio quality.

Dear Lazy web, Are there any 3G usb modem plans available in Finland that I can get for one week or a month? I'd need to buy the modem too. Hopefully it should work two hours drive from Oulu. Update: used a modem by Elisa. Worked well, even 3g. Closing comments because this post is getting spams every day - and blogger.com spam protection seems broken.

I just put up some code for doing templating with javascript(and jquery). Either on the server side or on the client side. If you open the html in your browser, it runs on the client side. If you first process it server side... then it runs server side(but not on the client side). http://github.com/illume/nodejs_jquery_templating This is a followup from you are using the wrong templating language - as a proof of concept. Why is this useful? No need for a server to process the templates. Either process them server side or client side. Data can be stored in a json file. No need for a database for testing. Just create json files in a text file. Can reuse knowledge of javascript libraries (like jquery), rather than learning one of 798394 different templating languages. Can keep one html file which front end web developers can edit without them needing a new template file. Can use jquery plugins on server side too(validation, etc). Seems to work ok so far... but it still needs a lot

Let's make a shit javascript interpreter! Part one. As a learning exercise, I've begun writing a javascript ECMAScript interpreter in python . It doesn't even really exist yet, and when it does it will run really slowly, and not support all js features. So... let's make a "from scratch", all parsing, all dancing, shit interpreter of our very own! Teaching something is a great way to learn. Also writing things on my blog always gets good 'comments', hints, tips, plenty of heart, and outright HATE from people. All useful and entertaining :) Tokenising So to start with, we need something to turn the .js files into a list of tokens. This type of program is called a tokeniser. From some javascript like this: function i_can_has_cheezbrgr () {return 'yum';}; Into a Token list something like this: [ {"type":"name", "value":"function", "from":0, "to":8}, {"type":"n

Over the last few months I've been fairly deep into javascript land. Both in my full time job, and in most of my coding side projects - I've been mostly doing javascript. Of course, like most web programmers I've been dabbling in javascript over the years... but never so intensely. On the way I've been collecting a 'reading list' of videos, articles, books, and projects. Here are some of the good links from the last few months. Crockford on javascript videos: http://www.yuiblog.com/blog/ 2010/04/08/video-crockonjs-5/? utm_source=feedburner&utm_ medium=feed&utm_campaign=Feed% 3A+YahooUserInterfaceBlog+% 28Yahoo!+User+Interface+Blog% 29 akihabara arcade: http://www.kesiev.com/ akihabara/ canvas javascript games: http://www.benjoffe.com/code/ javascript 3d engine: http://github.com/mrdoob/ three.js Aves Engine: HTML/ JavaScript Game Engine ( youtube.com ) http://www.youtube.com/watch? v=Ol3qQ4CEUTo http://ajaxian.com/archives/ aves-game-e

Use javascript + json + DOM for html templating. Front end developers know how to use this technology. They know this technology very well, so will be way productive. No weird new template language to learn and install on every single project. No dependency on a rapidly changing server side backend... or database, or 3rd party API required. No taking the html, and turning it into [$%%WEIRD TEMPLATE LANG%%$$] step. Instead you can use javascript, json and the DOM. Being able to use the templating system either via the browser, or via the server side means there is no dependency between the front end and the back end. Front end developers just need an example json file to work from. Then the back end developer just needs to create the json file. It does mean you are dependent on a DOM+javascript implementation on the server side if you want to do things like server side caching, and supporting Agents(browsers, bots etc) that do not use javascript on the client side. Just json files.

Continuing on from a previous article I wrote earlier in the year... ' Using a html form as the model ', this post describes using validation. With html 5 comes a whole bunch of validation you can use in your forms. This makes a html form even more useful as a schema than the html 4 forms. Since you can use types like email, telephone, time, date, and such. You can also use things like minimum, and maximum length. As well as the ever useful 'required' field. You can read about html 5 forms here .

Ctrl+c, as well as a single handed touchpad move is considered harmful. Try and do combinations of keys, mouse moves, and gestures one finger at a time. Why? RSI. RSI hurts people. Please consider people when designing your user interfaces. Either avoid single handed moves, or warn against them. Thanks. When using computers, consider resting regularly, stopping when you hurt(and before you hurt). Also consider using one hand or finger for each action. So rather than shift+A done with one hand, try doing it with two hands. Also consider moving your hand away from qwerty middle keys do get a less stretchy press of the shift/ctrl/alt/cmd keys. Thanks. For more details on RSI, and RSI prevention, please see: http://www.rsi.deas.harvard.edu/spread.html .

http://duckduckgo.com/

Is there a way to find the memory usage of python processes? Trying to find some portable way of doing this. However, so far I think a new module might be needed... I've got linux mostly covered, but maybe you know how with freebsd, OSX, windows(9x-7)? So is there something built into python already? Is there a X-platform third party module already? Or a module just for one platform available? update: here's the linux code I found and cleaned up a bit memory_usage.py if anyone is interested. bytes_resident = memory_usage.resident(). It reads /proc/PID/status... eg, like "$ cat /proc/PID/status | grep VmRSS" would. pympler : 'Pympler is a development tool to measure, monitor and analyze the memory behavior of Python objects in a running Python application.' psutil : 'psutil is a module providing an interface for retrieving information on running processes and system utilization (CPU, memory) in a portable way by using Python, implementing many functio

MacOSX can sometimes corrupt a drive (like most OSen). I've seen it happen a few times, in a couple of different ways. One way is if you interrupt some file transfers. Like by pressing 'stop' on a big transfer - this can trash your partition table. You'll likely see vfs errors in your log at this point. Anyway... linux, ubuntu and 'testdisk' came to the rescue. testdisk found the partition for me, and wrote it back... luckily it worked and my friend and I did a happy dance. The HFS+ partition was saved.

Ever wanted to get your multi media keys to do something other that play multi media? You could get these key events any number of ways. One way is through dbus. Here is an example: gnome_multimedia_keys.py Here is a version which does not block your mainloop. So it's useful for integrating with other libraries (like pygame :) I made a dbus with pygame example too.

pygame already has a sub-surface, which is part of a larger surface. Sub-surfaces refer to the same pixels as the surface they come from, and share the same Interface as a Surface. It's good for doing sprite sheets, where you save one file with many smaller images - but then being able to manipulate them as if you loaded the images as smaller separate surfaces. However, sometimes we would like to operate on a whole bunch of smaller surfaces stuck together. This is what I'd like to call a Super Surface - a collection of smaller surfaces which can act as one big surface. It's a complementary idea to the sub-surface, and intuitively it should be there... but it's not yet. Unfortunately it's a lot harder to code a Super Surface compared to a sub-surface. Since a Super Surface would need to have all the surface affecting routines changed to work with it. For example, everything in the draw modules would need to be redone. So would all of the surface methods. So whe

Why bzr and launchpad? Bzr AND Launchpad are open source. launchpad: open source github: closed source sourceforge: closed source (was open source in the past) bitbucket: closed source googlecode: closed source You can submit changes to launchpad at: dev.launchpad.net . As well as ( submit launchpad bugs ) and feature requests against it if you don't want to make the patch yourself. When there is a good open source alternative, I always choose the good open source option. I initially had problems with bzr a couple of years ago... but it has been quite good to me over the last couple of months. So I'm moving over all of my projects from other version control hosting services to launchpad. Of course bzr and launchpad are also written in python (with selected optional C optimizations), so that makes for happy hacking :) update: reported a bug here about the 'can not find source code easily on launchpad' issue. Any other issues with launchpad?

Details, and booking here: http://ldnpydojo.eventwax.com/7th-london-python-code-dojo """What is a coding dojo? This is a coding dojo . Last time we attempted to refactor the various adventure game solutions from the January dojo. Whilst interesting, perhaps refactoring isn’t that exciting an activity for a dojo :-). Nevertheless, people seemed to be having fun and we did achieve our goal of a “one true” adventure game code base that allows us to define and navigate around a game world. The code can be found here: http://github.com/ntoll/code-dojo/tree/master/adventure/week2/ . After discussion at the end of the February dojo (and later in the pub) we decided that this time round we’re going to try another small-groups based exercise with a “show and tell” at the end as we continue to build the world’s greatest adventure game. Problems we might want to tackle include: a command parser, keeping track of game state/score/objects, NPCs/AI, authoring tools, turning it

Made a quick little logo for a website called... Tried a few different versions... showed my girl friend a few, and stuck with this one in the end. Made it with a simple DejaVu Sans Mono Bold font. Ya for the DejaVu font project. Modified the zero(0) a bit, then inverted the colors (ya! negative space). Played around with the letter spacing... applied an old school guassian blur filter to the right side, resized for web... and done! What do you think? The brief is 'something quick for yet-another-culture blog/zine called uh0h... as in uh oh I dropped a hammer on my foot.'. update: Added some results in this image below. Which is updated with a pygame/freetype script. The idea is that the results in the image can be updated without the rss feed getting updated. Hoping to clean the script up and add it to pywebsite. The same technique seems to be used by some wordpress blogs with their 'comments:3' image links - so I think it will be useful for all sorts of things

Subversion(svn) merging is easy... iff you are using a modern svn version (1.6.x). Here it is in short: $ cd /dir/of/your-branch $ svn merge ^/trunk Where ^/trunk is the url of the branch you want to merge from. For more details, have a look in the svn book basic merging section. Also this article on svn merging explains it fairly well. The svn 1.6 release notes and 1.5 release notes also talk about various svn updates including merge enhancements amongst other goodies (like improved python bindings). Really, merging is not too bad in subversion now. I know this post won't stop everyone from using 2004 reasons arguing over version control systems... but what ever. Ok, bzr, hg, and git all have lots of nice features - but merging in svn is pretty easy now so please bash(zsh) it for other reasons if you must. Now, let's move back to the vi VS emacs argument ;)

Can you see the secret to my web development productivity in this photo? No it's not the red cup of coffee. It's not the pieces of sticky tape on my laptop. Follow the cable from my laptop... and have a look under the desk. ... ... I'll wait whilst you have a look before telling you the answer. ... That's right!!! ... it's a joypad. Using python and pygame, I've made a little app which listens for joystick events, and then deploys the website I'm working on. With a mash, kick or prod of my foot, up goes my website. Deployed. Deploy is just a fancy word for 'upload my website, do database migrations, restart app servers, run tests, rollback if there are failures... etc'. For 5 british pounds, $7.8 USD, or 5.74 euros, you can get one of these joypads delivered to most places these days. It's giving me too much pleasure pressing it every time I want to upload a new version of the website. Most live updates EVER today - and I have the joypad to tha

How to include a Genshi template inside of another genshi template? Use "py:match" inside the template you want to include into other pages (eg. a sitelayout.html). Then you can use "xi:include" in the pages you want the files included in. (eg. best_picture_of_my_cat_today.html) Now, it's not a "simple include this template here" kind of thing. The py:match can find various parts of the file, and then replace them how they like. For example, a page layout template will match the header and footer with "py:match". It is best explained with examples, and in the documentation: pylons genshi example Includes section of the genshi documentation where it explains "xi:include". Genshi documentation where it explains py:match . Hopefully this explains the genshi way of including a template into a template.

Projects that have X in the name are cool. Engines are cool (especially steam powered ones). Spelling engine 'ngin' gains 17.5 l33t points too. All combined, this makes the name nginx super schwet! I've been using nginx for a while now, and have found it quite good so far. Been moving a couple of websites to it, fooling around with it a lot. So far I've been able to use it for everything I've tried. Some of my apache configs are fairly long... so that is saying quite a bit. On my low memory (512MB) server it has saved quite a bit of memory - even though I've only moved over a couple of websites. Along with the cherrypy memory reduction work I did recently, my server has a bit more room to breathe... (and for me to waste memory on hosting other websites! ya!). Nginx has a good reputation as being rock solid - so I hope it holds true for me. Then perhaps I can get rid of apache completely (on this one server). I've tried to replace apache with other

Python already has a number of objects to represent c types. However, there is a need to improve interoperability between systems using these c types. Below I explain the need, and discuss existing efforts to address this need. Then ways to transparently translate between the various type systems without each system needing to know about each other are also discussed. In the ctypes library - you can represent an unsigned 32bit integer with ctypes.c_unit32. In the array, and struct modules there are different array type codes. For example, 'L' represents unsigned int with a minimum of 4 bytes on 32bit systems and 8 on 64bit systems. numpy, cython, pyopengl and other python extensions have their own types representing c types too. Most extensions which link up to languages which use static typing represent basic c types to python in some way. Not only libraries, but various compilers and translation tools also use c types. For example tinypyC++, cython, swig, etc. Also typ

will miss my little sister. wish we had more time together. so sad. shit

Another quality jquery release... jquery 1.4 . Some very handy changes, my favourite probably being able to use functions to set values. In short, some of the changes are: setting values with functions html5 form support better JSON/JSONP and ajax functionality easier element construction (just pass a dict of attributes) reverse indexing like in python, where -1 is the last item. better animation for multiple attributes at once performance improvements, code cleanups, and bug fixing Full details in the jquery 1.4 release notes .

What are worker pools anyway? Worker pools are an optimization to the problem of processing many things in parallel. Rather than have a worker for every item you want to process, you spread the load between the available workers. As an example, creating 1,000,000 processes to add 1,000,000 numbers together is a bit heavy weight. You probably want to divide it up between 8 processes for your 8 core machine. This is the optimization worker pools do. With a usual worker pool there is a queue of jobs/data for the workers to work on. The workers are usually threads, processes or separate machines which do the work passed to them. So for 1000 items on that queue, there are around 1000 accesses to that queue. Usually the queue has to be thread safe or process safe, so that pieces of data are not sent to many workers at once. This can be an efficient method to use for some types of data. For example, if each job can take different amounts of time, like IO tasks over the internet... this

There are hopefully a number of mini text based programming languages that non-programmers can understand. But what are they? One that I've used in the past is something like this: Which would parse into a python/javascript data structure like this: {name: 'Bob', gender: 'male/female', age:'22' } It's suprisingly common in things like search engines. Grandmas who occasionally check their email might not get it (but many do I'm sure!)... but I think a lot of others do. For things like search it is ok, if people know the magic 'terms'. If they do not know the terms, then they can just enter text to search normally. The mini language is used by advanced users. This is quite good for single line free form data entry. Since people only need to know the concept that you have 'key:value'. It's slightly easier than using urls, since separators can be different things. csv files Next up are comma separated files - csv. For example:

A quick cheat sheet for trying out pypy jit. This is for a ubuntu/debian 32bit machine translating the jit version of pypy-c. # install dependencies for debian/ubuntu. sudo apt-get install python-dev libz-dev libbz2-dev libncurses-dev libexpat1-dev libssl-dev libgc-dev libffi-dev # download from svn. svn co http://codespeak.net/svn/pypy/trunk pypy-trunk # Translate/compile the jit. This part can take a while. # Don't worry. Relax have a home brew. cd pypy-trunk/pypy/translator/goal ./translate.py -Ojit targetpypystandalone.py # Or for the low memory pypy-c use this translate instead... #./translate.py --gcremovetypeptr targetpypystandalone --objspace-std-withsharingdict The pypy getting started documentation has more about it if you're interested. pypy has most standard modules up to python2.5... and some from newer versions. I didn't need any external compiled extension modules... just sqlite, and cherrypy. So for this project I could use pypy! *happy dance* Didn

Tried out unladen swallow on two work loads today. After the announcement they are going to try and bring it into core. So I finally got around to trying it (last time the build failed). An hour or so later, the build finally finished and I could try it out. The C++ llvm takes ages to compile, which is what took most of the extra time. What follows is a review of unladen swallow - as it stands today. The good part? Extensions work(mostly)! w00t. I could compile the C extension pygame, and run things with it. Now to run code I care about, my work loads - to see if their numbers hold true for me. cherrypy webserver benchmark: crash pygame tests: some crashes, mostly work. pygame.examples.testsprite : random pauses in the animation. The crashes I've found so far seem to be thread related I guess. The cherrypy one, and some of the pygame ones both use threads, so I'm guessing that's it. Random pauses for applications is a big FAIL. Animations fail to work, and user in

I've mentioned this technique before, but I think it is worth repeating more clearly. Without any other code clouding the main message: ' All you need is love html . A HTML form can describe a model of your data quite well. This lets you go from HTML form design to a working Create Read Update Delete (CRUD) system. A CRUD can be thought of as a nice admin interface to edit your data (or a CMS if you prefer that acronym). For example: <form action='savepage' method='post'> title:<input type='text' name='title'> <br>textarea: <textarea name='content'></textarea> <input type='submit' name='submit'></form> That's all you need to create a CRUD. Things like validation can be defined in the html easily enough then implemented in your code to be checked server side, and client side. Parse the form to get the model, and go from there. The benefits are simplicity, and that designer

Here's an example for uploading multiple files with HTML 5 and newer browsers, like the firefox 3.6 beta5. You can shift/ctrl select multiple files, and even drag and drop them in. Makes for a much nicer user experience uploading files - rather than having to select one at a time, or having to load some java/flash thing. It uses the unreleased cherrypy 3.2, with it's new request entity parsing tool hooks. See http://www.cherrypy.org/wiki/RequestBodies for details about the new control allowed over the whole process. It's a lot easier to make custom request entity parsing behaviour now, and in a much less hacky way than before. http://rene.f0o.com/~rene/stuff/cherry_multi_fileupload.tar.gz With the tool in there, files come in as a list of files instead.

Feeling just a little nostalgic this time of year. Just made a little script to find the oldest python files on your hard drive. http://rene.f0o.com/~rene/stuff/oldest_python.py oldest_python.py [path] oldest_python.py mystuff/python oldest_python.py oldest_python.py ~ Update: Lennart mentions a unixy way in the comments of finding oldest files with this: find . -name '*.py' -printf "%T+ %p \n" | sort | more With that I found some really old python files of mine... woh! the oldest ones dated in 1998 . There are older C, C++, haskel, javascript, java, pascal, prolog, asm, sql, perl, etc, etc and heaps of other old junk files, but those are the first ones I could find written in python. I guess that means I've been programming python for around 12 years now . Python was at version 1.4 or so, and 1.5 was released not long after. New style objects did not exist, and it was not all too uncommon to be able to segfault the interpreter (ping could easily crash linux

Here's the london dojo google calendar for much of 2010. More details, and signup here: http://ldnpydojo.eventwax.com/5th-london-python-code-dojo/ . The idea is to practice, learn and teach python skills.