making apps, making webs.

Ages ago I made some python implementations of map. Map allows you to call a function on every element of a sequence. [2,3,4] == map(lambda x:x+1, [1,2,3]) One implementation I made used threads. Which made threaded programing easier for myself. However the interesting thing was comparing the speed of the builtin map to the map function accelerated by psyco. It turns out that for a whole bunch of cases the psyco version of map is faster than the CPython version. Here's some unittests and the map function which show the speed differences. http://rene.f0o.com/~rene/maps/map_unittests.py http://rene.f0o.com/~rene/maps/maps.py It shows how nicely loopy code can be speed up with psyco. Written by a Melbourne web developer . Available for your projects - python, php, mysql, e commerce, javascript, CMS, css, flash, actionscript, games, postgresql, xml.

Risk with spaceships. Galcon is kind of like the game risk - but with space ships. The most fun can be had playing multiplayer on the internet with others. Download the windows version Download the MacOSX version A linux version is also in the works. It should be up on the site within a week or so on the Downloads page. Galcon was made using Python, pygame and PGU. It uses some C extension modules for the graphically intense parts. Like the additive blending modes for particle systems, and custom nice fast space ship drawing code. It's great to see another quality game being made in python. The networking uses UDP so that network games are quite good. I can even play games here in Australia with people in the USA and elsewhere. Even with 400ms ping times. The single player game is fun too. With a whole bunch of different missions to play through. Different AI bots are used which give different challenges. There's a small group of Galcon multiplayer people giving the ga...

I have 10 pages of content. Each of those pages is complex, and will most likely be visited by the person when they arrive. 40 images are on some pages. 130 small images all up. The images don't change that often. So after the first visit they are not downloaded again. With well specified expires, and cache directives sent out by the web server - the browsers don't even check to see if the images changed all of the time. However a browser may only allow four requests at a time. This means it's going to take a while for all of the 130 images to download. The obvious solution is to batch all of the requests into one. Since web browsers are dumb, they don't do this. HTTP was poorly designed in this regard. They didn't allow batched downloads. eg 'GET image1.jpg,image2.jpg,image3.jpg' or something similar is not allowed with HTTP. Multiple requests in the same connection were not a good idea. Since there is latency involved for each file requested. So...

With gvim 7.x you can open files in tabs. However on the command line it follows the old behavior of editing one file at a time. eg: gvim file1.py file44.php file2.py Instead you have to use the -p command line option. eg: gvim -p file1.py file44.php file2.py Then you have all of your files opened up in separate tabs. The new tabs in gvim 7.x make using pypanel less necessary for me. pypanel is a little python app for X11 that works really nicely with WindowMaker (the minimal window manager). It gives you a task bar, which can be programmed in python. I used to use pypanel with gvim for when I have more than a few files opened in one work space. Now with tabs, I am able to have a lot more files opened at once. Gvim tabs, the pypanel task bar, and windowmakers workspaces make my desktop a lot more scalable, and less annoying :) Does anyone know of a tool to search through window names, and then focus the selected window? Now that I might have 100-200 files open at once a search fun...

John Popplewell has made a guide for compiling pygame on windows with the MS visual C compiler. http://johnnypops.demon.co.uk /pygame/index.html Looks like a pretty good guide to compiling pygame on windows. It shows you how difficult it can be. With unix it's much easier to compile. You can also compile pygame with mingw and msys using gcc. It's equally as complex - just in different ways. Written by a Melbourne web developer . Available for your projects - php, mysql, e commerce, javascript, CMS, css, flash, actionscript, python, games, postgresql, xml.

GET urls with side effects can be a security problem. For example, imagine the following urls: paymoney?user=joe&amount=3000.0 deleteSomething?file=important.xls Now if those urls are protected by a login system, then only those who login can use them right? Nope. It is easy enough to trick someone into visiting a web page so that those urls are called by their web browser. If that person is logged in when they are tricked, then the action happens. A person can be tricked either by using img, frame, redirect or some other tags or http/html/flash/javascript features. This is because according to a web browser it is ok to include or link to elements on other pages. In fact that's the whole point of hyper linking. In this way it uses the authorization of person viewing your well crafted page. You can now create a page so that you can delete files as someone else, or pay any amount of money you want to anyone. Whatever the badly designed GET urls allow you to do. This can eve...

I saw this in the latest development release notes of gimp 2.3.12: "- build a color-managed CMYK color selector if lcms is available" http://developer.gimp.org/NEWS So it seems the gimp is getting CYMK support(or has it already). That'll be good for those doing print work that requires it. Along with the gimps next generation image processing core (GEGL) things are looking good for the gimp. http://www.linux.com/article.pl?sid=06/10/16/1342216 http://www.gegl.org/ I really think the new versions are quite good. With most of the bugs that annoyed me gone, and with new features being added all of the time. There's still some font handling issues which would be wonderful if fixed. Like rotated fonts, and fonts that can follow a path. However I think those features will appear over time. This line may be interesting for those python users out there wanting to optimize their image production pipeline... "- many improvements to the Python bindings and the pygimp...

To design a font on linux you can use fontforge. http://fontforge.sourceforge.net/ As part of my learning graphical design, I have become interested in finding out how fonts work and how to design fonts. I think learning to design a font will give me insight into other fonts. Much like how learning assembly language gave me insight into how computers work. Creating a font on linux with fontforge is the way to go I think. I haven't found any other way to create a font on linux yet! It was quite hard to find fontforge for designing fonts on linux too. It's a time consuming thing - designing a font. That I think will take years to figure out. I don't expect to make a great font, however I think there is room for creativity in fonts yet. So I hope to make something useful and learn something on the way. I guess I have made bitmap fonts before including the animated one on f0o.com http://f0o.com/ . However crude it might be ;) True Type Fonts .(ttf) fonts contain a virtu...

Some python web frame works are using pickle to store session data. Pickle is a well known poor choice for secure systems. However it seems to be more widely known by those writing network applications, than those making web frameworks. Is your web framework using pickle for sessions despite the warnings in the python documentation about it being insecure? By using sessions with pickle people who can write to the database servers session table can execute code on the app server. Or people who can get data into the session file/memcache data store can execute data. This might be an issue if the database server is run by separate people than the app server. Or if the session table is compromised by an sql injection attack elsewhere. There are some more secure ways of storing pickled data. Pickle is deemed to be untrustworthy for data. In that it is not certain that code can not be snuck into the data that will be executed by pickle. So if some data from user input is put into the p...

It's been really interesting to see VMs over the last couple of years. Now there are emulators, and virtualisers which are capable of running x86 really quickly. The processors themselves don't run x86 natively anymore, it's a VM. Now Apple are using x86, and x86 is getting more common in the embedded world too. So now, rather than creating a VM like python does it seems to make sense to use the standard VM, and that is x86. Of course x86 is really complex, and still fairly slow to emulate on slow hardware. So using a simpler VM still has its advantages. However writing directly to the most common VM has its advantages too. You can make software which is 400 bytes big which can do almost the same as a 8000 byte program. That's a 10x saving in program size. The same program will run in 12KiB of memory, instead of 1.7MiB of memory. That's a 141x memory usage saving. Because the code size, and memory size is so much smaller you can get a lot more done with the s...

http://www.pyweek.org/3/ The theme voting has begun. Here are the themes for the week long python game jam. * Pick a card, any card * Watch me pull a rabbit out from this hat * Sawn in half * Spoon bending * The Disappearing Act A very short list of themes to choose from this time. Thanks to Richard for organising pyweek again! If you have some spare time coming up, and want to finish a game then you should enter pyweek . If you don't have much time, find a team and join that. Let's hope the server stays up mostly :) These competitions all ways result in a server going down. Lots of users, and wide spread attention amongst smart, young and demanding hacker types means it gets challenges lots of other websites do not get. Strain on the software, and on the hardware are two of the causes. Otherwise it is caused by a simple cracking in to the machine to take it down.

not a bit of good food for got home gut jill had her kill Oh, how I love bad poetry. I just had to share. You need to type it to appreciate how bad it is.

I replied to a blog post by Jason Huggins on his blog asking for a web frame work evaluation check list. I listed a number of common security issues, and finished my post with: "I can not see any python web frameworks that do not have a history of security problems. I also can not see one which was designed from the beginning with security in mind." Then James Bennett (a Django contributor) asked about problems with Django specifically. Saying 'if there’s some long bounding history of security flaws in Django, I sure haven’t seen it'. http://www.jrandolph.com/blog/?p=45 Unfortunately my reply seems to be caught up in the moderators queue, so I guess I'll have to put my answer below. A couple of posts after mine are there, so I am not sure why my post was not approved to be posted. ps. since my post on the 11th of August I have reported four security issues to the django project. Not one has even been replied to. Not even with a simple response like 'I go...

The third pyweek competition is starting in September. You can have group entries, or go solo. It is a competition of sorts, but it is all just for fun really. You have a week to work on your game. Most people work on it in their spare time. Other people take a week off to do it. http://www.pyweek.org/3/

Recently I was testing that pygame still works with the python2.5 beta. When I was doing this, I had a chance to play around with python2.5 and all it's new goodness. Amongst the other good python2.5 features is that python is performing better. Almost twice as fast a startup 'cold'. This is because it uses half the number of syscalls as python2.4 does. 755 syscalls in 2.4 and 461 in 2.5. Still not quite as good as perl with 40ish syscalls, but a massive improvement none the less. $ time python2.4 -c "print 'hello'" hello real 0m0.923s user 0m0.041s sys 0m0.025s $ time python2.5 -c "print 'hello'" hello real 0m0.417s user 0m0.043s sys 0m0.021s However compared to perl, python is still syscall heavy. Perl does around 50ish syscalls to start up. Python2.5 also frees up more memory that it isn't using. Python2.4 does not release all that much memory if you use up quite a lot. It holds onto it in it's memory poo...

If you have a non 64bit Debian machine (like a duron, p3 etc) installing libc6-i686 will speed it up. # note that doing this you may need to restart running programs otherwise they might crash. # rebooting might be an idea, if you do not know how to restart all your programs. apt-get install libc6-i686 After I had it installed I got a 7% faster pystone result. I also got a couple of fps quicker in some pygames. This was with an AMD duron 850mhz system, your milage may vary.

It makes sense I guess. The people who make firefox should be able to make a faster binary than a distro. The mozilla firefox build is i686 rather than i386, and probably uses some other methods for extra speed that debian doesn't. Debian stuff is usually made for stability and compatibility first, rather than for speed. However, so far on my machine the mozilla build has been running for a day or so. So it works nicely at least on my machine. It's made my browsing noticably faster on this duron 850mhz machine that I use daily. So I'm quite happy about it. The windows refresh faster, and the javascript heavy sites do their thing quicker. My pretendpaper site loads a whole 1.5 seconds faster using this build. It's probably the most slow, and intense javascript based site I know of. The load time is noticably slower on a slow cpu machine.

Pygame has moved to subversion. Thanks to our wonderful hosting friends at seul.org for setting it up for us. The sdl ctypes has been moving forward. This is a project sponsored by the google summer of code. A couple of SDL example programs are running, and the code is looking good. There is a manual, and a couple of tests. sdl ctypes is being developed in the pygame subversion. http://www.pygame.org/ctypes/ For pygame I have gone through most of the documentation comments on the website. Only 20 remain left to sort through. shreadwheat also added some more documentation enhancements. A request for the scaling functions to be able to accept a destination surface has been made. I've mostly completed this change. It allows for faster scaling when you are repeating the scale operation multiple times onto the same surface. This is because you don't need to allocate memory so often. However the destination surface has to be the correct size, and format. If it isn't ...

Will the arrival of google earth mean that hardware accelerated opengl be more widely spread on linux? It's a free fun toy that requires opengl. Hopefully it will do more for linux opengl than quake3, and xgl have done. I hope so anyway.

The last pygame release was August 16th 2005. So there are a lot of bug fixes and a few features in development that need to get out there into a release. With the new SDL being released last week it seems like a good time to get a new pygame out. Here is the release plan from the last release. We will follow it in the same way again for this new release. At the moment we are in the apply patches, fix bugs, get in any last minute features phase. This weekend I made pygame.image.save be able to save as .png and .jpg files. Since pygame already optionally links against libpng and libjpg to load .jpg, and .png it shouldn't be too much hassle for people, and only a tiny bit of extra code. So if pygame is linked against libpng, and libjpg it should be able to write .jpg and .png images. Saving as jpg/png is an often requested feature for people doing screen shots, caches, and level editors amongst other uses. Hopefully these save functions will eventually make it into SDL. That...

For pygame which uses SDL some parts are written in assembler. These parts detect which cpu they are running on and then use those mmx/sse/3dnow optimized assembly routines if those processor features are available. However much of pygame and SDL is still in C, not ASM. So compiling using processor specific features does give a very nice speed up for the C parts. For some parts a 33% speedup or more can be gained. Just by changing compilation flags. I think this is the best and easiest approach to getting parts of CPython to be sped up. Below are my notes, and thoughts about the topic. Mostly this is based around my experimentation with pygame compilation. This does not take into consideration python compilation itself, or any of its modules. However a similar methodology could be applied to speed up pythons modules too. Note that recompiling python itself to match your processor can give a large speedup too. So, I have started experimenting changing distutils to compile modul...

An update of what has been happening recently with pygame. Thanks to a kind person on the mailing list the macOSX version of the clipboard code was tested. It was a long process but eventually. A one week debugging cycle because I don't have a mac. HAHA. I also got around to testing out the windows version of the code on my win XP box. It worked fine there. Some blending functions have gone into CVS. So you can blit with the modes ADD,SUB,MIN,MAX,MULT. This allows more you to do lighting type effects, and is very useful for particle systems. These are the missing things that SDL doesn't have which are often asked for. The code was based on Phil Hasseys code which he wrote for 32bit software surfaces. He's also done some other code for lines etc. So I'll want to incorporate those changes so you can draw lines that blend too. I also need to add those blending modes to the surface.fill() method. That'll be very useful for fade to black/fade to white. Or f...

These are my favourite python ideas from the google summer of code. Ones that I think will benefit the most people, and affect the most change for the better. Ok, just ones I reckon would be cool. Implement ctypes support for GCC ARM platforms. The underlying issue is lack of closure API support for ARM in libffi. A patch available at http://handhelds.org/~pb/arm-libffi.dpatch , that should be hopefully a good starting point. ctypes CVS has a libffi_arm_wince directory, which also seems to support closure API. Make a python+pygame plugin for IE, netscape. CodingProjectIdeas/PythonWebPlugin Psyco for MacOSX. PPC, and universal binary versions. (also psyco for ARM would be cool!!) Research how to get python support into all the cheap webhosts. Security audit of python. Using as many automated processes as possible. Python speed up. Reduce memory usage, speedup startup time. The two main speed regressions of the 2.0, 2.1,2.2,2.3,2.4 releases. (438,453,499,771,880) syscalls vs 106 ...

Last week I checked into cvs a 'scrap' module. Which is basic copy/paste clipboard support for windows and X11. So you can copy things to and from the clipboard. It is based off the scrap code found off the SDL site. I still need to finish wrapping a couple of C functions, but it is almost finished. It has only been tested on X11 so far. However the original code was written to work on windows as well. So hopefully copy/paste will work there too. Then left to do is Mac OSX support. Today I did some code today on getting TinyGL to work with my character animation code. I managed to get a model animating, and at a decent frame rate. Of course there was no texture on this model. Which is the slow part. However even non textured 3d stuff is pretty useful done in software quickly. I did convert one simple texturing demo to work with TinyGL, and it ran ok. So I am hopeful for its performance. TinyGL only has about 100 functions which need wrapping. Most of the function...

I saw yet another 0.x piece of rapidly changing python code calling itself stable today. If your code meets any of these criteria your code is not stable. Is not even six months old, with a 0.x version number with massive changes planned. Your code is still old and changing rapidly with massive changes planned. Your api is changing causing breakages. A good test to see if your library/framework is stable is to take some old program and see if it runs correctly, as it used to. If it doesn't, then your library/framework is not stable. ah... words that have different meanings.

Zanthor is the game we made in one week as part of the pyweek2 competition. The idea of pyweek is to make a game in one week mostly from scratch. You need python and pygame to run our game. You can download it here: http://www.madecollective.com/~rene/stuff/zanthor_source.zip It works on Windows, Macs, linux, freebsd, and anywhere else python and pygame run. You can play with keyboard, mouse, or gamepad. Here is a screenshot: Please let me know if you like it, and of any cool ideas for it.

The Melbourne international Comedy festival is starting soon. My comedian friend is doing a show for it again. He has made a sudoku solver using an Excel spreadsheet. For something fun to put on his website. I thought it would be an interesting challenge to write one in python. Unfortunately there already appears to be 120million different sudoku solvers already written in python so I gave up. Such is life. After the pyweek2 finished I'm a bit shattered, and programming tasks seem a little hard. I need more sleep. If you're in melbourne, you might want to check out his comedy show: http://www.grouphug.com.au/tmr/ . Last years one was quite funny.

Pyweek 2 has begun. With the "it's powered by steam" theme winning the vote. 190 people have signed up to compete. Not sure how many people voted for themes. Hopefully quite a few people will actually compete. So here is one of our concept sketches that Aaron - aka did. We have a basic web server code done. Using wsgi, twisted.web2, and sqlobject. Since the game is going to have elements where you can affect the worlds of the players from a web page. So even if you don't play with the game, you can have fun messing around with other peoples game worlds. However the web element is going to be an optional part of the game. The game needs to be playable in single player mode. We also have the start of our client main game done. Using pgu, and an isometric game view. You will be able to play as a human who controls a robot castle which is powered by steam. We are currently figuring out what gameplay, and plot we want. But have the basics of what we want to do ...

Here are the themes for the pyweek game development competition. * Someone else's trash * A fraction too much friction * Mind the gap * Doorways * It runs on steam! Much more interesting than last pyweeks abstract ones. It runs on steam! is my favourite so far. It puts the most images into my head. Images of Victorian era steam races, and of a future world which runs on steam powered technology.

Today I did a little test app with PGU and pygame in preparation for pyweek . I have not played with PGU for a while, so thought I needed a little refresher course. PGU works with pygame, and has a gui library as part of it. It's a cross between html, and QT. It even has a basic html renderer. So if you know a little html, you can use some of that knowledge. The result is a simple RPG character attributes generation widget character_create.py It can generate these character attribute scores, and allow the user to assign points to them. I made it reusable so that people can plug in their own character attribute names. Eg - you could have 'Sense' instead of 'Dexterity'. You can put these widgets inside other containers, so that you can generate multiple characters at a time. Our pyweek team The Olde Battleaxe is going to do a few more practice sessions before the competition begins next week. Our team has one sound+music guy, a graphics+sound guy, and two co...

The new python website is up. It mentions web stuff, database stuff, and gui stuff. However it doesn't mention games or graphics stuff at all. Links to pygame.org, and pyopengl.sf.net should be added to the front page. As they are very popular python libraries used for games. Eve online, and the Temple of Elemental Evil are two retail success stories. The witches yarn is a shareware success story, and amongst the open source success stories are angry drunken dwarves, solarwolf, and pydance. Blender, the gimp, poser, truespace, are amongst the graphics programs that use python for scripting. Making learning python scripting for artists very useful. ILM use python for movie making.

When trying to figure out how to use transactions in sqlobject, I found this code snippet. As a bonus my init database code now runs twice as fast. Yah! def do_in_transaction(func, *args, **kw): old_conn = sqlhub.getConnection() conn = old_conn.transaction() sqlhub.threadConnection = conn try: try: value = func(*args, **kw) except: conn.rollback() raise else: conn.commit() return value finally: sqlhub.threadConnection = old_conn # do_in_transaction(some_function_with_sqlobject_stuff_in_it) # I needed to change my connection line to this. That is assign something # to the processConnection thing in sqlhub. # sqlhub.processConnection = db_connection = __connection__ = connectionForURI(config.conn)

It looks like pyweek registrations are open. Pyweek is a realy fun game development competition inspired by the ludumdare 48 hour competitions. Except this competition uses python, it goes for a week, and there can be teams or solo entrants! 0 minutes before the competition begins a theme is announced, and you need to make a game from scratch based on that theme. The idea is to have a lot of fun finishing a game in a week. You don't work for the whole week on the game, just spend a couple of hours a day, and perhaps more on the weekend. So for some people this is better than spending a whole weekend. For anyone interested in making games there is nothing more fun and educational than these competitions. Lots of the fun is all of the energy and excitement you get from looking at other peoples work, and chatting away on irc. Everyone is making a game all at the same time, it's a lot of fun. It is the game making equivalent of a jamming with people in a band. Except you are ...

After a couple of months of part time playing with the pretend paper code base I have put up a new version of this arts community site I work on for a hobby. The most visible change is the modified css design. Which gets rid of black lines, and makes it look more minimal. These changes were done by Dane with help from Aaron. A lot of the changes are not visible, as I rewrote things so that searching and editing are more doable. To make searching and editing how I wanted them to be I needed to be able to have multiple event listing tabs on screen at once. Which meant that I needed to change around the way it used ids to encode a tab id into each of the ids. Also the code for creating new tabs stopped using function callbacks so much, and instead used objects. By using objects I am more easily able to change, and query their behaviour. I cleaned up the javascript code a little, but it still needs a lot of work. However a lot of it is still prototype code, so I'm not finished ...

Yesterday I got some help from theoddbot on the latest changes on the gp2x scene. There is now a serial over usb driver. So I can log into my gp2x and get a bash prompt through minicom. Yah! much rejoicing. However the driver is buggy, and file transfers don't work. So it's still anoying to do some things. My old development method, pre serial over usb. ' close down gp2x, then go over to win box, take out card, put card in reader, copy file over from linux to sd, then take out card, then put card in gp2x, then boot, then select program'. Now I can make a bunch of changes, unmount the SD card through my bash prompt. Sneaker net them onto my windows box with the SD card reader. Then I can run the program, debug it, and make small changes with vi. I can also run code on the python interpreter. Yah! If the program crashes, or gets stuck somewhere, then I can kill it from the bash prompt. However ctrl+c doesn't seem to work, so I need to run the program in the ...

I packaged up Joe Wreschnigs Angry, Drunken Dwarves for windows today. Using py2exe and the NSIS installer. You can download it here: http://rene.f0o.com/~rene/angrydd_win_1_1.exe If you play it please leave a comment on my blog . update: it looks like piman has put the .exe on his page now.

A few interesting game related things. I have begun doing a series of pixel art tutorials in order to make my pixel art skills better. The result is this face. The face has been blown up 2x to make it easier to see. The idea is to do some pixel art tutorials every week this year. When I am learning something I like to do it by doing a small project. So my pixel art learning project is going to be to update my bleten game. Bleten was a game I made for a 48 hour game dev competition(using python and pygame to code of course). Due to lack of time(and poor pixel-fu) most of the pixel artwork in it is pretty average. Including some stick-sheep. Also game related, I am working on a milkshape character animation library written in C. I have converted it from some C++ code I had lying around. The conversion from C++ to C code is in order to make it easier to port, make it easier to use as a library, to simplify it, and to reduce code size. I will be wrapping it up with python binding...