Friday, December 23, 2005

Pygames Snakey is covered in snow!

Inspired by the ugliness of a certain other gamedev site, pygames Snakey is transformed with snow and a santa hat.

http://www.pygame.org/news.html

Also check out some of the cool new games and projects that people have been making.

Saturday, December 17, 2005

Of urls and single url DHTML sites.

Dynamic sites, and urls.

A website I am working on now, pretendpaper.com is one of those single url sites with lots of fancy DHTML where everything changes dynamically without changing urls. Everything is loaded with a python twisted, wsgi server sending xmlrpc into separate tabs.

A while ago I implemented a scheme which works with javascript to inspect the url, and open up different tabs. For example, this link opens up a review and an article into different tabs http://www.pretendpaper.com/?r=5&a=4

Putting in a site url box.

Now, to make people be able to share links of what they are looking at in the site, we are going to place a url box in there. So as they open up different bits of the site they can see a link for where they are. It will be like a browser inside of a browser. With a url bar, and tabs.


Saving the whole state of website as a url.

Now I am working on encoding the whole state of the page into a url. Including the scroll state of the bars in the tabs, and which tabs are focused. Then, periodically as a person visits the site it will save this state into a cookie. Then next time they visit the site they will be redirected to the saved state of the website as a url.

This gives many advantages. Especially once search is implemented. They will be able to arrive at the site, and have thier searches from last time appearing in the tabs. So if they searched for say noise gigs, and poetry readings in their suburb last time the same search tabs will appear with all the latest noise gigs, and poetry readings in their suburb.

Of course, this means that I need to finish off the UI for search...

Saturday, December 03, 2005

Save, compile, run cycle. Except with templates.

The save, compile, run cycle I reduced in my source code, why did I think it would be ok in templates?

I used the cheetah templates for http://www.pretendpaper.com/ They are a pretty nice way to handle templates. They are nice except for the compile that needed to be done when they are used. This is great for run time efficiency... I think. But not so good for development time efficiency.

However now I have removed the need for the templates to be compiled from my code. With the change of a config variable I can also use the compiled templates. Which will be good for the live version of the site.

Now I'm starting to think about the other parts in the save, compile, run cycle.

If I could get rid of either the save, or run cycles then that'd be great. No more pressing save to see what changes occur. Just change something, and then it will show me the results within a second or so.

How can removing the save and run parts be done? By having the editor in the browser. Or having the editor, automatically save things. Then once a change is noted, the thing tries to run itself automatically. If there is no error, then it will display the change.

I think there are probably nice editors written in javascript around. Ones that can do syntax highlighting for python, sql, javascript, css, and html. Easy modification would be good too.

Then I could type in an element to edit in a shell, have the file pop up in a html layer, or another window ready for editing. Once the changes are made, they are reflected in the site I am editing. As I edit things. No need to press save, or press run.

GP2X likes my batteries

I have been learning about the GP2X I received the other week. I have an arm gcc tool chain setup on my windows box, and my linux box. It could have been a much easier experience. However hopefully now the gp2x dev wiki has gathered enough information so new commers should have an easier time.

It has a bunch of potential to be a really nice device. It just needs a whole bunch of bug fixes. I think the people at GPH will eventually fix things up.

So I wait until I can afford a power supply for the unit. So that I do not have to recharge batteries all the time. Having to copy the file to the SD card via the card reader, then put it in the GP2X and reboot is no fun at all. Considering boot time is 25 seconds or so. Which means a lot of iterative development will not work if I need to wait 1 minute between changes. I am used to waiting 1 second between changes.

In the meantime someone else has already gotten python, and pygame working on the machine. Which is great! As soon as I get my power supply(about $40 AUD at dick smiths) I'm going to play around with that.

If someone can get tcp/ip over usb working with the machine, then iterative development with python should be a bunch easier. All the software is available for it. It just needs to be compiled into the kernel, and tested. Unforuntately GPH has not released the code for their modifications to linux. So this will have to wait.

I want to add a gp2x module to pygame. With all the extra things that can be done with the GP2X. For example, turning the red led light on and off. Putting the LCD to sleep when a game is paused. Nicely named constants for all of the joystick buttons. Putting the second CPU to sleep. Using the hardware overlays, scaling, and other abilities of the video post processor etc etc.

Monday, November 21, 2005

GP2X arrives.

My GP2X arrived this morning!

I ran down the stairs to meet the delivery man. Then ran back upstairs tearing open the packaging as I went up the stairs.

I was sent this one early by the lovely people at gbax so that I could port pygame to it.

This version of the firmware doesn't seem to be able to copy stuff to the onboard flash. The usb connection doesn't seem to connect to my computer. Hopefully it will work once an SD card is in there.

So all I can do with it at the moment is use it as an expensive flash light.

One thing I was dissapointed about... you can not use it as a usb host. Which means you can not connect your digital camera or other things to it. Oh well. Maybe something can be done for that through the EXT port. The EXT port will be used for TV output too. The cables for tv out should be available in a few weeks.

I've downloaded the available SDL development files, and been through the development wiki, and read all of the spec sheets. Now I wait until I get an SD card. How exciting!

Monday, November 14, 2005

query string using javascript.

I have just implemented query strings using javascript for my web page Pretend Paper

So you can make a link which will change the configuration of the dynamic page around.

This is an example of the link: http://www.pretendpaper.com/?r=5&a=3. It opens up the review with id 5 in a background tab, and the article with id 3 in the foreground.


It is really important to be able to get a url which will map to what is being shown on screen. This way people can post links, and bookmark pages they are in. Unfortunately with javascript one 'page' can be showing 600ish different things.

I still need to make it possible to open up news, and events via the url. As well as show people a url for the page they have opened up.

Wednesday, November 09, 2005

Fair evader.

[dang`r`us]: fair/fair evader?
[illume]: in Melbourne australia there is a tram company which is now privately owned. they have run a fare evader advertising campaign to guilt people into paying for tickets
[dang`r`us]: ah okay
[illume]: unfortunately they are pretty shit for a number of reasons... so a group of us are making games/websites to try and draw some attention to these problems



And so it begins.


The game will be about playing a tram inspector collecting commisions for each tram fine that they give people. You'll get bonuses for fining pregnant non english speaking ladies, old people, and poor people.

You can demand people hurry up and show them their tickets. You can sneak on in under cover clothes.

I think I'll even use the gui for speach dialogs. To present the user with multiple choice questions. 'fine', 'let go', 'twist arm'. Also for other speach dialogs like people trying to get out of tram fines, and general banter on the tram. etc.


8bit graphics, overhead view, using python and pygame.




I'm also using PGU for the map editor. Yesterday I spent a little while familiarising myself with it. Then I made a couple of modifications for quicker editing. So changes you make in the tileedit program appear automatically when you mouse over to the leveledit program.


Watch screen cast of tile editng... Exciting stuff ;)


Thanks to philhassey for the assistance in figuring out how PGU works.

I also came up with a plan to use the leveleditor for an animation editor. Sound crazy? Well it is a bit of a hack, but basically the same program can be used to place frames onto an animation strip. The tiles become frames, and the levels become the animation strips.

So I need to make some code to play the animations, which will read the level, and tile data making animations.

It should all be a bit of fun.

Friday, October 28, 2005

Pretend papering

I have recently moved a pretendpaper into subversion.

Before that it was in no revision control. Well, my own style of revision control. Which is a shell script that runs every few seconds, checks for changes in my local set of files, and backs those up.

All done with about 30 lines of bash. With time stamped directories showing all of my changes. Without me having to run 'svn commit' or some such thing.

Which worked quite well for my productivity, except I didn't really have an anotated history, apart from my own logs. I would make notes in a text file, and from there I could peer into an old directory to see what happened.

Which was all fine except now two other people need to make changes, and this system just can't work for others as easily as subversion can.


So now I have a remote dev server, as well as my local dev server. There were some issues with the website running from a different domain for some reason. Different timing issues showing up some bugs in apache2 mod_proxy mod_deflate/ and my code. I cleaned up some of my javascript before discovering it was really a problem with one of the mod_deflate settings. Well, maybe it was another part of the system interacting with that setting. I'm guessing it was a mod_proxy problem. Funny thing was, I never saw this problem at the other url.

However now my code is cleaner and more resilient. So that was probably worth it.

Pretendpaper now loads within 2 seconds on my fast machine. After the first load. Which can take up to 10 seconds. So I'm happy with its performance for now. The next performance improvement I think would be to do the xmlrpc part in an iframe.

There are a few other things I need to do to get things working nicely for me and the other two working on the site. I need to fix up the templating system so that you do not need to recompile templates once you change them. I also need a way to trigger a server cache clear. Which will most likely mean a server restart. However being able to clear various parts of the cache are probably a good idea for neatness.

Slowly getting there.

We made a few fixes to templates when we all met. We fixed up the article view and did some changes to the event template so that we could see the start and end dates in the view. Today I also made it sort the events by end date.

We decided to tackle one bit at a time. First make the events listing part really good, and then work on other parts. A few improvements things for events include search. So that people can search through events easily. Also search spiders, that search the web and other websites for information about events.

Saturday, September 03, 2005

A-profiling I a-go.

I spent a little bit of time recently profiling and optimizing some code.

This is not a usual thing to do for lots of programing tasks, because these days, often code is fast enough to do the job. However fast enough for which computer? One of the machines I run is an old duron 850 mhz from around 1999. Some programs are starting to get a little slow. Because developers are only testing to make sure their programs run on the latest hardware. Lots of people I know still have computers even older than this! So I want things to run really nicely on my slow machine at least. In the hopes that it will run nicely on other peoples slow computers. Luckily it often doesn't take too mucheffort to speed things up.

Most things I do run ok even on my old slow machine. However on two demanding applications I am working on there is a needfor profiling and optimization.

Here are some of the things I found...

Python game optimization.

My game holepit was leaking a little bit of memory between levels. It turned out that a destructor was not being called for the level, so a dislay list was not being freed. I think there may have been an error in the destructor(which doesn't tell python if there is an error there), or python was waiting to garbage collect it. Whatever. I just put an explit clean up call there.

Also the display list was getting compiled and executed every frame. Rather than reusing the previously compiled display list. This gave quite a big speed up too. The moral of the story... memory leaks are bad, and so is garbage collection in performance intensive situations requiring os cleanup(opengl resources, and file resources).

For mouse motion events, sometimes three or four could be delivered per frame. So my 'turn the avatar around' code was being called up to four times times a frame. This turned out to be a major cause of slow down in the intro sequence because there the time is sped up. The intro uses recorded mouse events to control a character which writes the word 'holepit' on the screen. It was getting up to eight a frame. Once I filtered these events to only update on the last mouse motion event every frame then the intro sequence ran a lot better. Such that it didn't skip between sections of the word. Much nicer looking :)

It had less of an inpact in the main game though.

One thing that did come up though was a place where I was using the dreaded 'exec'. This was in some code which interfaces with C++ code. Finding a class name encoded into a string pointer, and constructing that class. So I removed the exec call by storing the classes in a dict keyed by thier names, then constructed the classes from that instead.

This did make a noticable performance impact.


There were a number of other low hanging fruit in holepit for increasing the performance. However the game was now runing quite a bit faster than before such that it is running faster than vsync on my slow computer(over 75 fps). So now it is 'fast enough'.

Javascript website optimization.

Now the tactic I used to optimize www.pretendpaper.com was quite a bit different.

As of this writing the website still needs a bunch of work to make faster. But it is getting close to usable now. It takes 4-5 seconds to load on my slower computer, less on my faster one.

I already knew without profiling some of the slow bits. There is *lots I can do to pretendpaper.com to speed it up. But I knew of a couple of sure things which would give massive speedups. Also I knew that by chosing this one thing, it would only take me very little time, and not require much reworking of code.

I just put eight requests into one request. Those eight requests always happened anyway, so I might as well do them at once all the time. The overall time to request one big thing is less because 1) there is no latency between the requests. 2) there is better compression because some of the data in the requests is the same.


During changing the requests to use the same one I found quite a bit of duplicated code. As well as some things which could be easily done(and cached) on the server side. Saving the client side quite a bit of work.

More waffling on about the differences.

So even without a profiler, you can sometimes just know what bits are slow. However a profiler can definitely help you out. Especially for a large amounts of code, or when you did not write the code in the first place.

The other difference website/javascript optimization has with python game optimization was that I had to test on multiple browsers, and with multiple speed computers, and multiple speed internet connections. Lots of different variables. There are lots of variables with games as well. Such as different amounts and speeds of main memory and video memory, cpu speeds, video card speeds, harddrive speeds, and sound card speeds.

Optimizing towards fast enough is a good goal. As long as you optimize for fast enough on the computers that it will run on, not your brand new really fast machine. Just optimizing to fast enough is important when there are still lots of features needed to add, and bugs to fix.

Wednesday, August 17, 2005

Another pygame release at last.

There is finally a new release for pygame out. I was the one who did this release(with lots of help from others), and below are some notes on how it went, and what is next for pygame.

What is pygame?

Pygame is a is a set of Python modules designed for writing games. It is written on top of the excellent SDL library. This allows you to create fully featured games and multimedia programs in the python language. Pygame is highly portable and runs on nearly every platform and operating system.

Compared to the 1.7.0 release, there is a whole bunch of bug fixes. Compared to the commonly used pygame 1.6.2 there are heaps of changes. A different version of SDL, a different compiler used for the windows version, and a rewrite of the sprite classes.

Lots of new people have begun to submit patches and test things out. Which is really nice. We also followed a release plan. Which was made necessary because of the multiple people working on the release compared to one person previously.

There is also a new website which allows people to register, and show people their projects using pygame. Along with the upcoming pyweek competition with over 150 entrants, pygame continues to live and grow.


What is there to do for the next release?

For me I want to improve what is already in pygame. I want to speed up some areas, and fix up any remaining bugs. For example pygame on linux requires X libraries. Where it should be able to just use the frame buffer.

One place that could do with speed improvements is the transform module. For things like scaling, rotation, flipping, and rotozooming.

I would like to make them quicker by improving the api, and by optimizing the C code, as well as doing asm versions.

However first we need benchmark programs for all of these features. Games can be used, as well as writing programs which stress test the features. For example a rotation benchmark which rotates 30 different images of different sizes, and then blits them to the screen. Another part of the test would be to rotate different same size images without blitting them.

Repeatable benchmarks are important to test the speed on different computers, and to compare between different versions of pygame. The benchmarks can also be used of examples on how to use that functionality.


How ASM(mmx, sse, 3dnow, altivec) optimizations will be done.

I am going to do the asm versions just like the SDL blitters are done. That is detect what cpu features are available and then register the fastest function with the processor features available. This way the same version of the code will run on different processors.

def rotate_C(...):
pass
def rotate_mmx(...):
pass
def rotate_sse(...):
pass

if(has_mmx() and not has_sse()):
rotate = rotate_mmx
elif(has_sse()):
rotate = rotate_sse
else:
rotate = rotate_C

Using vectorizing compilers to make first versions of optimized code.

To start with I am going to use vectorising compilers to make the first pass at the optimized ASM. gcc4. vectorC, msvcc, and icc are all ones which can do auto vectorization. Some compilers are better at vectorizing than others.

Once the first pass is done with those compilers I can see which ones were able to successfuly vectorize the code. Then run them through the benchmark to see which is faster.

Then I can see if changing the algorithm can speed them up at all.

The final pass at optimization would be to change the asm code itself for optimizations.



Rotation as an example of API optimization.

As an example of an api change that could improve the speed, I will describe the rotate api and a way that it can be improved.

The rotate function could be made to do its thing inplace. So instead of this:
newsurf = pygame.transform.rotate(surf, 90)

A new api that could do things 'in place' could be made. ie:
pygame.transform.rotate_ip(angle, src_surf, dest_surf, dest_rect)

Where the destination surface could be reused on each call to rotate. Therefore saving memory allocations, and reusing memory. The given api could even be used to rotate directly onto the screen. Ie a rotating blitter :)

One problem with this api is that a rotated image can use up more space than the original image. Therefore we need to make sure that the destination surface is large enough to hold the rotated image.

Therefore we may need a helper function to calculate the maximum size of a rotated image.
arect = pygame.transform.rotate_maxsize(src_surf, angle = None)

If an angle is given then we get the size of the image that will be created for that angle.

Such a rotation function is going to be quite complex to implement in a way which is easy for the user.

Friday, August 12, 2005

Pretendpaper. Making pretend webs.

Pretend Paper is an arts/music, not for profit website meant to be a place where people can see what is on, and what people are doing. Pretend paper is also a place where you can tell people what creative things you are doing. As well as a place where people can post articles, and reviews of creative things.

Pretend paper is a site that has been made with two other people, and I. With contributions from many others. All of us have been taking a break from it since our launch party in July. Which means that it has been a month and a half since my last major code update.

What makes pretendpaper interesting from a web development point of view, is the new interface ideas. There are two sides of the site. A tab area on the left, and a tab area on the right. When you are viewing stuff on the left you can click to see something related on the right side of the screen. All without reloading the page.

One good use for the tabs is when you are reading the details for an event you will be able to click on a band, or artist listed in the event, and have it pop up the artists profile on the right side of the page.

Interface experiments, and standard controls.

Apart from the tabs, there are a bunch of other interface things in pretend paper too. Some more successful than the others, and some more developed than others. When making new interfaces you need to be able to have people be able to use them right away. So if you are going to include advanced things in your user interface then they need to act how things normally would act.

Standard behaviour is why for the events listing we made it click to change which event you are viewing. People are used to clicking to change things. So having mouseover change the event being viewed can really confuse some people. However, some of the people who used it really prefer viewing things that way. To view 30 different event descriptions with clicking requires 30 clicks. To view them with mouse over requires none. So we made a compomise. The default behaviour is clicking. However if you click on the same event that is highlighted, then it goes into mouse over mode. Give the event listing interface a try.

The scrollbars for example were done with javascript. The reason why they were done with javascript is for consistency accross browsers, and to avoid bugs with iframes in different browsers. Unfortunately the scrollbars are not quite up to par with the standard scrollbars. Standard scrollbars have so many features built in. For example mouse scrolling, variable width thumb controllers, variable speeds depending on the speed of the computer, and keyboard navigation. I have mouse scrolling in there, for those browsers that support it. IE, and Safari 2 support getting events from the mouse scroller. However firefox doesn't. Which annoys me the most as firefox is my current favourite browser.


What has been happening with pretendpaper?

There are still posts going up, and reviews being written. However each bit of content has a manual review process where I put stuff into the database. The interesting thing is that almost every time something is added I make the code a little more tolerant of errors. So that slowly it is getting better at doing what it does.

During this time I have been thinking of what things need to be done to fix up pretendpaper. I have spoken with the others breifly about it. The list of things to fix I have is a mile long, but there are two main things that need to be fixed.

Pretend paper needs to be faster, and it needs to work better at showing people what is going on.


Pretend paper needs to be Better at showing people what is on.

More events, and linking events to profiles are what is needed. As well as searching, and sorting.

Linking to profiles. There needs to be better links to profiles from the events. So people can find out who the people are that are playing. Being able to pop the band profiles up in the right hand tab set whilst skiming through the events would be great. Want to know if a band is uk pop influenced, a noise band, or a death metal band?


For more events to be in the database a few things can be done. The event adding interface needs to be *very* streamlined. So it is as easy as possible for people to add events. Including the site admins. Typing the band names/artist names, and venue into the description box should automatically fill in the fields for you. It should do a search for the venue, and finding any matches show you a list of them, with the most likely ones at the top. It should search through the profiles listed in the site, and

When more people start using the site, there should also be more people adding events to it. However it is the old chicken and egg thing.


Pretend paper needs to be Faster.

Pretendpaper loads slow. With only a couple of sections loading the method I used worked fast enough. However after all five sections were added it started to slow down. If another five sections were added, things would really start to crawl.

How it works is this; at the moment the website makes separate requests for each of the sections. Pretendpaper uses xmlrpc to do the separate requests. Xmlrpc takes quite a bit of cpu power to decode using javascript. So the slowdown is not on the server side of things, it is on the client side.

It should be able to load the whole default front page with one request. So I need to grab everything in one big request. Which is a batching/pipelining optimization. You can often do several things at once faster than doing them one at a time.

Also the front sections should be populated first. So the events tab, and the latest tab should be populated, and then the rest should be.


A work in progress continues.

Lots and lots of work left to do. The site itself is a work of art, one which is not nearly finished. There are so many planned features that have not been implemented, and so many little things that need to be improved.


Monday, August 08, 2005

Work on holepit continues.

Work on my shareware game holepit continues. After about a year break from it, recently I have decided to continue to do things on it when I have time.

The premise of the game is to move blocks into a hole. Every other part of the game comes from there.

To make the game more interesting I am adding a number of incentives to keep playing. One of them is this tunnel special bonus stage. It is kind of like a few different games have used before, where you run down a tunnel collecting things.


I am also adding another character modeled by Geoffrey Bantle. The transvestite cleaning lady. Complete with vacuume, and eventually stubble. She should add a bunch more game-play opportunaties, and make it more interesting overall. She will be the fifth character in the game. As the game involves playing against the other characters, there really does need to be more of them for holepit to be fun.

The character still needs a lot of work. Maybe another five hours painting. Then another 30 hours animating. Then I have to work on the characters AI. Which may take ten hours. Then I need to fit her into the character selection screen, as well as do any other graphical elements she may add to the game. All up, she will probably take a couple of months to get into the game in my spare time.

For the improvements I am listening very carefully to the advice of a select few people I think have a very good understanding of making games. Then I took this list of improvements and just started working on what interests me in my spare time.

An older version of the game got these review scores:
Graphics 69%
Sound 75%
Playability 33%
Longevity 20%

So with these review scores in mind I have been working on playability and longevity features of the game. The tunnel bonus round, and the extra character should make it a better game to play. I am also introducing scoring, and telling people which level they are on out of the total amount of levels. So that they have something to move towards. Not just playing the game for mindless fun. Instead they will have something to achieve.


Play testing the game has also been done during these changes. People play for the first time, and I watch them very closely to see what troubles they are having, and what game play options they like the best. Now I have chosen those options as the default.


The look of the game is one thing which needs improving a lot too. I have had a bunch of people tell me the game looks crude and not very well done. That the top down view is old, and does not provide a good look at the characters. You can't see much of them only looking at thier heads.

The top down view has been changed to a 45 degree view. Giving the player a much better view of the characters. I have also made the arena enclosed within the screen boundaries. This makes it easier for people to realise that there is a barrier there where the blocks will bounce off.


I am attacking each of the things which does not look so good one at a time. All these changes are small, and only make a small contribution to the look of the game. However when all added up they are slowly making the game look better. One of these things I have done is taken away the system mouse cursor, and replaced it with a cool looking 3D hand in the menu screen.


Another gameplay/graphical improvement has been to make the game play in window mode by default, and have the mouse cursor constrained to within the border of the window. By being in a window the graphics look better than full screen. I think this may have something to do with the way it was developed in a window. Windowed mode is the way I like to play it, so naturally I think this may be the way others like to play it too.

Many more things are needed for holepit. Lots and lots of polish. Then I *hope* other people will start to like it as much as me.

Sunday, March 27, 2005

pygame progress - one last piece.

One last thing needs to be done before we can release pygame 1.7 for windows python 2.4.

sdl_mixer needs to work. After much fiddling, I managed to get it working with everything except mod playing working. mp3s, oggs, midis, wavs all worked.

Hopefully next weekend there will be some goodness, and pygame 1.7 can be released for windows.

Tuesday, February 08, 2005

Pygame progress.

I got pygame compiling with python2.4 and mingw+msys today!! Not too many problems were encountered. It needed one distutils patch to make correct paths to feed to gcc. The msys shell needs paths like /c/Python24/lib not c:\Python24\lib. As well as a couple of little fixes in two source files for things that C can't do but a C++ compiler on windows can do. Pete was using the vc6 C++ compiler I think. After those couple of things I had most of the examples up and running :)

Now I need to do a few more fixes.

First is getting the numeric modules surfarray and soundarray working. I need to download the numeric stuff for that to happen.

Next is to fix up the config.py file and the config_mingw.py file to generate an acceptable Setup file. As well as detecting dependencies correctly with msys.

Then comes the task of finding all the api differences in the latest SDL. I also need to compile the latest SDL.

Finally we need integrate and test all the patches that have been flying in, and then give everything a big test.

Hopefully at the end of all that we will have a new pygame binary for windows on python 2.4 and SDL 1.2.8 :) Which means an alpha release for pygame 1.7!!!