making apps, making webs.

There's this worlds largest battery project going on in South Australia. Tesla sold a financial product. The government needed to cover some of the risk of when power lines go down, or if something goes wrong in another part of the network. It needs to cover this risk quickly for political reasons. The chance of another similar storm knocking out the power lines again, which now have bigger maintenance crews, is very small. But if there is another blackout and they didn't do anything? They'd be in big trouble with the newspapers. So Tesla really sold a risk product. Since SA could have spent the money on more generation. But not all people understand that. The 100MW stage of that wind farm cost $250 million, and took some years to reach agreement, and some years to build. By promising to build the battery quicker, they have covered that risk during the time they need it. They could have installed another wind farm the same size in

Drawing different types of pixels can be quicker if you know about the image you are drawing, and if you know that drawing parts of the image with specialised blitters is quicker. A good example is if your image is 25% areas of large blocks of either white or black. Using a specialised fill routine to just draw those big blocks of color is lots quicker. This is because there is usually an optimised, and hardware accelerated fill routine. See all this whitespace? (all the white parts on your screen) These can be drawn really quickly with ASIC fill hardware rather than a slow GPU running a general purpose image blitter. Another example is like this Alien image. The edges of the image are transparent, but the middle has no transparency. Since drawing transparent images is slow, using a different drawing routine for the middle part than the edges turns out to be faster. Alien graphic used in Pygame Zero teaching framework documentation.   Here is a proof of concept which d

There's four things I'd like to work on for faster pygame apps on slower computers (like the Raspberry Pi/Orange Pi). Dirty rect optimizations, to speed up naieve drawing. SDL2, for better hardware support. C/Cython versions of pygame.sprite Surface.blits, a batching API for drawing many images at once. The general idea with all of these is to take techniques which have proven to improve the performance of real apps on slower computers. Even though, for some of them there are still open questions, I think they are worth pursuing.  Even though more advanced techniques can be used by people to work around these issues, this should be fast even if people do things the easy way on slower computers. Anyway... this is a summary of the discussions and research on what to do, and a declaration of intent. Dirty Rect optimizations. First a couple of definitions. "Dirty Rect" is a technique in graphics where you only update the parts that changed (the dirty parts

Help? Since asking people to use pip to install things, I get a lot of feedback on pip not working. Feedback like this. "Our fun packaging Jargon " What is a pip ? What's it for? It's not built into python?  It's the almost-default and almost-standard tool for installing python code. Pip almost works a lot of the time. You install things from pypi. I should download pypy? No, pee why, pee eye. The cheeseshop. You're weird. Just call it pee why pee eye. But why is it called pip? I don't know. "Feedback like this." pip is broken on the raspberian pip3 doesn't exist on windows People have an old pip. Old pip doesn't support wheels. What are wheels ? It's a cute bit of jargon to mean a zip file with python code in it structured in a nice way. I heard about eggs... tell me about eggs? Well, eggs are another zip file with python code in it. Used mainly by easy_install. Easy install? Let's use that, this is all too mu

Some notes about the current state of comments, and thoughts about future plans are below. 0) Spam. So far there hasn't been comment spam on the new comment system(yet!)... but eventually some will get through the different layers of defense. Which are a web app firewall (through cloudflare) (which helps block bots and abusive proxy servers), user signups required, limits on the number of accounts and comments that can be posted per hour, making the spam useless for SEO(nofollow on links) and then a spam classifier. The spam classifier is pretty basic. It only uses the message text so far, and not other features like 'how old is the poster account', or 'does the user account have another other accounts linked'. Despite that, and only having been trained on a few thousand messages it seems classification works pretty well. It scores 0.97 for ham, and 0.76 for spam when it is cross validated on the test set. It's sort of weird having sour

"BULLSHIT! That bullet didn't even hit me!" they cried as the space ship starts to play the destruction animation, and Player 1 life counter drops by one. Similar cries of BULLSHIT! are heard all over the world as thousands of people lose an imaginary life to imperfect collision detection every day. Do you want random people on the internet to cry bullshit at your game? Well do ya punk? Bounding boxes are used by many games to detect if two things collide. Either a rectangle, a circle, a box or a sphere are used as a crude way to check if two things collide. However for many games that just isn't enough. Players can see that something didn't collide, so they are going to be crying foul if you just use bounding boxes. Pygame added fast and easy pixel perfect collision detection. So no more bullshit collisions ok? Code to go along with this article can be found here ( https://github.com/illume/pixel_perfect_collision ). Why rectangles aren

Sick of config file soup cluttering up your repo? Me too. However there is a way to at least clean it up for many python tools. Some of the tools you might use and the config files they support... flake8 - .flake8 , setup.cfg , tox.ini , and config/flake8 on Windows pytest - pytest.ini , tox.ini , setup.cfg coverage.py - .coveragerc , setup.cfg , tox.ini mypy - setup.cfg , mypy.ini tox - tox.ini  Can mypy use setup.cfg as well? OK, you've convinced me. -- Guido With that mypy now also supports setup.cfg, and we can all remove many more config files. The rules for precedence are easy: read --config-file option - if it's incorrect, exit read [tool].ini - if correct, stop read setup.cfg   How to config with setup.cfg? Here's a list to the configuration documentation for setup.cfg. coverage config pytest config flake8 config mypy config behave What does a setup.cfg look like now? Here's an example setup.cfg for you with various too

Type Tracing - as a program runs you trace it and record the types of variables coming in and out of functions, and being assigned to variables. Is Type Tracing useful for providing quality benefits, documentation benefits, porting benefits, and also speed benefits to real python programs? Python is now a gradually typed language , meaning that you can gradually apply types and along with type inference statically check your code is correct. Once you have added types to everything, you can catch quite a lot of errors. For several years I've been using the new type checking tools that have been popping up in the python ecosystem. I've given talks to user groups about them, and also trained people to use them. I think a lot of people are using these tools without even realizing it. They see in their IDE warnings about type issues, and methods are automatically completed for them. But I've always had some thoughts in the back of my head about recording types at run

tldr; you can do jobs, queues, real time change feeds, time series, object store, document store, full text search with PostgreSQL. How to, pros/cons, rough performance and complexity levels are all discussed. Many sources and relevant documentation is linked to. Your database is first. But can PostgreSQL be second? Web/app projects these days often have many distributed parts. It's not uncommon for groups to use the right tool for the job. The right tools are often something like the choice below. Redis for queuing, and caching. Elastic Search for searching, and log stash. Influxdb or RRD for timeseries. S3 for an object store. PostgreSQL for relational data with constraints, and validation via schemas. Celery for job queues. Kafka for a buffer of queues or stream processing. Exception logging with PostgreSQL (perhaps using Sentry) KDB for low latency analytics on your column oriented data. Mongo/ZODB for storing documents JSON (or mangodb for /dev/null replacemen