Include your tests in your package, and then whoever can install your package can install your tests.
At pygame we have had much luck with including the tests with pygame.
Then people can just do "import pygame.tests;pygame.tests.run()"
It makes distributed testing easy, since now any of your users can run all the tests without having to have anything else installed(like source code, and a development environment).
Then you need one machine to package your build for windows, and then it can be tested on lots of other windows machines. (same with mac). The builds are uploaded to the testing machines(that don't need a dev environment set up), and then tested there.
Cherrypy and numpy(numpy.test()) also include tests with their downloads, and including tests is in the original unittest white paper.
Then creating a network of test bots is trivial. Upload your package to them, or have them download it using http, and then get them to run the tests - then upload the test results to a central server.
Rather than them have to install a development environment - they can just download a pre-packaged version and run the tests.
Users are now equipped to be testers.
Now all of your users can also be testers. However they install your package they can run your tests. They don't require the source code to run the tests.
If they install with a apt-get install, an easy_install, a windows installer, or a mac installer, they can run your tests.
Someone posts a bug, you can make a bug fix, then ask them to download your latest pre-release and get them to run "yourpackage.test()". Saves you and the bug reporter time.
So rather than having 1-20 testing machines you can now have 1000's of machines testing your code.
Put your tests in setup.py install
Have your tests run at the end of your:
python setup.py install
Make them fail if they fail your tests. Every time someone installs, your tests are run :)
Then have your setup ask the user to upload to your test result collection web page.