Tuesday, September 16, 2008

pygame.test -- moving testing forward.

We are moving to including the tests with an installed python package... pygame.
>>> import pygame
>>> pygame.test.run()

Why?


Why include tests in pygame? Rather than only with the build process?

  • More people will run the tests.
  • people can run the tests to see if everything works in their own programs.
  • Can run tests on a persons computer to see if everything in pygame+ your program is working.
  • Which driver/function combination works, or works fastest? Run tests and find out.
  • Testing a py2exe/pyapp generated binary is much easier.
  • Reuse of our testing enhancements for their own tests.
  • Reporting bugs would be easier. Since everyone could run unittests, since they will be installed everywhere pygame is installed.
  • Result submission of unittests easy. This would result in a much larger base of computers running the unittests and submitting problems. This would be opt in of course.
  • Make the testing stuff more a library, than a framework.
  • Allow people submit unittests more easily. Since they won't require a source release of pygame to write unittests. eg people using a windows binary install of pygame, or a ubuntu binary install can run and submit unittests.
  • Make testing easier for people. The easier it is, the more worthwhile testing is.


A few other things it might do.


# load your own test into the test runner.
>>> pygame.test.load("tests/mytest.py")

# pass commands to the test runner.
>>> test_results = pygame.test.run(tags="interactive")

# Submit the test results for review.
>>> pygame.test.submit_test_results(test_results)

# submit a test you wrote for review to the pygame developers.
>>> pygame.test.submit_test("tests/my_new_test.py")


Nicholas and I have been discussing this for a while, and have now moved discussion to the pygame mailing list. However we would appreciate any thoughts from outside of the pygame community too.

1 comment:

kumar said...

I've been a long time proponent of distributing tests with a python package. I could not have said this better myself. Thank you.