python build bots down... maybe they need a spectacularly adequate build page instead?
Seems the python build bot pages are down. Maybe something simpler is needed instead of build bots? Something that requires less maintenance.
The beauty of this is that it can be easily de-centralized.
Could even use pypi infrastructure for this now. Each buildbot has a new project setup, which then updates the pypi project each time it builds. Have a special pypi tag(category) for python build bots, so people can easily search for them. To reduce the spam on pypi, just make it mark the releases as hidden... so they don't show up. The results are added to the pypi listing.
Probably only needs one cmd added to the python setup script to do the upload(based on existing pypi code).
No central authority, and very simple. Anyone who wants to run a 'buildbot' can without authority.
Can probably make it all work so the buildbot runner just does a special 'python setup.py buildbot' added to their cron, assuming they are already registered with pypi. So they don't need to setup a special buildbot, just use the existing python source code to do it.
work required:
As a bonus people could even use pypi to keep track of python, and not just python modules. It also turns python development into a series of continuous releases, stabalized when the tree decides to pass all the tests.
Something like this pseudo code would work:
Anyone else want to finish the script? To add building, and installing into a local directory, and the actual 'upload_build_results.py' to pypi etc.
update from svn
compile
run tests if compile completes
upload results to a simple page
(configure(stdout, stderr),
build(stdout, stderr),
test results(stdout, stderr),
binaries).
The beauty of this is that it can be easily de-centralized.
Could even use pypi infrastructure for this now. Each buildbot has a new project setup, which then updates the pypi project each time it builds. Have a special pypi tag(category) for python build bots, so people can easily search for them. To reduce the spam on pypi, just make it mark the releases as hidden... so they don't show up. The results are added to the pypi listing.
Probably only needs one cmd added to the python setup script to do the upload(based on existing pypi code).
No central authority, and very simple. Anyone who wants to run a 'buildbot' can without authority.
Can probably make it all work so the buildbot runner just does a special 'python setup.py buildbot' added to their cron, assuming they are already registered with pypi. So they don't need to setup a special buildbot, just use the existing python source code to do it.
work required:
- write a script to add a buildbot command. To be added to python source code when finished.
- uploads the results of the build/test into a .zip file to pypi. The zip contains stderr and stdout of each config, build, test stages. Also contains the resulting binaries. eg, .msi .exe .tar.gz etc.
- Only update binaries if all tests pass.
- Updates from svn, and adds the svn revision to the python version.
- uploads the results of the build/test into a .zip file to pypi. The zip contains stderr and stdout of each config, build, test stages. Also contains the resulting binaries. eg, .msi .exe .tar.gz etc.
- LATER - add pypi category for python build bot results.
- LATER - write code to parse results from any uploads into that category into one main page with all results listed.
As a bonus people could even use pypi to keep track of python, and not just python modules. It also turns python development into a series of continuous releases, stabalized when the tree decides to pass all the tests.
Something like this pseudo code would work:
./configure > config_stdout.txt \
2> config_stderr.txt && \
make > make_stdout.txt 2> make_stderr.txt && \
run_tests.py test_stdout.txt 2> test_stderr.txt && \
upload_build_results.py my_python_buildbot_pypi_page
Anyone else want to finish the script? To add building, and installing into a local directory, and the actual 'upload_build_results.py' to pypi etc.
Comments
Also, we *need* some amount of authority in this system. We need "official" build slaves representative of certain platforms, and we need some amount of dedicated time on those slaves.
Something lighter-weight than say, buildbot wouldn't be a bad thing - for example titus brown's pony-build system (http://github.com/ctb/pony-build/commits/master) aims to be a simpler, lighter weight CI server.
But we *still* need dedicated boxes for platforms, and ultimately someone to be on the hook to maintain the infrastructure. Offloading it to infrastructure that wasn't designed for it isn't the answer.
what you describe however
it looks like the end of your comment was cut off.
If not pypi, then a google apps webhost, or something else.
I think you are probably right about not putting load on pypi. However lots of python developers have accounts there, so maybe using those somehow would be good.
However, even with no authentication the build/test results could be useful.
If you have 30 core developers making changes every week, if each of them does 'python setup.py buildbot' at the end of their checkin, then you have 30 maintained buildbots.
Not to mention X other people who decide to submit their test results. X amount of people could easily be 1000s of people who decide to update their test results.
Anyway... this is the approach we working towards with pygame. We already have something like it for the last couple of years. Except the machines are maintained by 1-2 very nice people over a few different projects.
The step with this next pygame release is to distribute the testing/building more so - to take authentication out of the loop.
thanks,
--titus
After talking with Titus, I think his system will be able to solve just that problem.
cu,
I'm not sure about this. I'm considering the way things broken in this particular instance because it seems typical of how these things go. In this case, the entire system hosting the BuildBot build master was upgraded (I suppose because some hardware died and so the need to replace that was taken as an opportunity to change some other things - never a good idea, but there you go). As a result of that upgrade, the build master's configuration stopped working (I don't have the details of this, though - I just know that some people said the configuration needs to be updated before it will work again). Then, no one cared.
Does Titus's system solve the problem that humans can change systems in a way that causes them to stop working? If so, then I guess he has solved this problem. Otherwise, it seems not.
if the build results are coming from working developer machines, then yes. It doesn't matter if half the developers machines stop working - at least some will still be able to upload results. The server part of it should be simplified so any web server that can serve up files will work.
The old buildbot system python used required separate machines to be configured to do the building and testing. The proposed system makes use of existing machines for building and testing.
Anyone who can compile python should be able to post build results.
Anyone able to run a build of python should be able to run the test suite - and submit the test results.
A progressing Microsoft statement gives some comprehension of how the association designs the landing of new Office 365 ProPlus features. Office.com/setup Microsoft uses the equivalent "channels" language for MS Office 365 ProPlus incorporates revives as it does with Windows 10 feature invigorates. Office.com/setup To activate office setup you need to visit Office.com/setup where you can find what all you can do on office setup.office Remote turns your phone into a smart remote that interacts with Microsoft Office on your PC. Office.com/setup The app lets you control Word, Excel, and PowerPoint from across the room, Office.com/setup so you can walk around freely during presentations.