XML Feeds
Autmated distro testing for CentOS - Step 1
Everytime a new CentOS Release is built, it takes about 48 hrs before people can start testing these releases. What I would like to be able to do is run some install tests within the buildsystem / QA infrastructure so that we can catch major issues earlier and fix them faster. But I need some help from everyone out there in order to achieve this. Central to this idea is the need for kickstarts and test scripts that reflect usage out in the users systems and scenarios. Lets consider kickstarts first.
The idea here is that we have a couple of machines, close to the buildsystem, that can take a centos release, and run through installs in a loop, reporting success or failure after each install. Even if we only start with one machine for each primary arch, i386 and x86_64, we should be able to cycle through over 120 install cycles per arch within 24 hours with the right kind of automation. The automation is what I would like to focus on, I need your help with the kickstarts.
If you are able to write up some kickstarts, or even just go through an install and adapt the resulting anaconda-ks.cfg file - send them to me. Send me kickstarts for just about any role or deployment strategy you can think of - to get the ideas flowing I've made a short list of thing that would be good to have kickstarts for; dont, in any way feel restricted to these roles:
My initial wish list is something along these lines:
- General install, with no changes in the installer, just filling out whatever is the minimum required input
- A minimal install that only needs cd-1
- A LAMP server
- A Postgresql server
- GNOME Desktop
- KDE Desktop
- Xen Dom0 host
- KVM host
- DHCP and Static networked server installs
- Install from CDrom
- Install over http
- Install over NFS
- Install from local disk
- Office SME server, with samba and email
Here's how you could contribute the kickstarts:
- Post it online and leave a comment here with a url pointing to it
- Post it to the centos-devel list ( http://lists.centos.org )
- Drop into #centos-devel on irc.freenode.net and point us at it there
The kickstarts that we collect will be published online ( unless you specifically ask me not to ) along with results from each install cycle and details that came from it.
I know its not very organised at this time. But along with the automation I will build a web interface that lists the kickstarts along with a git repo that lets people directly pull from and push patches into.
If you need some help getting started with kickstarts, take a look at the wiki article at : http://wiki.centos.org/TipsAndTricks/KickStart and the docs at : http://www.centos.org/docs/5/html/5.2/Installation_Guide/s1-kickstart2-file.html . I would also highly recommend looking at the /root/anaconda-ks.cfg file that is left behind on any new centos install. Then there is always google.
- KB
30/Mar/2010 09:05:00 am, 8 comments
Along the same lines, do you have virtualized images of the various system configurations you listed (at the CentOS 5.4 level) such that those images could be "yum upgrade"d to CentOS 5.5 pre-release versions? Seems like VMware snapshots or something equivalent would be useful for exercising the upgrade paths as well. 
Don't forget trying to install to different kind of meta-devices: LVM, DM-RAID, Linux MD-RAID, multipath, iSCSI, etc..Also, what about using qemu/virtualbox/kvm for simulating other archs/storage configurations?
Good luck.
Luciano,Go ahead and send me some kickstarts which do these things and I will make sure they get added into the test pool.
A bit unrelated, but what kind of system do you use to test the centos releases? Or is this bit is going to be "Step 2"? Kyriakos,the QA team does all the testing, there are some - very few - automated tests but the bulk of the testing is done by people.
What I would like to do is ease the pressure off the qa team and move to using more and more automation. Not to replace what people are already doing, but to bring in more test cases and extend the functionality trust as it cascades down from upstream down through our build services to the users.
I have a network install server for CentOS/Fedora. I'll send you the kickstart files after I sanitize them (remove root/user passwords, etc). I've also setup/maintained install servers for several large organizations and can give you real world kickstart scripts. I may also be able to help with automation if you would like. Feel free to email me if you are interested. I have a few different ideas on how to automate your cycling installs (depending on what sort of tests you want to run post install, if any). Hi,the Automation is essentially a Continuous integration tool that has a set of 'sanity' tests which are run on each machine / vm instance that as its installed. The tests really depend on what the kickstarts contain. Ultimately being able to have a process that does both role and functional tests would be ideal but at the moment, its only a case of running through a set of sanity tests to make sure that system does - at a very basic level - what the user's default expectation would be.
What tools are used inn this CI process to verify the installation procedure? We have a lot of different kickstarts and scripts we run and I would really like to have some smart way to verify that the installation works correctly all the time (both after we do changes and when a machine is installed). It would be really cool if there were some tool I could add to hudson and get hudson to run lots of testes every time I update a kickstart or a script :)