The ARM plan for CentOS

ARM is an interesting platform for many reasons and many people around the CentOS ecosystem have often spoken about a native, maintained and in-sync CentOS ARM build might be. But a couple of major drawbacks have kept us ( and others ) from being able to deliver on this. The most important ones are that the CentOS-6 codebase is now old enough that a lot of the really cool things in recent ARM development were not included in there ( gcc ver, kernel ver, glibc etc ), and backporting large chunks of this code is truely beyond the scope of what we can do and have done in the past within the realm of the Core SIG. Also, making that level of change automatically forks the CentOS codebase, creating potential package orphans. This problem goes away with the CentOS-7 codebase, just looking at the RHEL7beta1 codebase, its clear that we should be able to build a reasonable ARM story around it.

Another challenge has been finding and working with reasonable hardware that we might be able to target CentOS ARM on. To that end, David Power and the guys at Boston UK have provided us a couple of instances in their Viridis, ARM as a Service Cloud to start the process off with. These are ARM32 HighBank nodes, that are able to run builds and tests at reasonable speeds ( the test gcc build that Johnny ran last week took ~ 8 hrs to finish ).

What we are then looking to do from here is bootstrap the rhel7beta codebase against fedora19 ( which runs really well on these nodes ) for an ARM32 build. Once we have that in place, we are going to try and see if we can get to a self-hosting state and prepare ourselves for the EL7 GA announcement. The aim being to try and build + deliver an ARM32 distro in sync with the mainline x86_64 distro.

This is going to be a fair bit of work, and towards that we are going to need people to come help with the builds, with the testing and we are also going to need more vendors to offer us hardware that we can use to build and test against. We have a mailing list on at called Arm-Dev, we will try and focus all the interest in ARM onto that list, so come and join the effort.

The present state is that Johnny is stress testing the nodes, doing some test builds, writing up the mock configs. Once this is done, we will get the reimzul builders running on there, and start accepting build requests, build patches and start pushing build logs etc.

One challenge that still remains is that the big Red Hat TM and branding hunt into the rhel7beta code has not happened – and this is blocking the i686 as well as the x86_64 rpms from being publicly visible. We do have a small plan for that, and it involves kicking off a community exercise in the second week to April and to build both a whitelist and a blacklist of code, rpms and to write some automated patching resources to handle the whitelisted content. Details about that in another post soon.

In the mean time, welcome to the ARM32 build effort on CentOS-7beta ( and ongoing ). We’ll see you on the Arm-Dev mailing list.

- KB

Downtime ahead for all services


While the internal network in the cluster ( which is part of ) has been on gigabit for years, the uplink from there has been on 100mbps. Today I’m hoping to go into the DC and upgrade that to a gigabit as well. While I am there, I also want to consolidate some of the infra. So there is going to be some downtime on services hosted there. Includes,, centbot, and some of our build services. Downtime is expected to start around 11:30 UTC and everything should be back online by 18:00 UTC today, Sun Mar 23rd.

Concerns, help, comments and all other kinds of feedback welcome in #centos-devel on

Shout out to the guys at Coreix ( ) who sponsor this rack. If you need hosting in the London area, include them in your short-list.

- KB

OpenStack Icehouse on CentOS-6

Openstack Icehouse, the next major version release in Openstack is coming soon ( target April 2014 ); and the RDO folks are hosting a couple of test days on the 19th and 20th of March so everyone can trial the code, the packaging, the update process and delivery mechanisms around this new release. If you have used openstack in the past, but not used the RDO distribution for it, these tests days can be a great intro towards that as well.

Details on the test days can be found on the RDO site at :, as a prep I highly recommend starting with a clean CentOS-6.5 minimal install and running through the RDO quick start guide.

Update: The test schedule got postponed to March 25th and 26th.

- KB

First community contribution

And there we have it, the first community contribution via is now live.

Andreas Thienemann’s offered to test and do some work of the work around the i686 CentOS-7 builds; and has gone through to propose mock configs that allow the i686 tree to be bootstrapped. I’ve just merged his pull request, and pushed it to the builders. The entire distro churn for i686 is now running, with details at and you can see the build reports published in realtime at

This should allow everyone to quantify the scope of work we might be looking at, if as a community, we decide to maintain a i686 tree + installer for CentOS-7.

- KB

Talking to Scientific Linux


Quite a few people have been getting in tough and asking if we are going to speak to the people at Scientific Linux to see how we might be able to collaborate with them. And I just wanted to say that yes, we are talking to them.

Its been quite a big change to the entire overall ecosystem and while we have had a few extra days to think about process proposals etc, they have’nt. So its going to take sometime to get expectations into proposals and into process; And we would like to have them come onbord in a way that works best for them, the organisation that backs them, and their user base.

- KB

We like ipv6

Here in the CentOS Project, we like ipv6.

In 2007 some of the private CentOS services started moving to ipv6; the entire buildservice has been ipv6 native since Sept 2008.

In 2009 we had running over ipv6, which meant essentially the entire CentOS mirror network had ipv6 native support. This isnt something for dualstacked machines – even machines without an ipv4 address were completely supported.

In 2011 we had offering an ipv6 hosted tracker service, and a couple of seeds offering content natively on ipv6.

Now, in 2014 Jan, the new CentOS website has launched with complete ipv6 support. This includes the CentOS forums and all services hosted behind the site.

Most, if not all, this ipv6 work is driven by the efforts of Anssi Johansson.

And I also want to take this opportunity to thank two of our long term supporters who have provided us the resources to offer these ipv6 services. since 2007 and more recently since 2011.

- KB is now open

As of about 30 minutes back, during the first ever CentOS OfficeHours session ( ) we announced that is nw open to public access. In the coming hours, you will start to see content flow in as its injected from the older private setup into the new public setup. In order to hit the points that people care about most, as early as possible – I am going to get the content being CentOS-7′s builds available.

This includes the mock configs, the metadata around those, the distribution blacklists and whitelists as well as the distro-tree management scripts.

User ACL’s are complex and the actual contribution acceptable policy is going to hit the centos-devel list for comments once its been through a board review. But in order to not let that hold us all back from being able to jump in and start contributing, I’m going to get repo mirrors on, allowing us to start acceping pull requests right away. Lets start with that, and if its an easier route for most people, we can just retain the mirrors for inbound content going forward.

For now, enjoy!

- KB

As a community, for the community.

Ten years ago, a few people came together to solve a larger ecosystem issue. Red Hat Linux had split into the more commericial enterprise user oriented RHEL and the more fast paced, innovation across the board platform – Fedora. People familiar with the Red Hat way of running linux, who did not fit into those demographics had to make a choice. At the same time another group of people came up with and delivered on an idea : A community oriented distro, but built from Red Hat’s enterprise linux sources. CentOS was born, growing to satisfy that niche and evolving into a complete ecosystem of its own.

These ten years have seen CentOS grow to be the platform of choice in Web Hosting, Voice over IP, Cloud, Virtualisation, HPC and many many homebrew efforts. As contributors to this process, we are all proud to have helped facilitate the social web movement, been the center of the devops mindset, been a catalyst to many infrastructure deployment, management and monitoring technologies emerging in the last few years – and many more wins from all over the world. Today you can find CentOS Linux everywhere. In almost every computing demographic, there is a fair chance that if you run into a self supported, onsite tweaked linux install the its likely be CentOS Linux. The logo is widely recognised, and every cloud vendor of note offers CentOS Linux images.

The characteristic resilience of the platform, the energy in the community and widespread adoption have made CentOS Linux an exceptionally productive base to work on emerging technologies, allowing the developers and contributors to focus on the bits they cared about most while having a safety net around them that the base platform isnt going to rapidly evolve away. And people have used it, extensively; to develop on, to innovate on, to deliver better, more user friendly interfaces to a vast set of solutions.

However, this momentum around the platform has been in silos, away from the core – an effect that has meant the larger community is unable to come together in one place to further, faster, broaden the scope of development and stabilisation around that development. Added to that is the overhead of every silo having to bootstrap its own community. Those of you who have met me, at conferences or otherwise, would know that this is a problem I consider to be the biggest failure of the CentOS Project – the failure to grow organically beyond the platform. And it is very much a project level issue since it stems from the way we’ve had to work and the processs we have had to adopt. But these were the choices we have had to make because of where we are, the code we consume and how we consume it, then map that over to how we’ve tried to protect the userbase and the community that has given us their trust. And even when there have been opportunities to tap the community energy into CentOS itself, we’ve not always been able to that – back to the way we work, the way we do things, and why we do them.

Looking for an example of what I mean ? Consider for a moment that there are in excess of fifty, relatively well known, repositories of rpm packages that exist today targetting CentOS consumer space directly; none of whom consume or base their work out of space.

The second biggest challenge we’ve faced has been the question of what-if the tap gets turned off. What if Red Hat becomes evil. Where does that leave the installed user base if all goes sour – and if it did, it will do so rapidly. Working and messaging around that threat isn’t easy, because it is very much a real and tangible threat. The small contributor base would never be able to either maintain or deliver on the established, expected user experience. Could we find the means to maintain it externally ? not easily and not without being disruptive to the existing user base – the people we care about most.

With the aim of addressing these issues, we are announcing today that the CentOS Project is joining forces with Red Hat. We can then make the next stretch of this exceptional journey alongside the giants whos shoulders we’ve stood on these last ten years. Our plan is to work together with the Red Hat OSAS team to remove the barriers we’ve had to build and live with in the past, create an open and publicly accountable meritocratratic governance group and to make the contribution process simple yet effective.

So how does this fit into the RHEL model and what does Red Hat have to say about this ? Personally, I still think the message we’ve promoted all along holds true even now: RHEL is a great commercial platform, over the last decade they have proven themselves repeatedly. The CentOS goal has always been to achieve a similar standing in the open communities. And its a goal that I will strive to ensure stays intact : To become the defacto community platform, by user choice.

Take a look at our official announcement at and come join us on irc, the mailing lists or the forums. I look forward to having a chat and answering any questions you might have about your CentOS story.

Here is a sampler of some of the things we hope to deliver in the coming months :
- A publicly accessible git repo that makes it easier for everyone to consume the code
- A SIG onboarding plan that allows anyone to propose and then curate a Special Interest Group
- An auditable process that allows a SIG to propose, build and then maintain their own CentOS Variant
- Better, bigger community outreach efforts
- Proposals to expand on the CentOS Extras/ and the CentOS Plus repositories
- A timeline with a tangible delivery plan, for the next 12 months
- Weekly board meetings, open to anyone who wants to come along and participate

And I’d like to welcome every one of you to come along with us, as we endeavor to expand the base platform into a bigger, more productive and more featuresome ecosystem. One that does not suffer the constraints of private privilege escalation and one that makes it possible for anyone to come along and help curate the bits they care about most. As a community, for the community.


Karanbir Singh
kbsingh or z00dax on #centos-devel @
tel: 0207.099.9389 / 0207.009.4455
Project Chair, The CentOS Project