Red Hat Talks CentOS Past, Present, and Future

The more lipstick you put on CentOS Stream, the more it looks like a nightly build.

That was the takeaway from a couple of "Ask the Experts" Q&A sessions on CentOS Stream during April's online Red Hat Summit, which saw experts at both sessions hit by a barrage of questions about the all but dead "CentOS Linux." That's how Red Hat now prefers to reference the operating system previously known merely as CentOS, a downstream clone of Red Hat Enterprise Linux that continues to be widely used by enterprises to avoid paying for RHEL when they don't need support.

The sessions were intended to be an opportunity for Red Hat to make the case that the new Red Hat Stream, which moves CentOS upstream from RHEL to perpetually serve as the build platform for the upcoming version of RHEL, would be more valuable to both current CentOS users and Red Hat devs than CentOS 8, which is now slated to reach end of life at year's end instead of 2029 as originally scheduled. CentOS 7 remains on schedule to be sunsetted in June 2024.

Red Hat's announcement in December that it was bringing CentOS Linux to an end immediately spawned two CentOS replacement projects: AlmaLinux, backed to the tune of $1 million annually by CloudLinux, and Rocky Linux, from CentOS co-founder Gregory Kurtzer. AlmaLinux saw its first release in March, and Rocky Linux came out with its first release candidate just days after the Red Hat Summit.

Everything Is the Same, Only Much Different

Both of the 30-minute Ask the Experts sessions were live, one timed for the APAC market and the other for North American audiences. Each featured three Red Hat experts answering questions that attendees typed into a text box and which could be upvoted by others. Questions were chosen by the number of upvotes.

At the APAC event, the first couple of questions seemed to be seeded to get the conversation Red Hat wanted, starting with a question about whether Stream would "send a stream of nightly releases or a more scheduled cadence," which was fielded by Brian Exelbierd, the community business owner for RHEL product management.

"There are two points of consideration here," he said. "The first is that new code and CentOS Stream is built continuously. As those patches are accepted, pass testing, and are ready to go, they are built and made available, so you will be able to in theory pull updates through DNF Yum [Red Hat's package manager] regularly and get access to that code."

In other words, instead of being the stable RHEL clone that only changes once every six months (after a new RHEL release), CentOS Stream is slated to be a constantly morphing OS that represents the work that Red Hat's engineers are doing building the upcoming RHEL release.

"The other side of this is install media, which is another concern that a lot of people have," he continued. "Currently the goal of the group is they want to build everything entirely and build out install media once a week, and then they will promote some of those upwards so that those would be the default pieces that you would install."

In spite of the fact that the distribution will change frequently enough to necessitate something like a weekly ISO, Red Hat's experts saw no problem with users swapping out CentOS 8 with CentOS Stream.

After Hervé Lemaitre, Red Hat's RHEL platform business strategist and chief technologist EMEA, answered a question on the difficulty of upgrading from CentOS 8 to Stream (it's easy, there's "like two commands to run"), Exelbierd interrupted to advise, "The only thing that I would add to that is, go for it. Our vice president of engineering loves to talk about how he could sneak into your data center at night and replace your CentOS Linux with CentOS Stream and there's a lot of cases where you just wouldn't notice."

Maybe. We would be surprised.

Features, Bugs, and Updates

Another question was whether the experts would categorize CentOS Stream as a stable, unstable, or a testing distribution.

"I characterize it as stable, and the rationale behind that is two things," Exelbierd said. "One, it's expected to run and it should run. It should not crash regularly; it should not fall over; it should not be like just random experiences, which I would expect out of an unstable operating system. It is not specifically geared at testing, because the code that goes into CentOS Stream has been tested. It has undergone testing; it has undergone validation; it is ready to go. It is the code that we would ship to our customers in RHEL were we shipping that day."

As stable as it might be, Exelbierd stressed again that CentOS Stream will not have a traditional release cycle, nor will it be exactly like the rolling releases that are popular with Linux desktop users, where changes are constantly introduced into an always up-to-date stable system. 

"The other thing to remember about CentOS Stream is, it is a continuous delivery of features, bugs, updates, etc.," he said. "There's no way to selectively easily pick the pieces you want, because the goal of the operating system is to give you all of that freshness, that tested code that's ready to go now."

Building Your Own Competition

At the discussion for North America, an early question was about Red Hat not getting the response from the CentOS changes that it had expected.

"I think the lesson we learned is that it is a mistake to sponsor your own downstream," Gunnar Hellekson, director of product management at Red Hat, answered. "Other people should be able to create downstream products. That's what open source is all about. But in order to create the kind of focus and coherence that you need, in order to create a successful project and a successful product, you really want to focus all of your effort as far upstream of development as possible."

This prompted a response from Mike McGrath, Red Hat's VP of Linux engineering.

"One of the big lessons I took away from this was that people at that point had not fully understood what CentOS Stream was," he said. "I think that we overestimated how much people had paid attention to and understood what Stream was, and we still get questions about it. I think a big takeaway for me was that people thought that Stream was completely different and incompatible in every way with what CentOS Linux was."

Perhaps the most revealing observation on what the real difference between CentOS Linux and CentOS Stream was came from McGrath, about a third of the way through the second session.

"With CentOS Stream you're going to be getting something as soon as we think it's ready for release. It will move much faster, but that's not to say that it gets rebases as fast as Fedora does," he said. "This is all still backporting and the normal development that we've done internally for years for RHEL. It's just now we're doing it out in the public. You know, we've had CentOS Stream internally since probably RHEL 2, it's just that we would call it a 'nightly build.'"