{"id":295,"date":"2022-02-12T13:03:04","date_gmt":"2022-02-12T18:03:04","guid":{"rendered":"https:\/\/www.paulreed.ca\/?p=295"},"modified":"2022-02-12T13:08:23","modified_gmt":"2022-02-12T18:08:23","slug":"295","status":"publish","type":"post","link":"https:\/\/paulreed.ca\/?p=295","title":{"rendered":"CentOS 8 is end of life (EOL). Now what?"},"content":{"rendered":"\n<p id=\"there-were-many-reasons-people-came-to-use-centos-as-an-alternative-linux-platform-to-redhat-enterprise-linux-rhel-centos-was-originally-built-as-a-downstream-release-of-rhel-which-was-free-to-use-without-support-centos-became-the-de-facto-standard-for-many-organizations-that-did-not-want-to-use-rhel-for-production-workload-since-it-s-basically-the-same-thing-just-rebranded\">There were many reasons people came to use CentOS as an alternative Linux platform to RedHat Enterprise Linux (RHEL). CentOS was originally built as a downstream release of RHEL, which was free to use without support. CentOS became the de facto standard for many organizations that did not want to use RHEL for production workload, since it\u2019s basically the same thing, just rebranded.<\/p>\n\n\n\n<p>While CentOS was originally a community project, over time RedHat began contributing substantially and eventually took over maintenance of the linux Variant. This allowed them to shift the direction of the CentOS project, much to the dismay of its users. Especially those who rely on it for their production workloads.<\/p>\n\n\n\n<p>With the shift to \u201cStream\u201d, RedHat has moved CentOS 8 from being a downstream release (built after RHEL) to an upstream release (built before RHEL). This means that instead of a stabilized release, CentOS Stream is now somewhat of a development\/QA release, which is unacceptable for production system use across most organizations.<\/p>\n\n\n\n<p>With all the vulnerabilities in the wild (and with numbers growing rapidly by the day), it\u2019s simply not an option to leave these systems as is. There are several options available including re-installation to a new OS, updating to CentOS Stream 8, or migration to another downstream RHEL variant (of which there are many available today).<\/p>\n\n\n\n<p><\/p>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"updates-not-available-the-process-is-broken\">Updates not available, the process is broken.<\/h1>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh6.googleusercontent.com\/IcOfKIMU0WAX9tIt4ij0ZyRNbI3zuHFiOTPSi2i0-pkv0ogMjmAamU1NAt_UO4c5cqW6cS0v4nxHObX2lywfxjchHQn16GElswG1FJQtZ3KoNvvBb_ka4pG7vu8LygN9FlIi_-4M\" alt=\"Error: Failed to download metadata for repo \u2018appstream\u2019: Cannot prepare internal mirrorlist: No URLs in mirrorlist\"\/><\/figure>\n\n\n\n<p>Not only did CentOS 8 go end of life, but the latest updates have been archived to the CentOS vault, removing the packages from mirrors. This caused a breaking change in the update manager for any systems using the internet based mirrors. If you\u2019re using an internal proxy system to stage the updates, you may not even realize that you are no longer receiving updates for CentOS 8.<\/p>\n\n\n\n<p>This means that a lot of CentOS 8 hosts can\u2019t even update to the most recent packages that are available, at least not without modifications.&nbsp;<\/p>\n\n\n\n<p><strong>This brings us to the first task\u2026&nbsp;<\/strong><\/p>\n\n\n\n<p>The first task in the example module addresses this issue at scale. Systems should be updated to the latest version of CentOS 8 that is available in the CentOS 8 vault, prior to migration to any other Linux variant.&nbsp;<\/p>\n\n\n\n<p>In the attached example puppet module, the `<strong>to_centos_vault<\/strong>` task will update the yum repos to point to the CentOS 8 vault so you can get the most recent updates that are available. While there will be no future updates, this will at least get you to the latest that were available up to the point of EOL.<br \/><strong>Note:<\/strong> If you\u2019re using any desired state (i.e. Puppet Code) to manage the current yum repos on your systems, you may (likely will) need to disable that prior to migrations and make modifications for the new target OS to be applied after migration.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"what-migration-options-do-you-have\">What migration options do you have?<\/h1>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh3.googleusercontent.com\/MquRosCY0Ujfw2kK0b6fAYkZgBMkxzA-u1WOGGbvs--S-Sa3kk71zy7ywYAAoYeulsQQRQkNuLwog1M74d3uj2n5grETQBw8S0Oft-kvZY-BkXmBJcT31HnuGk_uomV62kSkvLL4\" alt=\"\"\/><\/figure>\n\n\n\n<p>While this is not an exhaustive list, I will cover some of the most common options. The example module has tasks to convert to each of the following, which can be used at scale.<\/p>\n\n\n\n<p><strong>CentOS 8 Stream<\/strong><\/p>\n\n\n\n<p>If you\u2019re fine using an upstream release to RHEL, then CentOS Stream 8 is the path of least resistance. It\u2019s still EL8. It&#8217;s close to RHEL, but it is a few steps ahead, so maybe not the best choice for production workloads.<\/p>\n\n\n\n<p><strong>EL8 downstream alternatives (AlmaLinux 8 &amp; Rocky Linux 8)<\/strong><\/p>\n\n\n\n<p>Both Rocky and Alma are nearly identical alternatives to CentOS8. They are both downstream, rebranded, community supported variants of RHEL.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"automate-your-migration-with-puppet\">Automate your migration with Puppet<\/h1>\n\n\n\n<p><strong>Why use Puppet?&nbsp;<\/strong><\/p>\n\n\n\n<p>Scale, consistency, speed &amp; reduced risk.<\/p>\n\n\n\n<p>Simply put, if you\u2019re doing these migrations manually for a lot of systems, you may accidentally skip steps or end up with other inconsistencies, which could lead to your systems being left in an unstable, or even worse, an unrepairable state.&nbsp;<\/p>\n\n\n\n<p>Using Puppet Tasks allows for an efficient and consistent way to perform these migration tasks at scale. It will also save the output from the Tasks in a centralized manner for reporting and audit tracking purposes. If you\u2019re running these migrations individually from a shell, the output is typically lost as soon as you close the terminal, which can be disastrous for large migration projects that need to be tracked or migrations that are performed by a team of many individuals.<\/p>\n\n\n\n<p>If you have hundreds or even up to hundreds of thousands of systems to migrate, manual migration or using scripts on each host is nearly impossible. At the very least, it will take a significant amount of time and you may be exposed to multiple vulnerabilities before you can complete the project. Tasks can help significantly reduce the time it takes to migrate at scale.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"not-supported-but-it-works\">Not supported, but it works.<\/h1>\n\n\n\n<p>While this example module is fully functional and should work without issue, there are a few caveats and you are completely on your own for support.<\/p>\n\n\n\n<p>That said, If you\u2019re using CentOS 8 today, you\u2019re likely used to rolling on your own and are fine without official support anyway.<\/p>\n\n\n\n<p><strong>Use at your own risk<\/strong>&nbsp;<\/p>\n\n\n\n<p>There is no support for this workflow or module from Puppet, nor myself. <strong>You are on your own.<\/strong>&nbsp;<\/p>\n\n\n\n<p><strong>Applications may not support the target OS<\/strong><\/p>\n\n\n\n<p>Check your applications. This will convert the underlying OS. While it\u2019s still technically EL8, you may have issues with applications that don\u2019t support the target OS.&nbsp;<\/p>\n\n\n\n<p><strong>Have a backout plan<\/strong><\/p>\n\n\n\n<p>Use snapshots and backups before migration if possible. This is a potentially destructive operation, so act accordingly.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"the-centos8migrate-puppet-module\">The \u201ccentos8migrate\u201d Puppet Module<\/h1>\n\n\n\n<p>The example Puppet module can be found on GitHub here:<br \/><a href=\"https:\/\/github.com\/psreed\/centos8migrate\">https:\/\/github.com\/psreed\/centos8migrate<\/a><\/p>\n\n\n\n<p>With Puppet Enterprise, you can simply add this module to your Puppetfile in your current control repository to allow the tasks to become available in the PE Tasks interface:<\/p>\n\n\n\n<p>Puppetfile:&nbsp;<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table><tbody><tr><td><code>## centos8migrate<\/code><br \/><code>mod 'psreed-centos8migrate',&nbsp;&nbsp;<\/code><br \/>  <code>:git&nbsp; &nbsp; =&gt; 'https:\/\/github.com\/psreed\/centos8migrate.git',&nbsp;&nbsp;<\/code><br \/>  <code>:branch =&gt; 'main'<\/code><\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p>This module has been tested starting with a default CentOS 8.3 installation \u201cServer with GUI\u201d.&nbsp;<\/p>\n\n\n\n<p>These Puppet Tasks are simple, but functional. They should probably be expanded into Puppet Plans for use in production, which could include better error handling or adding workflows for your snapshot\/backup processes. At the very least, these can be used as building blocks for a more robust solution.<\/p>\n\n\n\n<p>The tasks include the following:<\/p>\n\n\n\n<p><strong>centos8migrate::to_centos_vault<\/strong><\/p>\n\n\n\n<p>This task will update YUM repos to point to the CentOS 8 vault so you can update to the last available updates. After running this task, followed by a `yum update`, you should have a system that\u2019s running CentOS 8.5 v2111<\/p>\n\n\n\n<p><strong>centos8migrate::to_centos_stream<\/strong><\/p>\n\n\n\n<p>This task will convert your system to CentOS Stream 8. It\u2019s the least impactful of the options, but if Stream is not a production option for you, consider one of the following tasks to convert to a different Linux variant.<\/p>\n\n\n\n<p><strong>centos8migrate::to_rocky<\/strong><\/p>\n\n\n\n<p>This task will convert CentOS 8 systems to Rocky Linux 8<\/p>\n\n\n\n<p><strong>centos8migrate::to_alma_linux<\/strong><\/p>\n\n\n\n<p>This task will convert CentOS 8 systems to AlmaLinux 8<\/p>\n\n\n\n<p><strong>Note<\/strong>: As a caution, you may want to temporarily disable the periodic Puppet run while these migration tasks are in progress. Yum will typically lock, however you could get into a race condition. Simply stop the puppet agent service before running these tasks, but don\u2019t stop the pxp-agent! You will still be able to execute the tasks using the pxp-agent and the puppet agent should be restarted after the migration\u2019s reboot (as part of the default process).<\/p>\n\n\n\n<p><\/p>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"shameless-plug-puppet-solutions-services\">Shameless Plug: Puppet Solutions &amp; Services<\/h1>\n\n\n\n<p><strong>Disclaimer:<\/strong> I currently work for <em>Puppet Inc.<\/em><\/p>\n\n\n\n<p>If your project is too big to handle on your own, or you would like to know how to manage your infrastructure more consistently and at scale, please reach out to Puppet at <a href=\"mailto:sales@puppet.com\">sales@puppet.com<\/a>\u00a0<\/p>\n\n\n\n<p>Puppet Enterprise and other Puppet Products and Services can help drastically reduce your technical debt and overhead through automated workflows. Let\u2019s discuss how to better get you to your goals of a stable, compliant, automated infrastructure whether it\u2019s on premise or in the cloud!<\/p>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"good-luck\">Good Luck!<\/h1>\n\n\n\n<p>Hopefully this will help if you have a CentOS 8 migration project upcoming or already underway!<\/p>\n\n\n\n<p>Feel free to reach out to us at Puppet if you have any questions on how our automation can help you!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>There were many reasons people came to use CentOS as an alternative Linux platform to RedHat Enterprise Linux (RHEL). CentOS was originally built as a downstream release of RHEL, which was free to use without support. CentOS became the de facto standard for many organizations that did not want to use RHEL for production workload, [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,25,22,54,49,38],"tags":[69,47,55],"_links":{"self":[{"href":"https:\/\/paulreed.ca\/index.php?rest_route=\/wp\/v2\/posts\/295"}],"collection":[{"href":"https:\/\/paulreed.ca\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/paulreed.ca\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/paulreed.ca\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/paulreed.ca\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=295"}],"version-history":[{"count":5,"href":"https:\/\/paulreed.ca\/index.php?rest_route=\/wp\/v2\/posts\/295\/revisions"}],"predecessor-version":[{"id":305,"href":"https:\/\/paulreed.ca\/index.php?rest_route=\/wp\/v2\/posts\/295\/revisions\/305"}],"wp:attachment":[{"href":"https:\/\/paulreed.ca\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=295"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/paulreed.ca\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=295"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/paulreed.ca\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=295"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}