In this article, we will examine Red Hat Linux Patch Management, how you can check available vulnerabilities list, security updates lists via yum and external sources, in LIVE production environment, and where you should get patches for RHEL Linux distributions. Patch management and steps to apply patch methods vary by distribution. If you’re paying for support from Red Hat or SUSE, you’re paying in part for support through their patch management systems to apply patch.
What Is Security Patch?
A security patch is an update to fix certain vulnerability. It incorporates changes in source code. Security Patches are normally applied to specific software components, such as the kernel, or a service, such as vsFTP. Security patches may fix bugs, address vulnerability issues etc
Identifying security vulnerabilities list
From time to time multiple security related vulnerabilities list are reported on Linux platform. If you’ve paid for a subscription to a Red Hat or a SUSE distribution, you can get email warnings about these vulnerabilities list and security updates lists.
You can use below pages to get the globally identified vulnerabilities list and CVE (Common Vulnerabilities and Exposures) list
Now there are 1000s of security vulnerabilities list identified on a daily basis so it is not possible for anyone to check these vulnerabilities list individually and then map them to your environment. Hence we perform security scan on our Linux machine to identify these vulnerabilities list which are impacting our system and then apply security updates list accordingly.
There are a number of tools available which can be used to scan your Linux environment, some of them are
We in our environment have used Nessus and Accunetix to scan our Linux system to identify all the vulnerabilities lists and apply patch accordingly.
Sample Nessus report
Here I cannot put the entire vulnerabilities list report due to contractual reasons but I have put some snippets from the report which shows the list of vulnerable rpms and CVE
Below table shows the list of impacted CVE with their description for kernel rpm which is installed on my RHEL Linux system:
This description list is followed by the list of CVE. This is just a short output what was there in the report
Below table shows the vulnerable kernel rpm and the one from security updates list which we should install to mitigate all the reported vulnerabilities list.
Security Patch Sources
There are several sources for security patches and upgrades. The best source is generally the upgrade repository pre configured for your distribution. However, there is often a delay when distribution developers process updates from other sources, such as the kernel, or services, such as the Apache Web server.
Depending upon your support contract you can request the developers to prioritise the patch delivery timelines. If you’re in a hurry, you can download packages from the Web site directly associated with your service. While not built for your distribution, it can help you get new features into service as quickly as possible.
Now depending upon your environment you can choose for online patch source or an offline patch source. We will discuss both these topics in depth later in this article.
Perform Patch Management in RHEL 6/7/8 Linux
Ideally Linux patch management refers when you have been reported with a mission critical vulnerability (since this article is all about security fixes we will consider vulnerability as our primary root cause) and customer is requesting for an immediate fix to apply patch.
In such case you may deliver a small hotfix which will apply patch on all the nodes as per security updates list in customer environment. Now this security hotfix can apply patch and security updates online as well as offline.
Apply Patch Online
To use online linux patch management your RHEL Linux system must be registered with Red Hat Network mapped with proper subscription channel to get the required security updates. If you have a substantial number of Linux computers, it may be cost effective to buy, configure, and dedicate one or more computers to the patch management task. For example, assume that you have a network of 100 computers, and linux patch management requires that each of these computers downloads 20MB per day. Downloading an additional 2GB per day, every day, can be expensive on business-level Internet connections.
In RHEL 7 and 8 this can be achieved using yum-security
plugin, for RHEL 6 you must install yum-plugin-security
rpm manually
On RHEL 6
# yum install yum-plugin-security
List Available Security Erratas
To list all available security erratas without installing them, run:
# yum updateinfo list available
RHSA-2014:1031 Important/Sec. 389-ds-base-1.3.1.6-26.el7_0.x86_64
RHSA-2015:0416 Important/Sec. 389-ds-base-1.3.3.1-13.el7.x86_64
RHBA-2015:0626 bugfix 389-ds-base-1.3.3.1-15.el7_1.x86_64
RHSA-2015:0895 Important/Sec. 389-ds-base-1.3.3.1-16.el7_1.x86_64
RHBA-2015:1554 bugfix 389-ds-base-1.3.3.1-20.el7_1.x86_64
RHBA-2015:1960 bugfix 389-ds-base-1.3.3.1-23.el7_1.x86_64
RHBA-2015:2351 bugfix 389-ds-base-1.3.4.0-19.el7.x86_64
<Output trimmed>
Security Updates List
To list all available rpms from security updates list without installing them, run:
# yum updateinfo list security all
RHSA-2018:3056 Moderate/Sec. samba-client-4.8.3-4.el7.x86_64
RHSA-2019:2099 Moderate/Sec. samba-client-4.9.1-6.el7.x86_64
i RHSA-2016:0006 Moderate/Sec. samba-client-libs-4.2.3-11.el7_2.x86_64
i RHSA-2016:0448 Moderate/Sec. samba-client-libs-4.2.3-12.el7_2.x86_64
i RHSA-2016:0612 Critical/Sec. samba-client-libs-4.2.10-6.el7_2.x86_64
<Output trimmed>
# yum updateinfo list sec
i RHSA-2014:0678 Important/Sec. kernel-3.10.0-123.1.2.el7.x86_64
i RHSA-2014:0786 Important/Sec. kernel-3.10.0-123.4.2.el7.x86_64
i RHSA-2014:0923 Important/Sec. kernel-3.10.0-123.4.4.el7.x86_64
i RHSA-2014:1023 Important/Sec. kernel-3.10.0-123.6.3.el7.x86_64
i RHSA-2014:1281 Moderate/Sec. kernel-3.10.0-123.8.1.el7.x86_64
<Output trimmed>
To get a list of rpms from the currently installed security updates list this command can be used:
# yum updateinfo list security installed
RHSA-2014:0678 Important/Sec. kernel-3.10.0-123.1.2.el7.x86_64
RHSA-2014:0786 Important/Sec. kernel-3.10.0-123.4.2.el7.x86_64
RHSA-2014:0923 Important/Sec. kernel-3.10.0-123.4.4.el7.x86_64
RHSA-2014:1023 Important/Sec. kernel-3.10.0-123.6.3.el7.x86_64
<output trimmed>
To know more about the advisory from the security updates list before you apply patch:
[root@rhel-fews-cc ~]# yum updateinfo RHSA-2019:2135 Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager =============================================================================== Moderate: qt5 security, bug fix, and enhancement update =============================================================================== Update ID : RHSA-2019:2135 Release : 0 Type : security Status : final Issued : 2019-08-06 08:04:56 UTC Updated : 2019-08-06 08:04:44 UTC Bugs : 1564000 - Rebase qt5-qtbase to 5.9.7 : 1564001 - Rebase qt5-qtcanvas3d to 5.9.7 : 1564002 - Rebase qt5-qtconnectivity to 5.9.7 : 1564003 - Rebase qt5-qtdeclarative to 5.9.7 : 1564004 - Rebase qt5-qtdoc to 5.9.7 : 1564006 - Rebase qt5-qtgraphicaleffects to 5.9.7 : 1564007 - Rebase qt5-qtimageformats to 5.9.7 : 1564008 - Rebase qt5-qtlocation to 5.9.7 <output trimmed> : refer to the CVE page(s) listed in the References : section. : : Additional Changes: : : For detailed information on changes in this : release, see the Red Hat Enterprise Linux 7.7 : Release Notes linked from the References section. Severity : Moderate updateinfo info done <Output trimmed>
If you want to apply patch only for one specific advisory:
# yum update --advisory=RHSA-2014:0159
To list all available security updates list with verbose descriptions of the issues they apply to:
# yum info-sec
===============================================================================
GeoIP bug fix and enhancement update
===============================================================================
Update ID : RHBA-2019:2224
Release : 0
Type : bugfix
Status : final
Issued : 2019-08-06 08:14:36 UTC
Updated : 2019-08-06 08:14:34 UTCDescription : GeoIP is a C library that enables the user to find the country
: that any IP address or host name originates from.
: It uses a file-based database that can be,
: optionally, updated on a weekly basis by
: installing the GeoIP-update package.
:
: For detailed information on changes in this
: release, see the Red Hat Enterprise Linux 7.7
: Release Notes linked from the References section.
:
: Users of GeoIP are advised to upgrade to these
: updated packages.
Severity : None
<Output trimmed>
View and Install Vulnerabilities list with CVE
To view Vulnerabilities List or CVEs which affect the system with:
# yum updateinfo list cves
CVE-2018-14633 Moderate/Sec. kernel-3.10.0-957.1.3.el7.x86_64
CVE-2018-14646 Moderate/Sec. kernel-3.10.0-957.1.3.el7.x86_64
CVE-2018-18397 Important/Sec. kernel-3.10.0-957.5.1.el7.x86_64
CVE-2018-18559 Important/Sec. kernel-3.10.0-957.5.1.el7.x86_64
CVE-2018-9568 Important/Sec. kernel-3.10.0-957.10.1.el7.x86_64
CVE-2018-17972 Important/Sec. kernel-3.10.0-957.10.1.el7.x86_64
<utput trimmed>
To install packages impacting a certain CVE Number
# yum update --cve CVE-2008-0947
Update all available security updates list
Run yum update security in the below syntax to download and apply all available security updates list from Red Hat Network hosted or Red Hat Network Satellite:
# yum -y update --security
To only install the packages that have a security errata use
# yum update-minimal --security -y
Apply Patch Offline
Most of the production environment are not connected to Internet hence online patch management is not possible. So in such cases there are two possible approach taken by customers.
- Create a local repo which is always in synch with Red Hat Network. If you can configure a proxy server, you could download Linux patch data once from the Internet, and then the 100 computers on your network could download the patches locally. You would then save the additional costs for your Internet connection.
- Create a security hotfix with all the packages. This hotfix will contain scripts to create local repo and update the packages locally on individual nodes or on some HTTP server. With this you do not need access to external network in your production environment and is the most secure method to perform patch management and apply security hotfix.
Steps to create offline security hotfix
Based on Nessus scan report you will get the list of CVE or vulnerabilities list which are impacting your Linux node. So you can download the rpms which fixes the respective CVE as explained under Online Patch Management
Place all the rpms from security updates list under one location on any Linux node, for example in our case we will keep all the rpms under /tmp/rhel_security_updates
# mkdir /tmp/rhel_security_updates
Next once you have the list of rpms which you need to download, you can then download these rpms from RHN along with their dependencies and keep it under the same path to apply patch.
Next execute createrepo
as shown below
# cd /tmp/rhel_security_updates # createrepo .
This will create the necessary repodata
files required to create an offline repo
Now our repo directory is ready to apply patch offline (security hotfix). You can create a script which can now do the below list of tasks
- Create repo file required to create a repo on individual node. A sample content is placed below
[rhel74_updates] name=rhel74_updates baseurl=file:///tmp/rhel_security_updates gpgcheck=1 enabled=1 gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
- Update all the rpms using (
yum update -y
) - Verify if the update was successful
Perform Rollback or Fallback after applying security hotfix
Now it is very important that in production environment you have an option available to rollback or fallback. So after you apply patch, to fallback or rollback to the older working state of your RHEL Linux host you must know the last state.
Here I mean that assuming the existing (before you apply patch or security hotfix) your RHEL system had 500 rpms then after fallback it is expected that you return the system to same set with 500 rpms.
Now before applying security hotfix I had below list of kernel rpms so after fallback also I should have same set of rpms
# rpm -qa | grep kernel kernel-tools-libs-3.10.0-957.21.3.el7.x86_64 kernel-3.10.0-957.21.3.el7.x86_64 kernel-tools-3.10.0-957.21.3.el7.x86_64
This is really tricky to handle because with every security update you install on your RHEL Linux host, there are a number of dependencies and additional rpms which gets installed.
Now for example I wish to update samba-common
rpm due to some vulnerability. Currently my RHEL system has samba-common-4.8.3-4.el7.noarch
# rpm -qa | grep samba samba-common-libs-4.8.3-4.el7.x86_64 samba-common-4.8.3-4.el7.noarch samba-client-libs-4.8.3-4.el7.x86_64
While there is a security update available with samba-common-4.9.1-6.el7.noarch
# yum updateinfo list sec | grep samba RHSA-2019:2099 Moderate/Sec. samba-client-libs-4.9.1-6.el7.x86_64 RHSA-2019:2099 Moderate/Sec. samba-common-4.9.1-6.el7.noarch RHSA-2019:2099 Moderate/Sec. samba-common-libs-4.9.1-6.el7.x86_64
So I plan to update samba-common
rpm
# yum update samba-common <Output trimmed> Dependencies Resolved ============================================================================================= Package Arch Version Repository Size ============================================================================================= Updating: samba-common noarch 4.9.1-6.el7 rhel-7-server-rpms 209 k Updating for dependencies: libsmbclient x86_64 4.9.1-6.el7 rhel-7-server-rpms 137 k libtevent x86_64 0.9.37-1.el7 rhel-7-server-rpms 40 k libwbclient x86_64 4.9.1-6.el7 rhel-7-server-rpms 111 k samba-client-libs x86_64 4.9.1-6.el7 rhel-7-server-rpms 4.9 M samba-common-libs x86_64 4.9.1-6.el7 rhel-7-server-rpms 170 k
As you see due to dependency reason I was supposed to also update additional rpms. But there is no such guarantee that while performing downgrade of samba-common to 4.8.3-4.el7.noarch
we will get the same set of dependency list.
In such cases you can manually download the individual rpm → identify the dependencies and then downgrade the rpm using "rpm
" command. But this is very hectic and not recommended.
I recommend using LVM Snapshot feature to perform fallback of such security hotfix. In such case if you wish to fallback then you can just revert back the using the LVM snapshot.
Starting RHEL 7.7 and RHEL 8 you can also boot your RHEL system using the LVM snapshot using BOOM.
It is the most reliable solution for such use cases. Although to perform LVM snapshot you need some mandatory prerequisites which I have explained in detail in a separate article.
Now showing you a step by step guide to perform LVM snapshot will be out of scope for this article so I have added hyperlinks to my other articles where I have explained this in detail with examples.
Lastly I hope the steps from the article to get an overview on linux apply patch, security errata, security updates list and performing linux patch management on RHEL Linux was helpful. So, let me know your suggestions and feedback using the comment section.
Is is possible to use rhel8.x iso to perform patch on rhel7.x. Thanks in advance.
No, you should first upgrade the server to rhel 8.x as the rpm syntax changes to el8
Moreover there are critical changes between both releases
Hi Manoj
So I updated my system with yum clean all; yum exclude=kernel* and my system went from Red Hat Enterprise Linux Server release 7.7 (Maipo) to Red Hat Enterprise Linux Server release 7.8 (Maipo). If a reboot the system, will I have a healthy system. I though that only by updating the kernel is when I will update a minor release update.
Thanks
Hi Luis,
This is a little tricky question. We have to understand that a minor release is made up of a bunch of rpms and cannot be defined by just one rpm. So just by updating a kernel (or updating all except kernel) will not tell you if you are at RHEL 7.7 or 7.8
# subscription-manager release –list
The file which shows the release information is updated by
redhat-release-X-XX rpm
so if you just install this rpm from RHEL 7.8 then you system would show newer minor release but in real there will be many rpms which will be still from RHEL 7.7The best approach to handle such minor updates are that you bind your subscription to respective release and either perform yum update or download the rpms locally and then choose the list of rpms you wish to update.
+——————————————-+
Available Releases
+——————————————-+
7.0
7.1
7.2
7.3
7.4
7.5
7.6
7.7
7.8
7Server
# subscription-manager release –set=7.8
# subscription-manager release –show
Follow below Red Hat solution article for more information
https://access.redhat.com/solutions/238533
Hi
thanks for the information for patching and vulnerability fixing in Redhat 6/7/8 .
can you please tell me what we need to do in case of centos 7 / 8 ?
Regards
Manoj Raul
Thanks for your feedback, the steps would be the same. The only additional thing required in RHEL is that you need to register your environment but in CentOS you should be getting these automatically. If you face any issues then please do let me know here for me to check further as we have RHEL in our environment so I have not explicitly tested CentOS