Cybersecurity threats are constantly escalating and the current landscape means the majority of successful cyber attacks exploit well-known vulnerabilities that can lead to system breaches and loss of sensitive information. The time between discovery of a system vulnerability and the start of malicious exploits is getting shorter, often a matter of hours before attempted attacks. Increasingly complex enterprise networks, use of bring your own device (BYOD) and other protocols for higher productivity, and the broader array of applications and devices leveraged for business use also provide a larger target for compromise.
System breaches and data loss can mean legal liability, substantial regulatory fines, and damage to an organization’s reputation. Consequently, implementing a proactive approach to patching vulnerabilities has become an essential IT process in today’s information systems environment.
One essential part of an overall vulnerability management program, patch management is the process of researching, testing and installing available patches to an organization’s production systems and applications. Patches are essentially code changes that are a fix for a system vulnerability discovered after devices, systems and applications are in production. Other benefits patch management provide include fixing bugs or adding features that support your systems and applications running smoothly and optimally.
10 Patch Management Best Practices
The following cover the full patch management lifecycle:
1. Develop an Inventory of Network Assets.
A typical IT environment can have dozens of servers and hundreds of workstations and PCs and includes devices used for remote access. A complete inventory of assets establishes what you have so you know what will need to be patched. This inventory will need to be updated regularly as part of your patch management process. Patch management products typically include tools for scanning a network to provide a thorough inventory of all network assets and provide updates as needed.
2. Categorize Assets.
Once an inventory is complete assets need to be categorized based on exposure and risk, which is usually a manual process. This process defines which devices require immediate critical patch deployment (in hours/days) and which require standard deployment (in weeks). When categorizing devices, the critical question is “what is the impact to your organization if a device is down or compromised.” Asking this question includes determining the sensitivity of information a device stores (patient information or payment card numbers), if a device supports a publicly visible element (a website) or supports an important service (payment processing).
3. Are all the operating systems in your environment supported?
This will determine what products you use for patch management as many of the most popular tools only support Windows and your system may also have Mac, Linux or Unix operating systems. Organizations may have legacy systems as well that can be impacted by patches so it’s important to consider all operating systems that are part of your network.
4. Gather information on updates and patches.
Vendors publish updates on their websites that include release notes detailing the latest fixes and additions, and provide email notifications to administrators. In addition, many patch management software solutions maintain their own patch information databases providing for a quick comparison of inventory to available patches. Using a software solution greatly simplifies information gathering and often provides additional patch information such as classifications and information on compatibility with business products.
Testing is an essential step in avoiding complications that impact productivity following patch deployment. A testing environment will need to be implemented that closely mirrors your organizations production environment. It is important to review patch descriptions, so any known issues can be included in the testing environment. System reboots should also be included in testing so unexpected reboots after patch deployment do not impact normal business operations.
6. Schedule a monthly routine for deploying patches.
Develop a strategy to efficiently patch all your systems in one big event over a weekend or set a percentage to complete each week over the month so any adverse impacts can be mitigated. Your unique organizational needs will determine what method works best for you.
7. Develop a roll-back plan.
If significant issues in performance or functionality are encountered after patch deployment, it is important to have an established roll-back plan to quickly reverse patches and return the system to a pre-patched state. The best patch management tools provide functionality for patch roll-back.
8. Deploy patches.
Now that you know have an inventory of assets, have categorized assets to prioritize critical patches, tested patches to avoid complications, and have a patch management schedule, you know what patches to install and are ready for deployment. The foremost challenge in this practice is deployment without disrupting production. There are many automated tools available for patch deployment and maintenance particularly for a large system and you can evaluate these tools to determine the best fit for your environment and budget.
9. Assess the patch deployment.
After deployment verification and reporting are an important step to assess patch deployment and discover issues that require mitigation. A post deployment assessment analyzes deployment logs and exceptions, and formally verifies that all planned deployments occurred correctly. If significant issues were encountered, the roll-back plan can be implemented to return the system to the pre-deployment state.
10. Mitigate for Exceptions.
It is often necessary to address exceptions during patch deployment. For example, if a patch to a core system component breaks something during deployment, an exception would be required. Leaving an application or system in an older or unpatched version requires additional security features such as locking down user permissions, removing direct internet access or whitelist application to stop untrusted payloads from executing.
Now that you’ve made it through a full patch management cycle, it’s time to start at the beginning and update your asset inventory. By having a robust, formal patch management program for your organization, you can ultimately protect your systems and data environments from the host of security threats attempting to breach your security and compromise your systems and sensitive information.
Today’s organizations increasingly use web applications to support necessary business operations, which introduces a broad array of vulnerabilities to system assets, often without a formal plan for remediating these vulnerabilities. Employees, vendors and consultants also use web applications, often without prior authorization, which introduces additional risk to system resources. This environment leaves organizations exposed to potential breaches at a time when web applications are increasingly a preferred target of cyberattacks.
A virtual patch, also referred to as a Web application firewall (WAF), uses an enforcement layer to analyze transactions to prevent attacks in transit so malicious traffic never reaches a vulnerable web application. An effective virtual patch prevents a malicious exploit without modifying an application’s source code.
Virtual patching provides benefits over conventional patching including:
- Critical components can remain online during patching so operations are not interrupted as occurs with conventional patching.
- Exploits are mitigated quickly reducing risk until an effective permanent patch is released by a vendor.
- Virtual patches need to be installed in a few system locations rather than on all network hosts as required in conventional patch management.
- Since source code is not modified, virtual patches are unlikely to introduce system conflicts.
Virtual patching is a security process requiring development of a methodical, repeatable process that provides the best path to successful protection of system resources. The following is a virtual patching methodology recommended by the Open Web Application Security Project (OWASP). You can read more about their methodology here: https://www.owasp.org/index.php/Virtual_Patching_Cheat_Sheet
OWASP Methodology consists of six phases:
- Virtual Patch Creation
- Recovery/Follow Up
Lays the patching foundation, ensuring everything is in place to respond when an incident occurs. Preparation involves the following critical steps:
- Vulnerability monitoring – Ensure your organization is signed up for all vendor alerts for commercial web applications in use.
- Pre-Authorization – Virtual patches do not modify source code and require quick implementation, so they can be categorized in the same group as anti-virus updates to expedite authorization and minimize testing.
- Deploy Virtual Patching Tool in Advance – Time is critical for effective virtual patching, so it is critical to have a tool in place ready for use, avoiding an extended approval process new software installation.
- Increase Audit Logging – Most web servers use Common Log Format (CLF) that does not provide adequate data for incident response. Ensure additional HTTP data is available:
- Request URI (including QUERY_STRING)
- Full Request Headers (including Cookies)
- Full Request Body (POST payload)
- Full Response Headers
- Full Response Body
Occurs when an organization is alerted to a vulnerability in a web application. Proactive identification involves an organization assessing their security posture using whitehat attacks such as penetration testing to identify flaws. Custom coded web applications require proactive identification as third-party notifications are not available. Reactive identification occurs when a vendor notification of a vulnerability is received, public vulnerability disclosure occurs (increased threat level as attackers know about the vulnerability), or an attack is active (requires immediate remediation).
Involves analyzing virtual patch management using the following steps:
- Determine patch applicability – Detailed analysis of a flaw should be performed to determine if the virtual patching tool implemented has effective detection capabilities.
- Utilize ticketing system – Enter vulnerability data into ticketing system for tracking and metrics purposes.
- Verify vulnerability name – Use a public vulnerability identifier, such as Common Vulnerabilities and Exposures (CVE) name/number. If vulnerability is proactively identified, assign your own unique identifier.
- Designate impact level –The criticality level involved with a vulnerability is important to understand to develop an appropriate response.
- Identify software versions impacted – Identify the software versions impacted by the vulnerability to determine if versions installed in your systems require the patch.
- Determine configuration that triggers the problem – Some vulnerabilities are only triggered for specific configuration settings so may not require patching in your system.
- List exploit code or payloads used during attacks/testing –Vulnerability notices often include exploit code that can be used for developing and testing virtual patches.
Two main principles are involved in patch creation, no false positives (do not block legitimate traffic), and no false negatives (do not ever miss attacks, even when configured to avoid detection). The goal is to minimize the impact to your operating systems and reduce risk. Automated tools are available to covert XML data collected during analysis into virtual patches.
Proactively identified flaws may require manual patch creation. A positive security patch management or whitelist model is a comprehensive mechanism that defines rules for every application parameter to provide additional security through patch management independent of the source code. A negative security or blacklist patch model defines rules that detect specific known attacks, then allow only valid traffic.
Testing newly created virtual patches involves the following steps:
- Implement patches in an initial “Log Only” configuration so normal traffic is not blocked.
- If a specific patch management tool was used to identify the vulnerability request a retest.
- If resting fails due to evasions, go back to the analysis phase to develop a better patch for the issue.
After virtual patch implementation, the follow-up phase involves the following steps:
- Update data in ticketing system – Virtual patching requires accelerated implementation, however, tracking should still follow normal patch management processes.
- Periodic re-assessments – Re-assessments should be performed periodically to determine when a patch can be removed if a web application has been updated with a source code fix. Many organizations will keep virtual patches in place, however, to provide additional identification/logging capabilities.
- Run alert reports – run reports to identify when any of your virtual patches have been triggered so data is logged and demonstrates the value for virtual patching.
Patch management is not an event, it is a continuous, complex cycle with a host of variables that are vital to your organization’s cybersecurity. Failure to patch vulnerable systems can lead to cyber crimes that result in legal liability, substantial regulatory fines, and damage to your organization’s reputation. Patch management not only provides security protection, patches also fix flaws and add features that ensure the optimal health of your IT environment.
Investment in a formal, proactive patch management program provides processes, procedures and automated tools that ensure the security and optimal operation of your organization’s core infrastructure, systems, which is vital, valuable support for your organization’s IT resources.