A few weeks ago, I woke up one morning to discover that Android had 34 software updates waiting for me. This was followed by my laptop wanting to reboot after installing the latest patches from Microsoft, my tablet needing a reboot after its latest firmware update, and my server screaming for me to put “yum” into action to install the latest patches available from Red Hat. All before 10:00 am in the morning!

With all of the technology that we have today, installing software updates has become a near-daily activity. That statement is true for all professionals in the technology industry, especially those who handle patch management for large-scale enterprise IT systems.

Recently, we wondered how organizations might be handling the influx of patches, so we decided to conduct some research on the topic. Not surprisingly, we found that many organizations are having trouble keeping up with the vast number of patches constantly being added to the work queue.

To many, installing a patch might sound easy. It’s just the simple click of a button, right? Not really. Patch installation difficulty varies across platforms, ranging from the most trivial of installation methods (clicking a button) to complex scenarios involving delicate sequences of events.

However, patch installation difficulty is not the only variable in this equation. Patch testing is another critical piece of the puzzle and, along with scale, is one of the more challenging aspects of patch management in the modern world of enterprise IT.

Enterprises cannot go about installing patches blindly without understanding potential impacts of the change brought by a patch. Patches have a history of breaking things and, when things break in the enterprise, chaos ensues.

In a recent survey, we asked 483 IT professionals their thoughts on patch management topics. We found that roughly half of those involved with patch management will always test patches before deployment, with approximately 30% saying it depends on the patch. Fewer than 20% never test their patches.

Another set of questions was related to the amount of time spent testing patches for desktops and servers. As you can see in Figure 1, most organizations spend approximately one week or less testing patches in their environment before deployment.

Figure 1: Time taken to test patches.

Last month, in an interviewed by “Padre” over at TWiT for an episode on the TWiET channel, the topic was “Enterprise Patch Fatigue,” and one of the questions Padre asked was related to the feasibility of thoroughly testing patches before enterprise rollout.

An organization’s ability to thoroughly test patches depends on scale and resources. Virtualization and orchestration technologies, coupled with good patch management and vulnerability management software, can help organizations create environments that enable extensive patch testing.

Still, testing every possible configuration is hard for any organization. As you scale, it becomes impossible. More nodes means an increase in the number of scenarios that need to be tested. Those considerations can quickly spiral out of control.

This leads us to the following conclusion: patch testing is currently done on a best effort basis and, as with most software-based testing, it only covers a small portion of the overall “state space” of test cases. An important question to ask is the following: will this scenario work in the future as more and more systems become highly interconnected?

Current trends, such as the Internet of Things, the Industrial Internet, and cyber-physical systems, are pushing the envelope of scale with an exponential explosion of devices coming online in the near future.

Will the failure of a patch installation in some datacenter in Australia cause my infotainment center to malfunction, possibly causing my navigation to go wacky? What other questions need to be asked? Technologists should be considering these types of questions.

Obviously, new techniques and innovations will surface to help alleviate some of our patch testing problems. I believe automation will play a huge role, and advanced research in automated testing processes will surely help us in this domain. It will be interesting to see what developments tomorrow will bring us in the realm of patching.

Via: tripwire