Security patching of Oracle databases can be a touchy and complex subject for some companies. It is perceived to be complex; companies don’t want the downtime; business is worried that a security patch can break the applications; the application vendor maybe says his application is only certified with a base patch set version and so on.
There is also this perceived lack of customer knowledge of what Oracle has actually fixed in a security patch. I remember detailed and heated discussions at conferences and online at the start of the CPU regime many years ago where people refused to patch (seemingly) because they didn’t know what the patch fixed. In one sense I can understand this; if a customer finds a functional bug in Oracle and raised an SR and a patch wings its way to them; they know for sure if it fixed the functional issue or not. They can test it.
There seemed to be this logic to security patches at the time and after that because a customer didn’t know what a security patch fixed in any detail, they could not test it or be sure it would not affect their application. Oracle or indeed other vendors are not going to give out a list of security vulnerabilities and hacks. This, of course would then be useful to hackers of databases that have not been patched. Oracle do credit researchers in their advisories for security patches but usually not enough details to show how the issue can occur. I have been credited a number of times on CPU advisories and I know what the issue was as I found it each time BUT the customers didn’t and I didn’t know what other fixes were for. Of course, Oracle do not credit their own staff who find and fix security bugs released in the same patches.
We as customers must just get over this fact and accept we will never know exactly what issues were fixed in every security patch release.
Researchers and hackers can reverse engineer each released patch. This is not trivial with C language binaries or DLL/SO but it is not impossible for skilled researchers. I don’t do this so please don’t bother to tell me the legalities in relation to the license agreement; I am just stating people can do it and have done it but not me. Where the patch involves PL/SQL packages then its even easier to reverse engineer the patched PL/SQL as there are unwrappers on the internet.
Why mention reverse engineering?
Well, we will come back to that in a minute.
I have been conducting detailed security audits of Oracle databases for more than 20 years. When I started there was little out there on securing Oracle or indeed hacking Oracle in terms of books or papers or products or tools. People and companies told me that they only had a budget for firewalls or desktop security and whilst when speaking to companies and people at these companies they agreed it’s good to focus on data security and Oracle security they didn’t have time or a budget for it.
At that same time 20 years ago, Oracle had not started releasing any security patches yet; first came alerts which are random security patches that got bigger and bigger up until the regimented quarterly security patches were started to be released.
Since the release of quarterly patches, I have asked groups of people at conference talks and also at my trainings whether they apply security patches. The results of these polls are usually between 10% and 30% of people apply patches. Of that group who have applied a patch I also ask who applies patches regularly; once a quarter or 6 months or even once a year. Always there is a larger subset of those who do patch but only once.
In other words, a smaller percentage of customers apply security patches regularly. This is not a good statistic but its not scientific and I didn’t talk to everyone who runs Oracle. Certainly, my experience of asking “do you patch or don’t you?” question is consistent across the years and it’s not changing.
What about databases I have seen; I have seen a lot! They sadly tend to follow a similar pattern as asking people these questions. A lot of customers don’t patch at all and some only patch once after the initial build or after a major upgrade but nothing in-between and some strangely patch the day before I come on site to do a security audit. A small number patch regularly but not enough do.
What should we do to secure data in an Oracle database; well at a high level we should focus on these elements:
- Patch the Oracle database software
- Implement access controls to the database; this could be network level or database level such as valid node checking or logon triggers
- Implement user security, least rights
- Implement data access controls
- Implement context based security to allow fine grained access to resources and data
Harden the database
The first three are about platform security where Oracle is the platform and the last three are data security design aimed at securing the actual data.
In terms of security patches what can you do at a high level:
- Patch regularly
- Patch randomly
- Don’t patch at all or maybe once the database was built or after a major update of an application or database
- Try and work around not patching
- Buy software that tries to replace the patch with a blocking or logging action
My experience is that most people don’t patch regularly or if they do it’s not regular. I do see some sites that do patch and have some semblance of regularity or at least not completely random. Some sites apply patches once a quarter or six monthly.
Not patching (or patching regularly) but also not even trying to work around the fact that there is no security patch is also not good enough. There are often no work arounds for security bugs fixed in quarterly security patches and some issues fixed are where an attacker can exploit the database without credentials or authenticating. These types of issues are dangerous and the only workaround in this case would be to prevent direct TNS access to the database; but what about the genuine database users who may need direct TNS access; my experience is that a large number of sites use open routing and blocking access would be difficult anyway. One step that can be done is limiting access to a list of known IP Addresses using tools such as valid node checking BUT this is a potential partial fix for one problem. What if a patch fixes issues in PL/SQL packages? Then valid node checking would not work; what could we do in this case; limit access to the package is possible but only if it’s not used genuinely in your application.
And the list goes on. Yes, we can do really good database security design and hardening and lock down access to everything and this may reduce the risk of exploitation of something fixed by a security patch but it’s never going to be perfect.
Another option that started to appear around 15 years ago is the idea of virtual patching. This is the idea that you cannot or don’t want to patch with Oracle’s security patch so you deploy network software that is a special version of an application firewall or intrusion detection / prevention system. The way this works is that network packets are sniffed (or shared memory attached and parsed) and attacks that could exploit issues fixed in Oracle’s security patch are detected. This is complex and prone to error or hacker bypass and requires the vendors of the virtual patch to reverse engineer (or guess!) what Oracle has fixed; then work out how that fix could be exploited and then how an exploit that could hack the database software can be detected.
As I stated above Oracle do not release details of what is fixed so this virtual patching is not perfect and involves a lot of work. Yes, I can see that a product such as this maybe be a vert short term barrier to when a patch of a particular system cannot be applied quickly but its not a perfect fix.
So, patch and apply Oracle’s security patches if you can; it makes sense; it builds belt and braces solutions and an underpinning of security of the Oracle software that hardening and data security design can be built on.
If your major concern is that an Oracle security patch may cause a performance issue or break something functionally if you apply it then one method is this; Wait until patch Tuesday when Oracle release the next security patch and then download the previous quarters patch and apply it; application testing as necessary. The logic of this is to allow everyone else to test Oracle’s patch for you over the previous quarter.
For me, it is important that security patches are applied. Security patches form part of the underpinning of data security (patch and harden) on which we build data security design (user controls, data access controls, context based security). I can appreciate the time needed if you must perform regression tests of applications but a suitable method must be created that allows regular security patching; this can be:
- Apply within the quarter every quarter
- Apply every quarter but apply the last quarter patch to allow for others testing it
- Apply regularly on a longer timescale such as every six months
Workarounds can reduce the risk but cannot replace patches and virtual patches may be fine for very short periods but cannot replace actual patches.
The most important message is patch and patch within a regular regime BUT don’t forget all of the other security tasks for your data; hardening, user security, access controls, data security and context based security. There is no point patching once and then never again or patching before a security audit and never again.
