In response to the Apache Commons library that OpenNMS uses, version 16.0.4 has been released to help secure against a remote exploit.

The exploit involves Java Remote Method Invocation (RMI) which listens on port 1099 by default. In my previous post I pointed out that if that port is inaccessible, then the exploit can’t happen.

What 16.0.4 does is limit RMI to only listen on localhost. While that will prevent remote exploits even in the event port 1099 is blocked via the firewall, it doesn’t completely solve the problem. To fix the root cause of the issue will require changes to Apache Commons, and we are ready to upgrade to the fixed version as soon as it is available.

We tend to be very internally critical of security issues within OpenNMS, and some people complained that my last post wasn’t technical enough. So I’m hoping to correct that with this one, but if you don’t care about such things you should probably skip it (grin). I have started updating the Security Considerations page on the wiki with details about securing OpenNMS in general, and that will have better information for people interested in security and OpenNMS than this blog post.

While blocking external access to port 1099 will secure OpenNMS against this attack for most people, it doesn’t prevent people who have access to the machine from exploiting the vulnerability. This is called a “privilege escalation” attack vs. a “remote exploit”, as a “normal” user can now have rights (i.e. root access) if they are locally on the machine. Most of our users tend to limit shell access to the server, so this shouldn’t be a problem, but in environments that rely heavily on directory services such as LDAP, the default may be to allow non-privileged access to certain users (say, the “IT Group”) that aren’t involved in maintaining OpenNMS.

And there is also the slim chance that there is a vulnerability in our webUI that could allow a user access to the system. We, of course, don’t know of any and we take great care to prevent it, but simply hoping to limit access to the server as a way to prevent this exploit is insufficient.

So, to prevent it entirely, we are removing RMI. It was introduced in the first iteration of the OpenNMS Remote Poller, but real world installation found that getting the proper ports open was a real pain. So instead the remote poller now talks over HTTP/HTTPS (with the latter being the most secure). Most networks have ports 80 and 443 open, so that made things a lot easier.

Until that is introduced (most likely with Horizon 17), it is still a good idea to limit access to the OpenNMS server to only essential people.

Note that Java Management Extensions (JMX) also use serialized objects and thus could be vulnerable. OpenNMS has a JMX port (18980) but it is bound to localhost by default. In fact, all ports are bound to localhost by default in 16.0.4 except for the webUI, port 8980.

There are a number of other steps you can take to harden your OpenNMS server. I’m planning on detailing them on the wiki, but start with only doing a minimal operating system install. The less software on the system, the smaller the chance one will have a vulnerability.

Also, OpenNMS currently runs as the “root” user. This is due to the fact that it needs access to ICMP traffic as well as port 162 for SNMP traps. Both of these require root by default. With some “stupid kernel tricks” you can run OpenNMS as a non-root user, but it has not been heavily tested. We have a detailed list of issues for running as non-root on our Jira instance.

Sorry to drone on about this, but we take security extremely seriously at OpenNMS. We also have to labor under the misconception that Java is inherently unsafe. It is not true, although people still have nightmares from the early issues with client-side Java applets. The Java in OpenNMS is server-side and we don’t use applets, and the language is used securely in a tremendous amount of software.

For comparison, WordPress, an application I love, is currently estimated to run 25% of the world’s websites. It is written in PHP, a language that has a huge track record of security exploits, and many of the spam e-mails I get link to compromised WordPress sites.

It is possible to secure WordPress (we use it for all of our websites as well) but it takes some diligence. We will remain as diligent as we can concerning the security of OpenNMS, and we will continue to take steps to make it even more secure.