What’s Old Is New Again

Today we launched a new look for OpenNMS, a rebranding effort that has been going on for the better part of a year. It represents a lot more than just a new logo and new colors. While OpenNMS has been around for over two decades now, it is also quite different from when it started. A tremendous amount of work has gone into the project over the past couple of years, and if you looked at using it even just a short while ago you will be surprised at what has changed.

New OpenNMS Logo

One of the best analogies I can come up with to talk about the “new” OpenNMS concerns cars. I like cars, especially Mercedes, and when I was in college I usually drove an older Mercedes sedan. I enjoyed bringing them back to their former glory (and old, somewhat beaten down cars were all I could afford), and so I might start by redoing the brake system, overhauling the engine, etc.

When I would run out of money, which was often, sometimes I’d have to sell a car. Prospective buyers would often complain that the paint wasn’t perfect or there was an issue with the interior. I’d point out that you could hop in this car right now and drive it across the country and never worry about breaking down, but they seemed focused on how it looked. Cosmetics are usually the last thing you focus on during a restoration, but it tends to be the first thing people see.

This is very much like OpenNMS. For over a decade we’ve been focused on the internals of the platform, and luckily we are now in a position to focus on how it looks.

Please don’t misunderstand: application usability is important, much more important than, say, the paint job on a car, but in order to provide the best user experience we had to start by working under the hood.

For example, from the beginning OpenNMS has contained multiple “daemons” that control various aspects of the platform. Originally this was very monolithic, and thus any small change to one of them would often require restarting the whole application.

OpenNMS is now based on a Karaf runtime which provides a modular way of managing the various features within the application. It comes with a shell that can allow even non-Java programmers access to both high and low level parts of the platform, and to make changes without restarting the whole thing. Features can be enabled and disabled on the fly, and it is easy to test the behavior of OpenNMS against a particular device without having to set up a special test environment to pore through pages of logs.

Another great aspect of OpenNMS is that much of the internal messaging can now take place through a broker such as Kafka. While this increases the stability and flexibility of the platform, users can also create custom consumers for the huge amounts of information OpenNMS is able to collect. For very large networks this creates the option to use that data outside of the platform itself, giving end users a high level of custom observablity.

The monolithic nature of OpenNMS has also been improved. The addition of “Minions” to provide monitoring at the edge of the network creates numerous monitoring solutions where there was none before. You can now reach into isolated or private networks, or monitor the performance of applications from various locations seamlessly. The “Sentinel” project allows the various processes within OpenNMS to be spread out over multiple devices with the aim to have virtually unlimited scale.

APM Example World Map

And I haven’t even started on the ability of OpenNMS to monitor tremendous amounts of telemetry data and to analyze it with tools such as “Nephron” or our foray into artificial intelligence with ALEC.

So much has changed with OpenNMS, much of it recently, that it was time for that new coat of paint. It was time for people to both notice the new look of OpenNMS at the surface, and the new OpenNMS under the covers.

One thing that hasn’t changed is that OpenNMS is still 100% open source. All of these amazing features are available to anyone under an OSI approved open source license. Plus we leverage and integrate with best-in-class open source tools such as Grafana for visualization and Cassandra (using Newts) for storing time series data.

Our new logo is a stylized gyroscope. For centuries the gyroscope has represented a way to maintain orientation in the most chaotic of situations. In much the same way, OpenNMS helps you maintain the orientation of your IT infrastructure which, let’s admit it, plays a huge role in the success of your enterprise.

Where the car analogy falls apart is that while the paint job is usually the end of a restoration, this new look for OpenNMS is just the beginning of a new chapter in the history of the project. Our goal is to create a platform where monitoring just happens. We’re not there yet, but check out the latest OpenNMS and we hope you’ll agree we are getting closer.

The OpenNMS Group Turns 15

Fifteen years ago today, on September 1, 2004, David Hustace, Matt Brozowski and I formed The OpenNMS Group, Inc.

This was the fourth business entity to steward the OpenNMS Project, and would turn out to be the one with staying power.

The original OpenNMS Group office was in a single 10 foot by 15 foot room with just enough space for three desks. The landlord provided Internet access. By adopting the business plan of “spend less money than you earn” we managed to survive and grow. Now the company has its main office in Apex, NC, USA as well as one in Ottawa, Ontario, CA, with a satellite office in Germany.

The OpenNMS platform is being used to monitor some of the largest networks in existence, many with millions of devices. With the introduction of ALEC the team is bringing artificial intelligence and machine learning technologies to network monitoring to provide the highest level of visibility to the most complex environments.

OpenNMS has always been lucky to have a wonderful community of users, contributors and customers. With their support the next fifteen years should be as great if not better than the first. I am humbled to have played a small part in its history.

Crash

It’s been even longer than usual since I’ve updated this site. I’m missing a ton of stuff, including the last day of Dev-Jam as well as my trip to this year’s OSCON conference in Portland. I wouldn’t be surprised if I lose one if not all of my three readers.

But I do have an excuse. This happened.

Crashed F150 Pickup Truck

On Friday, July 26th, I left my farm in Chatham County, North Carolina, to head to town. I needed to get the oil changed in the F150 and I was planning on meeting some friends for lunch.

About three miles from my house, another driver crossed the centerline on Hwy 87 and hit my truck nearly head-on. I suffered a broken rib, a fractured C2 vertebrae, and a fractured right big toe, but the major damage was that my left ankle was shattered.

I’ve spent the last 33 days at the UNC Medical Center in Chapel Hill, where I underwent two surgeries and was taken care of by some amazing staff.

I’m home now and plan to return to work (remotely) next week. I still have many months to go before I can approach normality, but a journey of ten thousand miles begins with a single step.

Thanks for your kind thoughts. One good thing that has come out of this is that I’ve spent the last 17 years trying to build OpenNMS into something that can thrive even without me, and the team has been amazing in my absence. I can’t wait to be at full strength again.

Meridian 2018

It is hard to believe that our first release of OpenNMS Meridian was over three years ago.

Meridian Logo

We were struggling with trying to balance the needs of a support organization with the open source desire to “release early, release often”. How do you deal with wanting to be as cutting edge as possible but to support customers who really need a stable platform? We did have a “development” release, but no one really used it.

Our answer was to model OpenNMS on Red Hat, the most successful open source company in existence. While Red Hat has hundreds of products, their main offering is Red Hat Enterprise Linux (RHEL). This is derived, in large part, from the Fedora Linux distribution. New things hit Fedora first and, once vetted, make their way into RHEL.

We decided to do the same thing with OpenNMS. OpenNMS was split into two main branches: Horizon and Meridian. Horizon was the Fedora equivalent, while Meridian was modeled on RHEL.

This has been very successful. While we were averaging a new major OpenNMS release every 18 months, now we do three or four Horizon releases per year. Tons of new features are hitting Horizon, from the ability to deal with telemetry data, new correlation features to condense alarms into “situations” based on unsupervised machine learning, to the first steps toward a microservices architecture.

We do our best to release code as production-ready as possible. Our users are very creative and use OpenNMS in unique ways. By offering up rapid Horizon releases it allows us to find and fix issues quickly and work out how to best implement new functionality.

But what about our users who are more interested in stability than the “new shiny”? They needed a system that was rock solid and easy to maintain. That’s why we created Meridian. Meridian lags Horizon on features but by the time a feature hits Meridian, it has been tested thoroughly and can immediately be deployed into production.

There is one major Meridian release a year, with usually three or four point updates. Anyone who has ever upgraded OpenNMS understands that dealing with configuration file changes can be problematic. With Meridian, moving from one point release to another rarely changes configuration, so upgrades can happen in minutes and users can rest assured that their systems are up to date and secure. Each Meridian release is supported for three years.

There is a cost associated with using Meridian. Similar to RHEL, it is offered as a subscription. While still 100% open source, you pay a fee to access the update servers, and the idea is that you are paying for the effort it takes to refine Horizon into Meridian and get the most stable version of OpenNMS possible. We are so convinced that Meridian is worth it, it is available without having to buy a support contract. Meridian users get access to OpenNMS Connect, which is a forum for asking questions about using Meridian.

It seems like it was just yesterday that we did this but it has now been over three years. That means support will sunset on Meridian 2015 at the end of the year. Never fear, the latest releases are just as stable and even more feature rich.

The main feature in Meridian 2018 is support for the OpenNMS Minion. The Minion is a stateless application that allows for remote distribution of OpenNMS functionality. For example, I used to run an OpenNMS instance at my house to monitor my devices. Now I just have a Minion. Even though my network is not reachable from our production OpenNMS instance, the Minion allows me to test service availability, and well as collect data and traps, and then forward them on to the main application. The Minion itself is stateless – it connects to a messaging broker on the OpenNMS server in order to get its list of tasks.

A Minion is defined by its “Location”. You can have multiple Minions for a given location and they will access the broker via a “competitive consumer queue”. This way if a particular Minion goes down, there can be another to do the work. By default OpenNMS ships with ActiveMQ as the broker, but it is also possible to use an external Kafka instance as well. Kafka can be clustered for both load balancing and reliability, and the combination of a Kafka cluster and multiple Minions can make the amount of devices OpenNMS monitors virtually limitless (we are working on a proof of concept for one user with over 8 million discrete devices).

There are a number of other features in Meridian 2018, so check out the release notes for more details. It is an exciting addition to the OpenNMS product line.

Welcome to 2018

I love New Year’s. Not exactly the party on New Year’s Eve, as I tend to spend it as a quiet evening with friends, but the idea of starting over and starting fresh.

It is also a good time to reflect on the year past. While 2017 was pretty tumultuous for the world at large, for OpenNMS it was a pretty good year.

Our decision to split OpenNMS into two versions is still paying off. We did three major releases of Horizon (19, 20, and 21) as well as point releases every month there wasn’t a major release, and Meridian 2017 finally came out, although later than I would have liked. Horizon users get to experience rapid advancements in power and features while Meridian users can relax knowing their system is very stable and secure.

While it is hard to pick out the best features added in 2017, I’d have to go with OpenNMS Helm and the Minion.

Helm allows you to combine and manage multiple instances of OpenNMS from a Grafana dashboard.

OpenNMS Helm

The Minion is our foray into the whole “Internet of Things” space with an application that can be installed on a small device and used to send remotely collected data to a central OpenNMS instance. Minions have minimal configuration and can be configured redundantly, yet they have the ability to collect massive amounts of monitoring data. We’re very eager to see what novel uses our users come up with for the technology (we have one customer that is “Minion-only”, i.e. they do no monitoring or collection from the central OpenNMS instance at all and instead just put two Minions at each location).

As for the OpenNMS Group, the company behind OpenNMS, we experienced modest growth but still had a record year for gross revenue. What is more exciting is that net income was also a record and several hundred percent above last year, so we are going into 2018 well positioned in our Business Plan of “Spend less than you earn”.

2018 should be exciting. The OpenNMS Drift project brings telemetry (flow) data into OpenNMS, and we are working on some exciting features regarding correlation which will probably involve new machine learning technology.

As always, these features will be available as 100% free and open source software.

Personally, I added three new countries to my list, bringing the total number of countries I’ve been in to forty. I had a great time in Estonia and Latvia, and I really enjoyed my trip to Cuba.

One last thing. If you are reading this you are probably a user of OpenNMS. If so, thank you. We are a small but dedicated group of people creating this platform and often we don’t get much feedback on who uses it and what they like about it. The fact that people do find it useful makes it worthwhile, and we wouldn’t exist without our users and clients.

So, Happy New Year, and may 2018 exceed your wildest expectations.

Service Outage Tomorrow, Saturday June 3rd

Wonder of wonders, Time Warner/Charter/Spectrum/whatever has finally delivered connectivity to our new office, albeit a month late.

So, we’ll be moving a number of servers from our old location to the new one, which means certain things, such as demo and Bamboo will be down for a few hours. Almost everything else is hosted elsewhere and redundant, so we shouldn’t have any other issues.

Sorry for the outage and thanks for your patience.

New Meridian® Releases Available

Just a quick note to point out that new Meridian releases are now available: 2015.1.5 and 2016.1.5

For those who aren’t aware, Meridian is a subscription-based version of OpenNMS built to complement Horizon, the cutting edge release. You can think of it as Meridian is our Red Hat Enterprise Linux to Horizon’s Fedora. There is one major Meridian release per year and each major release is supported for three years.

Before the Meridian/Horizon split it was taking us 18 months or so to do a new major release of OpenNMS. Now we do three to four Horizon major releases a year.

About half of our revenue comes from support contracts and so we had to be extra careful when doing a release, and even with that many of our customers were reluctant to upgrade because the process could be involved. This was bad for two main reasons: often they wouldn’t get bug fixes which meant an increase in support tickets, and more importantly they might miss security updates.

Updates to Meridian, within a major release, are dead simple. This is the process I used yesterday to upgrade our production instance of OpenNMS.

First, I made a backup of the /opt/opennms/etc and /opt/opennms/jetty-webapps/opennms directories. The first is out of habit since configuration files shouldn’t change between point releases, but the second is to preserve any customizations made to the webUI. I modify the main OpenNMS page to include a “weather widget” and that customization gets removed on upgrades. Most users won’t have an issue but just in case I like having a backup.

Next, I stop OpenNMS and run yum install opennms which will download and install the new release. The final step is to run /opt/opennms/bin/install -dis to insure the database is up to date.

And that’s it. In my case, I copy the index.jsp from my backup to restore the weather information, but otherwise you just restart OpenNMS. The process takes minutes and is basically as fast as your Internet connection.

If you have a Meridian subscription, be sure to upgrade as soon as you are able, and if you don’t, what are you waiting for? (grin)

Fifteen Years

On Sunday my mother celebrated her 75th birthday.

Although a happy occasion, why is this relevant to an open source blog? Well, it was soon after her 60th birthday in 2002 that I started my first company around OpenNMS.

I did not start OpenNMS, it began in the summer of 1999, with the first code posted on Sourceforge in March of 2000 by a company called Oculan. I started working with Oculan in September of 2001, and in May of 2002 they decided to stop contributing to OpenNMS. I saw the potential, so I asked Steve Giles, the founder and CEO, if I could have the OpenNMS project. He looked at his watch and said if I was off his payroll by Friday, he’d give me the domain names, a couple of servers, and he would sprinkle water on me and I would be the new OpenNMS maintainer.

That was actually the easy part. Explaining to my wife that I had quit my job and started a company “selling free software” was a bit harder.

sortova.com from archive.org circa May 2002

And thus Sortova Consulting Group was born. It was named after my farm. When Andrea and I decided we wanted to have a farm, we first bought raw land. In driving out from Raleigh to work on it we would pass this little farm with a barn, some cows, etc., and on the mailbox was a sign reading “Almosta Farm”. I joked that if that was “almost a farm” then what we had was just “sort of a farm”. Later, when we bought the place where we still live, the name Sortova Farm stuck.

We pronounce it “Sore-toe-va”. Only one customer ever pulled me aside and asked if it really meant “sort of a” consulting group. He laughed when I confirmed that it did.

Considering that I didn’t have any prior business experience, Java experience, or even real Internet access at my home, it is amazing that OpenNMS survived to this day. It is a wonder what you can accomplish with pure stubbornness.

Now my one true superpower is my ability to get the most fantastic people on the planet to work with me. The first group of those came from the OpenNMS community. When I was running Sortova it was the gang that later became the Order of the Green Polo that kept me going, mainly through mailing lists and IRC. In September of 2004 my good friend and business partner David Hustace and I founded the OpenNMS Group, and that corporation is still going strong. In 2009 we mortgaged our houses to buy the copyright to the Oculan OpenNMS code and thus brought all of it back under one organization, and two of the original OpenNMS team at Oculan now work for OpenNMS.

When I visit Silicon Valley I often get to meet some brilliant people, but the joy of this can be offset by the pervasive attitude of focusing on technology simply to make money. I know of a number of personally successful people who built companies, sold them, and then those products vanished into obscurity. Remember VA Linux? Their stock rose over 700% on the first day of trading, but where are they now? Did they ever deliver on their promises to the stockholders?

I want to build with OpenNMS something that will last well beyond my involvement with the project. I’ve gotten it to the point where I am not longer expressly required to make it thrive, but I am still working on its legacy. We want it to be nothing less than the de facto standard for monitoring everything, which is a high bar.

Note that I still would like to make a lot of money, but that isn’t the core driving force of the business. Our mission statement is “Help Customers – Have Fun – Make Money” in that order. If you have happy customers and happy employees, the money will come.

Fifteen years ago I made a leap of faith, in both myself, my family and my friends. I’m extremely happy I did.

OpenNMS Group Turns Twelve

Heh, it almost slipped my mind completely but The OpenNMS Group turned 12 years old today.

I did have to go give our co-founder, David Hustace, a hug and if we weren’t so slammed it would have been time for a beer. Raincheck.

I did spend a second reflecting on our wonderful customers who make all this possible, as well as all the people who contribute to and use OpenNMS. There are a lot of people who don’t believe a company can survive with a 100% open source model, but the funny thing is that we’ve outlived quite a few proprietary software companies in the last decade or so, thus we must be doing something right.

Our business plan of “Spend Less Money Than You Earn” and our mission statement of “Help Customers, Have Fun, Make Money” are as true today as they were in 2004. I look forward to getting ever better on delivering on both of them.

Kippis!

New Additions to OpenNMS

I am very happy to announce that Chris Manigan has joined the OpenNMS team.

Chris has been using OpenNMS since 2010 when he worked at Towerstream in Rhode Island. He gave us a very nice testimonial for the website, and has a lot of experience with using OpenNMS as scale.

Chris Manigan

He put that experience to use at Turbine, insuring that their infrastructure could deliver gaming content to users who demand performance. Now he’s going to use that experience to insure that OpenNMS is ready to take on the Internet of Things, for both our internal infrastructure and those of our customers.

I also want to announce that Jesse White, our CTO, and his wife Sara welcomed Charles White into the world early yesterday morning.

Charles White

Weighing 7 pounds and 11 ounces, he is already writing code in Python and we hope to have him making commits in Java in the next week or so.