OpenNMS, CSECS 2012 and VMWare Monitoring

A rather delayed post to let people know that Ronny Trommer did a presentation at this year’s Computer Science and Education in Computer Science conference on the work that he and Christian Pape are doing to add a very powerful monitoring feature to OpenNMS focused on VMWare’s vSphere.

I’ve also included a link to the paper itself.

We’ve had a sharp uptick in the amount of outside development being done on OpenNMS (especially with the GSoC and our developers in the German office) and I find this very exciting. I hope to be able to post more about it in the future.

Some Thoughts on the Apple/Samsung Silliness (#noapple)

My indentured servitude to AT&T ended recently and I decided to use that to jump in for another two years but also to get rid of my iPhone 4.

As my three readers are aware, last summer I decided to move away from Apple products toward freer alternatives. I still have a Macbook Air (running Ubuntu – natch) and up until last Thursday I had an iPhone.

I pretty much liked the iPhone, but it was mainly a consumer device (i.e. I didn’t create much using it) so I didn’t care so much, but I did get frustrated with the terms of service. It was easier for me to freakin’ buy the OpenNMS app than it was to spend 30 minutes or so every other month trying to update my project keys so I could check it out and build it. I settled on the Samsung Galaxy S3 as a replacement.

Having used it now for several days, I have to admit that I’m a little pissed at all of the talk about how Samsung (and implicitly, Google) ripped off Apple. Using the S3 is a greatly different experience from using the iPhone.

I almost wrote “totally” but I have to admit that, yes, there is a virtual keyboard, and yes, you can have a page of icons that you press to launch apps, but outside of that there is little in common between the two.

First, the phone just feels different. It is bigger, thinner and feels lighter to me (although in the interest of full disclosure I have a case on the iPhone 4 since without it my calls drop when I hold it in my left hand). The iPhone felt like a dense, solid slab whereas the S3 feels more like a bar of soap, all smooth and round edges. I am afraid that it might squirt out of my hand one of these days.

Next, the user experience is different. The way one navigates Android takes a little bit to get used to coming from iOS, but the fact that in addition to a physical home button I have two soft buttons (one for contextual menus and one for “back”) seem to make the UI experience a little cleaner (since there doesn’t have to be so many menu icons in the apps). Notifications are different, the way you can control placement of icons is different, and the idea of widgets seems pretty unique to Android. Widgets let you display information without having to actually open an app.

The one disappointment I’ve had is that the S3 doesn’t work with Banshee or Rhythmbox, so it is harder to organize my media files. I am hoping this gets fixed soon.

Android 4.0+ uses the Media Transfer Protocol (MTP) instead of just mounting the filesystem like a USB disk. I can get Ubuntu to mount the phone just fine, but when I launch Banshee it umounts the phone and then hangs. Under Rhythmbox it shows up as a Media Device, but the moment you try to access it (say, right click on it and choose Properties) it kills the app. There is an open bug on that one, but despite its use of Mono I much prefer Banshee.

Now, the S3 ships with the usual amount of kruft that you find on modern technology. Samsung has their own sync technology called Kies (no Linux client of course [sigh]) and I thought it might be interfering with libmtp. So less than 24 hours after I bought the phone I’d rooted it and installed Cyanogenmod (CM9 – not comfortable playing with the CM10 betas just yet).

Cool.

Now I don’t have any apps I don’t want, and I understand what all the apps I have installed are actually supposed to do. I haven’t seen any real performance problems with the exception of the camera crashing once and some browser issues that went away when I switched to Chrome.

With the exception of the issue managing my media, I am quite happy with this phone. The screen isn’t as crisp as the iPhone 4 but its large size really makes a difference with my aging eyes. But how anyone could confuse the two is beyond me. I hope this patent silliness goes away soon and in the meantime I’m going to vote with my wallet.

Digger and the Hugo Awards

Okay, no OpenNMS or open source content today, but since most free software geeks also like fantasy and science fiction literature I figure this might be of interest to the three people who read this blog.

One of the highest honors a writer of this genre can receive is a Hugo Award. They are given out every year by the World Science Fiction Society at their annual convention. This year I learned that anyone attending the convention can vote for the Hugo Award winners. I thought it was something like the Academy Awards where only other people in the business could vote. I was wrong. Better yet, I learned that by becoming a supporting member, anyone can vote even without attending the convention.

How great is that?

This is important to me, since a friend of mine, Ursula Vernon, has had her graphic novel series Digger nominated in the “Best Graphic Story” category. I think it would be awesome if someone who lives in Pittsboro, North Carolina, won a Hugo Award. Plus, her work is pretty fantastic on its own. And if Patrick Rothfuss can pimp out his editor, I can pimp out my friend.

Before I lose more readers with another “TL;DR” post, I just want to encourage anyone with a love of science fiction and fantasy to sign up as a supporting member and to vote. It’s US$50, but you get digital copies of most of the nominated work (DRM-free, and no, don’t ask me for a copy). If you bought just the “Best Novel” nominees it would be way more than fifty bucks, and you get exposed to amazing shorter work that rarely finds a market.

I always like to be an informed voter, so I am making a dedicated effort to read all of the nominees. Well, except for “Best Graphic Story” since my mind’s made up on that one. (Well, and Betsy Wollheim for “Best Editor – Long Form” since I trust Patrick’s judgement)

Digger is about a wombat. Wombats are marsupials native to Australia that dig extensive tunnel systems. The story starts out with our heroine digging (as wombats are wont to do) but she gets lost and emerges in a world both like and unlike her own. In an attempt to find her way back home, she enlists the help of a talking statue of the god Ganesh, unintentionally partners with a childlike shadow being (who gets her out of a couple of tight places involving hyenas) and listens to the prophecies of an oracular slug.

Cool huh?

The comics are available online, but I plan to buy the printed volumes. I am rationing them, one a month (I just ordered Volume 2 from Amazon). Check them out and then remember to vote! I also want to point out that the other nominees involve teams of people – Ursula both writes and illustrates her work – so that should be worth some extra consideration.

As far as the other Hugo Award categories, I’m working my way through the “Best Novel” nominees. The one to beat will be George R.R. Martin’s A Dance with Dragons which is the fifth book in the Song of Fire and Ice (Game of Thrones) series. I pre-ordered this and read it the day it came out and I wasn’t disappointed, so while it is a bit cliché it has my vote at the moment.

I just finished Embassytown by China Miéville last night. I enjoyed Kraken, but didn’t like this one as much. It starts off a lot like Stephenson’s Anathem, with a lot of linguistics that don’t make a lot of sense until you just plow through it for fifty pages or so. Unlike Anathem it is much more a book focused on the link between language and thought. Like pizza, when Miéville is good he’s really good and when he’s bad he’s still pretty good, I did enjoy the book and read the second half pretty much in one sitting, but if I am honest with myself I didn’t enjoy it nearly as much as A Dance with Dragons.

Tonight I start Leviathan Wakes by James S. A. Corey. I’ve never read anything by him but perhaps since he has two middle initials he can give Martin a run for the money. (grin)

UPDATE: Leviathan Wakes is awesome. At the moment it is my choice for the Hugo Award. I read it as non-stop as I could.

In part it was due to the writing style. “James S. A. Corey” is the pen name of a pair of authors, one who worked for George R. R. Martin. There is Martin DNA all over this book. It starts off with a rather brutal and shocking scene, but then they don’t return to it for several hundred pages. Every chapter is written from the point of view of one of the two main characters (although in third person) and most end in cliff hangers which makes you want to read the next one.

I ordered Caliban’s War, the second book in the series, halfway through this one.

Although this may make me sound a little like Harlan Ellison who, in his dotage, seems to be claiming to have written every science fiction story, I find myself making comparisons between any modern space opera that involves genetic mutation with Donaldson’s Gap series, Leviathan brings enough uniqueness and style to the genre that I’m certain I’ll devour the series.

OpenNMS and the Leap Second Bug

While this may be pretty old news for most, but I figured I’d post something about it anyway.

At midnight the morning of July 1st, an extra second was added to “official time” in order to keep clock in sync with the Earth’s rotation. This had a negative impact on certain machines, especially Linux machines running Java.

Since most OpenNMS installs are on Linux machines and it is written in Java, this could negatively impact OpenNMS performance.

We had one support ticket opened that was caused by this problem. In this case, OpenNMS was being run as a VM guest and it was the host machine that needed to be rebooted in order to clear the problem. The symptoms involved the CPU being pegged at 100% and OpenNMS never starting.

2012 Dev-Jam: Celebrating Community

When I became a maintainer of the OpenNMS project over ten years ago, for several months OpenNMS consisted only of me working on a laptop in my attic. One of the things that kept me going were my connections to, at the time, a group of strangers on the OpenNMS IRC channel and on the mailing lists. They kept me going at times when I wondered if anyone really cared about this project. With their help I was able to keep the project going until it could grow, and now I am very happy that OpenNMS is so much more than just one guy.

Moving forward to 2005, the business side of OpenNMS consisted of me, David Hustace and Matt Brozowski. We thought it might be fun to get together with other members of the community in person, and thus Dev-Jam was born. I invited anyone interested to fly out to Pittsboro, NC, to spend the week hacking on OpenNMS, and five people took me up on it: Bill Ayres, Craig Gallen, DJ Gregor, Johan Edstrom and Mike Huot.

It was a great week, and we learned a lot about the best way to get a group of disparate guys together. Everyone has different sleeping schedules, so it would be nice if people could set their own hours. Also, easy access to food would be cool. Finally, lots of bandwidth doesn’t hurt.

For the next Dev-Jam, Mike suggested we hold it at the University of Minnesota. And thus we decended on Yudof Hall.

It worked out so well that we have returned there for five of the seven Dev-Jams. Outside of the first one, we did Georgia Tech one year, and while it was okay it seems that Yudof is our home.

Things have changed a lot since that first Dev-Jam (although four of the original five people came this year as well). We have more money than we had back then, so this year I rented a bus and we all went to see the Twins play baseball. We had great seats in far right field, and while I’ve always pictured us as being in far left field, they worked out well and the weather couldn’t have been more perfect.

We even got our name up on the Jumbotron.

Photo credit Mike Huot

I had a couple of spare tickets left over, so we invited along some students. Ulf made some new friends.

Photo credit Mike Huot

And that’s pretty much what it’s all about: friendship. We got a lot of code written that week, but my main goal was to increase community involvement in the project. We’ve been lucky as a business that I’ve had such a great talent pool to pull from when hiring, but I worry when I hire a lot of community members that those I’m not paying will feel left out or less inclined to contribute. I really, really want a strong independent OpenNMS Users Group, and to that end I handed out copies of Jono Bacon’s “Art of Community” in hopes it would inspire people to stay involved.

OpenNMS is a great mixing bowl for bringing people together. We had people from seven countries (Canada, France, Italy, Germany, UK, USA, Venezuela). The seven Germans sat next to the one Italian as Italy once again knocked Germany out of a major soccer tournament.

Photo credit Mike Huot

One of our oldest fans, Ronnie Counts, who has been using OpenNMS longer than I have, got to meet one of our youngest developers, Ronny Trommer, or as we call him in this context, Mini-Me.

Another German, Markus Neumann, was awarded the Order of the Green Polo for his work on the code and in building the community, especially in Germany (he’s mentoring two Google Summer of Code students).

Everyone seemed to have a great time, and I am already looking forward to next year.

Photo credit Alex Finger

2012 Dev Jam: Celebrating Abundance

I am in Minnesota this week attending the seventh annual OpenNMS Developers Conference. With the exception of one year, we’ve been having them annually since 2005 when five people (outside of opennms.com) flew to Pittsboro crash at my house and to spend the week hacking on OpenNMS. This year we have 26 people from seven countries descending once again on Yudof Hall at the University of Minnesota for a week of community, coding and just having a good time. What’s even cooler is that four of those original five guys are still here.

This is one of my favorite times of the year. While I work almost every day with someone who is not local to me, this event brings us all together so we can work face to face. It’s an amazingly fun week.

I was trying to label what the Dev-Jam experience is like and I came up with things like “enjoying being a geek” or “engaging in free culture” but the best phrase I could come up with was “celebrating abundance” (especially when it comes to food at Dev-Jam).

I was once taught that the definition of “economics” what the study of how limited resources are applied toward unlimited desires. This makes sense in terms of food, water, cars, gold, etc., but it seems to break down when it comes to digital products, such as music, video and software. While good software is definitely scarce, once it is created the cost to produce more of it is close to zero.

Since creating software is both a iterative and evolutionary process, it seems to follow that the more software that is available to developers, the faster it can be produced. Think of it like bittorrent – the more people share the faster it happens.

And since I brought up bittorrent, I have been following with great interest the recent debate concerning digital music, sparked by a post by a 21 year-old woman on NPR’s “All Things Considered” blog. In much the same way that fauxpen source companies try to redefine “open source” to mean anything from the “freemium” to the “open core” model, media producers (music, video, books) are struggling to figure out how to deal with the fact that their traditional way of generating revenue is being threatened. It seems similar to what the horse carriage industry felt with the arrival of the automobile, or the telegraph industry when the telephone came along. You can fight it for awhile, but eventually you’ll lose.

(Note: I want to point out that I am extremely happy to see Simon Phipps appointed as the president of the OSI as I believe he will do a lot to fix this misappropriation of the term “open source”).

I love it when content creators, instead of trying to preserve the status quo through legal threats or legislation, actually embrace change for their benefit. This digital revolution creates an environment where the people who create the content can be the ones who directly profit from it. People like Louis C.K. and Jonathan Coulton seem to “get it” and their lives (and pocketbooks) are enriched accordingly.

Which brings me back to abundance. The traditional model would say that you can’t get qualified programmers to write code unless you pay them lots of money. I am currently surrounded by a number of amazing coders who are working on OpenNMS for free. The traditional model says that you can’t make something that appeals to a limited number of people, but numerous Kickstarter projects have shown that to be wrong and that there is a lot of money out there, but traditional markets haven’t been able to efficiently tap into it.

We are living in a time of abundance for those who are willing to look for it. I feel extremely blessed that OpenNMS has both survived all of these years and prospered, and I am exciting to see what the future brings.

Ubuntu (64-bit) and Amazon MP3 Downloader

I am a big fan of Amazon and I tend to buy all of my music from them, mainly since they were the first to offer legitimate music downloads without DRM.

I was on their site today to buy The dB’s first album as a band since 1987’s The Sound of Music: Falling Off the Sky.

I hit a snag. Now that we are surrounded by “the Cloud(tm)”, Amazon will store your purchases so you can always get them, but the bad news is they want you to use a piece of proprietary software called the “Amazon MP3 Downloader” in order to get them to your system.

For Ubuntu, they only have a version that was written for 9.06 and only in 32-bit mode. I am running 12.04, 64-bit and I got a lot of errors trying to install their .deb.

Hunting around I came across a way to deal with this. When you try to download the files from the Amazon Cloud you will be prompted to download the Amazon MP3 application. In small print under that should be a “click here if you have already installed it” link. That sets a cookie on your machine that will allow you to download the .amz file which is needed to access your mp3s.

You can then use clamz or pymazon to download your music by feeding it that .amz file. I used clamz since it was already included in Ubuntu.

A Cautionary Tale: Rogue Amoeba and Apple

At the OUCE people were teasing me because I kept saying “Apple is Evil”.

Now I really don’t believe in good and evil as absolute constructs, but the phrase gets my point across. If you are a casual user of Apple’s products you probably don’t care, or more likely just don’t notice, how tightly Apple is trying to control both the information you create and what you can do with it.

[Note: for those of you who point out that Apple is a fine example of capitalism, they aren’t. In functioning markets, profits like Apple’s are not sustainable because competition will arise to drive prices down, to the benefit of consumers. With the double edged sword of stupid software patent lawsuits and a hegemony on components brought about through having container ships full of cash, Apple is preventing this from happening, or at least slowing it down, but I guess that’s a topic for another rant].

Some people think that I hate the idea of paying for software. Not true, I spend a lot of money each month on software. I prefer free alternatives, but I am more than willing to purchase commercial software as long as the vendor looks out for my privacy and the software works (or better yet, I prefer spending my money supporting free software).

One company I’ve spent money with over the years is Rogue Amoeba. They write wonderful audio-focused utilities for OS X. Recently I used Audio Hijack to digitize a collection of vinyl albums and then processed them through Fission. Through their Airfoil products I could stream music to any of my devices (well, before I started using Ubuntu – I really wish someone would work to make Linux tools that can leverage Apple’s Airplay/Airtunes technology or create a free alternative). The apps were inexpensive, easy to use, and got the job done.

Yesterday on the Rogue Amoeba website, they posted the following: “Apple Has Removed Airfoil Speakers Touch From The iOS App Store”

Airfoil Speakers was a little app that allowed you to stream music from your computer to any iOS device. It had been around since 2009. No reason was given for why this application was pulled and nothing had recently changed to act as a trigger for its removal.

Rogue Amoeba isn’t some little flight by night software company. They’ve been around a long time, been very focused on OS X (even when it wasn’t cool) and are all around good guys. While they are probably somewhere down the list of Apple business partners from, say, Samsung, my guess is that they are much closer to the top than most. The fact that Apple would just yank their app with no justification shows how little Apple cares about those who develop on their platform.

Think about it – Rogue Amoeba, and thousands of other software companies – have bet the farm on being able to supply software for Apple devices. These are good companies employing smart developers who have mortgages to pay. Now, to a large extent, their livelihood is being threatened by Apple’s fickle control of its marketplaces.

The rumor is that Apple is creating a similar product in the next iOS release, and apparently the terms of service prohibit applications that duplicate Apple-provided functionality.

Control is the main point here. One reason I left Apple last summer was that I saw that company taking more and more control over what I could and could not do with the devices on which I create. I could envision some point in time where Apple would make a decision I didn’t like, and then it might be too late or too expensive for me to back away.

I could see Apple moving to standardize everything, including their personal computers, on an iOS platform. With terms that prohibit competing products to anything Apple itself creates they are stifling competition, and I could foresee a future event where their policies might arbitrarily screw me over, although I didn’t really have a concrete example.

I do now, even if it didn’t happen to me directly.

Nifty Trick for Auto-Acknowleding Notices

OpenNMS has a built in notification system that can act like a mini-trouble ticketing system. It’s triggered by events. Once an event creates a notice it “walks” a destination path where various actions can occur (send an e-mail, send a page, call a phone number, etc.)

The path has various “targets” and there is an escalation delay between each one. If the notice goes unacknowledged, the next set of targets is triggered.

Each path also has an “initial delay”. This is some amount of time where no notices are sent. It is useful combined with a feature that allows certain events (specifically nodeUp, interfaceUp and nodeRegainedService) to automatically acknowledge corresponding “down” events.

I hate getting a page at 2am that something is down. What I hate more is when a minute later I get a second page that the problem has been resolved. To minimize alerting on these transient outages, I use an initial delay of two minutes (or more) for most of my paths. This allows OpenNMS to make multiple attempts to see if the problem is resolved before notifying me.

Now, there are some issues with the notification system. First, it is triggered by events. This can be a pain for, say, a pesky trap that comes in every minute. Those can be reduced using the alarm sub-system within OpenNMS, so the plan is to eventually move the notification system to trigger on alarms. Since we already have a trouble ticket API built in to alarms, rather than modify notifd we plan to move that functionality to a separate product in the future.

The second is that this auto-acknowledgement feature only works if each event can be refined with a nodeid, interface or service name. If not, there is no way to differentiate similar events, and thus the system may acknowledge the wrong notices.

I just finished working on a support ticket to implement this auto-acknowledgement functionality using alarms, and I thought I’d share my solution.

First the events in question have to become alarms. This is done by adding an <alarm-data> tag to each event. The down should look something like this:

<alarm-data
  reduction-key="%uei%:%dpname%:%parm[#1]%"
  alarm-type="1"
  auto-clean="false"/>

Note that I just chose an generic %parm[#1]% to use the first event parameter to uniquely identify this event, but it could be any parameter, or a combination of parameters, depending on the event in question.

The up would thus look like:

<alarm-data
  reduction-key="%uei%:%dpname%:%parm[#1]%"
  clear-key="[UEI of Down Event]:%dpname%:%parm[#1]%"
  alarm-type="2"
  auto-clean="false"/>

Note the addition of the “clear-key”. This must match the reduction key of the “down” event.

Once these alarms are in place, it is pretty simple to create an automation.

Edit vacuumd-configuration.xml and add an automation:

<automation name="ackMyNotices"
            interval="60000"
            active="true"
            trigger-name="selectMyUpAlarms"
            action-name="ackMyDownNotices" />

with a trigger and an action. The trigger should fire when the “up” alarm is generated:

<trigger name="selectMyUpAlarms" operator=">=" row-count="1" >
  <statement>
    SELECT *, now() AS _ts
    FROM alarms
    WHERE eventuei = '[UEI of Up Event]'
  </statement>
</trigger>

So if there is at least one “up” event, we’re good.

Finally, perform the action:

<action name="ackMyDownNotices" >
  <statement>
    UPDATE notifications
      SET answeredby='auto-acknowledged', respondtime=now()
      WHERE eventid = (SELECT lasteventid FROM alarms WHERE reductionKey = ${clearKey})
      AND answeredby is null
      AND pagetime <  ${lastEventTime}
  </statement>
</action>

This will update the entry in the alarm table to show it as being “auto-acknowledged” if it isn’t acknowledged already. The tricky bit is that join where the eventid from the down alarm is matched with the eventid used for the notice.

Note that this works best with alarms that aren’t reduced, i.e. there aren’t multiple downs before the up event.

2012 OpenNMS Users Conference Europe

Schnapps is Evil.

That isn’t the only thing I learned at the fourth annual OpenNMS Users Conference – Europe (OUCE), but it was the last thing so it stuck in my mind.

But I’m getting ahead of myself.

After spending a great week in Montréal, I was home for around 33 hours before heading back out to the airport. Even though there is a direct flight from RDU to London, I try to avoid Heathrow whenever possible, so I flew two hours in the opposite direction to catch a flight from DFW to Frankfurt. While the flight was uneventful, I didn’t manage to sleep so you can imagine that I looked and felt my best upon arriving at 8am on Monday.

I met up with Jeff, who had flown in from Atlanta, as well as Ronny and Markus, and we headed to the OpenNMS office in Fulda.

Since it is associated with the University, we then headed to a student run place called Cafe Chaos for coffee and to catch up.

Over time a number of other OpenNMS folks joined us. Christian is working on our VMWare integration, and Dustin is one of our Google Summer of Code folks. Sven brought us a cool little plush toy representing the Planck Epoch, or the universe immediately after the Big Bang.

I’m not sure if it was to scale – those things are pretty hard to measure.

I was also introduced to Club-Mate, the official drink of European hackers. It’s an ice tea-like beverage with a high caffeine content and relatively low sugar. I needed caffeine at the time so I drank a half litre, and I can understand the origin of the drink’s tag line which is “You get used to it”.

After lunch we wandered around campus and looked at the facilities. Since Ronny teaches part time at the school, Markus’s impersonation of a young student calling out to him “Herr Trommer” in a high pitched voice became a meme for the week.

Due to a mixup in scheduling, the OUCE was scheduled the same week as a huge emergency service vehicle convention, so all of the hotels in Fulda were booked, thus we got rooms just outside of town at the Hotel Gasthof Rhönblick. It was actually quite nice for the price, and I always like to stay a place where bikers are welcome. That afternoon we checked in and then headed back to Fulda for dinner.

Monday evening we met up with Uwe from Nethinks and his bride Daniela. While the company was awesome at dinner, by this time it was getting close to 11pm and Jeff and I had been up for 34+ hours, so we called it a night.

On Tuesday the OUCE started with the first of two training days. I did a one day seminar on the basics of OpenNMS. While our formal training offerings are a little more structured and go into greater detail, this is a good introduction to the application. Roughly 90 minutes are spent on each major facet of OpenNMS: provisioning, events, service assurance and data collection. We had a little over 30 students in attendance and everyone seemed to enjoy it (I didn’t see anyone sleep or look to terribly lost).

Wednesday was the second, or advanced, training day. This was added after last year’s conference when people wanted to explore the more complex features of OpenNMS. Jeff and Ronny taught this day (to let my voice rest) and it included things like the remote poller, the trouble ticketing interface and JasperReports. We had a few more people show up to for this class since they had been there the year before, and the only complaint I heard was that it was a lot of material to cover. Even when some of the hands-on stuff is removed we are trying to squeeze a five day class into two so it can be a bit much to take in.

On Thursday the OUCE kicked off in earnest. We decided to split the two days into a standard conference agenda on Day One and then a Barcamp on Day Two.

I started the day off with a presentation on the State of OpenNMS, which included a review of the new features in OpenNMS 1.10 as well as the future direction as we build toward 2.0. I also was able to present a mockup of a new user interface based on Vaadin.

This was followed by a great presentation by Alex Finger on project management for large NMS deployments. As someone who has been creating network management solutions for 25 years I can’t stress enough the need for some formal project management structure in order to guarantee the success of such projects.

It was during this talk that I started to structure the agenda for the OpenNMS Users Conference Americas (OUCA) to be held at the end of June. While a lot of people involved with OpenNMS are technical, many tend to have more political than technical challenges to overcome, so I thought it would be cool to have a track focused on the politics of getting a solution like OpenNMS into an organization instead of just how to use it. Alex is going to give this presentation again in Minneapolis as part of that track.

The next four talks were focused on integration. Jeff and Ronny talked about integrating OpenNMS with Puppet. Markus did a very technical talk on instrumenting Java applications with JMX so that OpenNMS could be used to monitor metrics from inside the applications themselves. Michael Batz from Nethinks demonstrated how he was able to use NagViz, a tool designed to integrate with Nagios, to present data from OpenNMS in a graphical format (i.e. maps), and I closed the day with a “Stupid Data Collector Tricks” presentation that included, among other things, a detailed description of my OpenVND project where we used OpenNMS to monitor a drink machine.

After seeing how well these talks were received, I invited the presenters to repeat them for the OUCA. It kinda solidified the theme of “Integration” for the conference. Although, perhaps we’d have more people show up if we unpacked our suitcase of “happy cloud”.

At the end of my presentation we got together to plan Friday’s barcamp. Everyone got up and introduced themselves, and we took down suggestions for topics and placed them on the wall. People would then vote on the talks they most wanted to see, and the top talks would be presented.

We had a little time for a break before everyone headed to the local brewery for an evening event, and no, this is not where the Schnapps comes in. I really like the Wiesenmühle – the beer is excellent and the food is good – although I was unhappy that this wasn’t the right time of year for Dunkel.

After eating, I excused myself for a couple of hours so that I could head to the office and catch up on e-mails and phone calls. Two hours later when I went back there were some hardcore OpenNMS folks still at it, but our designator driver was able to drive everyone back to the hotel.

Friday we split the day up into two tracks. The first session found me discussing some useful Net-SNMP tricks while Ronny discussed some ideas for using OpenNMS to monitor IPMI interfaces.

The second session was combined to discuss developing the European user community. While it didn’t get a ton of votes, I felt it was important enough to warrant the slot, and considering that we went 30 minutes over the time budget, we had a lot of good discussion. I really, really want to see the development of an independent users group for conferences such as this. I have no problem with The OpenNMS Group acting as a sponsor, but this should be something for the users by the users and I am certain that our partner Nethinks, who carries the majority burden of organizing this thing every year, could use some help. This session resulted in the creation of a new opennms-europe mailing list and I’m hoping to see the discussion continue there for next year’s conference.

After lunch we had three more sessions. In the first one, Ronny talked about Christian’s VSphere integration for monitoring VMWare while Markus talked some more about JMX integration (it was a popular topic). After that I did a presentation on making OpenNMS highly available, while Jeff did a session on Drools integration. His talk went almost an hour over time (well into the afternoon break) so we shifted the last session out even more. In that one, Jeff talked about backup strategies and Michael revisited maps.

During the afternoon we said goodbye to a number of people who had trains to catch, and the overall atmosphere for the conference was very upbeat. People seem to like OpenNMS and are eager to get most out of it, which does require an investment in time and energy even if the software is free.

There was still a core group of us left after everyone else had gone, and by this time I was totally shattered. After being on the road for most of six weeks, and the last two being out of the country, I just needed to unwind.

This is where the Schnapps comes in.

We dropped our bags off at the office at went to a Cuban-themed bar for drinks. Things started off pretty simply, but I am certain I was the one who called for the first round of Schnapps. We relaxed and talked for a long while, and then the subject of dinner came up. Places were still busy, it being a Friday night as well as the other convention being in town, so we decided to head back to the Wiesenmühle for dinner.

And more beer. And more Schnapps.

I forget how much we drank that night. I remember Ronny teasing me around 10pm since that was when I wanted to be in bed, but although my memory is a bit hazy I’m certain that was a couple of hours before we actually stopped for the night. Both Ronny and Markus abstained so they could drive the rest of us back to the hotel.

I don’t really remember that ride.

I do remember getting to the hotel, saying goodbye to Ronny and then remembering, too late, that my laptop bag holding my room key was still in his car. Needing sleep, I found that the doorway to my room was comfortable enough. I then I remember sometime later being wakened by Alex who had managed to get the master key from the innkeeper to let me in my room (these locks used old school physical keys and not key cards).

The next thing I know it was 5am. I’m awake, on my bed, completely clothed, and all the lights are on in the room (I haven’t done that in decades). I set my alarm for 7am, turned off the lights and got undressed for a two hour nap before heading to the airport.

Ronny showed up Saturday morning to drive me, Alex and Ian (another attendee) to the airport. I never saw Jeff. My trip home had me going through Heathrow as I wanted to visit some friends in the UK, and I am happy to report that I managed to stay awake as Martin drove me from the airport to his home in Lyndhurst.

There, his wife Sue plied me with the restorative powers of proper English tea, and after what seemed like about two pots I was pretty much myself. On a walk around Lyndhurst I found the next company car, if I can get it with left hand drive.

Craig visited from Southampton and we had a nice dinner, and I managed my first real good night’s sleep in weeks.

On Sunday, Sue drove me back to the airport in her new Fiat 500 (what a fun little car) and I had an uneventful trip back to Raleigh (where I was happy to find two Global Entry kiosks at customs).

It was a great week, and I hope to capture some of that energy for the conference next month. We have a number of folks signed up already, and remember that the early bird special ends this week. Space is limited due to the capacity of the rooms we’ve rented from the University, so if you are thinking about coming, register soon or drop me a note so I can save you a space.

And remember, Schnapps is Evil.