Working from Home

I made it to Frankfurt without incident. Well, except that the taxi driver took me to the wrong hotel (same name, different part of town). I’m now ensconced in my fashionable Euro-hotel with expensive Internet access, and ready to work on getting into the local time as fast as possible.

International travel has certainly changed in the 10 or so years since I was last here. The rise of international data networks means that I can get here solely with my passport and electronic tickets, use my ATM card to get local currency, and once on the Internet it is almost like I’d never left home. No more travelers checks, paper tickets, expensive phone calls or time spent in line at the currency exchange.

These networks have also changed the way companies are organized. At The OpenNMS Group we are often spread out geographically, yet it doesn’t seem to matter. This week I’m in Germany, Dave is up at Quantico, Matt will be in Durham part of the week, Ben is in Vancouver (Canada) part of the week and Jeff is in Atlanta. But through IRC, Jabber, e-mail and Skype it is just another day in the office. This ability to run a company with a distributed workforce translates directly into the ability to run a distributed project such as OpenNMS, and I thought I’d share some of my thoughts on what makes the arrangement successful.

Trust: When I worked at NORTEL the phrase “Working from Home” was code for “I’ll be checking e-mail occasionally, but for the most part I’m slacking off for the day”. It takes a lot of trust in your people to know that you can leave them alone unsupervised and know that the are actually working. This is usually easy for the most part at OpenNMS because we only hire people who are extremely passionate about what they do. When you are selling services, your reputation is in your people and it is tied directly to your success. Thus I rarely worry about my guys not working (in my mind they work too hard), but that does lead into the subject of …

Time: At OpenNMS we are results driven. I’m writing this at about 10pm local time on a Sunday, which is well outside of normal business hours, and my team tends to work just as hard. Work gets done at all hours of the day and night. But it is important to set aside some period of time during the day where everyone is available. There is nothing more frustrating than needing some help and not being about to get it. In a traditional business office, one could just walk over to another office or cubical to ask for help. When everyone is separated by considerable distances and sometimes oceans, being able to quickly reach someone is important, and to have some kind of standardized work hours helps with that.

Tools: Another important aspect of running a distributed company is the choice of tools. For me, e-mail is king. My life is driven by my Inbox, as it also serves as my “to-do” list. During a normal business day I can expect a response to an e-mail sent to the team within 10 to 15 minutes.

If I need something sooner, or if it is just a quick question, we also use the Jabber instant messaging protocol. We set up a private Jabber server for just employees. While I love chatting with people, when I’m on public IM services I sometimes get overloaded with “Dude! How’s it going?” messages. It is key to have the IM equivalent of the “red phone” where people are quick to respond and the Signal to Noise ratio is high. We’ve had to actually implement strict “auto away” policies where if you are away for even a couple of minutes you status will reflect it, because unlike e-mail being unresponsive on IM is frustrating.

We also use a number of other collaborative tools. We have an internal mailing list, shared calendars and an internal wiki. The goal is to make sure that everyone in the company has access to the information they need, and no one is the sole source of any information that might be needed to address our client’s issues. We are also starting to use video more, but as anyone who has seen me can attest, this is not necessarily a good thing (grin).

Targets: You can’t run any company without targets, especially when your goal is to provide the world’s de facto network management platform and your opponents are IBM, HP, CA and BMC. We set deadlines, and although we don’t always meet them we always seem to work better and more cohesively when they approach. When your team is separated, it is important that they all know what the immediate goal is so they know what to work on and feel that the work they are doing is useful and important.

Temperament: One last piece to the puzzle of running a successful distributed company is finding people who can work well outside of the traditional office environment. Social scientists often talk about the concept of a “Third Place” meaning a place to gather outside of home and work. But what if home and work are the same place? There are a number of people who do not work well in isolation. The work/life balance becomes harder to maintain. When I started working with OpenNMS I actually had to set up a room in my house devoted just to work. Otherwise I found myself working all of the time and workplace issues started to bleed over into my personal life. With the growing availability of Internet access in a number of public places, it is becoming easier to find a place to work that removes that sense of isolation that can arise from working from home.

No matter how well your team works when apart, it is still important to get together once in awhile (we aim for once a month). I can tell in myself and amongst my team that prolonged periods of working without real world social interaction causes the bad parts of the job to seem worse than they really are, and the good parts less good. Even if it is just sitting together in a room working and listening to music from AirTunes, optimism and pride in what we are doing is just easier to spread that way.

Which is one of the reasons I travel so much – to spread the love (grin). I am really looking forward to meeting OpenNMS fans in Frankfurt this week as there is nothing that can replace meeting face to face. It is one way to build the community which is so important to success. I’ll be sure to post more as the week unfolds.

Ooh la la, Gimme a Litre of GPL

I saw this in France and I thought it was cute:

While I understood that this was a pump for Liquid Propane Gas, my first thought was the GNU General Public License.

I wonder if anyone else has this problem.

I remember once I was at a conference in Boston, and I saw an older, extremely well-dressed woman carrying a bag that said “AIX“. I was so curious as to why she would have such a bag that I started to follow her. When I got closer I saw it was “A/X” for Armani Exchange.

Happy Anniversary

Today marks the third anniversary of The OpenNMS Group. Since we started with no venture money and pretty much nothing but a strong will to make OpenNMS the main management platform of choice, this is a nice milestone.

I have told the “OpenNMS” story a number of times, but never on this blog, so perhaps now would be a good time to tell it again.

I didn’t start OpenNMS, nor am I responsible for much of its beauty. Brighter people than me came up with the original architecture and design, and even brighter people than that keep it growing today.

Since I wasn’t there, my knowledge of how OpenNMS got started is pieced together from what various people have told me, so let me apologize in advance if I get things wrong or overlook someone that I should have mentioned.

OpenNMS was started in 1999 by Steve Giles, Luke Rindfuss and Brian Weaver. Steve was one of the founders of Onion Peel Software, which had just been sold to a company called Netplex. Onion Peel provided software and services around the HP OpenView suite of products, and a map management tool called Amerigo was probably their most famous product. Luke and Brian had both worked with Steve at Onion Peel.

Steve was eager to start another company (he’s one of of them “serial entrepreneurs“) and wanted to stay in the network management space. Now at this time companies like Riversoft had raised £83 million in investment, so it was almost impossible to compete in that space without a lot of money. Steve and company saw open source as an opportunity to develop the next-generation of management software without having to raise tons of capital.

They named the company PlatformWorks and started hiring. Ben Reed, one of the current OpenNMS employees, was employee number 5.

PlatformWorks was acquired by Atipa. Atipa was trying to follow in the footsteps of VA Linux, which had a record IPO. Both companies were trying to become the Dell or Compaq of Linux. This was before it was realized that software drivers aren’t really the issue, and that it’s more about the supply chain and getting that disk drive from Malaysia on time.

Anyway, with the market crash of 2000, Atipa needed a new business plan. They had a warehouse full of 1U servers, and this network management software, so why not create a network management appliance for small to medium sized businesses? Thus Oculan, “Your Eye on the Network”, was born.


The original Oculan OpenNMS team

Oculan used OpenNMS as the basis for its management application, but added quite a number of additional features and interfaces to make it easier to use. They still worked on OpenNMS as a separate project, as even though the bubble had burst people still needed to manage their networks and there was a lot of interest in an open source project for this.

I entered the picture in 2001. I joined Oculan on September 10th in order to build a services and support business around OpenNMS. The next day the world changed, and it became a lot harder to sell anything, much less this new idea of open source network management.

OpenNMS 0.8 had just been released. In the following months I worked to figure out this whole services business and to get some commercial customers. In 2002 we released OpenNMS 1.0, and I managed to sign up our first four customers (two of which are still customers).

Then in May of 2002 Oculan got new investors who wanted to make two major changes. They wanted to focus solely on the appliance and they didn’t want to open source any software. As copyright holders they are free to publish the code under any license they choose, although once something is published under a license such as the GPL it isn’t possible to “unpublish” it.

I believe I could have stayed at Oculan, but in the months I’d been working on OpenNMS I started to really see its potential. To be honest, when I first took the job I thought, hey, HP OpenView without the cost. But OpenNMS is much, much more than that, and without someone pretty much dedicated to working on it, it would die.

So I made one of the toughest decisions of my life. I went down to Steve’s office and asked for OpenNMS. Oculan would keep the copyright to 1.0, of course, but I would become the project’s maintainer and get the domain names, etc. We worked out the details and suddenly I was much more wedded to open source than I ever thought I would be.

That was the easy part. The hard part was telling my wife.

We live on a horse farm. The previous spring her company, Union Carbide, was purchased by Dow, and rather than move to Michigan she decided to leave, spend the summer working with the horses, and get a job in the fall. After 9/11 it was much harder to find a job, so she was working in a tack shop when I decided to do this. I had literally bet the farm on OpenNMS.

The really crazy part was that I didn’t know Java – at all. Sure, I had programmed before, but not seriously for several years. The guys at Oculan were forbidden to work on OpenNMS, since at that time the code base was so similar that there were IP issues, so I was on my own.

Or so I thought.

I started Sortova Consulting Company (pronounced Sore-toe-va even though it meant Sort Of A), bought The Big Black Book o’ Java, and got to work. Slowly I was able to fix some of the more grievous bugs, and slower still a community starting to gel around the project. I got more customers and was able to make my mortgage payment. The community kept me going when I really had no one else to help me, and even today I am humbled by their involvement.

On the farm I didn’t have access to any real form of broadband Internet access. I used DirecPC satellite, which was iffy in the best of times. Getting something like a T1 to the house was prohibitively expensive, and it was actually cheaper to rent an office downtown with DSL. So that’s what I did.

I decided to dump the satellite to save money (my “spend less than you earn” philosophy had gotten me this far) so I needed a dial-up provider. As one of those folks who buys local when I can, I called up the local ISP, Blast Internet Services. Always looking for new customers, I asked the guy who answered the phone what they used to monitor their network. He didn’t know, and suggested I “Talk to Lyle”.

Lyle Estill is one of the most interesting people I know. He’s one of the few people who can make a decision faster than me. To make a long story short, he liked the idea of open source, and since Blast was services company and Sortova was a services company, why not have two struggling companies struggle together?

So I put Sortova to bed and began working at Blast. It was great to have other people working with me. Business grew and OpenNMS kept getting better. In early 2004 we were able to hire two more people to work on OpenNMS, David Hustace and Matt Brozowski.

David and I have worked together at four different companies. There is no other person I’d rather have by my side in business. Even though he had to take a large pay cut and a great risk to come work on OpenNMS, he did it anyway.

Matt Brozowski has forgotten more about Java than I’ll ever know. He worked at IBM when they licensed Java from Sun, and he also did a lot of programming on NetView, so he understands network management. Matt has five kids, so the decision to come work on OpenNMS was also not an easy one, but he also did it anyway.

The three of us now formed the core team around OpenNMS, but a number of other people in the community had stepped up too. The lists now had a life of their own, and I didn’t feel the need to answer every single question.

In 2004 Lyle left Blast to work on sustainable energy in the form of biodiesel. I am honored to know that his exposure to open source communities has helped shape his own. With Lyle gone the three of us approached Blast to acquire the OpenNMS assets, and on September 1st, 2004, The OpenNMS Group was born.

Shortly after that we started the Order of the Green Polo to give an official form to the community, and together we set off to do great things.

So on this anniversary let me say thanks to everyone who made this possible. You helped write this story, now let’s go make history.

And the winner is …

I just got back from the Sourceforge Community Choice Awards party. So without further ado the winner of the Best Project for Sysadmins is:

phpMyAdmin

The winner for Best Project for the Enterprise is:

Firebird

Oh well. It was nice to be nominated in such fine company. I want to thank everyone who voted for us. It means a lot to me to get such feedback from the community. Also kudos to Ross Turk and company for a nice party, although I didn’t win the iPhone.

In too few hours I will be on my way to Minneapolis for Dev Jam 007. I leave here at 8am and get there about 5pm. Envy me.

Forget the Bobs, I'm Going to See the Alisons

In the past I’ve been pretty critical of LinuxWorld Expo, and I may have to take some of it back.

I love the .org Pavilion at the LinuxWorld shows. We went to both US shows in 2005, and we won a Product Excellence Award at the San Francisco show. So we were really surprised when we weren’t invited back.

This year I received an e-mail from Alison McCormack with an offer to submit OpenNMS for consideration for this year’s San Francisco show. This was a nice change from the previous year where the process was shrouded in mystery, so I put our name in the hat.

Today I received an e-mail from Alison Dwelley that read in part:

Dear Tarus,
It is my pleasure to inform you that your organization has been chosen for this year’s .org Pavilion at the LinuxWorld Conference & Expo being held in San Francisco August 7-9, 2007

w00t.

This happens to be the week after Dev-Jam and right after I have to be in a wedding, so I had to price a plane ticket from RDU -> MSP, MSP -> ATL, ATL -> SFO, and SFO -> RDU. If we get a spot at OSCON I do believe my wife will kill me.

If you can’t make it to Dev-Jam, hope to see you in San Francisco.

Slashdot Spanking

The server for opennms.org has been around for almost five years now. It’s nothing fancy, but lately we’ve been running out of room on the disk drives so we went out and bought a new (refurb) server: lots of disk, lots of RAM, and two fast processors.

Unfortunately, I didn’t get the website moved fast enough.

The opennms.org site uses Mediawiki, which is written in PHP and had a MySQL back end database. It’s not the most efficient site in the world (although we have used various tricks to make it faster) but it has served us well. The usual load average is around 0.10 to 0.20, with occasional peaks when the nightly tape dump runs or someone is doing a lot of mail synchronization.

This week has been a big press week for OpenNMS. SearchNetworking.com issued its annual user survey and OpenNMS was awarded Gold in the network management platform category. This was thrilling for the team for a variety of reasons. First, this was a user survey. As an open source project we don’t really know if anyone is using our work. We have a wall of postcards but we don’t get many, and we have a large number of downloads but we can’t really tell if they ever get installed much less used. The fact that it was end users that honored us with this award means that someone, somewhere must find it worthwhile to use it.

Second, this was a survey of all tools, not just open source applications. People often want to associate open source with “cheap” or “bargain” and it is really the wrong way to look at it. OpenNMS can be better to use simply because it is a better tool for the job. When deployments are measured in man-hours, an OpenNMS deployment can take a lot less time than a comparable OpenView or Tivoli deployment. That’s where the real cost savings can be had, although the fact that there are no licenses fees involved is just a bonus. (grin)

Finally, it was OpenView and Tivoli that OpenNMS beat out for the Gold. We have always positioned the product against these platforms. Having used those applications in the past, I think OpenNMS is much easier to use and deploy. Many new users are overwhelmed by the sheer amount of flexibility OpenNMS offers, and it can scare some of them off. But OpenNMS was never designed to compete against tools like What’s Up or Nagios, which many find easier to install. However, if OpenNMS can monitor 50,000 devices at one site, it can really scream on a network with 500.

Another bit of press for OpenNMS came from Network World. I was in a face-off where the question was “Can You Trust Your Network to Open Source”. I, of course, took the “yes” position against Roger Greene from Ipswitch (makers of the aforementioned What’s Up product). It was easy to write the piece but much harder to do the podcast. See, What’s Up isn’t designed to be an enterprise-grade management platform. It is built to be a real easy, pointy-clicky application that small to medium businesses can use to get a view into the status of their network. For many, the time spent installing What’s Up may be less than working with, say, a Nagios installation. Note I am not saying “for all”, but I’m all about “if it works, use it” and if What’s Up works for you, then great. I liked talking with Roger and could see his point, although some others had to point out the flaws in his arguments a little more strongly. (grin)

But I do stand by my comments on the podcast that it’s ease of use makes it a commodity. And it is priced as such – a basic install of What’s Up is much less than an OpenNMS support agreement. But for those who need a high end solution for either complex systems or large networks, or simply a system that allows for easy customization, it’s going to require something like OpenNMS.

All is well and good until Monday afternoon when my e-mail stopped working. I tried to SSH into the box and couldn’t reach it, so I opened a support ticket and tried to figure out what happened.

What happened is that we were on the front page of Slashdot, and it was spanking our little five year old white box server. This is the first reply I got from my provider:

At console, your server was very slow, and very hard to work with; I was unable to determine why SSH failed to respond to remote connections. Your server is under a lot of stress, the last load average I was able to get:

load average: 162.58, 171.32, 149.41

Heh.

I was finally able to get them to stop httpd which freed me up to work on the box, and I quickly moved the site to the new server. I upgraded to MediaWiki 1.9.3 and we installed squid front end for acceleration, and things look much better now (although it could be due to the fact that we’re not getting pounded like we were yesterday as much as the move to a new server).

To anyone who tried to visit us and failed, I apologize. And when you actually get to our sites, let me apologize again, because they aren’t the prettiest in the world. As we say on opennms.com: Professional Software, Amateur Marketing.

Memory Lane

I talk a lot about how projects like OpenNMS really depend on the community. Well, we were chatting on the IRC channel (#opennms on freenode) today about classics from the past, and through the magic that is gmane we were able to dig up some of the more classic posts.

First off, at OpenNMS we really try to be nice, and I think we succeed compared to some other lists. I don’t think anyone ever really gets flamed, although we may reply with humour. For example, last month I replied to the following:

From: Tarus Balog
Subject: Re: [opennms-discuss] customize opennms web console
Date: 2006-11-09 18:01:35 GMT

> Hi all, does any know how to customize opennms web console ?

Oooh! I do, I do.

-T

Okay, that was a bit snarky, but what kind of question is that? It’s like I’d asked “does anyone know how to program in Java?”. Having done this for a long time now I read it as “Hey, let me interrupt here and due to my laziness can someone stop what they are doing and create a long tutorial on modifying that there web-thingy?”

Sometimes questions like that just hit me wrong. Had it been phrased “Hello. Sorry to bother, but I’ve looked all over and I can’t seem to find any documentation on the OpenNMS webUI. I understand how web pages work, although I haven’t worked with Tomcat before, so if someone could point me in the right direction I would appreciate it.” I would have crafted a slightly more useful answer. Seems like I’m not the only one, here is the infamous “masochistic coding slaves” post:

Continue reading

Ben Reed

When I first started working at Oculan, oh so many years ago, I didn’t know too many people there. I wasn’t sure if I was going to fit in or like it. However, in the afternoon of the first day someone made popcorn in the microwave. Imagine if you will a large room with low-wall cubes, with a refrigerator and a microwave in one corner and you pretty much have the geek corral at Oculan. So when someone makes popcorn, the smell is going to permeate the place.

Of course someone mentions that the popcorn smells good, to which someone else points out that it’s not ordinary popcorn, but “Boy Scout Popcorn”. This leads to the usual jokes of “made with real Boy Scouts”, etc.

But then I hear Ben Reed from my desk doing his best Charlton Heston “Boy Scout Popcorn’s People! It’s Peeeeeeople!”.

I knew I’d found a friend.

Ben is a member of the Order of the Green Polo and is mainly responsible for OpenNMS on OS X.

Today Ben and Cynthia Fulbright were married, and should be either on their way to Scotland soon for their honeymoon. Congratulations to both of them. I am very disappointed to have missed the wedding, as I’m still suffering from the cold/flu/whatever I picked up in Portland. Suckage. I sure it was beautiful and we at OpenNMS wish them all the best.