Month: June 2008

Farewell, Mr. Gates

Posted on by tarus

Today Bill Gates officially retires from Microsoft. As much as we might like to hate his policies, without him I doubt our industry would be what it is today. He took the power out of the hands of IBM and put it into the hands of everyday consumers. He amassed a great fortune, and it looks like he’ll be using the rest of his life to spend it, mainly on charitable causes. By doing so he has created an example for others, like Warren Buffett. I wish him well.

The first piece of computer software I ever bought was published by Microsoft. It was the “Colossal Cave Adventure” for the TRS-80.


A picture of me yoinked from my high school yearbook

The TRS-80 was my first computer. I got it for Christmas 1978 when I was 12 years old. It had 4K of RAM and a black and white screen. By April of 1979 I had upgraded it to 48K of RAM (the other 16K of addressable memory was in ROM) and added two 80K floppy drives. That system ran “TRS-DOS” which shared a lot of similarities for what would become MS-DOS.

Now, back then most software was “open” in some sense. Most programs for the TRS-80 were written in BASIC, which is an interpreted language so it was not possible to hide the code. We didn’t have the Internet back then, so my main contact with others was through magazines, such as Creative Computing and Kilobaud. They would often publish program listings that I would stay up late typing in and modifying. Of course I’d make mistakes, but that would help me learn to troubleshoot code.

My Dad was working for General Electric at the time, and I sometimes got to play on their PDP 8. It ran FORTRAN, but it was similar enough to BASIC that it was easy to pick up. In 1982 he bought one of the first IBM PCs: CGA monitor, dual floppies, 256K soldered to the motherboard.

Also in 1982 I went off to high school. They had a VAX, and so I was introduced to the concept of multiuser systems. We had e-mail and shared space were we could put up documents – sort of a precursor to the web.

In 1984 I went off to California to attend Harvey Mudd College, one of the many colleges I eventually was asked to leave (I’ve been kicked out of some of the finest schools in the country). There I met my first VAX running UNIX (Berkeley I believe) and experienced the Internet for the first time (as well as wasting hours playing rogue).

When I eventually returned to North Carolina in 1986 I didn’t have a computer, but a friend had bought one of those fancy “Macintosh” computers. It was pretty cool, but I couldn’t afford one so I used a PC. Several friends of mine from Mudd had gone off to work for this company called Microsoft outside of Seattle. They would send me software, including a Paint program launched on a system called Windows (version 1.0). It really didn’t compare to, say, MacPaint on the Mac, but little did I know how it would catch on. I also got Windows version 2 and eventually 3.0. I could run an app called Procomm that would let me use a modem to communicate with other systems.

Using a modem you could dial in to bulletin board systems (BBSes) and post messages, send mail and download files. I tended toward those systems running WWIV, and in just day or two I could get a message to a friend of mine across the country through Fidonet and other grassroots networks.

Some BBSes were multiuser, but that required multiple, expensive phone lines. So flame wars were funny: you’d have to post, log off, wait awhile until the other guy could log in and post, then repeat the process.

I can remember when 2400 baud modems came out because it was the first time the text would download faster than I could read it. True Internet access was only available on campus, but I eventually found a way to dial in to the campus network (I had friend in IT) so I could use it from my apartment.

Ah, those were the days. Anyway, what are your first memories of Microsoft and what was your first computer?

Gettin' Our Marketing On

Posted on by tarus

At OpenNMS it seems that we are always doing things backwards. We didn’t spend any time working on a business plan and seeking investment, we just started to provide services and running a profitable business. We didn’t hire any dedicated sales people, because we found that our technical people are the best at sales. And we pretty much let the project market our business.

I don’t like marketing. While I enjoy finding out about new things I might want to buy, or things that may help solve a problem, too much of marketing seems to be about creating an artificial need than actually helping customers. I know that’s a little harsh, but I’ve seen too many marketing campaigns that either rely on superlatives (“We’re the best, everyone else sucks, buy me!”) or fear (“You’ll lose your data, you’ll lose your job, you’ll lose your money unless you buy me!”) that I just get turned off by the whole mess.

And we’ve done well without any marketing. I still wince when someone posts to the opennms-discuss list with “Is there commercial support for this and where do I find it?” but I’d rather keep OpenNMS the Project distinctly separate from OpenNMS the Services Business than gain a sale or two.

But something happened that made me rethink our need for marketing. We were talking with a potential client in the UK. They had talked to most of the commercial players as well as the commercial “open source” players and then they came to us. David talked with them a bit, and this is the reply I got:

Talking with Dave was an eye opener for $COWORKER and myself, I was under the right impression already from the email discussions I have been having with Jeff and yourself. We find you guys very honest and easy to discuss with about our environment and the challenges we currently face.

We got the PO.

What we provide is a very personalized, results focused experience. The fact is that if OpenNMS is not a right fit for your organization, we don’t want you as a client. We’d just end up spending hours upon hours trying to get OpenNMS to do something it isn’t designed to do. It doesn’t work for us, it doesn’t work for the client. So we’re very eager to learn about the goals for a potential OpenNMS deployment as well as being quite honest if it doesn’t fit. A software company, on the other hand, hopes you’ll buy the product and not use it. If you use it, you might need support, and in their business model one of the best cases is if it becomes shelfware, since support costs money and reduces margins. At OpenNMS we have to provide great service since you can always just continue to use the product without us.

But how do I get this across?

Since I suck at marketing, I needed to find someone who understood it and who understood open source. Then I remembered a woman I met at BarcampESM.

As luck would have it, Michelle Greer understands both and she had just started her own company, SimpleSpeak Media.


Dave, Matt, Ben, me and Michelle

So for once I got to be the customer and order me up some of that consulting mojo. Michelle came up to metropolitan Pittsboro for a couple of days and really helped us work out a plan. For one thing, she pointed out that the .org site gets 50K unique visitors a month, and the .com site … 2500.

We still have a number of things to finish, but I think that soon we’ll have a … well, I think it’s called a “campaign”. Something that stays true to our free and open roots and makes it easier for our community to help overcome any resistance they may find in their organizations to FOSS.

If you’re in the same boat with respect to marketing, give Michelle a shout.

Twitter

Posted on by tarus

Okay, so I don’t always rush out to try the latest and greatest meme to sweep the intarwebs. I’m slow.

Apparently, not as slow as Twitter.

Gah.

I’m an old guy. I don’t twit. I still think e-mail is pretty “keen”.

But I decided to try it. I set up my account and downloaded Twitterific. Several guys in the office had accounts so I started to follow them. This is what I got for my trouble:

This sounds like a job for OpenNMS. Hey – Twitter dudes – drop me a “tweet” @Sortova and we’ll see if we can help with things like this:

since you can see I haven’t done 70 tweets in my life, much less an hour. I”m not Coté. (grin)

But hey, if you like this sort of thing, feel free to follow “OpenNMS“.

If you can.

George Carlin, RIP

Posted on by tarus

I started working on OpenNMS on September 10th, 2001. Needless to say, my second day on the job is easy to remember.

Since then I start my day by hitting my RSS feeds to see what is in the news. Today I awoke to find out that George Carlin had died.

The news services are starting to pick up the story.

I am a big fan of Carlin. Part of it is due to his brilliance, but another has to deal with my admiration for people who are willing to take a chance and do something different, even if it is met with scorn, and in Carlin’s case, lawsuits.

My first date with my wife was to see Carlin from the second row at Memorial Auditorium in Raleigh. I am such a romantic.

So may we all hum the Seven Words today, and I hope that George is watching it all on Heavenly CNN.

Customer Service Stories: Etymotic

Posted on by tarus

I started maintaining OpenNMS in May of 2002. By early 2003 the company was doing well enough that I decided I needed a little reward for all of the 80 hour weeks, so I bought a 12″ Powerbook the day it was announced, and I also ordered an iPod.

Back then I traveled as much as I do now, and having something like an iPod helped make the trips more pleasant. However, I wasn’t happy with the default earbuds when it came to dealing with the noise of the aircraft cabin. All the road warriors were buying the Bose noise-cancelling headsets so I decided to check them out.

I didn’t like them for two main reasons. First, they were large. When you schlep bags through airports for a living you want things as small and light as possible. Second, they required a power source. I am constantly being let down by my battery powered devices and I didn’t want to have to deal with another one.

It was then I learned about a company called Etymotic. They pioneered the idea of audiophile-quality in-ear headphones. Think of it as earplugs with amazing sound. They basically block out all sound waves except for that which comes through the bones of your skull, plus they are lightweight, small and require no external power. Most of the reviews I read had the Eymotics beating the Bose hands down.

Ben Reed had gotten a set of ER-4Ps and he let me try them. They were as amazing as I had been lead to believe. While they don’t block out all sound, as long as you have something playing through them you really can’t hear anything else.

So I bought a pair. At the time they were a little more expensive than the Bose (around US$350) but they saved my sanity on more than one occasion. Now when I sit down on a plane and notice the infant (or infants) in the row next to me, or the loud group of college students, or practically any other possible source of noise pollution, I smile, put in my 4Ps, and relax.

Now something like 300,000 air miles later, my 4Ps had seen better days. The insulation had rubbed off in a few areas, but the death knell came on my return from Europe when the right earphone went out.

Grrr.

So I called up Etymotic and found out I could get them repaired for about US$50. Off they went, and a week later I received some bad news. After the cables were replaced, they tested the 4Ps and found some slight distortion on the right earphone. Since they have to repair them in matched sets, they just decided to send me a whole new set of 4Ps.

Wow.

That’s what I call customer service, especially with a product this expensive (although they’ve come down in price about half since I bought mine originally). If you’re looking for some high quality headphones, please check these out.

The OpenNMS Contributor Agreement (OCA)

Posted on by tarus

As the OpenNMS Project has matured, it has become apparent that we need some sort of Contributor Agreement.

The most important reason to have an OCA is to get the contributor to validate that they actually own the code they are contributing. While I believe it is much more likely for a closed source company to abuse open source code than the other way around, we need to have some sort of paper trail to help prevent this.

The second reason is to assign copyright. It makes things much easier for the steward of a project to hold 100% of the copyright. However, we want to encourage contribution, but the assignation of copyright can be a sticky subject for many companies (usually less so for individuals). It is hoped that The OpenNMS Group has proven itself an acceptable maintainer of the Project and that contributors won’t have a problem in assigning the copyright to them.

We approached this as we usually do with community issues, with a healthy discussion on the OGP mailing list. We examined a number of traditional open source contributor agreements, but many people complained about the transfer of copyright. It was then that DJ brought up the Sun Contributor Agreement (SCA) [pdf].

The SCA is unique in that it introduces the concept of “shared copyright”. Basically, the contributor gives copyright to the project but maintains every other right, with the exception of being able to revoke the copyright given to the project (see the SCA FAQ for details). This seemed perfect.

However, I couldn’t figured out under what license the SCA was published. I sent a number of notes off to people I know at Sun, but never heard back. I wanted to borrow it, pretty much verbatim, but I needed to know if I could (and they say us open source folks don’t care about IP).

I noticed last week that the SCA had been updated to include a Creative Commons license. Yea!

So today we announce the OpenNMS Contributor Agreement. We will be asking past contributors to sign it as well as all new contributors, and within the month anyone with write access to subversion will need to have signed.

I believe this is best for the Project as well as maximizing the rights of those who did the work. I especially like Item 4, which means that all contributions will remain free and open.

I am eager for comments, so let me know what you think. I’d also be interesting in hearing from folks who have signed other such agreements and how the OCA compares.

"Sustainable, Open Source Software and All"

Posted on by tarus

In an earlier post I mentioned Lyle Estill’s book “Small is Possible“. It is the followup to “Biodiesel Power” and focuses on the efforts of a small community in North Carolina as it tries to “feed itself, fuel itself, heal itself and govern itself.”

People are probably tired of me going on and on about the community being the heart of open source. I know I sound like a broken record, but with the current bastardization of the term “open source” by commercial software companies I feel compelled to point out that you don’t get the benefits attributed to open source software unless you have an empowered community. I saw a press release from one such company the other day that was going to “unveil” its latest software release at Red Hat Summit. Unveil open source software? You wanna see the latest OpenNMS, check it out of subversion and build it. You don’t have to wait for someone to allow you to do it. There is nothing hidden, nothing to “unveil”.

The community has been key to OpenNMS. This year is our fourth Dev-Jam developers conference. We (the .com side of things) spend a lot of money on Dev-Jam, but we get every cent back in the form of a better OpenNMS. It is the one time of year where the core of the community gets together, face to face, to decide on the future direction of OpenNMS. It isn’t dictated by the commercial side of the business, and there are many more people in attendance than are employed by the OpenNMS Group. The OpenNMS Group is strictly a services company, while the software is owned by the community at large.

[Note: we’re still looking for Dev-Jam sponsors (grin)]

I am often surprised at how well this model translates to other new endeavors, such as Lyle’s biodiesel operation. It started out as an experiment in a blender, grew to form a coop, and then became a commercial enterprise. Now that he is producing over a million gallons a year, one would assume that he’d just want to build a bigger plant. No, he’d rather see others replicate what he is doing, and he and his team will be glad to help you do it. In much the same way that I sell expertise with OpenNMS, Piedmont Biofuels will sell you their expertise to get your biofuel operation underway. Lyle has created a “reference implementation” of a biodiesel plant, just as we are trying to create a reference implementation of a management framework that is completely open source.

Apparently I’m not the only one who sees the parallels. Michael Tiemann read Lyle’s book and had some cool comments on his blog. I’ve never met Michael but we seem to share the same philosophy when it comes to open source. Together, it *is* possible to build powerful and sustainable solutions, both within software and within our lives and communities.

[Note: I’m storing a bunch of “Small Is Possible” books for Lyle in a spare office so he gave me a box. All new support contracts and renewals will get a copy. While supplies last.]

Why People Need Support

Posted on by tarus

I like to think that the people who use our services get value for their money, but I sure many more ask the question “why do I even need support?”

At OpenNMS, we don’t sell software (all our software is free). I like to say we sell time. At the moment, anyone who has found out about OpenNMS, installed it and decided to use it obviously possesses well above average intelligence, impeccable taste and is most likely devilishly attractive. They are capable of figuring out issues without a support contract, either by experimentation, using the free resources such as the mailing lists, or both. But do they have the time?

Normally, most of the trouble tickets we get concern configuration, a few involve actual bugs with OpenNMS itself, and more than you would think are the result of vendors not honoring standards. We spend a lot of time figuring out issues with things like poorly written SNMP agents and even operating system problems.

And then there are the bad MIBs.

Recently I got an e-mail from a person who uses the Anevia Flamingo product. They wanted some help using mib2opennms to convert Flamingo SNMP traps into a format they could use.

Usually I have to politely decline helping people who contact me privately about OpenNMS issues. It wouldn’t be fair to our paying clients if I spent time helping people one-on-one for free, so I point them to free resources like the mailing lists. When I have time I try to help out there, as that gets archived publicly and might help others. The catch is that you may or may not get a timely answer to your question on the list, whereas you can always pester us about support tickets.

But this question involved mib2opennms. I’ve been using that tool for six years and my mib2opennms-fu is strong, so I took the Anevia MIB I was sent, cranked it through the tool and sent back the output.

I received a reply that it wasn’t working and the user was still getting unformatted trap errors like:

Received unformatted enterprise event (enterprise:.
1.3.6.1.4.1.20967.1.12.1.30 generic:6 specific:2). 3 args: .
1.3.6.1.4.1.20967.1.12.1.30="" .1.3.6.1.4.1.20967.1.12.1.30.1="1" .
1.3.6.1.4.1.20967.1.12.1.30.2="10.180.1.232"

I went into the file I had created and noticed that the enterprise id was missing the last “.30”, which is why it wasn’t matching, so it was off to look at the MIB.

It started off normally enough, with some object definitions:

anevia OBJECT IDENTIFIER ::= { enterprises 20967 }
anevia1 OBJECT IDENTIFIER ::= { anevia 1 }
tsnmp OBJECT IDENTIFIER ::= { anevia1 1 }
manager OBJECT IDENTIFIER ::= { anevia1 12 }
aneviaManager1 OBJECT IDENTIFIER ::= { manager 1 }
aneviaManagerTraps1 OBJECT IDENTIFIER ::= { aneviaManager1 30 }

and then later in the MIB came the trap:

inputDownTrap TRAP-TYPE
  ENTERPRISE aneviaManager1
  VARIABLES { streamerInputIndex, streamerAddress }
  DESCRIPTION
    "This trap is sent when an input on a streamer becomes unavailable,
     and can no longer provide any useful data, the provided index is the
     index of this input."
  ::= 2

At least the mystery of the missing “.30” was solved. The “ENTERPRISE” value for this trap should be “aneviaManagerTraps1” instead of “aneviaManager1”. Easy enough to fix. But then I noticed that instead of the two varbinds listed in the MIB, the agent was sending three (see above) where the first one was blank (as well as being just the enterprise OID).

Grrrr.

The second varbind value of “1” could easily be the streamerInputIndex and “10.180.1.232” could be the streamerAddress but these won’t be correctly reflected in the events file since they’re off by one due to the mystery blank initial varbind.

This is the case of a poorly written MIB and a poorly implemented agent, and there is little we can do about it but work around it in configuration. I asked the user to make sure we had the latest Anevia MIB and was told we did. I wrote Anevia support but since I don’t have a relationship with them I never got a reply.

This happens way more than you might imagine, and we’ve gained a lot of experience in diagnosing and either correcting or working around such issues. Because we’ve seen stuff like this before, we can do this quickly, which is why I like to say I sell time. It only takes a few issues like this to have a support subscription pay for itself.

[Note: This post isn’t meant to be a pitch for services but a rant about the time I wasted playing with the Anevia MIB, but if it helps sell a support contract, that’s cool too (grin)]

It's Like Christmas

Posted on by tarus

When I talk to investors I’m often grilled on things like download numbers, Google hits, things like that.

I really don’t care. I try hard to focus on our customers, our community and if others find OpenNMS useful, so much the better.

My guess is that other projects associated with a commercial enterprise have investors that are a little more demanding, which is why I have to give a name and e-mail to access either software, documentation or both. I don’t mean to imply that there is something wrong with that, it’s just that it is different experience than I get with projects like OpenNMS.

One of the bonuses about being rather ignorant about who is accessing the software is that I get to be pleasantly surprised when a company I know turns out to be using it. It was amazing when a Fortune 5 company called us a couple of years ago about the project, and I’ve written about companies like Checkfree who I find out use OpenNMS.

So it was cool when it was pointed out to me that Orbitz not only uses OpenNMS, they decided to present it at JavaOne. This is a show we’ve never been to, so perhaps we should.

To those at Orbitz, we salute your excellent taste.

OpenNMS on OpenJDK and Fedora 9

Posted on by tarus

Okay, I started mucking around with OpenJDK on Fedora 9 today, and it turns out it is pretty easy to build OpenNMS against OpenJDK and get it to work. There are some tricks, so I thought I’d document my steps.

On a base Fedora, install the following packages:

yum install subversion gcc postgresql-server postgresql-devel rpm-build
yum install autoconf automake libtool
yum install java-1.6.0-openjdk java-1.6.0-openjdk-devel

Then create a place for your subversion files:

mkdir svn
cd svn

and check out OpenNMS. This is the latest testing branch:

svn co https://opennms.svn.sourceforge.net/svnroot/opennms/opennms/branches/1.6-testing testing
cd testing

Now, one of the big issues is that in a lot of places OpenNMS relies on the “jdk” package, which is what Sun’s RPM called the JDK. Since that package doesn’t exist (yet) in Fedora, edit the .spec file:

vi tools/packages/opennms/opennms.spec

and change:

BuildRequires:          jdk

to

BuildRequires:          java-1.6.0-openjdk

Then build:

export JAVA_HOME=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0
./makerpm.sh

After a long time (maven has to check out a lot of dependencies) you should have a bunch of RPMs in

~/svn/testing/target/rpm/RPMS/noarch

Now in order to run OpenNMS, both jicmp and iplike are required. So in your svn directory:

svn co https://opennms.svn.sourceforge.net/svnroot/opennms/jicmp/trunk jicmp
svn co https://opennms.svn.sourceforge.net/svnroot/opennms/iplike/trunk iplike

Build jicmp:

cd jicmp
m4/autogen.sh
./configure

Now at this point I just did:

make
make install

but this will not create RPMs. To get RPMs to work you’d run

make dist
make rpm

But this will crap out due to there being no “jdk” package. While I hate using “–nodeps” for this experiment I just decided to install jicmp and iplike outside of package management (I could modify the .spec file as above but I’m lazy).

Build iplike:

cd ../iplike/
m4/autogen.sh
./configure
make
make install

Then install OpenNMS:

rpm -Uvh --nodeps opennms-1.5.92-0.9251.snapshot.noarch.rpm
                  opennms-core-1.5.92-0.9251.snapshot.noarch.rpm
                  opennms-webapp-jetty-1.5.92-0.9251.snapshot.noarch.rpm

Now, create the postgresql database:

service postgresql initdb

And then open up permissions

vi /var/lib/pgsql/data/pg_hba.conf

and set

# TYPE  DATABASE    USER        CIDR-ADDRESS          METHOD

# "local" is for Unix domain socket connections only
local   all         all                               trust
# IPv4 local connections:
host    all         all         127.0.0.1/32          trust
# IPv6 local connections:
host    all         all         ::1/128               trust

and start PostgreSQL:

service postgresql start

Then, set the Java version:

/opt/opennms/bin/runjava -s

and run the installer

/opt/opennms/bin/install -dis

Now you should be ready to run OpenNMS, but there are a couple of changes still to be made.

For some reason I got an error in Statsd.

An error occurred while attempting to start the "OpenNMS:Name=Statsd" service
(class org.opennms.netmgt.statsd.jmx.Statsd).  Shutting down and exiting.
javax.management.RuntimeMBeanException: org.springframework.beans.PropertyBatchUpdateException;
nested PropertyAccessExceptions (1) are:
PropertyAccessException 1: org.springframework.beans.TypeMismatchException:
Failed to convert property value of type [java.lang.String] to
required type [org.opennms.netmgt.statsd.RelativeTime] for property 'relativeTime';
nested exception is java.lang.IllegalArgumentException: Unsupported value: YESTERDAY at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrow(DefaultMBeanServerInterceptor.java:877)

This could be an OpenJDK issue or just a testing issue, so I disabled that daemon for now:

vi /opt/opennms/etc/service-configuration.xml

and comment out:

<!--
        <service>
                <name>OpenNMS:Name=Statsd</name>
                <class-name>org.opennms.netmgt.statsd.jmx.Statsd</class-name>
                <invoke at="start" pass="0" method="init"/>
                <invoke at="start" pass="1" method="start"/>
                <invoke at="status" pass="0" method="status"/>
                <invoke at="stop" pass="0" method="stop"/>
        </service>
-->

Now OpenNMS will start, but the webapp will be disabled because of a JVM check. To override this:

vi /opt/opennms/etc/opennms.properties

and set

opennms.skipjvmcheck=true

Then just start OpenNMS:

service opennms start

Go to the webapp and set up a discovery range and you’re good to go.

One weird thing is that I don’t think OpenJDK uses the same fonts as Sun’s JDK, so the graphs look a little funky:

Very steampunk.