Headset Boy

I hate phone calls. I think it is because they are synchronous. With Twitter, instant messaging, IRC, RSS feeds, e-mail, etc. it is real easy for me to only pay attention to them when I have the cycles. A phone call, on the other hand, demands my immediate attention.

But, man, I have done nothing but talk on the phone for the last two days. Support calls, demos and sales calls – my ears actually hurt from wearing my headset. I’ve talked my way across the US: from Boston to Chicago, Wisconsin, Texas, to Oregon, and across the world, including Italy, Chile and Australia.

I don’t know if it is just OpenNMS or if things are picking up all over, but it’s a seriously crazy time.

If I wasn’t so tired I might actually be enjoying myself. (grin)

So let me apologize in advance if I’m a little slow in replying to e-mails, LinkedIn requests, or notes on Facebook. It’s nothing personal, and on the bright side I might be coming, soon, to a town (or country) near you.

Order of the Blue Polo Profile: Tyler Mills

Many thanks to those people who have already filled out their testimonials for the Order of the Blue Polo. I’ll be placing my first order for shirts tomorrow, so I should be able to ship them out in about two weeks.

The next person I wish to highlight in my series on OBP members is Taylor Mills.

The company he works for, Pavlov Media, supplies communication services to apartment complexes, thus scalability is a primary concern. He is using OpenNMS to monitor over 6500 devices, and by using it he does not “need to worry about the scalability of the platform”.

This is one thing we strive for with OpenNMS: scalability. Coupled with our determination to provide the software without “per device” software licenses, I think it would be hard to find a better solution. Think about it: even at US$10 a device they would be paying US$65,000 a year to monitor their network, and considering that many proprietary and neo-proprietary solutions run upwards of US$100 a device, the savings are considerable.

Outside of scalability, OpenNMS was designed to be an open platform. Tyler writes, “we have been able to integrate it with our ticketing system, export graphs to web pages to make available to certain parties, and mold it to our needs”. Exactly what OpenNMS was designed to do.

Please check out his testimonial where he also gives out the specs for his system hardware. Fast disks are always a plus for data collection, and having a separate database server can provide serious benefits in performance on a network of this size.

Neo-Hippies?

I live in Chatham County, North Carolina, USA. This is surprising to me, since I grew up less than an hour away in a town called Asheboro, and when I turned 18 I wanted to be as far away as I could. I moved to Los Angeles. I didn’t expect to end up this close to where I grew up.

Chatham is an interesting place to live, and I often describe it as one-third redneck, one-third hippie, and one-third Latino. I can find a lot to love in all three aspects of the culture.

Working on open source software in Chatham appeals to the hippie aspect of the culture, but I really don’t view myself as a hippie. However, I do embrace such hippie ideals as personal freedom, concern for the environment, community and cooperation.

These ideas are not incompatible with capitalism, although many think they are, especially the environmental aspect. The problem is that often there is no direct way to evaluate the costs or value of things like “clean air” and “community” when trying to determine the best way to maximize profit, so these things can be exploited.

I’m trying to come up with a term to describe people who are attracted to hippie ideals but can combine them with the capitalist ideal of maximizing return on investment. It’s sort of like that scene in A Beautiful Mind where John Nash demonstrates that if he and his friends cooperate the overall benefit to the group is greater than if they act alone.

What should we call ourselves? Neo-Hippies? Pragmatic Idealists? The practitioners of conservatism popularized by the American President Reagan and Presidents Bush are called Neo-Cons – I’m looking for something like that.

What does this have to do with open source software? True open source software provides just such a cooperative benefit, since both the creators and the consumers of software have an ownership stake in the project.

While not always true of consumer software, enterprise software has tall barriers to entry and exit. If you have a large company running on Windows, it is not easy to switch to, say, OS X. Enterprise software also tends to be expensive. I once worked with a large bank, and the network manager spent nearly US$1 million on an application called Concord Network Health. While not a bad app, it was expensive, and when I was hired to install it I realized that the manager had made a bad choice. The information he was interested in was not possible to easily acquire with the software.

But there was no way they were going to scrap the project after spending so much money on it. In fact, they kept spending money on it in yearly maintenance costs in order not to “lose” their investment due to lack of upgrades and bug fixes. Even after the manager was fired a couple of years later, he still defends the decision.

In the meantime, Concord made gobs of money and made their investors happy. While not strictly a zero-sum game, in this case (and in many, many others I can name) the software vendor profited much more than the end user.

While it is rare for a vendor to seek to actively screw their customers, it is the profit aspect of proprietary software that really appeals to the open core folks. They want the benefits of open source and community, but they also want the advantages of vendor lock-in and software license revenue. My claim is that having that business model within the framework of maximizing profits means that at some point the community will suffer, and we’re back to the zero-sum game.

True open source software gets rid of this dichotomy. By lowering the barriers to entry and exit, it changes the dynamics of the vendor/customer relationship. The benefits to the vendor increase proportionally to the benefits to the community (and thus the customers), and not tangentially.

Sure, open source is not without cost, but without huge up-front license fees it is much easier to choose a different solution. Without required maintenance fees it costs a lot less to own. And being open source, it is often easier and less expensive to add the functionality you need than to seek out a different solution.

But many people seem to think that adopting a pure open source business strategy goes against capitalism. Sure, I’ll grant that it is a lot harder to make money fast, but in the long run I believe the payoff will be greater and more sustainable.

I didn’t start this post with the idea of ending up rehashing my open source v. open core beliefs (I plan to prove the validity of the pure open source business model by making OpenNMS incredibly successful, not by writing blog posts) but instead to focus on a businessman with a social agenda vs. a pure profit one.

Specifically, Lyle Estill.

Lyle is a local legend. At times he is pure hippie. Catch him on a work day and you’ll find him in dirty overalls. At other times he is pure capitalist. Catch him at a business meeting and he’s dressed in “dry clean only” finery rivaling my own collection of Armani.

He’s a salesman, a metal sculptor, a chess player, a writer and founding member of the grassroots biodiesel movement. I have rarely met a tougher negotiator or a person more focused on the bottom line, yet in almost everything he does there is a social angle to it.

He was recently quoted in USA Today (an OpenNMS client, I might add) on the resurgence of a local currency we have called The Plenty. Unfortuntely, the quote was wrong. He said we are a “whacked-out little town,” meaning a little odd but in a good way, which got translated as “wiped-out little town,” which doesn’t sound good at all. Now, when the Telegraph in the UK runs the article, the headline is “Struggling US towns print their own currency“.

(sigh)

We’re not struggling. In fact in the face of this recession I’d say we’re doing pretty well. Part of that is due to our emphasis on community.

Communities in general are becoming more prevalent. With the widespread availability of broadband internet the geographical barriers are gone and sites likes Facebook are seeing record traffic. In software especially, the role of the user community is getting a lot of attention. I don’t think that anyone will disagree that at least for enterprise software, in the near future there will be some component that is open source in order to foster just such a community.

But a traditional capitalist might see community as something to be manipulated, if not exploited, much like clean water or air. That could result in some short term gains but they will be at the expense of long term gains as the community learns that they are being manipulated.

With the rising importance of communities, there will be those of us that see that vibrant, independent and empowered communities go hand in hand with maximizing profit in the long term. That money and ideals don’t have to be mutually exclusive. But what should we call ourselves? Neo-hippies doesn’t seem quite right.

Oh, and in case anyone is counting, Lyle is rather financially well off. He demonstrates that wealth and community is not an either/or situation. I’m just trying to follow in his footsteps.

OpenNMS Could Aggravate Bug on Cisco Switches Running 12.2(44)se1

Tom Powers posted on the install list today information he discovered about HTTP polling on Cisco 3550 switches.

On Cisco 3550 switches, running 12.2(44)SE1, the standard OpenNMS HTTP service polling will kick off a memory leak in these switches.

This is resolved with se2 and higher (se6 is out now).

On a switch with 30+ mb ram…it took about 3 days to kick off the memory leak to where HTTP, HTTPS, Telnet and SSH failed. The switch still runs…but the memory pool shows full (steady decrease from point of monitoring)

This article talks about the http core services memory leak fixed first in SE2.

We found this in an installation with 56 switches and every one of these 3550s (eight of them) with 12.2(44) SE1 leaked out. Older and newer switches all were fine.

Just a heads up. Note that this is not an OpenNMS bug. The fix is to upgrade the Cisco software, and a workaround would be to unmanage HTTP on these devices. To see if you have any Cisco 3550 switches in your network, if you have SNMP enabled on all your devices you should be able to find them in the database:

psql -U opennms opennms

SELECT nodeid, nodelabel, nodesysdescription
     FROM node
     WHERE nodesysdescription LIKE '%3550%';

q

I don’t have any of these switches on my network, so I can’t verify it but my guess is that the sysDescription will include the software version as well.

UPDATE: Tom wrote more on this:

After talking with Cisco this morning, and explaining our situation and findings … this issue is not confined just the 3550 switches…it is all Cisco catalyst switches with the 12.2(44)se1 revision IOS that are affected. We were just lucky enough to have all 3550s in the environment we caught this in.

Turns out the rest of the series switches, 2900’s and such are affected as well since they all use the same IOS.

Cisco also confirmed that polling of the http interface like OpenNMS does would aggravate this leak.

Here’s a picture of what happens.

The Masters

I am not a golf person. I’ve never played a round. Even though we named the OpenNMS Order of the Green Polo after the Masters green jacket, I had not ever watched more than just a small amount of that tournament.

Well, this year the Masters happened to land on Easter, so I drove to my parents house, stuffed myself with the annual Easter feast and planted myself on the couch. The tournament happened to be on and I ended up watching.

The whole thing.

I found it quite entertaining. There’s lot of numbers involved, and I like numbers. Plus, this year the last round was pretty exciting (and ended with a two-hole sudden death playoff). Another thing I found amazing is that while I know a lot of people who play golf, I don’t know, personally, anyone who has ever, in their life, shot a par round of golf. Yet these players were consistently shooting under par.

The only downside to the afternoon was the guy who keeps shouting “It’s in the hole!” after every shot, especially those by Tiger Woods. Who is this guy? More importantly, is there something we, as a group, can do to him physically to convince him to stop? I’m sure we can think of something.

It was probably funny the first 300 times, but seriously dude.

The Sharecropper Model for Commercial Open Source

As I was running through my RSS feeds this morning, Roberto Galoppini pointed me to a post by James Dixon (the CTO of Pentaho) on the Beekeeper Model for Commercial Open Source (PDF).

It references Geoffrey Moore’s Crossing the Chasm which I’ve used for years to drive our OpenNMS business, so it caught my attention. I haven’t blogged much about open core for awhile, but I thought this article deserved a closer look.

Since I’ve been labeled a hippy open-source purist, let me state again my contention that companies that label themselves “open source”, including “commercial open source,” need to produce products where 100% of the software code is available under the OSI Open Source Definition via an OSI approved license. If the business model involves generating the majority of revenue by selling “enterprise” software via traditional closed licenses, that software doesn’t meet the definition of open source and should not be called such. I’m perfectly happy with “Commercial Open Core” but let’s not confuse a neo-proprietary software model with open source.

My bullshit meter jumped slightly when I read his list of open core companies:

Companies using the single-vendor or open-core commercial open source models include MySQL, Ingres, Compiere, Open Bravo, Liferay, SugarCRM, Mule, Alfresco, JBoss, Digium, and Zimbra.

The meter only jumped a little because he used the term “or” instead of “and”, but on reading the rest of the article it seems like they are equivalent, so I need to point out that not all of these companies are open core.

Everyone who calls their company “open source” likes to be compared to JBoss since they were purchased for so much money, but I want to stress that JBoss isn’t open core. From their website

you can see that there is no code differences between JBoss Community and JBoss Enterprise. Sure, you get patches, and hot fixes and gobs of benefits by being an Enterprise customer, but special code is not one of them.

Digium is very similar. We love the gang over Digium as they strive to make sure their code is 100% open, and we use their software in-house as well.

It is also my understanding that MySQL, another successful open source company, made 100% of its code available until a year or so before their acquisition by Sun. In that last year I believe that “enterprise” customers got to see code six months or so before it was released into the wild, which while unfortunate, still resulted in all of the code being open. Prior to that you could get MySQL under an open license or you could purchase a proprietary license and imbed it in your commercial product. As long as the open and proprietary version are basically the same code I think this is a legitimate way for an open source company to generate revenue from software licenses.

Someone please correct me if I have my MySQL facts wrong.

In two if not all three cases they pass the CentOS test, and thus I don’t consider these companies to be open core.

Mr. Dixon did score points with his statement on open source software vs. free software:

Open source is not free. In 1998 the term ‘open source’ was coined to replace the term ‘free software’ because many people assumed ‘free’ to mean ‘zero cost’ whereas it was always intended to mean ‘freedom’.

That’s it in a nutshell. Open source requires a different kind of cost, but you don’t get a free (gratis) solution. However, people have taken it a step further to mean that commercial software can be called open source since commercial software is not free, too. “Syllogisms are only partially convertable. While Alma Cogen is dead … only some of the class of dead people are Alma Cogen”.

His entire section on The Principles of Open Source is pretty spot-on. I found myself warming to Mr. Dixon.

Then we get to the meat of his document, his four models of software development:

  • The Wild Hive Model for Open Source Projects
  • The Maple Syrup Farm Model for Proprietary Software Companies
  • The Beekeeper Model for Single-Vendor Commercial Open Source
  • The Honey Gatherer Model for Services/Support commercial Open Source

This is interesting stuff, so please read it for yourself. It is so elegant and comes with nice little drawings that it took me awhile to understand why my bullshit meter was pegged.

Then it dawned on me. Instead of the Beekeeper model it should be called the Sharecropper model. The single vendor controls everything while benefitting from the community, while the community only exists to serve the single vendor. When he writes “In the Beekeeper Model the bee farm provides land, hives, and flowers etc.” it is just like a plantation owner in the old South owning all the land and means of production. The bees are not in control of their destiny, much like some sharecroppers were told when, where and what to plant.

This is at the heart of my problem with open core software. In a vibrant open source community it is the community that controls the product, not the vendor. Mr. Dixon states:

This explains the common practice of the Beekeeper companies to offer some kind of ‘Enterprise Edition’ that includes features not available to the community. These are high-end features that only larger organizations find of value.

Who decides what features are of what value? As I mentioned in my Hyperic post awhile ago their community is screaming for a feature that only exists in the “enterprise” version, yet their needs go unmet. It is obvious that Joe User needs the feature but because it drives software revenue it will never be open. According to Dixon is necessary because

It is clear that the single-vendor model is more costly to set up and operate than the services/support model. It is logical that companies using the Beekeeper Model need to generate more revenue to recoup these costs than a company using the Honey Gatherer Model

No, it is logical that if the only way you can meet your revenue needs is by selling commercial software then your open source business model is broken. Don’t say “pure” open source doesn’t work if the problem is you can’t run your business properly.

There are several other things that just slapped me upside the head:

The community gains open source software they can use for their own purposes. This software has more functionality and more resources than a ‘pure’ open source project could provide. In this way the community profits directly from the company and its customers.

If he means that an open source project with a commercial backer has more functionality, then I’d say “well, duh, of course”. But there is no reason that making money on an open source project is in conflict with being “pure”.

The customers gain higher quality software at a better price. The customers profit from the open source community’s ability to produce high quality software.

In the first statement he implies that in order to produce high quality software you have to have a commercial entity producing it, but then here he states it is the community that produces the high quality software. Which is it?

As far as price, nothing could be better than free, I agree. But if he is talking about customers having to buy enterprise versions of “open source” software the math gets a little hazy.

For example, in our space OpenNMS provides unlimited Standard Support for US$14,995 a year. Zenoss, an open core company, charges US$150/device for a minimum of 250 devices for its “enterprise” software, or US$37,500 a year, over twice as much as we do and limited to 250 devices.

However, if you take our average commercial install of 2000 devices, the Zenoss price would be US$300,000 per year. That is insane – you might as well buy OpenView or Tivoli. Over 5 years the cost will be must less than the US$1.5 million Zenoss will charge. Of course, no one should be paying list price for their software, but it is so wrong to call it open source even if you can haggle it down.

A prospective customer should not have to learn about open source in order to become a customer. The sales and marketing materials should neither hide their open source model nor require understanding of it by the market.

Why shouldn’t a prospective customer “not have to learn about open source”? Aren’t there some serious advantages to open source? That’s like saying Toyota shouldn’t educate its clients on the hybrid synergy drive on its cars. True, the customer shouldn’t have to be able to build one or understand exactly how it works, but I would think the basics of why you would want one should be explained early and often.

Educating the market will be the downfall of the open core business model once people realize that there may be “pure” open source offerings that do it better. After listing the benefits of open source so well in The Principles of Open Source why does Dixon feel it is not important to the buying decision?

Customers are not bees, bees are not customers, and you cannot convert one to another.

With OpenNMS a good portion of our clients are also actively involved in the community. In fact we encourage this. Many, if not all of our customers came to us from the community. With our commercial support offerings, however, there isn’t a requirement of community involvement. What I get out of this statement is that the single vendor company sees the community (the bees) and their customers as separate things, and will focus on the needs of the customer, in order to generate revenue, versus the needs of the community.

Some people assume that all commercial open source models are flawed because the company does not have direct control over the direction of, and development of, the software … The services/support model does suffer from this. The company might pay for full-time developers to work on some of the open source projects that it utilizes but it does not have the same level of influence that the single-vendor model provides.

This has not been my experience. The statement that single-vendor (open core) models have more influence goes back to my plantation owner analogy – the sharecroppers/bees/community are told what to do since the plantation owner controls everything. At the OpenNMS project, influence is based on merit. We have a number of full time developers who both create code and help integrate the contributions of the people outside of our company. Since we get to work on it full time we produce more code which earns us influence. However, every major project decision is governed by the Order of the Green Polo, which got there also on merit (and not always by writing code).

It seems like in the Beekeeper model influence is bought. You, the community, do what I say since I pay for everything. And, by the way, I have to sell software in order to pay for it. It seems to be the antithesis of the open source communities I’ve known.

In summary, I do think the need for a “whole product” is valid, but I don’t think it is necessary to sell commercial software licenses in order to deliver it. The “Honey Gatherer” can deliver the whole product without resorting to commercial software licenses.

Part of my antagonism toward the model comes from the fact that Pentaho sells an “enterprise edition” and thus is an open core/neo-proprietary company, and since those companies have co-opted the term “open source” I am naturally distrustful. It’s like saying “we love the bees, we need the bees, but no royal jelly for you” and I think that is wrong within an open source environment.

But this model is slick and well written and it did make me think, which is always welcome.

Buy vs. Build

I was talking to a potential client the other day and the question of “buy vs. build” came up.

I have been saying for several years now that software (all software) will eventually become either open source or a commodity product. The days of expensive “enterprise” software will be over, perhaps not tomorrow, but within the next ten years.

But the “buy vs. build” decision won’t go away. Organizations will still have to decide if they want to try and purchase the software to solve a particular problem (the commodity side) or do they want to build their own solution (using open source).

In the past, “build” meant to build the system from scratch and “buy” included taking off-the-shelf software and configuring it to work in the environment. With the increased popularity and power of open source software, it has become a whole lot easier to build a solution in house (since one doesn’t have to reinvent the wheel), and with the availability of broadband networks, the buy solution now includes outsourcing the whole thing to another vendor.

So I’m talking with this client and it was really hard to determine if OpenNMS would be a good fit. Since we are a services company, going through our sales process is a bit different than with buying software. If you buy a piece of software and it doesn’t meet your needs, the vendor still profits, as does the salesperson, so they have the incentive to sell as many licenses as they can to anyone they can.

With our business model, if we try to put OpenNMS into a place where it doesn’t belong, we just end up doing tons of support. Compared to a situation where OpenNMS is a good fit, these clients would end up costing us money. While we like money and money is good, it is not worth forcing OpenNMS to fit where it doesn’t.

There are several factors that go in to the buy vs. build decision for a management solution, and often it is not possible to determine them on a phone call. Size plays a big role, as a small organization, especially one without a technical focus, is much better off either outsourcing the whole thing or buying a basic solution like WhatsUp Gold. Larger institutions should opt for something like Solarwind’s Orion or if they are heavily Windows focused, Nimsoft’s Nimbus.

But for many larger organizations and especially those at the carrier level, the business network is so unique that no off-the-shelf product can be made to scale or to fit their business processes. For them there really isn’t a “buy” solution, although in the past the enterprise software vendors have tried to sell one.

But size isn’t the only factor, nor is it even the main one. The true measure of a buy vs. build decision comes down to your people.

I’m not sure when “people” became a bad word, but as far back as I can remember companies have been trying to reduce the number of people in their IT departments. Perhaps it was because they had a roomful of “screen monkeys“, or perhaps it was because, especially during the bubble, anyone in the industry was very expensive. But in my experience having a few good IT folks coupled with the right tools is the best way to manage your information infrastructure.

But how to do you ask that on a conference call? How do you ask “Hey, do you have anyone on your staff who really enjoys solving problems with technology and they have the skills to take something like OpenNMS and run with it?”

If you read our Order of the Blue Polo testimonials (keep ’em coming) you’ll see some that praise the “ease of use” of OpenNMS. While I find OpenNMS extremely easy to use, our biggest criticism is the learning curve. Yet there are people talking about how easy it was to get running. The only conclusion I can come to is that OpenNMS users are just smarter, more hard working, witty, and physically attractive than your average IT person (grin).

Those organizations that choose “build” and have the staff to pull it off will be the big winners. I saw a great post yesterday on The True Cost of Migrating to Open Source. Mark Taylor presents a solid argument that the cost to migrate to a free and open solution (as opposed to a shareware, open core, neo-proprietary, or “commercial” open source solution) is the same as migrating to another proprietary product, yet the long term cost savings are great. While I’m sure this will get him labeled as some hippy purist, it is hard to argue with his conclusions.

I just wish the buy vs. build conclusion was as easy to reach.

Order of the Blue Polo Profile: Ho Trong Dat

When we started the Order of the Blue Polo, I wasn’t sure what to expect. It’s been interesting, since most of the OBP members so far are people I either didn’t know or haven’t heard from in a long time, and some people I figured would be the first to jump on it haven’t (probably due to the fact we require a company name).

Back in 2002, OpenNMS was pretty much just me in my attic (well, the room over the garage). The economy was still depressed post-9/11 and sometimes I wondered if it was all worth it (luckily I don’t give up too easily).

However, something happened in September of that year that gave me my first glance into how powerful an open source community could become, and it strengthened my resolve to make OpenNMS the de facto management platform used by everyone.

It was a letter from Vietnam.

Hi MAN.
I am calling you MAN because I consider you as a great man in NMS field. You and your band have made an wonderful product as I’ve ever seen and used. I am using OpenNMS 1.0 ( from 0.8.x) at my sites for approximately a year and it keep running well. In future, we will use NMS to monitor almost our VNPT’s network ( VNPT – Vietnam Post and Telecomunication ).

Here our information:
CDIT – Center for Development of Information Technology-VNPT
http://cdit.com.vn/
VietNam

I hope some day I will have a chance to contriubte my little knowledge for OpenNMS’s development.
Best regard.

It was from a man named Ho Trong Dat and it seriously made my day, if not my month. Here I was, working by myself in a little town in rural North Carolina, getting a great letter from a guy in freakin’ Vietnam (and while some readers will probably chuckle at the English, Dat’s English is a hell of a lot better than my Vietnamese).

Note: In most Asian cultures the family name comes before the given name, and I make sure I am aware of this when I travel. However, especially in Japan, some business cards will compensate by reversing them on the “English” side of the card. Note that I said “some” and since I don’t read Japanese I often can’t figure out if it has been done or not. It’s an example where accommodations made for Westerners cause more problems than they solve. So let me apologize in advance if I ever get your names backward.

Another Note: Vietnam has become a popular tourist destination. Think about it – in 40 years the in place to be might be Bagdad.

I was both surprised and extremely happy to see that Dat was to become one of the founding members of the OBP.

Dear OpenNMS users

My name is Dat and I am working for CDIT, a subsidiary company belong to PTIT (Post & Telecommunication Institute of Technology). We are R&D centre. We know OpenNMS when we researched about open source software.

We have been using OpenNMS in CDIT since 2002. From that time, OpenNMS did not release the first version, still 0.9, I remember. We had to install and reinstall OpenNMS a lot of time to get familiar with it. After six month, we finally and totally control OpenNMS in administrating our network. We deployed an WAN for our mother company (VNPT) with over 50 network nodes (router, firewall), a lot of servers (30). Using OpenNMS, we can monitor the status of links between subsidiaries, the performance of the link, of the server. We also monitor a lot of services such as : email, web of collaboration – our internal website, file servers, We think OpenNMS is a very convenient, flexible and highly configurable network management software. We feel happy when using OpenNMS.

Thanks you guys who have been developing such a nice and beautiful software.

So here is a guy from halfway around the world who used OpenNMS back when it first came out, and nearly seven years later he’s still using it (plus Dat’s English is much improved).

It’s letters like these that make me feel happy making OpenNMS.