Open Core Returns from the Dead (sigh)

The last 18 months of my life have been delightfully free of “open core” companies. These were companies who pretended to be “open source”, at least in their marketing materials, yet their business model was based on selling “enterprise extensions” which consisted of proprietary software that actually had the features you wanted. Basically, the open source piece was a loss leader to get you to buy the commercial edition, and as Brian Prentice pointed out so eloquently there was no real difference between “open core” and traditional closed source software. We like to call these businesses “fauxpen source“.

Customers realized this as well, which lead most open core companies to switch their tactics. While many still maintain an open source project, they have removed the term “open source” from their websites and most of their marketing (often replacing it with “open architecture”). I’m happy with this, as it allows true open source companies like OpenNMS and Nextcloud to differentiate ourselves while allowing these other companies to still produce open source software without misleading the market.

But lately I’ve been introduced to two new licenses that offer access to the source code without meeting the ten requirements of the Open Source Definition. These licenses further muddy the waters due to giving access to the code without including the freedoms of truly open software.

The first case was from Monty Widenius, who announced a proprietary Business Source License (BSL) for some of the MariaDB products. Monty was the guy who earned €16.6 million by selling MySQL to Sun and then got upset when Sun got bought by Oracle. Apparently, he seems to be unhappy that he isn’t earning enough money from his fork of MySQL products so he wants to create commercial software but not call it that.

The BSL, or as I call it, the “Rape of Large Companies License” allows the developer to offer the code up for use for free unless you cross some sort of arbitrary threshold, also set by the developer. In three years that code will revert to an OSI approved license, in this case the GPLv2, and if you are above the usage threshold then you don’t have to pay anymore.

I’m not sure what his goals are here, outside of running a commercial software business while paying lip service to open source software. Perhaps he hopes to get people to contribute to BSL licensed projects as long as their use case is small enough not to cross the “pay me” threshold, but more likely he just wants to ride on the coattails of the success of open source software without committing to it.

I learned of another such license called the Fair Source License (FSL) from a post by Ben Boyter who writes the Searchcode Server. Ben, at least, is a lot more up front about his reasons for adding a “GPL Timebomb” to his code. Initially, the code is published under the FSL but with a switch to the GPLv3 in three years. He isn’t expecting contributions and instead has offered up the code simply so it can be audited for back doors. As this is one of the more powerful features of open source software I applaud him for doing it, but I really wish he hadn’t used the term “bomb”. I have to deal with terms like “GPL poisoning” enough in my business that negative words like that just tend to scare people. He should have called it “Happy Fun Lucky GPL Gift Giving Time!”

Look, I’m all for anything that gets more code out there under an OSI-approved license but, c’mon, three years is a lifetime in this industry. Enterprise customers, who would be most affected by this license, will still have to approach the buying decision as if a BSL or FSL licensed application were commercial software. Even with the three year revenue window, it is unclear what happens if, say, a huge security bug is discovered three years out. Does the code to fix that bug restart the clock?

The whole process is confusing and doesn’t help the cause of open source software. I think open source is awesome and extremely powerful, and when I see things like this I’m almost insulted, as if the developer is saying “when I’m done you are welcome to my leftovers”. Instead of announcing a future switch to an open source license years in advance, they should just open it when they are ready, like id Software does with the Doom engine.

I’m giving a talk at All Things Open about running a truly open source business, the core point of which is that you can’t have an open source business with a business model based around selling software. No matter how you dress it up by either calling it “open core” or “business source” it is still proprietary software.

Nextcloud, Never Stop Nexting!

It’s been awhile since I’ve posted a long, navel-gazing rant about the business of open source software. I’ve been trying to focus more on our business than spending time talking about it, but yesterday an announcement was made that brought all of it back to the fore.

TL;DR; Yesterday the Nextcloud project was announced as a fork of the popular ownCloud project. It was founded by many of the core developers of ownCloud. On the same day, the US corporation behind ownCloud shut it doors, citing Nextcloud as the reason. Is this a good thing? Only time will tell, but it represents the (still) ongoing friction between open source software and traditional software business models.

I was looking over my Google+ stream yesterday when I saw a post by Bryan Lunduke announcing a special “secret” broadcast coming at 1pm (10am Pacific). As I am a Lundookie, I made a point to watch it. I missed the start of it but when I joined it turned out to be an interview with the technical team behind a new project called Nextcloud, which was for the most part the same team behind ownCloud.

Nextcloud is a fork, and in the open source world a “fork” is the nuclear option. When a project’s community becomes so divided that they can’t work things out, or they don’t want to work things out for whatever reasons, there is the option to take the code and start a new project. It always represents a failure but sometimes it can’t be helped. The two forks I can think of off hand, Joomla from Mambo and Icinga from Nagios, both resulted in stronger projects and better software, so maybe this will happen here.

In part I blame the VC model for financing software companies for the fork. In the traditional software model, a bunch of money is poured into a company to create software, but once that software is created the cost of reproducing it is near zero, so the business model is to sell licenses to the software to the end users in order to generate revenue in the future. This model breaks when it comes to free and open source software, since once the software is created there is no way to force the end users to pay for it.

That still doesn’t keep companies from trying. This resulted in a trend (which is dying out) called “open core” – the idea that some software is available under an open source license but certain features are kept proprietary. As Brian Prentice at Gartner pointed out, there is little difference between this and just plain old proprietary software. You end up with the same lack of freedom and same vendor lock in.

Those of us who support free software tend to be bothered by this. Few things get me angrier than to be at a conference and have someone go “Oh, this OpenNMS looks nice – how much is the enterprise version?”. We only have the enterprise version and every bit of code we produce is available under an open source license.

Perhaps this happened at ownCloud. When one of the founders was on Bad Voltage awhile back, I had this to say about the interview:

The only thing that wasn’t clear to me was the business model. The founder Frank Karlitschek states that ownCloud is not “open core” (or as we like to call it “fauxpensource“) but I’m not clear on their “enterprise” vs. “community” features. My gut tells me that they are on the side of good.

Frank seemed really to be on the side of freedom, and I could see this being a problem if the rest of the ownCloud team wasn’t so dedicated.

On the interview yesterday I asked if Nextcloud was going to have a proprietary (or “enterprise”) version. As you can imagine I am pretty strongly against that.

The reason I asked was from this article on the new company that stated:

There will be two editions of Nextcloud: the free of cost community edition and the paid enterprise edition. The enterprise edition will have some additional features suited for enterprise customers, but unlike ownCloud, the community and enterprise editions for Nextcloud will borrow features from each other more freely.

Frank wouldn’t commit to making all of Nextcloud open, but he does seem genuinely determined to make as much of it open as possible.

Which leads me to wonder, what’s stopping him?

It’s got to be the money guys, right? Look, nothing says that open source companies can’t make money, it’s just you have to do it differently than you would with proprietary software. I can’t stress this enough – if your “open source” business model involves selling proprietary software you are not an open source company.

This is one of the reasons my blood pressure goes up whenever I visit Silicon Valley. Seriously, when I watch the HBO show to me it isn’t a comedy, it’s a documentary (and the fact that I most closely identify with the character of Erlich doesn’t make me feel all that better about myself).

I want to make things. I want to make things that last. I can remember the first true vacation I took, several years after taking over the OpenNMS project when it had grown it to the point that it didn’t need me all the time. I was so happy that it had reached that point. I want OpenNMS to be around well after I’m gone.

It seems, however, that Silicon Valley is more interested in making money rather than making things. They hunt “unicorns” – startups with more than a $1 billion valuation – and frequently no one can really determine how they arrive at that valuation. They are so consumed with jargon that quite often you can’t even figure out what some of these companies do, and many of them fade in value after the IPO.

I can remember a keynote at OSCON by Martin Mickos about Eucalyptus, and how it was “open source” but of course would have proprietary code because “well, we need to make money”. He is one of those Silicon Valley darlings who just doesn’t get open source, and it’s why we now have OpenStack.

The biggest challenge to making money in open source is educating the consumer that free software doesn’t mean free solution. Free software can be very powerful but it comes with a certain level of complexity, and to get the most out of it you have to invest in it. The companies focused on free and open source software make money by providing products that address this complexity.

Traditionally, this has been service and support. I like to say at OpenNMS we don’t sell software, we sell time. Since we do little marketing, all of our users are self selecting (which makes them incredibly intelligent and usually quite physically beautiful) and most of them have the ability to figure out their own issues. But by working with us we can greatly shorten the time to deploy as well as make them aware of options they may not know exist.

In more recent times, there is also the option to offer open source software as a service. Take WordPress, one of my favorite examples. While I find it incredibly easy to install an instance of WordPress, if you don’t want to or if you find it difficult, you can always pay them to host it for you. Change your mind later? You can export it to an instance you control.

The market is always changing and with it there is opportunity. As OpenNMS is a network monitoring platform and the network keeps getting larger, we are focusing on moving it to OpenStack for ultimate scalability, and then coupled with our Minions we’ll have the ability to handle an “Internet of Things” amount of devices. At each point there are revenue opportunities as we can help our clients get it set up in their private cloud, or help them by letting them outsource some or all of it, such as Newts storage. The beauty is that the end user gets to own their solution and they always have the option of bringing it back in house.

None of these models involves requiring a license purchase as part of the business plan. In fact, I can foresee a time in the near future where purchasing a proprietary software product without fully exploring open source alternatives will be considered a breach of fiduciary responsibility.

And these consumers will be savvy enough to demand pure open source solutions. That is why I think Nextcloud, if they are able to focus their revenue efforts on things such as an appliance, has a better chance of success than a company like ownCloud that relies on revenue from software licensing sales. The fact that most of the creators have left doesn’t help them, either.

The lack of revenue from licenses sales makes most VCs panic, and it looks like that’s exactly what happened with the US division of ownCloud:

Unfortunately, the announcement has consequences for ownCloud, Inc. based in Lexington, MA. Our main lenders in the US have cancelled our credit. Following American law, we are forced to close the doors of ownCloud, Inc. with immediate effect and terminate the contracts of 8 employees. The ownCloud GmbH is not directly affected by this and the growth of the ownCloud Foundation will remain a key priority.

I look forward to the time in the not too distant future when the open core model is seen as quaint as selling software on floppy disks at the local electronics store, and I eagerly await the first release of Nextcloud.

OmniROM 6.0

For the last few days it has been hard to remain true to my free and open source roots. I guess I’ve been spoiled lately with almost everything I try out “just working”, but it wasn’t so with my upgrade to OmniROM 6.0 on my Nexus 6 (shamu).

I’ve been a big fan of OmniROM since it came out, and I base my phone purchases on what handsets are officially supported. While I tend not to rush to upgrade to the latest and greatest, once the official nightlies switched to Android “Marshmallow” I decided to make the jump.

Now there are a couple of tools that I can’t live without when playing with my phone. They are the Team Win Recovery Project (TWRP) and Titanium Backup. The first lets you create easy to restore complete backups and the latter allows you to restore application status even if you factory reset your device, which I had to do.

[NOTE: I should also mention that I rely on Chainfire’s SuperSU for root. It took me awhile to find a link for it I trust.]

When I tried the first 6.0 nightlies, all I did was sideload the ROM, wipe the caches, and reboot. I liked the new “OMNI” splash screen but once the phone booted, the error “Unfortunately process com.android.phone has stopped” popped up and couldn’t be cleared. Some investigation suggested a factory reset would fix the issue, but since I didn’t want to go through the hassle of restoring all of my applications I decided to just restore OmniROM 5.1 and wait to see if a later build would fix it.

Well, this weekend we got a dose of winter weather and I ended up home bound for several days, so I decided to give it another shot. I sideloaded the latest 6.0 nightly and sure enough, the same error occurred. So I did a factory reset and, voilà, the problem went away.

Now all I had to do was reload all 100+ apps. (sigh)

I started by installing the “pico” GApps package from Open GApps and in case you were wondering, the Nexus 6 uses a 32-bit ARM processor.

I guess I really shouldn’t complain, as doing a fresh install once in awhile can clean out a bunch of kruft that I’ve installed over the past year or so, but I’ve come expect OmniROM upgrades to be pretty easy.

One of the first things I installed from the Play store was the “K-9 Mail” application. Unfortunately, it kept having problems connecting to my personal mail server (the work one was fine). The sync would error with “SocketTimeoutException: fai”. So I rebooted back to Omni 5.1 and things seemed to work okay (although I did see that error when trying to sync some of the folders). Back I went to 6.0 (see where TWRP would come in handy here?) and I noticed that when I disabled Wi-Fi, it worked fine.

As I was trying to sleep last night it hit me – I bet it has something to do with IPv6. We use true IPv6 at the office, but not to our external corporate mail server, which would explain why a server in the office would fail but the other one work. At home I’m on Centurylink DSL and they don’t offer it (well, they offer 6rd which is IPv6 encapsulated over IPv4 but not only is it not “true” IPv6 you have to pay extra for a static IP to get it to work). I use a Hurricane Electric tunnel and apparently Marshmallow utilizes a different IPv6 stack and thus has issues trying to retrieve data from my mail server when using that protocol.

(sigh)

I tried turning off IPv6 on Android. It’s not easy and I couldn’t get any of the suggestions to work. Then I found a post that suggested it was the MTU, so I reduced the MTU to 1280 and still no love.

So I turned off the HE tunnel. Bam! K-9 started working fine.

For now I’ve just decided to leave IPv6 off. While I think we need to migrate there sooner rather than later, there is nothing I absolutely have to have IPv6 for at the moment and I think as bandwidth increases, having to tunnel will start to cause performance issues. Normal traffic, such as using rsync, seems to be faster without IPv6.

That experience cost me about two days, but at the moment I’m running the latest OmniROM and I’m pretty happy with it. The one open issue I have is that the AOSP keyboard crashes if you try to swipe (gesture type) but I just installed the Google Keyboard and now it works without issue.

I have to say that there were some moments when I was very close to installing the Google factory image back on my Nexus 6. It’s funny, but the ability to shake the phone to dismiss an alarm is kind of a critical app with me. Since the last time I checked it wasn’t an available option on the Google ROM, I was willing to stick it out a little longer and figure out my issues with OmniROM.

Heh, freedom.

Avoiding the Sad Graph of Software Death

Seth recently sent me to an interesting article by Gregory Brown discussing a “death spiral” often faced by software projects when issues and feature requests start to out pace the ability to close them.

Sad Graph of Death

Now Seth is pretty much in charge of managing our Jira instance, which is key to managing the progress of OpenNMS software development. He decided to look at our record:

OpenNMS Issues Graph

[UPDATE: Logged into Jira to get a lot more issues on the graph]

Not bad, not bad at all.

A lot of our ability to keep up with issues comes from our project’s investment in using the tool. It is very easy to let things slide, resulting the the first graph above and causing a project to possibly declare “issue bankruptcy“. Since all of this information is public for OpenNMS, it is important to keep it up to date and while we never have enough time for all the things we need to do, we make time for this.

I think it speaks volumes for Seth and the rest team that OpenNMS issues are managed so well. In part it comes naturally from “the open source way” since projects should be as transparent as possible, and managing issues is a key part of that.

Capitalism and the Open Source Way

I’m supposed to be on vacation today. My 50th birthday is coming up and I’m taking some time off to celebrate and reflect. But Jan Wildeboer posted a link to a critical article about a recent Paul Graham essay, and it touched a nerve. I wanted to write down a few thoughts about it while they were fresh.

In the essay, Graham boasts about increasing income inequality. It’s the new version of “greed is good“. He proposes that the best method for modeling democracy is that of the startup. I can’t agree with that.

Look, I work at a ten-year-old startup, but that isn’t what Graham means. He means the Silicon Valley startup which follows this basic model:

1) Come up with an idea
2) Get some rich people to give you money to pursue the idea

If you get past Step 2, this is considered “a success” because if a rich guy wants to give you money your idea must be good, right?

3) Burn through that money as fast as you can in search of turning your idea into something people will watch, download, share or buy
4) Run out of money
5) Get more money
6) Go back to step 4, eroding your share of the idea until the rich people own it

Success is then measured by an acquisition or IPO. Failure is that you can’t get past step 5 at some point.

I can’t remember who told me this, so I do apologize for not being able to credit you, but it was pointed out to me that a lot of startups tend to hit the US$5MM revenue mark and then stall. The reason, she said (and I do believe it was a she) was that startups are aimed at the culture of Silicon Valley, and quite frequently an idea that works in the Valley doesn’t work elsewhere.

The Valley consists mainly of young, white and Asian males. I’ve spent a lot of time in the Valley, and while I’ve met a lot of amazing people, I’ve met an equal number of assholes. The latter seemed to measure value strictly on wealth, and they pursue money above all else (“go big or go home”). Look, I think money is great, it can provide options and security, but the sole pursuit of money is not a good way to live. If I have any wisdom to impart after 50 years it would be to buy experiences, not things. The former will last a lot longer.

And this shameless pursuit of money, in both the Valley and on Wall Street, is creating a huge wealth inequality. From what I could find on the web, the average software engineer in the Valley makes around US$150K. Meanwhile, for the same year the average household income was a little over US$50K, so a third of that probably with more than one person working.

People will defend those salaries because they say they are valuable, but if we are talking about a startup-driven economy, most startups both lose money and eventually fail. So I’m not sure it can be defended on value creation. Plus, as the wealth gap gets larger and larger, there is a real, non-zero chance of a whole lot of people with baseball bats storming those gated communities.

When I was younger and took my first Spanish class, the teacher told us that many countries in South and Central America, where Spanish is spoken, had turbulent political histories. She explained that it was often due to wealth inequality. When you have a small but significant group of rich people and a whole lot of poor people, those at the “top” don’t tend to stay there. She then pointed to the US and its large middle class, and argued that it was one of the reasons we’ve been around for 200+ years.

Also, back in the “old days”, if you asked a kid to list jobs you’d get things like teacher, policeman, doctor, janitor, nurse, mailman, lawyer, baker, fireman and, my favorite, astronaut.

Those are wonderful, productive roles in society. Sure, the doctor and lawyer made more money, but we didn’t look down on the janitor (I can remember really liking the janitor at our elementary school and thinking he was so nice to keep our school clean). But somewhere in the last ten to twenty years, we’ve seemed to lose our way as a culture and we look down on a lot of these jobs. The message seems to be “be scared and buy shit” and success is measured on how much shit you can buy.

It’s not sustainable. In finance the idea of “grow, grow, grow!” is considered the goal. In nature it’s called “cancer”.

This is one reason I love my job. At OpenNMS our business plan is simple: spend less than you earn. The mission statement is: help customers, have fun, make money.

A lot of that comes from the fact that we base our business around open source software. One of the traditional methods for securing profit in the software industry, especially the Valley, is to lock your customers into your products so they both become reliant on them and are unable to easily switch. Then you can increase your prices and … profit!

In order to do this, you have to have a lot of secrets. Your code has to be secret, your product roadmap needs to be secret, and you have to spend a lot of money on engineering talent because you have to find highly skilled specialists to work in such an environment.

Contrast that to open source. Everything is transparent. The code is out there. The roadmap is out there. This week is the CES show in Las Vagas where products will be “unveiled”. We don’t unveil anything – you can follow the development branches in our git repository in real time. While I am lucky to work with highly skilled people, they found OpenNMS, not the other way around, because they had something to offer. Our customers pay us a fair rate for our work because if it isn’t worth it to them, they don’t have to buy it.

This has allowed OpenNMS to survive and, yes, grow, over the last decade while a number of startups have come and gone.

This transparency is important to the “open source way“. It promotes both community and participation, and it is truly a meritocracy, unlike much of the Valley. In the Valley, value is measured more by how much money you make and who you know. In open source, it is based on what you get done and how well you advance the project.

[Note: just to be fair, I know a number of very talented people in the Valley who are worth every penny they make. But I know way more people who, in no way, earn their exorbitant salaries]

Another comment that triggered this post was a tweet by John Cleese about a quote from Charlie Mayfield, the Chairman of the John Lewis Partnership which is a huge retail concern in the UK. He said “… maximisation of profit is not our goal. We aim to make sufficient profit.”

Sufficient Profit Tweet

What a novel idea.

I’m sure my comments will be easily dismissed by many as just the ranting of an old fart, similar to “get off my lawn”. But I have always wished for OpenNMS to be, above all else, something that lasts – something that survives me and something that provides value long after I’m gone. Would I like more money? Of course I would, but for longevity the focus must be on creating value and providing a great experience for those who work on the project, and the money will come.

After all, it is the experience that lasts.

♫ Don’t Call It a Comeback ♫

Welcome to 2016. My year started out with an invitation to join the AARP. (sigh)

As my three readers know, when it comes to this business of open source we are pretty much making things up as we go along. We are lead by our business plan of “spend less money than you earn” and our mission statement of “help customers, have fun, make money” but the rest is pretty fluid.

In 2013 we mixed things up and tried a more “traditional” start up path by seeking out investment and spending more money than we had. It didn’t work out so well.

Thus 2014 was more of a rebuilding year as we tried to move the focus back to our roots. It paid off, as 2015 was a very good year. We had record gross revenues, and although we didn’t make much money on the bottom line, it was positive once again. At the moment we are still investing in the company and the project so pretty much every extra dollar goes into growth.

And we had a lot of growth. The decision to split OpenNMS into Meridian and Horizon paid off in three major Horizon releases. Horizon 17 was an especially large and important release as it brought in the Newts integration. At the moment we are working with it on a customer site using a ScyllaDB cluster capable of supporting 75K inserts per second. The technologies introduced in 2015 will make it in to Meridian 2016, due in the spring, and it should solidify OpenNMS as a platform that can really scale.

In 2015 we also received orders from two of the Fortune 5 companies. I’ll leave it as an exercise to the reader to guess which two and you have a 1 in 16 shot at getting it right (grin). The fact that companies that can choose, literally, any technology they want yet they choose OpenNMS speaks volumes.

One of these days we’re going to have to figure out a way to talk about our customers by name, since they are all so cool. We are working on it, but it is surprisingly difficult to get permission to publicly post that information. Above all we respect our clients’ privacy.

I have high expectations for 2016 and the power of the Open Source Way. Thanks to everyone who has supported us over the last decade and more, and we just hope you find our efforts provide some value.

Happy New Year.

Reflections on Paris and My Cowardice

I was on a bus in Ireland when I heard the news about the Paris attacks. I had gotten up early to head to the opposite coast as I wanted to see an Ireland that wasn’t Dublin, and I don’t think I could have picked a better spot than Doolin, in County Clare.

Today was to be a particularly gray day and it was dark when I started out. It didn’t get much lighter as we rode to Galway, and when I changed buses the driver was playing the news from the radio. Of course the only story was about the more than one hundred people killed in senseless violence overnight.

Peace Symbol by @jean_jullien

I have some friends in Paris and so I immediately reached out to them. As I waited for a response, I pretty much sat, stunned, as the Irish countryside passed by outside my window.

Once I got to my B&B, I dropped my bag and took a long walk, looking for lunch. The day reflected my mood perfectly. It was like nature itself was in mourning. At high noon the sky wasn’t much lighter than at dusk. A roaring wind came off the sea, churning up angry whitecaps. The clouds drizzled rain like tears.

By the time I was getting cold, I found the recommended pub and went in. It was packed, as this is a popular tourist location and they drop people off by the bus load. Since I was alone, I offered to sit at the bar to make room for the next coach, which arrived about five minutes after I did.

A boisterous crowd of mainly young people came in and crowded around the bar where I sat. They were laughing and joking, blissfully unaware of how quickly that can change. I took a little comfort in the normalcy of that moment: people ordered food, the Indian guy asked about vegetarian options, and drinks were poured (including an inexplicable request for a bottle of Miller beer).

As I ate my meal, a nice smoked salmon salad and a wonderful seafood chowder stuffed with mussels, I was reminded of the last time I had mussels this good, which just happened to be in a Belgian restaurant in Paris called La Gueuze.

And I struggled with a dilemma. The Paris Open Source Summit is next week and I am supposed to be there. Heck, I lobbied hard for the opportunity to participate. But while the chance of anything happening is very slim, I can’t say I’m eager to be in Paris at the moment, especially as part of a large crowd.

So I decided not to go.

There were a number of factors. Part of it was concern for my wellbeing. Part of it was concern for my family. I travel a lot and I know they worry no matter where I’m going, and they have been very understanding when I’ve gone to places that don’t exactly have a reputation for safety. I refuse to put my decision on them, but it did play a role.

But I think the deciding factor was actually how much I enjoyed Paris on my last trip. It is an amazing city, and I didn’t want that memory ruined by seeing soldiers on every corner or having to go through intrusive screening at every point of entry.

It makes me feel like a coward. The terrorists have won.

And I can’t understand it. Of all the countries in Europe, the French bend over backwards to be accommodating to different views and ways of thinking. The French motto “Liberté, égalité, fraternité” leads with the word for freedom, and they go to great lengths to explore all the weird corner cases to insure their society is as free as possible.

And that’s what makes me the most angry. I’m certain these acts are going to change that. Not only will it move France to be more restrictive, it will give the more aggressive countries reason to step up military action in the Middle East. A lot more people will die, and most of them will have darker skin. This will create more terrorists, and the cycle will continue.

I hope France and the rest of the world shows some restraint. I’m not, in any way, shape or form, suggesting justice not be sought out, but I’m reminded of something I saw many years ago.

I was living at my parents’ house and my two-year-old nephew was staying with us. It was a beautiful day and so the windows were open, and there was a gentle breeze throughout the house. One strong breeze caught the door behind the boy and slammed it shut. It scared him, so he reached out and smacked the door, as if to punish it. It struck me as a perfect example of a childish reaction – I’m scared and angry so I need to strike out at the nearest thing, whether is makes sense or not.

I hope the world remembers that we are not children.

I don’t have any answers on how to make things better. The best I can do is to promote free and open source software. I know it sounds silly, using FOSS to cure the world’s problems, but in every place I’ve visited (and I’ve been to 37 different countries) I’ve found like-minded people in that community with a strong desire to create new things through cooperation. It creates an environment where anything is possible. In a small way, it creates hope.

I am writing this sitting on my bed at the B&B. It’s cold, and the wind is whipping around the house, but I feel cozy and safe. Here’s a wish that everyone can find a place to be cozy and safe, as well as the hope that tomorrow will be a better day.

OUCE 2015: Bad Voltage Live

Every year at the OpenNMS Users Conference (OUCE) we have a good time. In fact, learning a lot about OpenNMS goes hand in hand with having fun.

At this year’s SCaLE conference, the team behind the Bad Voltage podcast was there to do a live version of the show. You can watch it on-line and see it went pretty well, and this gave me the idea to invite the gang over to Germany to do it again at the OUCE.

Since there may be one or two of my three readers who are unaware of Bad Voltage, I thought I’d post this little primer to bring you up to speed.

Bad Voltage is a biweekly podcast focused on open source software, technology in general and pretty much anything else that comes across the sometimes twisted minds of the hosts. They deliver it in a funny manner, sometimes NSFW, and for four guys with big personalities they do a good job of sharing the stage with each other. As I write this they have done 47 episodes, which is actually quite a nice run. For anyone who has done one or thought about doing a periodic podcast or column, know that after the first few it can be hard to keep going. It is a testament to how well these guys work together that the show has endured. Believe it or not, I actually put time into these posts and even I find it hard to produce a steady amount of content. I can’t imagine the work needed to coordinate four busy guys to create what is usually a good hour or three of podcast. (grin)

Bad Voltage as The Beatles

Anyway, I want to introduce you to the four Bad Voltage team members, and I thought it would be a useful analogy to compare them to the Beatles. As I doubt anyone who finds this blog is too young to not know of the Beatles, it should aid in getting to understand the players.

Bad Voltage - Jono Bacon Jono Bacon is Paul. If you have heard of anyone from Bad Voltage, chances are it is Jono. He’s kind of like the Elvis of open source. He was a presenter for LugRadio but is probably best known for his time at Canonical where he served as the community manager for Ubuntu. He literally wrote the book on open source communities. He is now building communities for the XPRIZE foundation as well as writing articles for opensource.com and Forbes and occasionally making loud music. He’s Paul because is he one of the most recognizable people on the team, and he secretly wishes I had compared him to John.

Bad Voltage - Bryan Lunduke Bryan Lunduke is John. He gets to be John because he has heartfelt opinions about everything, and usually good arguments (well, arguments at least) to back them up. He has passion, much of which he puts into promoting OpenSUSE. I’ve never met Bryan in person, but we’ve missed each other on numerous occasions. I missed him at SCaLE, he missed the Bad Voltage show I was on, and I missed him again at OSCON. And I’ll miss him in Fulda, as his wife is due to deliver their second child about that time, but he will be there virtually. He adds depth the the team.

Bad Voltage - Jeremy Garcia Jeremy Garcia is George. Although none of these guys could be described as “quiet”, he is the most reserved of the bunch, but when he opens his mouth he always has something interesting to say. You can’t be part of this group and be a wallflower. I’m not sure if he has a day job, but fifteen (!) years ago he founded Linuxquestions.org and has been a supporter of open source software even longer. He adds a nice, rational balance to the group.

 

Bad Voltage - Stuart Langridge Stuart is Ringo, known to his friends as “Aq” (short for “Aquarius” – long story). He is pretty unfiltered and will hold forth on topics as wide ranging as works of science fiction or why there should be no fruit in beer. He was also a member of LugRadio as well as an employee of Canonical, and now codes and runs his own consulting firm (when he is not selling his body on the streets of Birmingham). If there was a Bad Voltage buzzword bingo, you could count on him to be the first to say “bollocks”. He adds a random element to the group that can often take the discussion in interesting directions.

They have been working hard behind the scenes to plan out a great show for the OUCE. Since many of the attendees tend not to be from the US or the UK, it is hoped that the show will translate well for the whole audience, and to make sure that happens we will be serving beer (if you are into that sort of thing). If you were thinking about coming to the conference, perhaps this will push you over the top and make you register.

But remember, you don’t have to attend the OUCE to see the show. We do ask that you register and pony up 5€. Why? Because we know you slackers all too well and you might sign up and then decide to blow it off to binge on Regular Ordinary Swedish Meal Time. Space is limited, and we don’t want to turn people away and then have space left open. Plus, you’ll be able to get that back in beer, and the show itself promises to be priceless and something you don’t want to miss.

If that isn’t enough, there is a non-zero chance that at least one of the performers will do something obscenely biological (and perhaps even illegal in Germany), and you could say “I was there”.

Case Study: Why You Want OpenNMS Support

I wanted to share a story about a support case I worked on recently that might serve to justify the usefulness of commercial OpenNMS Support to folks thinking about it. As always, OpenNMS is published under an open source license and so commercial support is never a requirement, but as this story involves commercial software I thought it might be useful to share it.

We have a client that handles a lot of sensitive information, to the point that they have an extremely hardened network environment that makes it difficult to manage. They place a separate copy of OpenNMS into this “sphere” just to manage the machines inside it, and they have configured the webUI to be accessed over HTTPS as the only access from the outside.

Recently, a security audit turned up this message:


Red Hat Linux 6.6 weak-crypto-key
3 Weak Cryptographic Key Fail "The following TLS cipher suites use
Diffie-Hellman keys smaller than 1024 bits: *
TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA (768-bit DH key) *
TLS_DHE_RSA_WITH_AES_128_CBC_SHA (768-bit DH key)" "Use a Stronger Key If
the weak key is used in an X.509 certificate (for example for an HTTPS
server), generate a longer key and recreate the certificate. Please also
refer to NIST's recommendations on cryptographic algorithms and key
lengths (http://csrc.nist.gov/publications/nistpubs/800-131A/sp800-131A.pdf
) ." Vulnerable

and they opened a support ticket asking for advice on how to fix it.

I had some issues with the error message right off the bat. The key used was 2048 bits, so my guess is that the algorithm is weak and not the key. The error message seems to suggest, however, that a longer key would fix the problem.

Anyway, this should be simple to fix. The jetty.xml file in the OpenNMS configuration directory lets you exclude certain ciphers, so I just had the customer add these two to the list and restart OpenNMS.

And then we waited for the nightly scan to run.

This fixed the issue with the TLS_DHE_RSA_WITH_AES_128_CBC_SHA cipher but not the first one. Nothing we did seemed to help, so I installed sslscan on my test machine to try and duplicate the issue. I got a different list of ciphers, and since openssl uses different name for the ciphers than Java, and it was a bit of a pain to try and map them. I couldn’t get sslscan to show the same vulnerabilities as the tool they were using.

We finally found out that the tool was Nexpose by Rapid 7. I wasn’t familiar with the tool, but I found that I could download a trial version. So I set up a VM and installed the “Community Edition”.

Note: this has nothing to do with open core, which often refers to their “free” version as the “community” version. Nexpose is 100% commercial. They use “community” to mean “community supported”, but it is kind of confusing, like when Bertolli’s markets “light” olive oil which means “light tasting” and not low in calories.

I had to fill out a web form and wait about a day for the key to show up. I had installed the exact version of OpenNMS that the client was using on my VM, so my hope was that I could recreate the errors.

First, I had to increase the memory to the VM. Nexpose is written in Java and is a memory hog, but so is OpenNMS, and it was some work to get them to play nice together on the same machine. But once I got it running, it wasn’t too hard to recreate the problem.

The Nexpose user interface isn’t totally intuitive, but I was able to add the IP address of the local machine and get a scan to kick off without having to read any documentation. The output came as a CVS file, but you could also examine the output from within the UI.

The scan reported the same two errors, and just like before I was able to remove the “TLS_DHE_RSA_WITH_AES_128_CBC_SHA” one just by excluding it in jetty.xml, but the second one would not go away. I found a list of ciphers supported by Java, but nothing exactly matched “TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA” and I tried almost all of the combinations for similar TLS ciphers.

Then it dawned on me to try “SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA” and the error went away. I guess in retrospect it was obvious but I was pretty much focused on TLS based ciphers and it didn’t dawn on me that this would be the error with Nexpose.

It was extremely frustrating, but as my customer was being beat up about it I was glad that we could get the system to pass the audit. While this was totally an issue with the scanning software and not OpenNMS, it would have been hard to figure out without the help we were happy to give.

It may not surprise anyone that a large number of OpenNMS support issues tend to be related to products from other vendors. Usually most of them can be classified as a poor implementation of the SNMP standard, but occasionally we get something like this.

Our clients tend to be incredibly smart and good at their jobs, but having access to the folks that actually make OpenNMS can sometimes save enough time and headache to more than offset the cost of support.

2015 O’Reilly Open Source Conference

I think this year marks the eighth OSCON I’ve attended. I’m not sure of that, but I am sure that every year I can meet up with a number of interesting people that I just don’t see elsewhere.

I used to get the conference pass so I could see the presentations, and while they tend to be of a very high quality, I often found myself spending most of my time outside of those rooms, either on the Expo floor or just sitting and talking, so this year I just got the Expo pass.

OSCON 2015 - Entrance

I have a love/hate relationship with OSCON. It seems to be skewed toward large companies, and this year was no exception.

I got to see the jugglers at Paypal:

OSCON 2015 - Paypal

(Note: Jason, who used to work with us at OpenNMS, is now at Paypal and so I get to hear about some of the stuff they are doing around open source it is pretty exciting).

and Microsoft was back with the photo booth:

OSCON 2015 - Microsoft

There were also some smaller companies in attendance. I had to go by and say “hi” to the Atlassian team as we happily use a number of their products to make OpenNMS happen, such as Bamboo and Jira:

OSCON 2015 - Atlassian

and it was nice to run into Chris Aniszczyk, the open source guy at Twitter.

OSCON 2015 - Chris Aniszczyk

I had not talked to Chris since last year’s OSCON and it was cool to learn that he’s doing well.

One thing I’ve been looking at for OpenNMS is the best configuration platform with which to integrate. It is hard to choose between Puppet, Chef, Ansible and Salt (and we should probably do all four) but if the choice was solely based on the friendliest staff Chef would probably win.

OSCON 2015 - Chef

I never did get the full story on what happened with their booth.

Right around the corner was the Kaltura booth with its incredibly shy and withdrawn Director of Marketing, Meytal:

OSCON 2015 - Meytal Burstein

She was also at CLS and our paths crossed a lot, and I’m certain I’ll run into her in the future. Oh, and if you want her opinion, you’ll have to drag it out of her.

(Note: some of the above is not true)

OSCON 2015 - CDK Global

It was also cool to see a booth for CDK Global. CDK was formed by merging Cobalt and ADP Dealer Services, and the latter uses OpenNMS. Sam (the guy in the middle) was also a Frontalot fan, so we got along well.

I spent most of my time off to the side of the Expo floor on a row I called the “Geek Ghetto”. These are booths that OSCON offers to open source projects and organizations. It was cool to see that it was almost always packed with people.

OSCON 2015 - Geek Ghetto

I got to talk to the team at the Linuxfest Northwest. This is one conference I have yet to attend but I’m going to make an effort to get there next year. I’m hoping to convince the Bad Voltage guys to come along and do a live show (they will be with us at the OUCE this September in Germany)

OSCON 2015 - Linuxfest Northwest

Next to them was a booth from the EFF. Maggie, who was at the anniversary show in San Francisco, was also doing booth duty at OSCON.

OSCON 2015 - EFF

I believe in what the EFF is doing so it was nice to get to talk with them.

Last year I spent a lot of time learning about Free Geek:

OSCON 2015 - Free Geek

and it was nice to chat with them again. If you are in a Free Geek city, you should get involved.

It was good to see a large number of women in attendance, although it was still not reflective of the population as a whole. One group working to change that is Chicktech:

OSCON 2015 - Chicktech

Note that my picture got photobombed by “Open Source Man”.

Also in the Geek Ghetto was the Software Freedom Conservancy, run in part by Bradley Kuhn and Karen Sandler. I think highly of them both and enjoyed the time I got to spend with them.

OSCON 2015 - Karen Sandler

Now, I should probably explain my shirt.

Bryan Lunduke is one-fourth of the Bad Voltage team. While I have known Jono Bacon for some time, I didn’t get to meet Jeremy Garcia or Stuart Langridge until this year’s SCaLE conference. I never got to meet Bryan. To be honest, a lot of these “meetings” happened in bars and Bryan doesn’t drink, and I did try to get his attention on the show floor but he obviously didn’t hear me.

Then I was on the Bad Voltage podcast talking about OpenNMS. This was an episode where Bryan was ill, so outside of signing in to say he couldn’t do the show, I didn’t see much of him.

Finally, we are planning on having Bad Voltage come out to the OpenNMS User’s Conference this September. Bryan is expecting the arrival of his second child, so he had to beg off.

Now I just see these things as coincidences, but the guys in the office suggested the real reason is that Bryan hates me. Jessica, our graphic designer, took the bait and made up a graphic, and my friend Jason at Princredible printed a few really nice shirts.

I wanted to meet up with him in Portland, but he was only at CLS the second day (I was there the first). He was at OSCON on Wednesday. I wandered around the Expo floor trying to find him but we could never meet up.

It started to become amusing. People would stop me and say “Bryan was just here looking for you”. After awhile I thought it might be even funnier if we never met, just circled each other at the conference and to this day we still haven’t stood next to each other (he and Jono did call me later in the day, but I had already left).

Anyway, if you think Bryan Lunduke hates you too, you can get a nifty shirt just like mine. Jason will take orders until 10 August. These are high quality shirts that are actually printed – the image is dyed into the fabric and not screened on top were it is likely to crack and peel.

OSCON 2015 - Jono Bacon

Speaking of Jono, he did an “Ask Me Anything” session and I was very eager to get some of the burning questions off my chest. Unfortunately, it was subtitled to limit the questions to things like “community management” and “leadership”. Mine were, to a fault, all obscenely biological.

I want to end this note with a picture of one of my favorite people, within or outside of open source, Stephen Walli.

OSCON 2015 - Stephen Walli

I usually only see him at OSCON, and while in his sunset years he has quieted down a bit (grin), I always welcome the time I get to spend with him.

Hope to see everyone in Austin in 2016, if not sooner.