One thing that all open source projects struggle with is getting users. Most people in IT and software are overwhelmed with a plethora of information and options, and matching the right material to the right audience is a non-trivial problem.
Last year my friend Chris suggested that I speak at a Network Operators Group (NOG) meeting, specifically AusNOG. It was a lot of fun. I felt very comfortable among this crowd. so I decided to reach out to more NOGs to see if they would be interested in learning about OpenNMS.
The thing I like the most about NOGs is that they value getting things done above all else. While “getting things done” is still important with the free and open source crowd, there seems to be more philosophy and tribalism at those shows. “Oh, that’s written in PHP, it must suck” etc. As a “freetard” I live for the philosophical and social justice aspects of the community, but from a business standpoint it doesn’t translate well into paying customers.
At NOGs the questions are way more business-focused. Does it work? Is it supported? What does it cost? While I’m admittedly biased toward OpenNMS and its open source nature, the main reason I keep promoting it is that it just makes solid business sense for many companies to use it instead of their current solution.
Plus, these folks are pretty smart and entertaining while dispensing solid advice and knowledge.
Anyway, with that preamble, at AusNOG I learned about the New Zeland NOG (NZNOG) and submitted a talk. It got accepted and I found myself in Queenstown.
The main conference was spread out over two days, and like AusNOG it consisted of 30 to 45 minute talks in one track.
While I know it won’t work for a lot of conferences, I really like the “one track” format. It exposes me to things I wouldn’t have gone to otherwise, and if there is something I am simply not interested in learning about I can use that time to catch up on work or participate in the hallway track.
The conference started with a presentation by the Honorable Clare Curran, a newly minted Member of Parliament (they recently held elections in New Zealand). I’m slowly seeing politicians getting more involved in information technology conferences, which I think is a good thing, and I can only hope it continues. She spoke about a number of issues the government is facing with respect to communications technology.
Several things bother me about the US government, but one big one is the lack of understanding of the importance of access to the Internet at broadband speeds. Curran stated that “lack of reliable high-speed network access is a new measure of poverty”. Later in the day John Greenhough spoke on New Zealand’s Ultra-Fast Broadband (UFB) project, where on one slide broadband was defined as 20Mbps download speed.
Where I live in the US I am lucky to get 10Mbps and many of my neighbors are worse off, yet the government is ceding more of the decision making process about where to build out new infrastructure to the telecommunications companies which have zero incentive to improve my service. It’s wonderful to see a government realize the benefits of a connected populace and to take steps to make it happen.
Because we all need Netflix, right? (grin)
There was a cool talk about how Netflix works, and I didn’t realize that they are working with communications providers to provide low-latency solutions distributed geographically. This is done by supplying providers with caching content servers so that customers can access Netflix content while minimizing the need for lots of traffic over expensive backhaul links.
I did find it cool that one of the bandwidth graphs presented was obviously done using RRDtool. I don’t know if they collected the data themselves or used something like OpenNMS, but I hope it was the latter.
With this push for ubiquitous network access comes other concerns. New Zealand has a law called TICSA that requires network providers to intercept and store network traffic data for use by law enforcement.
I thought the requirements were pretty onerous, but I was told that the NZ government did set aside some funds to help providers with deploying solutions for collecting and storing this data (but I doubt it can cover the whole cost, especially over time). The new OpenNMS Drift telemetry project might be able to help with this.
There were a couple of talks I had seen in some form at AusNOG. The ever entertaining Aftab Siddiqui talked about MANRS (Mutually Agreed Norms for Routing Security) but unlike in Australia he was hard pressed to find good examples of violations. Part of that could be that New Zealand is much smaller than Australia, but I’m giving the NZ operators the credit for just doing a good job.
The Facebook folks were back to talk about their NetNORAD project. While I have a personal reluctance to deploy agents, there really isn’t a way to measure latency at the detail they want without them. I think it would be cool to be able to gather and manage the data created by this project under OpenNMS.
What I like most about these NOG meetings is that I always learn something cool, and this one was no different. Geoff Huston gave a humorous talk on DNSSEC and handling DNS-based DDoS attacks. While I was somewhat familiar with DNSSEC, I was unaware of the NSEC part of it.
Most DNS DDoS attacks work by asking for non-existent domains, and the overhead in processing them is what causes the denial of service. The domain name is usually randomly generated, such as jeff123@example.com, jeff234@example.com, etc. If the DNS server doesn’t have the domain in its cache, it will have to ask another DNS server, which in turn won’t have the domain as it doesn’t exist.
The NSEC part of DNSSEC, when responding to a non-existent domain request, will return the next valid domain. In the example above, if I ask for jeff123@example.com, the example.com DNS server can reply that the domain is invalid and, in addition, the next valid domain is www.example.com. If implemented correctly, the original DNS server should then never query for jeff234@example.com since it knows it, too, doesn’t exist.
Pretty nifty.
One talk I was eagerly awaiting was from Rada Stanic at Cisco. She also spoke at AusNOG but I had to leave early and missed it. While she disrespected SNMP a little more than I liked (grin), her talk was on implementing new telemetry-based monitoring protocols such as gRPC. OpenNMS Drift will add this functionality to the platform. Our experience so far is that the device vendor implementation of the telemetry protocols leaves something to be desired, but it does show promise.
It was nice being in New Zealand again, and our mascot Ulf seemed to be popular with the locals. Can’t imagine why.
