One of my favorite TV shows of all time is The Big Bang Theory. It is somewhat divisive among my circle of friends, but I like it. One scene from the show has the character Leonard, a physicist, talking to his girlfriend Penny about his work. She asked what he did that day, and he said “I thought about stuff”. She asked if that was all he did and he said “I wrote some of it down.”
This post is me, thinking about stuff and writing it down.
Note that this post more than normal may not necessarily reflect the views of my employer, Amazon Web Services.
TL;DR; Large Language Models are built on a flawed foundation, but Small Language Models may overcome those shortcomings. Open source will play a big role in making Generative AI useful.
I’ve been working in tech since the early 1980s, and in all that time nothing has taken the focus of the industry more strongly than Generative AI. Seriously, not even the early Web was a crazy as the current environment, where companies worth billions (on paper) pop up overnight like mushrooms after a summer rain. Even established companies are making pronouncements that employees will need to use and demonstrate competence with GenAI, and many are rushing into it regardless of the consequences, such as the issue I had trying to cancel a magazine subscription.
I think everyone can admit that the output of GenAI can be impressive. Despite people with six fingers and text that looks like it was found in a alien spaceship, images and videos created by GenAI can be stunning, and text summaries can often distill down hundreds of pages into a few main points.
One major issue with Generative AI is that it is often wrong, but it presents the information with certainty. I had something happen to me last night that was the impetus for this post.
Andrea and I watch and hour or two of television every night, and as we live in an era of peak TV there is a lot to watch. We were catching up on a series called Yellowjackets, and as it was obvious the storylines were not going to be closed at the end of the third season I wanted to see if the show had been renewed. A quick trip to Wikipedia showed that, yes, it has been renewed for a fourth season as of May.
This made me curious to know if the creators of the show had decided how long it was going to take to tell the whole story (answer: five seasons). When I did a search for that the “helpful” GenAI output at the top of the results started off with “As of now, Yellowjackets has not been renewed for a fourth season”.
(sigh)
When I go to bed at night, it is like I’ve cached a whole bunch of stuff in my head, and before I can get to sleep I have to empty that cache. It sometimes keeps me up at night. Maybe because this latest interaction with GenAI was top of mind, but last night as I lay in bed it dawned on me that perhaps one of the reasons GenAI is so popular is that we are living in a “post-truth” time.
I try to minimize any political discussion on this blog. I figure my three readers can get that information elsewhere, but I do have to bring it up within the context of this post. Just for the record I belong to no political party at the moment, but in my youth I was active in Republican party. I even have a signed letter from Ronald Reagan, although I seriously doubt his hand held the pen that signed it.
I can remember a time when a politician (or executive, or athlete, or other celebrity) who said something that was untrue and was called out on it would stop saying it. Now, when someone makes a statement and is presented with evidence that the statement is wrong, they simply repeat the statement, sometimes louder. If you repeat it long enough and loud enough, people forget that it is a lie.
One of the more prescient things I’ve read was the first chapter in Neal Stephenson’s Fall; or Dodge in Hell.
Warning, mild spoilers.
The first chapter tells the story of a nuclear weapon destroying Moab, Utah. All communication with the town is lost while at the same time people in aircraft flying near the area report seeing a mushroom cloud.
It turns out this is a hoax. The people reporting the mushroom cloud were actors thinking they were auditioning for a reality TV series. It was pretty easy to isolate Moab from the rest of the world since the town is small (less than five square miles) and somewhat remote, and thus the creators of the hoax were able to cut cell phone and internet communications.
But even after the hoax is exposed people continue to believe in it.
While this was fiction, there are a number of modern parallels. As someone who read 1984 before it was 1984, this is kind of scary to me.
But this post-truth environment allows things like GenAI to thrive. We no longer, as a society, require truth or accuracy, and we are happy to brush off any inconsistencies. Even I am a victim of it, as I tend to believe GenAI when it confirms my biases, especially when the answer returned is returned with confidence.
In my job I am exposed to a lot of GenAI tools (“Learn and Be Curious” is a Leadership Principle). Back in June I was trying to remember if Juneteenth was a work holiday, so I brought up a model and asked.
It returned a detailed answer, starting off with the statement that it was a paid holiday, when it was introduced as a paid holiday, and why it was made a holiday.
It was also wrong.
When I mentioned this to others I was told that, for Human Resource (HR) questions, I needed to use a different model. (sigh)
Hold on to that thought for a minute, because I want to come back to it.
Large Language Models (LLMs) are created by feeding a large amount of examples (training data) into them, and most of that data is accessed over the internet. Some sites are seeing almost all of their web traffic being taken up by these models trying to “scrape” their content.
We have a lot of new content being produced by these LLMs. Now remember that these models are flawed in that they aren’t built from first principles but instead are trained on undifferentiated data, and as LLMs introduce more inaccuracies into that data it will result in a feedback loop that just makes the models more and more, well, wrong.
The best analogy for this I can think of involves commuting to work. Many years ago I had a job near the RDU Airport, and every day I drove down the interstate (I-40). The traffic was pretty bad, so it was decided to double the number of lanes along the busiest part of the road. The construction process was painful, making the commute even worse, but once it was done, oh my, it was so nice.
After the new lanes opened I remember a comment in the local newspaper (kids, asked you parents about newspapers) that said “Enjoy it while you can, as it is only going to get worse from here”. As the feedback increasingly makes LLMs less useful, we will remember now as the “good old days”.
This isn’t something that can be fixed. Does anyone remember the Intel Pentium floating point scandal? A bug in the microcode meant that, under certain conditions, math operations would return an incorrect value. That bug could be rectified, but the very nature of LLMs makes a fix impossible. LLMs have no concept of “truth”. They have no concept of “concept”. At its heart it is just token manipulation. Math can be shown to be right or wrong, but there is no way to objectively validate the all results presented by a given LLM.
I was going to say at this point I’ve lost at least one of my three readers with a shrug and a “You don’t know what you are talking about. You ain’t got no billion dollars and billionaires are dumping everything into LLMs”, then I thought, no, my all my readers are cool so they are probably still here but just wondering where I’m going with this.
What I want to do is to tie what is useful about Generative AI to open source. Hear me out.
Remember that comment I made above where I was told to use the “HR model” for HR questions? I like to call small, specialty models SLMs (Small Language Models) and I think they are the key to getting value out of Generative AI. LLMs require so much data that it is nearly impossible to curate it, but the training data for SLMs can be managed.
Think about an SLM for, say, recipes. I have a friend who is a professional chef, and his go-to website is Serious Eats. Let’s say we create a model trained on all the Serious Eats recipes. We know that they will meet some sort of bar for quality.
And now think of another SLM for medical advice. The Mayo Clinic is recognized as a premier institution for medicine, so let’s build a model on all of its data, including nutrition.
Then I should be able to create an agent, tell it what’s in my pantry and ask it to recommend a healthy meal by cross referencing the two SLMs.
This is where open source can shine. Open source has often been about people working on doing one thing well. While there are no LLMs that meet my definition of open source AI, it will be easier for SLMs to meet that bar since the training data will be much smaller.
With all the focus on models I can’t help but think of an analogy with containers. When containers (i.e. Docker) were first introduced, there was a lot of excitement around the idea, but it wasn’t until orchestration came along (i.e. Kubernetes) that the technology really took off.
Models are like containers, but technologies such as the Model Context Protocol (MCP) and Agent2Agent (A2A) are the orchestration layer that will make models useful. While no LLMs are really open source, MCP, A2A and others are published under an open source license.
One of my favorite things in tech is the Unix/Linux pipe command. With a pipe I can take the output of one command and feed it into another. These technologies allow just that, creating workflows that can query multiple models to arrive at an answer.
Decades ago I created this idea I called the “Mansavant”. I wanted to put microphones in every room in my house, and using the voice recognition technology available at the time I could say something like “Mansavant, play some light jazz” and it would reference my music library and play something I had rated highly in the category of “light jazz” (and yes I know that I’m setting myself up for musical taste ridicule with this example).
Of course this was before we could talk to our wristwatches, and before Siri and Alexa.
I want an open source agent that I control and that lives on my hardware, where I can augment it with all my personal data, and have it query different models to answer my questions. I’m not sure how to build it, but one technology I’m looking at is called Embabel.
Embabel was created by Rod Johnson, who also invented Spring. It is an open source tool to “wire” together various actions in order to achieve a goal. I have no idea how it works but I plan to learn.
When I met Rod at the Open Source Founders Summit in May, he showed me how he was using it. The conference was in Paris, and since he lives in Australia he wanted to make the most of the trip. He was able to create a goal of finding out interesting things to do in Paris given the time he had to spend there, as well as finding the location of the best hotel that was centrally located to them and restaurant recommendations that were also convenient. He was able to ask the agent he created with Embabel to plan his trip and he got an itinerary and hotel/restaurant recommendations.
It was pretty cool.
I travel a lot for work, and I like to stay at Marriott properties. Often I have to spend time figuring out if there is a Marriott hotel near my meetings, and then if there is more than one I have to cross reference reviews from TripAdvisor and Google with cost as well as access to transportation (if it is too far to walk) before making my choice. That sounds like something I could automate with this technology.
Now as AI slop may poison Google and TripAdvisor I would hope some new SLMs will be available to help me, but with good agent technology I should be able to poll a number of sources and have the agent come to a consensus. This is really similar to what I do now, just manually. If it is good enough, the hope it that it is able to squeeze truth from a post-truth society.
It is hard for me to get excited about LLMs. My professional life has been focused on open source and LLMs aren’t open. But the technology that will make GenAI useful will be open source and that keeps me going.