On Non-Determinism // BrXnd Dispatch vol. 35
Hallucinations, brands, brands as hallucinations?
You’re getting this email as a subscriber to the BrXnd Dispatch, a (roughly) weekly email at the intersection of brands and AI. 2024 conference planning is now in full swing. I’m looking for sponsors and speakers (including cool demos!). The initial round of conference invites went out to previous attendees. If you’re interested in attending, feel free to be in touch. As space is limited, I’m prioritizing attendees from brands first. If you work at an agency, just bring a client along. Ok … onto the newsletter.
Carrying on my theme of intuition-building in AI, one of the conversations I’ve been having a lot lately is around the non-determinism of these systems.
What do I mean by that? The canonical example is math. Ask a computer to multiply two numbers like 33,321 x 343,211, and without skipping a beat, it will tell you 11,436,133,731. This is surprising to none of us: we know that computers are overpowered for the job of basic multiplication.
But what happens when you ask an LLM? It’s been well-established that these things aren’t good at math, but to see it in this stark contrast is striking (and first introduced to me by Tim Hwang in his Hallucinations talk at last year’s conference). The AI’s answer (bottom half of the image below) looks right until you dig in a bit and realize that the numbers in the middle are off. It’s not off by that much and would constitute a good guess if I asked you to do the math quickly in your head, but the idea that a computer is guessing answers to math problems is just … weird.
The reason for this is simple. Math is a deterministic process. No matter your method, you will follow a predetermined algorithm and come up with the same answer each time. There is only one correct answer, whether you’re multiplying five numbers or five million. Computer programming works the same way: if/then statements, functions, and loops are all configured to produce a predictable result depending on the input. For instance, a function designed to tell you if an integer is even or odd will always return the correct answer as long as it’s given an integer as input. That last bit is particularly important. If you try to give that function a float (decimal) or a string (word), it won’t know what to do and will throw an error. Determinism requires reasonably tight bounds, and the idea of deciding whether a word is even or odd doesn’t make much sense.
The LLM returns a reasonable-looking but incorrect result because it’s doing none of that. It’s not running a deterministic process to calculate the answer, it’s just predicting what’s most likely to be the answer based on the data on which it has been trained. This predictive process is inherently probabilistic, meaning the model uses statistical methods to guess the most likely output. This means each time the model responds to an input, it has the ability to generate a different response. Telling you 2 + 2 or 10 x 10 isn’t much of an issue for an LLM for the same reason it’s not much of an issue for you—mainly exposure. When I ask you the answer to those math questions, it's almost certain that you’re not doing any calculating either—you’ve just memorized the answer and are reciting it back.
A big thank you to our first 2024 sponsors, Brandguard, the world's #1 AI-powered brand governance platform, and Focaldata, who are combining LLMs with qualitative research in fascinating ways. If you’re interested in sponsoring the 2024 event, please be in touch.
Back to the point at hand, this kind of non-determinism is super weird because we don’t have much to compare it to. We are all used to asking our computers to do basic calculator stuff, but we would never imagine asking it to write a sonnet or design a shoe. The very notion sounds absurd. At the end of the day, AI is good at everything computers are bad at and bad at everything computers are good at.
But this also gets into the topic of hallucinations, something I’ve been thinking about a bunch lately. As you’ve almost certainly heard, hallucinations are what folks call it when the model says something untrue. Here’s Mixtral (running via Ollama) making up some quotes from me:
It’s nice that the model added the caveat at the end, but I never uttered those things. With that said, I might have. The model clearly has a general sense of the topics I speak and write about—marketing, strategy, and data—and it’s not all that hard to imagine that I might have said or written something similar. Similarly, one of my favorite examples from my brand collabs project was how many sneakers the AI imagined that included a swoosh regardless of the brand requested.
In both these cases, the model technically hallucinated by adding a swoosh. But, like the quotes, while they were technically inaccurate, they were perceptually correct. In other words, the mistake the AI was making, in this case Dall-E 2, was not that it imagined some impossible thing, but rather that it drew a connection between Nike and sneakers that is incredibly common amongst the general population. The hallucination wasn’t an act of deceit or even necessarily a mistake, but rather a totally reasonable illustration of American brand associations.
That’s what makes hallucinations such a hard-to-pin-down idea. Some people have argued we should call them lies or confabulations instead of hallucinations, but the problem is that there’s no LLM without the hallucination. “They are dream machines,” Andrej Karpathy wrote on X a few months ago. “We direct their dreams with prompts. The prompts start the dream, and based on the LLM's hazy recollection of its training documents, most of the time the result goes someplace useful. It's only when the dreams go into deemed factually incorrect territory that we label it a ‘hallucination’. It looks like a bug, but it's just the LLM doing what it always does.”
Tools like ChatGPT code interpreter, Bing Copilot (or whatever they’re calling it now), and Perplexity aim to combine the determinism of computing with the indeterminism of AI to lower/eliminate the eventual hallucinations. The approaches are interesting and, in my experience, work pretty well at ensuring the model returns factually accurate information. But I’m not sure we should conceive of them as solving the “problem” of hallucinations because I’m not convinced it’s really a problem.
Beyond that, though, all of this stuff is why I keep finding myself fascinated by this intersection between AI and marketing. In a way, brands are also a hallucination. There’s nothing inherently real about them: they’re just a bundle of associations and ideas held in the minds of consumers about a company and its people, products, and communications. Great brands can better contain those perceptions, ensuring that most people carry around the same conception.
If you have any questions, please be in touch. If you are interested in sponsoring, reach out, and I’ll send you the details (we’ve already got a few sponsors on board).
Thanks for reading,
Noah