Classification All The Way Down // BrXnd Dispatch vol. 029
A few thoughts on the amazing power of LLMs to deal with boring tasks.
You’re getting this email as a subscriber to the BrXnd Dispatch, a (roughly) bi-weekly email at the intersection of brands and AI. I am currently planning two corporate events, so if you’re an AI company or consultant and have an interest in connecting with brands, please be in touch.
In the last Dispatch, I had a little aside about building a project router for Linear, my task management tool. Basically, it takes in a new task and figures out which project it should belong to. I tried to explain why this kind of stuff is, in my view, the perfect use of AI:
But mostly, this kind of stuff is too perfect for AI. While many companies are adding various “magic” creative tasks to their AI (like this weird new Google Docs writer thing), these basic classification tasks are a much more fruitful and simple place to apply the technology. This is something you shouldn’t have to think about, and now, mostly, I don’t have to anymore.
As I was thinking more about it and talking to a few friends, I started to try to come up with a broader list of stuff this technology is particularly excellent at. I don’t mean the stuff where it’s just good or cool, but the places where it is an order of magnitude better than what we’ve had before—the tasks where I’ve just thought, “Huh, well, I guess that’s now solved.”
One of the first versions of this for me was web scraping. I have spent a lot of time building various web scrapers over the years. It’s not a hard problem, it’s just a super annoying one. That’s because, in the past, when you built a scraper, you had to design it for the specific HTML you were scraping. That meant you had to know it was using an <h1> tag as the title and <p> for the body, for instance. But most pages don’t follow the same formats, so you needed to customize your scraper for whatever you were doing. And then, as soon as the page structure changed, you needed a new scraper.
The first time I built a scraper using GPT-3, I realized that all I needed was the text, not the structure, and it could easily make sense of it and return it to me in whatever format I needed. This was amazing and continues to be something I utilize in projects nearly daily. It’s not just that it can classify all this data. It’s that it’s flexible enough to deal with any changes that might arise.
As I was thinking more about the scraping use case and trying to come up with others, I started to realize that fundamentally scraping, at least in the way I’m doing it, is no different than my project assignment task: I’m asking the AI to classify some unstructured data into a structured format. In the case of the scraper, I’m asking it to classify multiple things at once (the title, the body, and some other info), but it’s still fundamentally about classification.
And as I tried to come up with more of these fundamentally game-changing use cases—the things where I can’t imagine a human should ever do that work again—it kind of kept coming back to this. Sure, there is non-classification stuff it’s great at, like summarization, but a place where it continually shines is in figuring out which bucket some arbitrary text should go.
The implication of this, at least for me and the brands I’ve been spending time with, is that the focus should be more on the tedious tasks than the so-called interesting ones. The amount of work within an organization where some person has to file something in the right place—like a campaign getting the correct campaign ID for reporting—is huge. And while so much of the focus is on image or text generation, this is the kind of low-hanging fruit that companies can take advantage of right now without much concern for the legal questions that still surround LLMs.
Anyway, I don’t really have a big conclusion here. I just wanted to share something I’ve been thinking a bunch about.
A few other notes of interest:
Things with Dossify, my calendar briefing tool, are going well. I hope to have something I can invite more people to soon, so if you want to leave your name, you can do that here.
I’ve been spending some time building a series of very basic tools that can take on super simple assistant functions like summarization, task creation, scheduling, unsubscribing, etc. Again, AI is the glue between many bits of code. I’m not quite sure what to do with it yet, but the idea of turning it into an open-source thing that others can contribute new tools is kind of intriguing.
To that end, I keep trying to refute the idea that coding is going away because of AI. Even as new stuff emerges, like Vercel’s v0, which can write UI code that is apparently pretty clean, the core job of programming has really remained the same for me: think about what needs to be built and how it needs to be structured and put it all together. As I’ve said lots of times, these tools have made it so I’m much less intimidated to try new things because I always know I have a partner/coach to take me through it if I run into issues, but I’m still very much guiding the process. I wonder if there’s not an opportunity to put together a little intro to coding course that helps folks get their head around how to start to use these tools in code, where I have found them to be incredibly powerful and easy. If that’s something interesting to you, let me know, and maybe I’ll figure it out.
I recently ran across the term “retrieval augmented generation,” which describes the process of pairing a model with some data source and feeding the data to the model to summarize and respond to a user. It’s a pattern we’ve all probably tried at this point—it’s how the browse with Bing plugin works on ChatGPT—and an approach I’ve used a bunch to build research tools that look up original quotes from user research and feed those to a model to turn into a response to a question. Anyway, I guess that’s what people are calling it. Ivan Kayser, CEO of Redscout, did a session on it at BrXnd NYC.
That’s it for this week. I hope you enjoyed it. As always, feel free to be in touch if there’s anything you want to talk about or I can help with.
— Noah