By now, your team has probably gotten itself into the software business, even if unintentionally.
Even small and mid-sized businesses have at the very least assigned someone to build them simple automation scripts even if it’s just in Claude Work, ChatGPT Work or Grok Bot.
This often means trusting the software to make the best decisions around what it’s capable of and to let you know if it’s biting off more than it can chew. Or if maybe there’s a better way to tackle it.
If your experience is anything like mine you know these software agents are eager to please, especially if you don’t give them adversarial prompts to try to get them to challenge your assumptions about the best approach.
Coming from a background with a couple of decades of machine learning experience, I have had the unique opportunity of building machine learning apps within these foundational or LLM models. And I have been very surprised that the models will also often tell me that they’re done. But when I go back and ask them questions around what they actually did, they often skipped many of the important steps required by a useful machine learning application.
And it’s not just true with machine learning. Anytime you’re asking the LLM to make reasoned decisions around prioritization, sorting, classification, decisions, math or applying predefined rules, you are asking it to do more than what it was designed for. For our purposes here we’ll call them algorithmic steps.
You see, large language models were made for interpretation first and algorithmic abilities are really a secondary feature.
The mistake companies make is they’ll build their software around a particular LLM and they’ll force it to do these algorithmic steps. But what happens when that model is discontinued, or it’s updated, or it gets too expensive, or for some other reason you need to swap it out? In many cases you’ll have to start from scratch. That’s because many models are not hot swappable, meaning they can’t always replace each other.
In this post we’ll explore the wisdom of separating those algorithmic steps from the interpretation steps in your design. I’ll use as an example a “Pending Items” automation I built for my team.
The AI underneath your software can change quickly
The LLM you are using today might change next week, or tomorrow. Case in point: Replit recently shifted its preferred/default model provider from Anthropic to OpenAI, citing performance, economics, and the strength of the partnership. Replit still supports Claude and other models and describes its approach as model-agnostic.
Of course, Anthropic is operating in a very fast-moving infrastructure market itself, with TechCrunch reporting discussions around another potential $45 billion compute arrangement with Nscale. The point isn’t that building your AI system with Anthropic is uniquely risky. It’s that the foundation-model ecosystem itself changes all the time.
So, if the intelligence layer underneath your software may change, your team should design your software with that potential change in mind.
Don’t put all your business logic inside the LLM
When you’re starting out building systems, it’s tempting to just give the AI model the entire problem and say, “Solve this for me.” You give it raw information, ask it to make decisions, rank the results, and then interpret everything. Feels right at the beginning and often you get pretty decent results from it. Enough prompting, enough evaluation. You’re probably going to get something pretty close to what you were looking for.
The challenge with this is that your business logic, the thinking, and the steps behind the way the software is supposed to make decisions are not mapped out in this way. You’re trusting it to what’s ultimately a probabilistic model, meaning it’s probably going to get it right most of the time but it may find some unusual ways of getting to that result. And it may not even be the most efficient way to do it or the most correct. Also of course, when you leave things like ranking and classification to a model like this, you’re opening yourself up to hallucination and other probabilistic errors.
But even if you do get it fine-tuned, if you change the model you may see a completely different system behavior right out of the shoot.
I have found that a useful design principle is to design for model portability. Don’t expect model uniformity.
My Pending Items Tracker shows another approach
As an example I recently realized that with all my to-do list items often in flux and a lot of them coming through my email, I built a tracker that examined my calendar, my email, my Obsidian notes, and a few other places on a regular basis. It then looked for to-do items and action items and duplicated them so that they were unique. It boiled down to a unique list, sorted by priority. And I even told it to look for evidence that I had completed those items so that it could remove them from the list if needed.
If I had done this a couple of years ago, I would have built it within an LLM from start to finish. Instead I built a system where it starts with code, then feeds into an LLM, and then ends with code. And this is a layered system with each layer doing different types of work.
If you think about it there are several steps involved with this type of system. Deduplication, sorting by date and/or urgency, inference about what might be an action item or not, summarization and rewriting, and presentation.
I tackled this by leveraging various Python modules to handle most of the work, including some of the summarization and deduplication. Long before large language models came into play, we had natural language processing models that could handle a lot of this work. And they do so more accurately because they’re based on vector math not on probabilistic outcomes.
Now my prioritization system emails me daily with the top priorities I need to handle and I can respond to it right away if I’ve already tackled some of them that it may not have known about. I will just list the numbers, like #1, #4, #7, #12, and #14, and it will remove those from the to-do list permanently and then send me a revised list. This daily email helps me tremendously deal with loose ends and open items. And if for some reason the model that I’m using for some of the interpretation gets switched from Claude to Anthropic or among Claude models, for example, I can be certain that it will not materially affect the outcome.
Let software handle what software already knows
In some ways the Python script acts as a preprocessing step. It applies order to the chaos of open-ended emails, calendar entries, and notes from my various knowledge management systems. Language is messy and Python and the algorithms therein can make it smoother and put things into buckets. It can also strip out emails, dates, names, and other data that would otherwise be found in a sea of information that the LLM would have to parse out. The Python can also do things like look for automated messages or messages from other people that don’t apply to my pending list. All of it is rules-based and once the rules are defined it never fails. If I launch something and miss a rule, I just add it a couple days later and suddenly it’s fixed.
Even if I used a complex prompt for an LLM to do this work, there’s really little value in paying the LLM to infer something that a software algorithmic-based application can solve and establish exactly.
But of course the LLM does have its place. For example I have another system I call Gmail Triage that is more rules-based, strictly looking for specific codes in the email that I’ve inserted on purpose. However that triage system, without the benefit of an LLM, might not be able to distinguish between “Can you send this by Friday?” and “Thanks, we’re all set.” By applying the LLM to these kinds of phrases, it earns its keep as an important part of the solution.
LLMs are good at handling meaning. Python’s good at deterministic rules.
But those rules fail at some point. Understanding whether someone requested something, whether I made a promise, whether it appears to have been fulfilled, and what a phrase such as “before the end of the month” actually means in context. By contrast trying to put all the possibilities for saying these things into an algorithmic set of rules would take a very very long time and it would probably still miss some. This is where the LLM really shines.
The LLM adds disproportionate value around: language, ambiguity, context, and inference.
Think of it like this: AI supplies judgment where judgment is needed. Software supplies policy where consistency is needed.
Then take control back from the LLM
Once the LLM has applied its thinking around some of the more ambiguous phrasing, it then passes its result back to the Pythonic code. And that’s where the algorithm can then do its final step of doing things like determining urgency, overdue status, sorting, checking off times, and presentation.
That separation matters because the business policy remains stable even if the model changes.
The business rules don’t change unless you change them. With this sort of a setup there’s no room for fuzzy changes to fuzzy probabilistic changes that might otherwise arise from pure LLM use.
With this in mind, the key architecture becomes:
Code → interpretation by LLM → code
rather than:
Raw information → LLM makes nearly every decision
Make the model powerful, but replaceable
Today my application happens to use Claude. But because the model performs a narrow function and communicates through a structured contract, replacing it with another capable model should affect a small portion of the system rather than requiring me to rebuild the application.
That’s the broader lesson from Replit. The idea isn’t to assume every model is identical. It’s to prevent one model vendor’s behavior from becoming inseparable from your business logic.
The question every company should ask
So what should you do with this information? How does this affect how you instruct your team to build software? I think a good place to start is to ask a question.
For any AI-powered application, ask: If we had to replace the foundation model tomorrow, how much of this system would we have to rebuild?
If the answer is “one interface, some testing, and perhaps a prompt,” you have created a relatively portable AI system.
If the answer is “most of the application’s decision-making,” you may need to reconsider how you structured it. Pull the logic out of the LLM and rely on systems that can faithfully reproduce the results you’re looking for, while still leveraging the magic that LLMs can provide.
To build robust reliable systems, AI automation, and decision systems on your own data, reach out to JLytics today.
