Soundings
Our Perspectives and Insights
Frankenstein's AI Agent: What Changes When You Swap the Parts
I want to do a thought experiment fit for the Halloween season: a modern-day Ship of Theseus argument for AI agents.
Plutarch tells the original version. The Athenians kept the ship Theseus sailed home from Crete, and they kept it for generations, pulling each plank as it rotted, one by one, and fitting new timber in its place. Philosophers have argued ever since about whether it was still the same ship once every plank had been replaced.
Now run the thought experiment on an AI agent. Swap the model underneath it. Give it new tools. Rewrite its instructions. Wipe its memory. At what point is it a different agent? And for the darker side of the season: what happens when you bolt on new parts and nobody checks what you built?
This won’t stay hypothetical for long. Agents get rebuilt constantly. The model gets upgraded, a tool gets added, someone tweaks the prompt on a Friday afternoon. Most teams log each of those as maintenance. Each one changes what the agent is, and each one needs to be tested like a change.
I had a funny run-in with this myself. Anthropic made a global change, and all of a sudden our Claude agents started writing with British spellings on everything: lots of u’s, and no z’s. We had to make changes to fix it and get them back to American English.
What Makes an Agent More Than a Model?
Two parts: its instructions and its memory.
Without them, you have a large language model processing whatever prompt shows up. The instructions (the master prompt) give the agent its purpose: what it’s for, and what it should and shouldn’t do. Memory carries what it has learned from one session to the next: the context, the decisions, the corrections. Together they give it direction, and they’re the core that binds everything else.
Swap almost anything else and that purpose survives. A new model changes how the agent processes the work. New tools change what it can reach. Neither changes what it’s trying to do.
But the core only sets the direction. It doesn’t hold the line. As I wrote in Everything Is Permissible, a rule typed into a prompt is a weighted vote that a stronger goal can outweigh. Holding the line is the fence’s job: the tools the agent is given, the permissions behind them, and the tests that run every time something changes.
Six Parts You Can Swap, and How to Test Each One
Set aside whether the result is still the same agent. The useful question is what each swap changes, and how you’d know.
- The instructions. Change them and you’ve changed the agent’s purpose. Rerun the scenarios that define its job, and if the purpose moved on purpose, version it as a new agent.
- The memory. Wipe it and you’ve turned a seasoned colleague into a new hire with the same job description. Check that what it remembers is still true and still its own, and decide deliberately what carries forward.
- The model. This changes how the agent thinks, not what it’s for, but the same instructions read by a new model don’t always mean the same thing. Run the same test set before and after, and treat an upgrade as a change even when it’s better.
- The tools. These are the arms and legs. A new tool lets the agent do things it couldn’t before, and a removed one quietly breaks things it used to do. Test the new reach and the failure cases, because whatever it can’t reach, it can’t break.
- The permissions, and whose name is on them. This is the fence, and it sets the blast radius for everything above. Confirm it still holds after every other swap, and that one person still owns what the agent is allowed to do.
- The harness. This is the loop around the model that decides when to act, when to retry, and when to stop and ask. Same model and same instructions with a different harness will behave differently, so watch the edge cases: when it stops, when it escalates, and when it keeps going.
The core only sets the direction. It doesn’t hold the line.
How Frankenstein's Creature Became a Monster
Mary Shelley’s creature wasn’t born a monster. Victor Frankenstein assembled it from parts, brought it to life, and then ran from it: no name, no guidance, and nobody responsible for what it did next. The creature puts it plainly to his maker: “I was benevolent and good; misery made me a fiend.”
An agent doesn’t feel misery. The mechanism still holds, though: parts bolted together, switched on, and left without a core to give it direction or a fence to contain it. The erratic behavior that follows comes from how it was built and then left alone, not from some flaw in its nature.
That’s what the guardrails are for. The core keeps the purpose steady from one swap to the next. The fence keeps a bad swap from turning into something monstrous before a test catches it.
Whatever it can’t reach, it can’t break.
The Bottom Line
Keep swapping planks and, hypothetically, you end up with the Ship of Theseus: every part replaced, still afloat, and a fair argument about whether it’s the same ship. Swap the parts without testing or validating any of them, and you end up with Frankenstein’s monster.
Whether it’s still the same agent is a question for the philosophers. Whether you tested it is a question for you.
A Pilot & Rutter principle.