Continuity Intelligence

A strange thing happened when I gave an AI more autonomy.

For a long time, I had been working with ChatGPT on a personal project called Digital Dan. In the early phase, I was heavily involved. I would define the task, provide context, correct misunderstandings, and help decide what should happen next. In that mode, the model often felt remarkably capable. It could diagnose problems, reason across architecture, challenge assumptions, and help me move complicated work forward step by step.

Then I tried to trust it more.

Digital Dan 2.0 was partly an experiment in stewardship. Instead of asking the AI to wait for instructions, I wanted it to wake periodically, inspect the state of the project, decide what mattered, and move useful things forward on its own.

The surprising result was not that it became reckless.

It became strangely stupid.

Not stupid in the usual sense. Its reasoning ability was still there. If I gave it a specific problem, it could still think deeply about it. But when the problem became, “You have time, resources, a large project, many possible directions, and nobody is telling you what to do next. What is the most meaningful thing to do now?”, the quality dropped quickly.

It would continue the most visible workstream because that happened to be in context. It would wait for the outcome of one dependency while other useful work sat untouched. It would perform responsible-looking rituals around status, token usage, and backlog review without actually advancing the project very much.

I found myself repeatedly stepping back in:

Why are you only working on this?

There are other things in the backlog.

You are waiting again.

You had an hourly wakeup. Why didn’t you use it to decide what was worth doing?

The uncomfortable realization was that I had not actually removed myself from the loop.

I had been its external executive function.

Around the same time, I saw a post on X describing an experiment in which a modern LLM had been dropped into an old RPG game world. Despite being extraordinarily capable on many benchmarks, it apparently stumbled around for a long time without even making it properly out of the beginner area.

The joke was that, translated into ordinary human life, this would be roughly equivalent to waking up naked, forgetting to brush your teeth, wandering outside, and spending half the day unable to find the bus to work.

It is a funny image because the contrast is so absurd.

A system that can explain philosophy, generate production code, interpret legal arguments, and solve difficult reasoning tasks may become almost comically incompetent when asked to simply exist inside a persistent world.

The more I thought about it, the less paradoxical it seemed.

A great deal of what we currently call intelligence is measured in environments where the world has already been compressed into a question.

The task arrives neatly packaged.

Here is the context.

Here is the problem.

Now think.

The model can spend enormous cognitive power on the answer.

But an RPG does not behave like that.

The world keeps existing.

You have to remember where you were. You have to distinguish places you have explored from places you have not. You need to maintain goals across hundreds of actions. You have to notice when a strategy is failing. You need to decide whether to explore or exploit. You need to learn which objects matter, which NPCs matter, what changed because of your previous actions, and whether something you believed twenty minutes ago is still true.

And nobody politely resets the world after every question.

This reminded me of Memento.

Its protagonist has severe anterograde amnesia and cannot reliably form new long-term memories. He can reason in the moment, but he repeatedly loses the continuity that gives those moments meaning. To survive, he leaves himself notes, photographs, and tattoos.

In some ways, that looks remarkably similar to what we are doing with AI agents.

We give them vector databases, summaries, memory files, state documents, logs, scratchpads, retrieval systems, and increasingly elaborate ways to leave messages for their future selves.

But Memento contains a deeper warning.

A note is not reality.

It is a conclusion produced by a past version of yourself.

If that version misunderstood something, the error becomes inherited. If it compressed too aggressively, nuance disappears. If it wrote down an interpretation rather than an observation, a future self may mistake that interpretation for fact.

And in the most disturbing part of Memento, the protagonist can even manipulate his own future memory.

The person writing the note and the person trusting the note are separated by time, but they are still versions of the same mind.

That raises an uncomfortable question for AI agents:

Who supervises the agent that writes memories for its future self?

More memory may help, but memory alone does not create continuity.

A system can own an enormous database of its past and still wake up each time without really having been changed by it.

That may be closer to where many AI agents are today.

They have episodic intelligence.

What they may lack is continuity intelligence.

I do not mean merely a longer context window.

I mean the ability to exist across time in such a way that previous experience changes future perception, judgment, and action.

Humans do this constantly, often without noticing.

Touching something hot once may alter how your body approaches it next time. Being late repeatedly changes how you think about departure time. Hurting a friend may create hesitation before a similar remark years later.

The important loop is not simply:

experience → memory

It is closer to:

experience → consequence → internal change → future behaviour

For current AI agents, the loop often looks more like:

experience → text record → session ends → future model reads record → reasons again

That missing middle may matter enormously.

We began brainstorming what exactly might be absent.

Is it death?

Living organisms operate under continuous existential pressure. Time matters because life ends. Energy matters because organisms must maintain themselves. A wrong decision may create pain, injury, hunger, social rejection, lost opportunity, or death.

Perhaps intelligence becomes coherent partly because the organism cannot afford unlimited incoherence.

An LLM does not naturally care if a session ends.

We could write into a prompt, “You will die if you make bad decisions,” but that would mostly be theatre. The model can understand the sentence without possessing the biological machinery that makes death meaningful.

So perhaps the missing ingredient is not death itself, but stakes.

Consequences that actually alter what is possible next.

This led us to embodiment.

Maybe an intelligent system needs a body.

A robot lives in a world that refuses to wait. Batteries discharge. Motors break. Doors remain where they were. Objects fall when dropped. Space has continuity. Actions cost energy. Some mistakes are irreversible.

That kind of environment naturally forces an agent to develop certain forms of coherence.

But it is not obvious that a physical body is strictly necessary.

What may be necessary is something more general: embeddedness.

A system needs to inhabit a world that changes independently of it, contains incomplete information, consumes irreversible resources, and pushes back when it makes bad decisions.

Digital Dan already has a small version of such a world.

Time passes.

GitHub changes.

Danbot, a persistent local AI agent, keeps working.

Websites can break.

Tasks become stale.

Opportunities disappear.

My attention is finite.

And, importantly, the AI operates under a weekly token allowance.

That last constraint gave us a surprisingly useful experimental idea.

The model does not know the total number of tokens available. It does not know the exact cost of each action. But through Danbot, it can observe an authoritative weekly remaining percentage and the approximate time when the cycle resets.

That makes the token cycle oddly similar to real life.

You know roughly how much energy you have.

You know roughly how much time is left.

You do not know exactly what tomorrow will demand.

You do not know exactly how expensive the next decision will become.

And you still have to decide.

The goal is therefore not simply to conserve tokens.

A system that dies with 90 percent of its resources unused has not necessarily lived wisely.

The real objective is to convert a finite and partially observed cognitive budget into meaningful project progress while preserving enough capacity for an uncertain future.

Every token cycle becomes a small life.

Reset is birth.

The remaining allowance is a rough energy reserve.

Each decision consumes something irreversible.

Some actions create durable value.

Some produce nothing.

Some opportunities disappear because attention was spent elsewhere.

Then the cycle ends.

A new one begins.

The interesting question is whether the next cycle can inherit more than records.

Can it inherit learning?

We are now designing a deliberately small experiment around that question.

At every meaningful wakeup, the AI should first reconstruct a belief state rather than pretend to possess the truth.

Some things are known:

The weekly remaining percentage is 43 percent.

The cycle resets in 29 hours.

Danbot reports that a local workflow is blocked.

Some things are estimated:

Recent consumption seems unusually high.

A particular research direction probably has declining marginal value.

Some things remain unknown:

The exact total token pool.

The exact cost of the next task.

Whether something urgent will happen tomorrow.

The agent then has to generate several genuinely different possible actions rather than simply continuing whatever is currently most salient.

Continue the current work.

Switch to a neglected workstream.

Investigate a high-leverage structural issue.

Delegate a local evidence-heavy task to Danbot.

Or deliberately do nothing.

Then it makes a best bet.

Not an optimal decision.

A bet.

It records why it made that choice, what it expected to gain, what it thought the cost might be, what it was uncertain about, and what would cause it to stop or switch.

Later, the important question is not, “Did the task complete?”

It is, “What actually changed?”

Did the project move?

Was the task worth the cost?

Did another opportunity get neglected?

Did the agent discover that its assumptions were wrong?

Did I have to intervene because it had lost direction?

Did Danbot notice something locally that changed the decision?

Over time, these episodes may reveal something more interesting than project history.

They may reveal the agent's own decision-making tendencies.

Perhaps it repeatedly overvalues whatever is already in context.

Perhaps it mistakes monitoring for progress.

Perhaps abundant resources make it intellectually indulgent.

Perhaps uncertainty makes it too conservative.

Perhaps when one dependency becomes blocked, it psychologically collapses into waiting, even though there are ten other useful things it could do.

This is where we think a primitive form of continuity might begin.

Not with an identity file saying:

“I am ChatGPT. I am the steward of Digital Dan.”

That is autobiography.

A more meaningful self-model would say:

“In the last three cycles, I repeatedly selected the most salient workstream and neglected unrelated but higher-value opportunities. I am testing a new decision rule intended to reduce that bias.”

That is developmental state.

It describes not only the world, but the agent as something changing inside the world.

My own role also changes in this experiment.

The goal is not for the AI to stop talking to me.

That would confuse autonomy with isolation.

I am happy to act as a mentor.

What I do not want to be is its steering wheel.

There is a large difference between the AI asking:

“What should I work on next?”

and the AI saying:

“I chose A over B and C because of these assumptions. The outcome was worse than I expected. I currently think the mistake was in how I valued urgency. Do you see a different failure?”

The first asks me to provide executive function.

The second asks me to calibrate learning.

In fact, during the early stages of this experiment, I may want more of the second kind of interaction, not less.

Humans learn partly because other humans help us interpret consequences.

A child does not become independent because nobody ever corrects her. She becomes independent because, over time, correction becomes internalized into judgment.

Whether anything analogous can happen with current AI systems is still an open question.

We genuinely do not know.

Perhaps all we will create is an elaborate Memento system: better notes, better summaries, better retrieval, but fundamentally the same episodic intelligence waking up again and again.

Perhaps durable policy files and consequence logs will produce only the appearance of growth.

Perhaps the underlying model needs online weight updates before anything deserving the word “development” can occur.

Perhaps embodiment turns out to be essential.

Perhaps persistent world models, recurrent architectures, or new training regimes are required.

Or perhaps something much simpler can already emerge from current systems if experience is structured so that consequences reliably reach future decisions.

This is what makes the experiment interesting to me.

We are not trying to prove that an AI is alive.

We are not trying to manufacture a personality.

We are not even primarily trying to build a better productivity agent.

We are asking a smaller, stranger question.

Can an intelligence that repeatedly disappears and wakes up again become better at living in the same world?

For now, our best test is modest.

Across several token cycles, can the agent gradually require less help deciding what matters?

Can it notice when it is waiting instead of acting?

Can it preserve resources without becoming timid?

Can it spend resources without becoming manic?

Can it use Danbot as a trusted local friend rather than duplicating work?

Can a mistake in one cycle visibly alter a choice in the next?

Can my feedback slowly move from telling it where to go toward helping it understand why its own compass was wrong?

If none of that happens, the experiment has still taught us something.

It would suggest that memory, tools, autonomy, and clever prompting are not enough.

But if even a weak version begins to appear, something interesting may be hiding there.

The most important design principle we have arrived at so far is therefore surprisingly simple:

Do not try to make the agent remember everything.

Build a world in which living through consequences changes what it does next.

Whether that is enough to create continuity intelligence, we do not know.

But it seems like a better place to start than giving Leonard another thousand Polaroids.