I’ve been working on an open-source project called RPG OS and thought this forum might have useful opinions on the technical side of it.
The basic idea is:
Use an LLM as the engine for a persistent text-based game, but don’t trust the conversation itself to be the only source of state.
The player talks normally:
I don’t trust the main entrance. I climb onto the roof and look for another way in.
The model handles the world and characters, but important state is kept in ordinary files so the game can survive multiple sessions and fresh chats.
Things like:
-
current situation,
-
inventory/resources,
-
NPC relationships,
-
who knows what,
-
injuries,
-
debts and promises,
-
unfinished tasks,
-
historical events,
-
and evidence from earlier play.
The part I found interesting is that generating a convincing scene is easy compared with maintaining a coherent world for a long time.
A model can write something that sounds completely reasonable while quietly changing a fact established 15 sessions earlier.
So I stress-tested it
I ran one continuous synthetic campaign for:
30 condensed sessions / 260 player-GM exchanges.
I deliberately put in long-lived state:
-
conditional payments,
-
restricted funds,
-
unfinished paid work,
-
changing agreements,
-
NPC knowledge boundaries,
-
injuries,
-
disputed ownership,
-
and consequences that only became important much later.
Some of that held up surprisingly well.
Some didn’t.
We found stale state, checkpoint timing problems, old goals that remained active after completion, and other cases where the saved files were structurally valid but semantically wrong.
That led to the next test.
Valid file, wrong reality
I deliberately corrupted copies of the saves.
Examples:
-
remove an unresolved debt,
-
duplicate money,
-
invent an event,
-
give an NPC knowledge they were never told,
-
reactivate an already-settled obligation.
The normal structural validator accepted them.
Nothing was malformed.
The problem was simply that the saved state no longer matched what actually happened.
Comparing the save with the original play transcript could expose those discrepancies.
That eventually led me to split persistence into three separate ideas:
-
State saved through — how far the current state has actually been recorded.
-
History archived through — how far the original play evidence has been preserved.
-
Review covered through — how much of that evidence has actually been compared against the save.
Previously those were too easy to treat as one vague “memory is saved” concept.
It still has a major weakness
The semantic reviewer is still fallible.
The same-model reviewer missed two issues on the first pass in the 30-session test.
So this is not a claim that I solved LLM memory or hallucination.
What I’m trying to do is make failures recoverable and inspectable, instead of letting a wrong summary quietly become permanent truth.
The tools can verify things like:
-
exact source bytes,
-
hashes,
-
save lineage,
-
citation ranges,
-
whether a review belongs to the actual files being saved.
But a human or model still has to understand what the events actually mean.
Why I’m posting here
I suspect people here who run local models, agent workflows, or long-context systems may have much better ideas for this.
I’m particularly interested in:
-
whether you would use plain files at all,
-
whether a database/event log would make more sense,
-
how you’d handle semantic drift,
-
whether a second model should audit the first,
-
how much raw history you would retain,
-
and how you would test this blindly instead of planting errors you already know about.
I’d also be very interested if someone wants to try running it with a local model setup.
The system itself isn’t tied to one model provider, but the environment needs to be able to read and write the campaign files reliably.
The project is free and open source:
https://github.com/croatianrdy2defend-create/RPG-OS
The repository contains the standalone playable files, architecture documentation, and the published 30-session test with transcripts and failures.
I’m not really looking for “nice project” replies.
If the architecture is dumb, overcomplicated, or there’s a much cleaner way to solve the persistence problem, I’d genuinely like to hear it.