I changed Hermes’s USER.md while a session was still running.
The file changed immediately.
The next provider-bound request did not.
It still contained the old user profile.
Then I compressed the session.
Same session ID. New system prompt. The updated profile finally appeared.
That small experiment exposed the boundary I wanted to understand:
A file on disk is not the context in the current turn.
To prove this without relying on model interpretation, I used meaningless marker words in an isolated test profile. AMBER meant “old profile.” COBALT meant “new profile.” Similar markers let me trace project context and skills through the runtime.
The markers were instrumentation, not the point.
The point is simpler:
Hermes stores several kinds of state, but the model sees only the representation assembled for the current call.
A durable file can change without changing the active prompt. A project rule can enter through conversation history instead of the system prompt. A complete session record can differ from what reaches the next model request. A prompt can remain byte-identical while provider-cache behavior changes independently.
That is what this post is about.
Tested against Hermes Agent v0.20.1, using an isolated Hermes profile and a real provider-backed
openai-codex / gpt-5.6-solrun. I captured 22 provider calls across 16 user turns and two sessions. The companion repository contains the fixtures and sanitized evidence for readers who want to reproduce or challenge the results.
If you want the raw experiment rather than just the architecture, the Part 2 companion lab includes the disposable fixtures, scripts, reference receipts.
The 60-second mental model
Think of one durable value, such as something stored in USER.md, moving through four different states:
durable source → cached prompt snapshot → selected live history → provider-bound request
Those objects are related, but they are not interchangeable.
A file can change without changing the cached prompt.
Conversation history can change without rebuilding that prompt.
The database can preserve more information than the next model call receives.
And an unchanged prompt does not prove that the provider served it from cache.
Once you separate those states, a lot of “agent memory” behavior stops looking mysterious.
The Prompt Is Built, Not Discovered by the Model
Hermes does not ask the model to discover its operating context from scratch on every turn.
When a system prompt is built, the v0.20.1 prompt assembly implementation joins three ordered tiers:
Stable
Context
Volatile
Those names are about lifecycle and cache locality.
They are not model-enforced authority levels.
The stable tier contains relatively reusable identity and operating guidance, such as SOUL.md, tool and model guidance, environment information, and platform hints.
The context tier carries workspace-scoped material, including caller instructions, startup project context, and workspace state.
The volatile tier contains things that vary more often across sessions, including the skills index, MEMORY.md, USER.md, external-memory context, plugin prompt sections, and runtime identity such as session, model, provider, and timestamp.
One easy mistake is reading “volatile” as “recomputed every turn.”
It is not.
The complete system prompt is cached on the agent and reused through ordinary turns. A compatible resumed session can also restore the persisted prompt snapshot rather than rereading every source file.
If you remember one picture, make it this one:
This is the ownership model behind the article.
Durable files live outside the model.
The prompt builder turns some of them into a cached snapshot.
Tool activity can add context later through live history.
Request assembly chooses the model-visible representation for the next call.
The provider sees that assembled request, not “everything Hermes knows.”
A compact ownership table makes the distinction clearer:
The important idea in that table is lifecycle.
The same piece of information can exist durably in one place, be copied into a session snapshot at another point, and become model-visible only when request assembly includes it.
That is why “context” is not one bag of text. USER.md, a nested AGENTS.md, a loaded skill, session history, and provider-cache state all have different sources of truth, load boundaries, and refresh rules.
Ownership shows up in who controls those transitions. Durable files hold the underlying content. The Hermes runtime owns prompt assembly and the live session representation. The session store owns persisted history. The context engine owns how history is transformed during compression. The model provider owns its own cache state.
Once you separate those responsibilities, the behavior I saw with USER.md becomes much easier to explain.
A Snapshot Is Not a Live View
The USER.md experiment made that visible.
I started Session A with the old test value in the user profile.
The provider-bound system prompt contained that old value.
While the session remained open, I edited USER.md to a new value.
Then I sent another ordinary turn and prevented Hermes from reading the file through a filesystem tool.
The next provider request still contained the old profile.
The system-prompt digest had not changed.
I then repeated the experiment in reverse.
A fresh Session B started with the new profile value and correctly loaded it.
While Session B remained open, I changed the file back to the old value.
The next ordinary request still carried the value from Session B’s original prompt snapshot.
Two sessions. Two directions. Same result.
Hermes was not treating USER.md as a live database lookup.
It was treating it as an input to a session-level prompt snapshot.
That choice has useful properties.
If every external file write immediately mutated the prompt, one session’s system message could change beneath an in-flight request. Prompt bytes would also churn more often, which matters for reproducibility and caching.
A snapshot creates a cleaner lifecycle.
But it creates a contract the operator needs to understand:
writing durable state is not the same thing as making that state model-visible.
A fresh prompt build is the reliable general rehydration boundary.
Compression has a narrower refresh behavior.
When I compressed Session A, Hermes reloaded its built-in memory and user-profile state. The newly loaded USER.md no longer matched the profile contained in the cached prompt, so Hermes rebuilt the prompt.
The next provider request now contained the updated profile.
The session ID did not change.
That does not mean compression rereads and validates every durable prompt source. It should not be described as a universal refresh of SOUL.md, project context, skills, plugins, and configuration.
It is a specific lifecycle boundary with specific reload behavior.
That distinction matters for any agent runtime exposing durable writes.
A good write receipt should answer two separate questions:
Did the durable source change?
When will that change reach the model?
Project Context Arrives Through Two Paths
Project instructions have another split lifecycle.
The first path runs when the prompt is constructed.
At v0.20.1, the context-file loader checks supported project-context types in priority order:
.hermes.mdorHERMES.mdAGENTS.mdCLAUDE.mdCursor rules
The first supported type with usable content wins.
For AGENTS.md, startup loading can construct a chain from the Git root toward the current working directory, letting broad repository guidance appear before more local guidance.
Hermes can deterministically choose those sources and place their text in a deterministic order.
That does not make the resulting natural-language instructions a deterministic authorization system.
Prompt order is architecture, not a policy engine.
The second context path is more interesting.
Hermes discovers some nested project guidance progressively.
The subdirectory tracker watches path-bearing tool calls. When execution reaches a new area of the workspace, Hermes can discover relevant local context there and append it to the tool result.
That is what happened in my test repository.
The root AGENTS.md rule was present in the startup system prompt.
The nested backend/AGENTS.md rule was not.
Then Hermes read backend/probe.txt.
The resulting read_file output was enriched with the nested backend rule, and that enriched result was persisted into history.
The cached system prompt did not change.
I reproduced that behavior in two fresh sessions.
This is a useful architecture pattern.
A large repository can have specific instructions close to the code they govern without putting every local rule into every session’s fixed prompt.
But it creates another debugging boundary.
If the model misses a local rule, the question is not simply:
Did Hermes load
AGENTS.md?
The better questions are:
Which
AGENTS.md?Was it startup context or progressively discovered context?
Which tool access caused it to become relevant?
Did it enter the system prompt or conversation history?
Is that history still part of the next model request?
That is a much more useful 3:00 AM checklist.
A Skills Index Is Not the Skill
Skills use a similar progressive-disclosure pattern.
I created one synthetic test skill with:
a distinctive marker in its description
another distinctive marker only in the full
SKILL.mdbody
At session start, the provider-bound prompt already contained the description marker.
The full-body marker was absent.
Then I explicitly loaded the skill.
The full body appeared through the tool result and entered conversation history.
The system prompt remained unchanged.
So saying “this skill is in context” is ambiguous.
Before loading, the model has skills index metadata.
After loading, it has the full procedure as part of live history.
Those two representations have different token costs and different lifecycles.
The live run also uncovered a small observability bug.
/context all reported no separate skills attribution even though the exact system-prompt bytes contained the skills index and history contained the loaded skill.
The v0.20.1 context-breakdown implementation explains the mismatch: the prompt builder places the skills index in one tier while that breakdown path looks for it in another.
The model still had the data.
The inspector categorized it incorrectly.
That is another useful distinction:
an observability view is not automatically the source of truth for what the provider received.
Under larger context pressure, loaded skill content can also be reduced and later reloaded from the durable skill source. I verified that behavior through the test suite rather than forcing the live session large enough to reproduce it manually.
Subagents introduce one more boundary worth mentioning.
At this release, a delegated child receives a separately constructed prompt and explicit task context rather than simply inheriting the parent’s complete project prompt, profile, and transcript.
The details belong in Part 4.
The Part 2 takeaway is enough:
Shared runtime does not imply shared context.
A Cached Prompt Object Is Not a Provider Cache Hit
This is another place where one word hides several mechanisms.
Hermes has a cached system-prompt object.
A model provider may have provider-side prompt caching.
A local inference server may have a KV or prefix cache.
Those are not the same cache.
A cached prompt object is not a provider cache hit.
The live experiment let me separate them.
Fresh-session requests on my Codex route reported zero cached input tokens.
Repeated requests commonly reported 15,872 cached tokens.
Those values came from the provider’s usage metadata.
That is evidence about provider caching.
The system-prompt digest answered a different question.
When progressive backend context entered through history while the prompt digest remained unchanged, I could conclude that Hermes had not rebuilt the system prompt.
I could not conclude from that digest alone that the provider used its cache.
Likewise, I did not verify whether a local inference-level KV cache existed in this environment.
So if I were instrumenting an agent runtime, I would not expose a single metric named cache_state.
I would keep at least these separate:
prompt digest
selected-history digest
model and provider
session identity
provider cache-read tokens
provider cache-write tokens
compression count
Same word. Different owner. Different evidence.
Compression Is a State Transition
Long-running sessions eventually create pressure.
The obvious problem is context-window size.
But there are two other costs: resending more input on every call, and asking the model to reason through growing amounts of stale or low-value history.
Hermes handles that through an explicit compression lifecycle.
The manual path I tested, /compress here 2, is particularly useful because it makes ownership visible.
The CLI first separates older history from a selected recent tail.
Only the older head goes to the compressor.
The compressor can prune large older tool results, preserve its required internal boundaries, and summarize that older portion.
Once that compacted head returns, Hermes reloads the built-in user profile and memory, decides whether the system prompt can be retained or needs rebuilding, commits the compacted head, and rejoins the selected recent tail in live memory.
Here is the observed sequence:
The most interesting finding was not that summarization occurred.
It was what happened to the recent tail.
Before compression, the session database contained 28 active rows.
Immediately afterward, it contained 12 active rows and 28 rows marked compacted.
Yet recent synthetic facts from the selected tail were still present in live memory and reached the next provider request.
Their original SQLite rows were not reinserted as active rows.
They remained compacted and recoverable.
Nothing was lost.
But the live model-facing representation and the database’s active-row view were not identical.
The model-visible live history after compression was not identical to the SQLite active-history view.
That is the kind of detail that gets lost when every state surface is casually called “the conversation.”
A session can have:
durable original history
active persisted history
compacted history
live in-memory history
selected provider-bound history
Those representations are related, but they do not have to be identical.
The compression boundary also refreshed the user-profile snapshot.
Before compression, the current system prompt contained the old USER.md value.
After compression, it contained the updated value.
The session ID stayed the same.
Compression is a lossy state transition, not housekeeping.
It is also not durable memory.
A compression summary is a smaller representation of conversation history.
MEMORY.md is separately persisted state intended for later reuse.
They may both contain summaries of the past, but they have different owners and different contracts.
The failure behavior reinforces that point. The implementation and targeted tests cover both abort and fallback behavior when summary generation fails. Whether the runtime preserves the existing representation or continues with a fallback is an explicit policy decision.
Forward progress and fidelity are different guarantees.
Where It Breaks
The session starts in the wrong workspace
If Hermes starts from the wrong working directory or Git root, it can assemble perfectly valid project context for the wrong project.
The model may then appear to “ignore instructions” when the actual problem is earlier: the runtime selected a different context source.
At 3:00 AM, inspect the session CWD, Git root, startup context selection, and path-bearing tool calls before debugging prompt compliance.
The design lesson is simple: workspace identity is session state.
The durable file changed, but the prompt did not
This was the failure I reproduced directly.
USER.md contained the updated profile, while the next provider request still carried the old snapshot.
Both states were correct within their own lifecycle.
The operational mistake would be assuming that a successful durable write also means immediate model visibility.
Write APIs should make that distinction explicit.
The model saw recent context that SQLite did not mark active
After /compress here 2, the selected recent tail survived in live memory and reached the next provider request.
Its original SQLite rows remained compacted rather than active.
An operator looking only at active database rows could incorrectly diagnose data loss.
The right inspection surface is not just the session database.
Compare persisted history, live selected history, and the actual provider request.
A loaded skill can become a reload pointer
Loaded skill content is not the durable source of truth.
The skill file is.
Under context pressure, Hermes can reduce older loaded skill content and retain a path for reloading the procedure.
That is a good general pattern: if context is lossy, preserve a deterministic route back to the authoritative artifact.
Summary failure forces a fidelity decision
If summarization fails, a runtime has to choose.
Preserve everything and abort compaction?
Or continue using a smaller fallback representation?
Hermes has explicit behavior for those cases.
Agent builders should too.
“Compression failed” is not enough of an operational contract.
What I Would Steal
1. Give every context source a lifecycle
For each source, define:
who owns it
where the source of truth lives
when it is loaded
where it enters model-visible context
when it refreshes
how an operator can inspect it
The invariant is:
A durable write is not model-visible until the runtime copies or retrieves it into a request.
That one rule eliminates a surprising amount of fuzzy “memory” language.
2. Separate the prompt snapshot from dynamic context
Stable identity and operating guidance do not need to churn every time a nested file becomes relevant.
Progressive context can enter through another channel.
That helps keep the prompt stable while still letting the model receive local information when execution reaches it.
The cost is more lifecycle complexity.
That is acceptable if the transitions are observable.
3. Discover project specificity when it becomes relevant
Loading every local instruction from a large repository into every system prompt does not scale well.
Progressive discovery is a useful alternative.
Start broad.
Add specificity when the runtime reaches that part of the workspace.
But always preserve evidence explaining why that context was selected.
4. Treat compaction as a real state transition
Compaction deserves the same engineering discipline as any other state mutation.
It needs:
a trigger
protected boundaries
a clear commit point
recovery
observability
a definition of what the next model request will contain
If persisted active history and live request history can differ, make both visible.
Otherwise, debugging a long-running agent eventually becomes guesswork.
Run the Companion Lab
This article is designed to stand on its own.
If you want to inspect the receipts or reproduce the experiment, the Part 2 companion lab contains the synthetic fixtures, platform setup scripts, reference results, and submission format.
A useful reproduction should capture small, sanitized evidence such as:
which profile value reached the provider request
whether the system-prompt digest changed
where nested project context appeared
where the full skill body appeared
session ID before and after compression
active and compacted row counts
provider-reported cache metrics
Do not submit full prompts, credentials, personal memory, or private session data.
A result that disagrees with mine is more interesting than another unverified PASS.
Recap
Hermes does not send “everything it knows” to the model.
It sends an assembled representation for one call.
A durable profile file can change while the current prompt remains stale.
A root project rule can live in the system prompt while a nested rule arrives later through tool history.
A skill can exist on disk while only its index metadata is initially visible.
A complete session record can differ from the live history sent to the next model request.
A cached prompt can remain unchanged while provider-cache behavior has its own independent state.
The common mistake is collapsing all of this into one word:
context.
The more useful questions are:
Who owns this state?
When was it loaded?
Where did it enter the model request?
What event refreshes it?
And what evidence would prove that at 3:00 AM?
Part 1: Gateway, Sessions, and the Agent Loop traced how one message becomes a stateful run.
Part 3 will follow durable state in the opposite direction.
It will separate facts, user preferences, and reusable procedures, then look at how memory and skill writes are created, corrected, and reused across future work.
Version Note
Tested against Hermes Agent v0.20.1, using an isolated Hermes profile and a real provider-backed openai-codex / gpt-5.6-sol run.
The experiment covered two sessions and 22 provider calls. The article incorporates the one state-boundary mismatch found during validation: after manual compression, selected recent context remained model-visible in live memory even though its original SQLite rows were compacted rather than active.
Full fixtures, validation receipts, cross-platform instructions, and the claim matrix are in the Part 2 companion lab.
References
Hermes Agent Architecture
Part 2: Prompt Assembly, Context Files, and Compression
Part 3: Memory, Skills, and the Self-Improvement Loop
Part 4: Tools, Plugins, Delegation, and Persistent Work
Part 5: Security Boundaries, Profiles, and Safe Deployment
Run the companion lab and submit a sanitized result if you want to compare behavior.
When your result differs, which boundary changed first: durable state, prompt snapshot, live history, persistence, or provider-cache behavior?
Subscribe to The Agent Stack for Part 3, where I will trace how facts, preferences, and reusable procedures become durable state.
Thanks for the writeup. The marker word test is the convincing part, a durable write is not model visibility until the snapshot rebuilds. Compression doing the reload makes that timing depend on context length.
I turned these concepts into a runnable tutorial using Python, now with 500+ GitHub stars: https://github.com/hardness1020/awesome-agent-architecture/tree/main/sections/10-system-prompt
This touches something most agent frameworks quietly sidestep: whether prompt assembly is session-scoped or invocation-scoped. Binding config at session start gives you consistency within a run, but it also means the operator's mental model of the system diverges from the agent's actual behaviour the moment a file is saved. The harder design question is what happens when you do hot-reload mid-session — you need the agent to reason about the discontinuity in its own instructions, and most context-window management today has no mechanism for that.