Thanks for the writeup. The marker word test is the convincing part, a durable write is not model visibility until the snapshot rebuilds. Compression doing the reload makes that timing depend on context length.
one nuance: in my test the refresh happened at an explicit compression boundary, so I’d call it rebuild-boundary dependent rather than context-length dependent; automatic compression can make context length one trigger.
I also like how your tutorial contrasts this with assembling from live state each turn, that makes the consistency vs. caching trade-off very clear. Thanks for sharing it. I'll check out.
This touches something most agent frameworks quietly sidestep: whether prompt assembly is session-scoped or invocation-scoped. Binding config at session start gives you consistency within a run, but it also means the operator's mental model of the system diverges from the agent's actual behaviour the moment a file is saved. The harder design question is what happens when you do hot-reload mid-session — you need the agent to reason about the discontinuity in its own instructions, and most context-window management today has no mechanism for that.
Exactly. That discontinuity is the interesting part. Hot-reload is not just a cache-invalidation problem, it changes the instruction contract mid-session, so the runtime needs an explicit policy for when new state becomes authoritative and how that transition is recorded.
Thanks for the writeup. The marker word test is the convincing part, a durable write is not model visibility until the snapshot rebuilds. Compression doing the reload makes that timing depend on context length.
I turned these concepts into a runnable tutorial using Python, now with 500+ GitHub stars: https://github.com/hardness1020/awesome-agent-architecture/tree/main/sections/10-system-prompt
yep, on durable write ≠ model visibility.
one nuance: in my test the refresh happened at an explicit compression boundary, so I’d call it rebuild-boundary dependent rather than context-length dependent; automatic compression can make context length one trigger.
I also like how your tutorial contrasts this with assembling from live state each turn, that makes the consistency vs. caching trade-off very clear. Thanks for sharing it. I'll check out.
This touches something most agent frameworks quietly sidestep: whether prompt assembly is session-scoped or invocation-scoped. Binding config at session start gives you consistency within a run, but it also means the operator's mental model of the system diverges from the agent's actual behaviour the moment a file is saved. The harder design question is what happens when you do hot-reload mid-session — you need the agent to reason about the discontinuity in its own instructions, and most context-window management today has no mechanism for that.
Exactly. That discontinuity is the interesting part. Hot-reload is not just a cache-invalidation problem, it changes the instruction contract mid-session, so the runtime needs an explicit policy for when new state becomes authoritative and how that transition is recorded.