Playground experiment 01 · context window
How much can an agent keep in mind?
Every AI agent has a context window: everything it can see at once. Pick a task and watch the agent work it out on its own, then redo what a reviewer sends back. Switch a skill on and run the same task fresh, or see what twenty sessions add up to. Every number was measured.
1 Pick a task
2 Skills that fit
Pick a task first. The skills that fit it will pop up here.
⚙︎ How many skills your agent knows
≈0 tokens
of a 200K context window
Each square is 500 tokens · 400 squares fill the window · one session
tokens to keep a one-line summary of 500 skills in view
tokens to load all 500 skills’ instructions at once: 5.7× a 200K window
tokens to an answer a reviewer accepts, across six measured tasks with skills: more on the first try, less on the redo
So that’s why skills scale
A skill is written in layers. The agent always sees the top layer, one short line per skill. It opens the middle layer, the SKILL.md, only when a task matches. And it reads the bottom layer, the reference files, only when a step asks for one. Know fifty skills or five hundred: the agent still only reads what the task needs.
How we measured it. The sizes are real: measured from the public skills on LLM SkillHub, most-downloaded first, and each task offers skills the site’s own search ranks as strong matches. Files are estimated as characters ÷ 4, the usual rule of thumb for English. The work is measured: we ran an agent on all six tasks, alone and as a fresh session with the first skill offered, counted how its context grew at each step, graded both answers blind against five points the skill teaches, and had a reviewer send back what each missed. That redo is part of the run. A second session cost within 5% of the first, so later sessions multiply. A real agent also holds its own instructions and tools (about 44K tokens in our runs), which this leaves out.
🧪 Just for fun, not a benchmark. These are small experiments we ran ourselves to make the idea easy to see: six tasks, one model, one session each way, two of them repeated. Every number here was checked against the runs that produced it, so it is verified, but it is not proof of how skills will behave for you. Your agent, your model and your tasks will differ. Read it as a picture of how skills work, not as a verdict on any of them.