💡 An agent that forgets every run is a contractor with no notebook. An agent that dumps every trace into the next prompt is a contractor with a junk drawer. WikiSkill is the claim that you need a compiler in between: immutable traces, a compounding wiki, and skills you can execute — and roll back.

🧱 The missing compiler

Agent skills are already a product surface: a directory with SKILL.md, scripts, and progressive disclosure so the model only loads what it needs [3]. Most of them are still hand-authored. The 2026 skill-evolution papers — EvoSkill, Trace2Skill, SkillOpt [4][5][6] — close that gap with the same outer loop: roll out, analyze successes and failures, patch the skill, gate on validation. Fine. The bug is where the learning lives.

Those methods keep insights scattered across optimizer histories, rejected diffs, and last week’s traces. That is Slack-as-a-wiki. The next proposer cannot see the postmortem. It re-proposes the thing you already rejected. WikiSkill’s move is to insert a persistent knowledge layer between raw experience and executable procedure, then let that layer compound across iterations while skills stay gated.

Karpathy’s framing is the right one: compile experience into knowledge that pays rent next week, not a longer context window that pays rent until the next compaction [2]. Horizontal progress is copying last run’s prompt. Vertical progress is a compiler. See Zero to One if that split already lives in your head.

📚 Three layers, one workspace

WikiSkill splits the agent workspace the way a grown-up platform splits logs, runbooks, and deployable code [1]:

• 📦 Raw layer (raw/) — immutable execution traces. Reasoning, tool calls, outputs, answers. You do not edit history. You compile it.
• 📚 Wiki layer (wiki/) — pattern pages, an evolution log, and a skill-impact tracker. Failure modes, successful strategies, rejected diffs, acceptance outcomes. This layer is never rolled back.
• 🎒 Skills layer (skills/) — the only thing the inference agent is allowed to read. SKILL.md is the procedure. PURPOSE.md maps the skill back to the wiki patterns that justified it. Provenance, not vibes.

flowchart TB
  RAW["📦 Raw layer
immutable traces"] --> WIKI["📚 Wiki layer
patterns · logs · impact"]
  WIKI --> SKILL["🎒 Skills layer
SKILL.md + PURPOSE.md"]
  SKILL --> INF["🤖 Inference agent"]
  INF --> RAW
            

Translate that onto the contracts I already use for agents. Traces are evidence. The wiki is the compiled postmortem. Skills are policy. Mixing those three in one prompt is how you get a god-prompt with a README. I said this in Agentic AI Fundamentals: memory is evidence, never policy. WikiSkill is that sentence with a filesystem.

🔁 The loop — and the one-way wiki

Each iteration has four jobs [1]:

1. 🤖 Inference agent rolls out on training tasks with the current skills injected. It cannot see the wiki. That restriction is the point, not a bug — more below.
2. 📝 Wiki Maintainer samples traces (they cap at ~8 per iteration: more fails than passes), writes or patches pattern pages, updates the index, appends the evolution log.
3. 🛠️ Skill Proposer is a ReAct agent [7]. It is not force-fed a dump of traces. It reads wiki/index.md, skill-impact.md, then picks which pattern pages and traces to open. One atomic proposal per iteration: create a skill or patch one.
4. 🧪 Gating scores the candidate on a held-out validation split. Strict improvement only. If the score drops or ties, the skill set rolls back. The wiki stays. The rejected diff is logged so the next proposer does not retry the same bad idea.

flowchart LR
  I["🤖 Rollout"] --> W["📝 Wiki maintainer"]
  W --> P["🛠️ Propose skill"]
  P --> G{"🧪 Val better?"}
  G -->|yes| A["✅ Keep skill"]
  G -->|no| R["↩️ Roll back skill"]
  A --> L["📜 Log to wiki"]
  R --> L
  L --> I
            

That last sentence is the whole paper, dressed as an algorithm. Skills are a release. The wiki is a ledger. You can revert the deployable. You do not revert the postmortem. If you have ever shipped a model behind a canary and kept the incident doc after the rollback, you already know this shape — see MLOps Deployment Strategies.

Their ALFWorld case study makes it concrete. Iteration 0 proposes a vague “goal-directed action” skill. Validation rejects it. skill-impact.md keeps the diff. Iteration 1, informed by that rejection, ships Never return an item to its origin location and it lands. Later, new loop variants show up in the wiki, and the skill gets a tighter rule: each operation type once per item. That is not “the model got smarter.” That is a compiler that did not forget.

🚫 Do not let the worker read the wiki

The ablation is the part I would tattoo on an on-call runbook. With Gemini-3.5-Flash, giving the Skill Proposer wiki access is worth about +15 points average across their benchmarks. Giving the Inference Agent wiki access during training rollouts hurts the skills you end up with (LiveMath 72.6% → 64.8%) [1].

Their hypothesis matches production intuition: if the worker can cheat by reading the wiki, the traces stop teaching you what the skill file is missing. You optimize a system that will not exist at serving time — unless you also ship the wiki into production, which is how you recreate the god-prompt you were trying to kill.

Map:

• 📚 Wiki → compiler / reviewer / postmortem author
• 🎒 Skills → what the worker is allowed to load
• 📦 Traces → evidence the compiler reads, not a second policy channel

Same rule as long-term memory: retrieved facts are evidence. Security rules, stop conditions, and tool permissions do not get learned from a conversation, and they do not get learned from a wiki page the model wrote about itself. If a pattern is ready to govern behavior, it graduates into a skill, gets gated, and gets a version. Until then it is a note.

📈 Skills are a scaling law, not a participation trophy

WikiSkill beats the other skill-evolution methods on average across five models and five benchmarks — math, search, spreadsheets, long-context docs, embodied household tasks. The interesting curve is not “we won the table.” It is who benefits [1].

• 📏 Inside Qwen, WikiSkill’s average lift grows with scale: +12.3 / +17.5 / +23.9 points at 4B / 9B / 27B. Stronger models write better skills and follow them.
• ⚖️ Skills can buy you scale you did not pay for: Qwen-3.5-9B with WikiSkill (47.4%) beats Qwen-3.6-27B without skills (39.4%).
• 📊 The lift is not uniform. Spreadsheets and ALFWorld move a lot; long-context OfficeQA is stingy, and the 4B model actually gets slightly worse — it cannot execute the multi-step search workflow the skill describes, so it reverts to “just read the document.”

That last point is the one vendors will skip in the tweet. A skill is not a spell. It is a procedure the inference model has to be strong enough to run. If your 4B tool-caller cannot follow a four-step grep plan across a Treasury bulletin, evolving a prettier SKILL.md will not save you. Climb the agent ladder only when the rung below fails a measured eval — same rule as Agentic AI Fundamentals.

🔀 Discovery and execution are different jobs

The transfer table is the product insight. Skills evolved by one model often help another — sometimes more than the target’s self-evolved skill. Qwen-3.6-27B skills take Qwen-3.5-9B from 63.4% → 70.2% on ALFWorld. Qwen-3.5-4B skills jump Gemma-4-31B from 33.9% → 73.1% on LiveMath. Gemini skills help Qwen-27B on SealQA more than Qwen helped itself [1].

flowchart LR
  D["🔬 Discovery model
writes the skill"] --> S["🎒 Skill artifact"]
  S --> E["⚡ Execution model
runs the skill"]
            

Self-evolution conflates two capabilities:

• 🔬 Discovering a procedure from traces — diagnosis, naming the pattern, writing a rule.
• ⚡ Executing that procedure at inference time — following the rule under a long context, a tool budget, a noisy environment.

Those do not have to be the same model, the same family, or the same price. That is a platform shape: a stronger (or cheaper, or overnight) compiler writes skills; the serving fleet runs them. It is also how you should think about Cursor / Claude skills and internal playbooks. The author of the runbook is not the on-call who executes it at 2am.

Transfer is not free. Qwen-3.5-4B spreadsheet skills destroy Gemini-3.5-Flash: 50.5% → 18.1%. The 4B skill encodes crutches — single-line Python, string conversion rules, fragmented diagnostics — that help a small model not fall over, and that prevent a strong model from writing an end-to-end script before its interaction budget dies. General procedures transfer. Model-specific workarounds are tech debt. Label them that way in PURPOSE.md or you will ship a 4B hack into a 27B prod path and call it “knowledge reuse.”

🏭 What I would actually ship

I would not paste WikiSkill’s research harness into production. I would steal the compiler shape and put it behind the LLMOps contracts I already want:

flowchart TB
  T["📡 Replayable traces"] --> W["📚 Wiki / postmortems"]
  W --> P["🛠️ Propose skill patch"]
  P --> EV["🧪 Eval gate"]
  EV -->|pass| REL["📦 Versioned skill release"]
  EV -->|fail| RB["↩️ Rollback skill"]
  RB --> W
  REL --> REG["🔎 Skill registry"]
  REG --> AG["🤖 Serving agent"]
            

Where I would not copy the paper:

• 📥 Full skill injection. They dump every active skill into the system prompt so retrieval cannot confound the study. Correct for a paper. Fatal for a library of fifty skills. Keep classify → search registry → permission filter → load, as in Agentic AI Fundamentals.
• 🧹 No wiki pruning. They admit it. Permanent memory is a liability portfolio: stale patterns, poisoned procedures, prompt bloat, cross-tenant leakage. Retention, review, and deletion are not optional just because a markdown folder feels harmless.
• 📏 Strict > on a tiny val set. Neutral patches that unlock the next iteration never land. Small val splits make the gate noisy. In production I want: bootstrap the gate, require a minimum delta or a human for high-impact procedural writes, and never let the model promote a skill that changes permissions.
• 🧑‍⚖️ Unreviewed procedural memory. If a bad workaround gets written as a pattern and then compiled into a skill, you have automated the incident. High-impact wiki pages (anything that sounds like policy) get a human, the same way irreversible tool calls do.

The honest limitation they flag — no long-horizon online adaptation inside a single multi-hour rollout — is the next compiler, not this one. WikiSkill is offline skill evolution between batches of tasks. Online skill patches mid-incident are a different blast radius. Do not conflate them.

🪙 My 2 cents

Stop asking “should the agent remember?” Ask the questions that force a compiler:

• 📦 What is immutable evidence vs compiled knowledge vs executable policy?
• 🚫 Does the worker ever see the wiki, or only the skill?
• 🧪 What eval fails the skill release?
• 🔀 Who writes the skill, and who runs it?
• 🧹 When does a pattern expire, get pruned, or get promoted?
• 🔐 Which skill loads are permission-scoped, signed, and retrieved — not dumped?

Answer those and the shape almost picks itself:

• 📡 Traces you can replay
• 📚 Wiki the compiler reads and a human can audit
• 🎒 Skills the worker loads on demand
• 🧪 Gate that can roll back the skill without erasing the lesson
• 🔬⚡ Discovery ≠ execution — transfer is a candidate, not a merge

Contracts that are non-optional:

• 🎒 Skills over god-prompts
• 📜 Memory is evidence, never policy — the wiki is memory
• 📦 Version the skill the way you version the graph, the tools, and the prompt
• 🧪 Evals you can fail a release on; traces you can replay
• 🧹 Prune the wiki; label workarounds so they cannot masquerade as general procedures
• 🛡️ Permission envelope outside the model — always

🌱 Treat agent experience as something you compile — not something you paste into the next prompt.

📚 References

1. Liyan Tang, Cyrus Rashtchian, Chun-Sung Ferng, Andrew Tomkins, Da-Cheng Juan, and Tu Vu, “WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution,” arXiv:2608.27454, Aug 2026, arxiv.org/abs/2608.27454
2. Andrej Karpathy, “LLM Wiki,” GitHub Gist, 2026, gist.github.com/karpathy/442a6bf555914893e9891c11519de94f
3. Barry Zhang, Katelyn Lazuka, and Mahesh Murag, “Equipping agents for the real world with Agent Skills,” Anthropic, Oct 2025, anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills
4. S. Alzubi, N. Provenzano, J. Bingham, W. Chen, and T. Vu, “EvoSkill: Automated Skill Discovery for Multi-Agent Systems,” arXiv:2603.02766, 2026, arxiv.org/abs/2603.02766
5. J. Ni et al., “Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills,” arXiv:2603.25158, 2026, arxiv.org/abs/2603.25158
6. Y. Yang et al., “SkillOpt: Executive Strategy for Self-Evolving Agent Skills,” arXiv:2605.23904, 2026, arxiv.org/abs/2605.23904
7. Shunyu Yao et al., “ReAct: Synergizing Reasoning and Acting in Language Models,” ICLR 2023, arxiv.org/abs/2210.03629