SkillOpt

microsoft/SkillOpt

A text-space optimizer that trains reusable natural-language skills for frozen LLM agents

https://github.com/microsoft/SkillOpt

SkillOpt improves an agent skill automatically. A skill is a text file of instructions, like the SKILL.md files Claude Code loads. SkillOpt treats that file the way deep learning treats model weights: it runs the agent on tasks, proposes small edits to the file from the results, and keeps an edit only if it scores better on tasks the optimiser never saw. The model itself never changes.

The output is 1 file, best_skill.md, usually 300 to 2,000 tokens, that you drop into your agent. It adds no model calls at run time.

For business people

The problem: agent skills are written by hand, generated once by a model, or tweaked whenever something goes wrong. None of these reliably makes a skill better over time. A fix for one case often breaks another, and nobody notices.

What SkillOpt does differently: it applies the discipline of machine-learning training to a text file.

• Test before you trust. An edit only survives if it improves the score on a separate set of tasks.
• Small steps. Each round allows a limited number of edits, and that budget shrinks over time, like a learning rate.
• Memory of failures. Rejected edits are kept, so the optimiser doesn't propose them again.
• No model training. The model stays as it is. Only the skill file changes, so the result works with the model you already pay for.

The results, as reported by Microsoft: across 6 benchmarks, 7 models and 3 agent setups (direct chat, Codex and Claude Code), SkillOpt was best or tied for best in all 52 combinations tested. On GPT-5.5, it raised average accuracy by 23.5 points in direct chat, 24.8 in Codex and 19.1 in Claude Code, compared with no skill. Trained skills also carried over to other models and harnesses without retraining.

The limits: training needs a set of tasks with a score, and many real skills, like writing a sales email, don't have an objective score. Every training round costs model calls. And the benchmarks are academic: spreadsheets, documents, search, maths and household tasks.

SkillOpt-Sleep: a nightly cycle for your own agent

The part closest to daily use is SkillOpt-Sleep, released as a preview in June 2026. It gives a local coding agent a "sleep cycle":

1. Harvest your past Claude Code, Codex, Copilot or Cursor sessions from disk.
2. Mine the tasks you do again and again.
3. Replay them offline, on your own API budget.
4. Consolidate what worked into edits to your CLAUDE.md and skills, kept only behind the same held-out gate.
5. Stage a proposal. Nothing changes until you review and adopt it, and every adoption backs up the old file first.

⚠️ WARNING: harvesting is local, but a real run sends excerpts of your session transcripts to the model provider for mining, replay and judging. The README says outgoing prompts aren't guaranteed to be free of secrets. Review or redact the harvested tasks before running it on sensitive projects.

For technical people

How the training loop works

• A fixed agent runs the current skill on a batch of training tasks and records scored trajectories.
• Optimiser models read mini-batches of those trajectories and propose atomic add, delete or replace edits.
• The edits are merged, deduplicated and capped by an edit budget that decays over training.
• The candidate skill passes a validation gate: accepted only if it strictly improves the held-out score. Rejected edits go to a buffer that informs later proposals.
• After each epoch, a slow update compares the old and new skill on the same samples, and updates a meta-skill for the optimiser: which edits helped, which failed, which failures remain.

Install and run

pip install skillopt

# The nightly cycle for your own agent
skillopt-sleep dry-run      # harvest, mine, replay, report only
skillopt-sleep run          # full cycle, stages a proposal for review
skillopt-sleep status
skillopt-sleep adopt --skill NAME
skillopt-sleep schedule     # nightly cron entry for this project

In Claude Code, the repository also ships a plugin, installed from a local clone:

/plugin marketplace add ./plugins/claude-code
/plugin install skillopt-sleep@skillopt-sleep
/skillopt-sleep dry-run

The training side supports OpenAI, Azure, Claude, Qwen, MiniMax and any OpenAI-compatible endpoint, plus the Codex and Claude Code command lines as targets. A web dashboard is optional.

Value

This hits the weakest point of my own setup. I run about 75 skills, and only 4 have evals (see The NicAI skills catalogue ). Today a skill improves when I notice a bad result and correct it by hand. SkillOpt shows what the disciplined version looks like: a scored task set, small edits, and a gate that rejects any change that makes things worse.

Where I'd start:

• The gate, not the whole framework. Even without SkillOpt, the rule "no skill edit without a held-out test that still passes" would stop the silent regressions I get today.
• Skills with a checkable output first. Data and document skills, where a script can score the result, suit it. Voice and writing skills don't, because my scoring there is a blind test, not a number (see The NicAI voice stack ).
• SkillOpt-Sleep in dry-run only, on a project with no client data, to see which recurring tasks it finds in my sessions.

Further Reading

SkillOpt: agent skills as trainable parameters

Microsoft Research blog feature, July 2026

https://www.microsoft.com/en-us/research/blog/skillopt-agent-skills-as-trainable-parameters/

SkillOpt: Executive Strategy for Self-Evolving Agent Skills

The research paper on arXiv

https://arxiv.org/abs/2605.23904

SkillOpt project page

Visual walkthrough of the loop, docs and demo video

https://microsoft.github.io/SkillOpt/