A few weeks ago I mentioned how I started writing agent skills for use with my LLM-based coding assistant. It turned out that I encountered a few rough edges pretty quickly.
Being accustomed to “normal” programming, I naturally extended my normal approach to code to skills. Of course, being non-deterministic, they are not really “executable code” like Lisp, Python or JavaScript. Still, I figured that applying the usual practices to them is a good idea. This means (among others) keeping them in version control and commenting them.
Unfortunately, both are difficult with skills. First of all, they should reside in ~/.your_favorite_coding_agent/skills. Having a Git repo there is possible, but I don’t like the idea. For example, what if I have all my personal coding projects under one directory and my job projects under another? Thanks to such a structure I can use conditional includes and have my personal email address attached to commits in side projects, and my job email address in job projects. Then a place like ~/.agents/skills is yet another directory where some of my projects would reside.
Of course, I could probably symlink the directories where I develop my skills to the place the LLM sees them – but that would not help with another issue I have. When working on code, I can play around in my Git repo, not worrying about messing stuff up – I can always git
reset --hard, and what happens in the working directory stays in the working directory and doesn’t influence the production environment. Having separate “development” and “production” environment for skills seems a natural thing to do – but that means that I have now two copies of each skill, and when I modify the “development” environment, I need to “deploy” any changes to “production”. If that seems like an overkill, it’s because it probably is – but consider this scenario. Imagine that several people work on a project, and they’d like to share the project skills. This is perfectly doable (and in fact, the several coding assistants I checked support that), but what if different team members use different coding agents? Most of them, but not all, look for skills in .agents/skills – so depending on what agent a person uses, they should copy the skill file(s) to a different place. And when there are two copies of a file, there is always a danger that someone will make some edits in on of them and other edits in the other one, leading to confusion (and data loss when the skill is blindly copied from “dev” to “prod”).
Another issue I have with skills is that they are supposed to be written in Markdown, which has serious problems. Of course, they are more or less free-form text, but Markdown is expected – if only because their entry point is always called SKILL.md. This means that if you want to write your skills in a superior format – for example, Org – you’re out of luck: SKILL.org will not be used by the agent (and writing a one-line SKILL.md telling to look in SKILL.org feels really stupid!).
All these problems are annoying, but they are not deal-breakers. The last one, however, is – at least for me. You just cannot comment skills! Markdown has no notion of “comments” (well, of course, there is no notion of “Markdown”, either – it is just a common name for a large group of similar but subtly different formats…). If you want to mark things like decisions made and their rationales, or decisions rejected, or “to-do” items mentioning stuff you’d like to add to your skills later – the only thing you can do is put them in a separate file (or live with the fact that the “comments” which were supposed to be only read by humans will be also ingested by the LLM, potentially confusing it and using up the precious context).
Of course, being a programmer, I decided to do something about these problems. Some time ago, I set out to write a set of tools for working with skills, which would help with these issues, and this way Skill shed was born.
It is a Node.js script whose main feature is the “deploy” command – copying the skill file(s) from their “source” directory to the “target”. Optionally, it can transform Markdown files and strip comments from them (comments use the well-known HTML syntax). The script takes care not to overwrite anything in the target directory not put there by itself, so if you accidentally edit your skill in the target directory, the deploy command will warn you about it and abort. Alongside SKILL.md and other files meant for the LLM, you need to create an .env file telling skill-shed where the target directory is – and that file may be git-ignored (and thus be different for each team member). The creation of this file can be automated. Also, you can lint skills, that is, check for a number of potential issues with them (like malformed frontmatter, too long description, unclosed Markdown fences or HTML comments, files in the skill’s directory not referenced in SKILL.md, dead URLs in the skill etc.) The plan is to also allow writing your skills in the SKILL.org file and have them converted to Markdown on deployment (that, however, is not yet implemented).
The project itself is an experiment – it is one of my first projects written with an LLM (although I would not call it “vibe-coded” – I carefully reviewed every single commit!). One of the things I usually do when I delegate coding to an LLM is an extensive test suite. In fact, in LLM-assisted projects I often insist on 100% test coverage, which sounds crazy but actually makes sense. Also, I don’t use things like Lodash or Ramda – coding in vanilla JS is less convenient for a human but doesn’t make that much difference for an LLM, and thus I can afford the luxury of having as few dependencies as possible. (That is way less important in Python, which adheres to the “batteries included” philosophy and has a really extensive standard library; using JS is more like buying a piece of furniture but without the screws and glue necessary for assembling it…)
Anyway, if you find this project useful, use it, star it on GitHub, fork it, open a PR or issue – I really hope it’s useful not only for me! I cannot promise I will review your PRs etc., but I’m definitely willing to make this tool more useful.
See you next week!