My company ran a test and found that they reduce token output on flagship model by something like 2-4x, and that number has been increasing with newer models. I suspect the increased subagent usage is driving this trend, because this means we're relying on models to do their own prompt engineering.
Yes, they are just text, and can therefore be replaced with good prompting. However, this also means they confer a real benefit: a good set of skills creates a transferable baseline, raising the skill floor and offering a more consistent experience across the organization.
I am somewhat confused by takes like this. Of course skills are just prompts, this is the whole point.
A skill is just a stored prompt you want to put more information into than you're likely to type out every time you intend to do that thing. Documentation of a business process.
What are you confused by? You're saying the same thing they said.
They added the additional claim that writing the skills down (apparently) prevents the models from having to self-prompt on the fly and therefore reduces token consumption.
Yes, skills that are actually used count toward token consumption.
The question is whether the number of tokens required to achieve a certain behavior/intelligence/quality is equal between you manually providing those tokens via skills versus the model "deriving" the "skills" it needs on-the-fly in order to produce the outcome you want.
The claim above is that the former requires far fewer tokens.
Also skills only consume tokens when they are used, and part of the value is that the model will dynamically find and disclose only what's needed (assuming the skill is "well-designed").
Their claim is not about the prompt or skill tokens, it's about output tokens - skills can help the model bypass some thinking tokens or avoid reasoning deadends, and that way reduce output token usage. That's what they seem to have found empirically from their testing. (If it's truly 2x-4x, the time savings in waiting for the output is a pretty nice benefit too.)
It's not just a stored prompt, you can attach re-useable scripts to them to offer more determinism. ex: a script that validates that a PR follows exactly the template you want, with a max of N lines per entry.
The more determinism you have, the more consistent you can be and the more leverage you can build. (yes I understand that skill calls are non deterministic).
We do that, but keep the scripts in the code and just tell them in the markdown where the scripts are, same with "references" (docs/) for us. It never made sense to me to put those in a skill dir, many are useful across skills and for humans (many written for humans before agents were a thing)
One of the more interesting benefits to skills is that many harnesses now run the inline command(s) in backticks, shortcutting the model needing to make a tool call. This is helpful for deterministically building up context content for the skill before the agent ever sees it.
We take this further in some instances and have workflows that (1) does deterministic context gathering (2) invokes an agent (3) processes a file the agent is told to produce. This has made our PR review agent much better and removed it's access to all credential files. We have a step that gathers the diff + existing pull request comments into a .review dir, let the agent process that and create a comments.jsonl, then run a script in a new step to apply the comments against the API
The “recursive” part comes from the fact that you have an AI which was developed by an AI (that was developed by an AI (that was developed by an AI (…)))
Sounds like "recursively" walking to the grocery store by putting one foot in front of the other (that put itself in front of the other (that put itself in front of the other (...)))
Is it a serious question? IQ tries to quantify the positive correlation between the results of all intellectual tasks a person takes (AKA positive manifold).
I wanted to highlight that we're actually looking at a trio of concepts: intelligence, IQ, and value. Strongly correlated concepts, yes, but also meaningfully distinct!
Whatever the people who came up with IQ intended isn’t really relevant to the question of whether IQ measures intellect. At best is is loosely correlated. Very loosely.
IQ strongly correlates with many real world outcomes that people tend to associate with being more "intellectual".
IQ has precise definition. It's what the tests measure. Intellect has only fuzzy handwavey definition. Correlation between IQ and intellect is about as loose as the definition of the intellect. The way people make the correlation even looser is by defining intellect in even more fuzzy and nebulous manner.
It is kind of nuts how little attention this more-than-controversial-enough aspect of it gets compared to anything else, to the point people don’t even know this about it.
I happen to hate Omarchy for the precise reason I don’t want that sort of interface, but apparently everyone else does, and if they do that’s up to them.
Can you say what you prefer? I'm always down for alternative UX flows and with hyprland being so user specific it's hard to see how people leverage it across the spectrum.
Yes, they are just text, and can therefore be replaced with good prompting. However, this also means they confer a real benefit: a good set of skills creates a transferable baseline, raising the skill floor and offering a more consistent experience across the organization.
reply