We test whether the tools behind AI agents earn their keep
Right now that means one question: do MCP and agent skills actually make agents better, and by how much? We run the experiments, we measure the results, and we publish what we find. Including the things that don't work.
Three threads, one question
Tool access via MCP
Connecting an agent to your tools through the Model Context Protocol is easy to set up and hard to evaluate. We test whether tool access actually raises task success, or just adds cost and new ways to fail.
Agent skills
Packaged, reusable skills promise more reliable agents than prompting alone. We measure whether that holds up across real tasks, where it helps most, and where it quietly breaks.
Measuring efficacy
A good demo is not evidence. We build evaluations around the numbers that matter in production: success rate, cost per task, latency, and reliability under real load.
When should an agent know about a tool?
Adaptive tool context as a managed working set
Today's agents keep every skill's description resident in context and lazy-load the heavier skill bodies on demand. That resident layer grows with the catalog, and past a few hundred skills it becomes both a token cost and a distraction, where the model reaches for a plausible near-miss instead of the right tool. We ask whether treating that resident layer as a dynamic working set, loading and evicting descriptions as their odds of use shift, beats the static all-resident standard.
We run a scaling sweep across two axes, catalog size and semantic density, against a fair lazy-loaded-body baseline. Failures split into two kinds a single success rate would hide: distraction, where the model picks a visible near-miss, and omission, where the needed skill never surfaces to be called. The output is the crossover point, where adaptive gating starts to win and where the simpler static approach is still the right call. The study is built to disconfirm itself, and mapping that boundary is the contribution either way.
Papers and findings
Read it before anyone else
The Hyix AI lab plans to make much of our research public in the near future. Add your name and you'll be among the first to read it.
Get early access