shipfeedAI news, curated daily

23:05:07 CET
20 MAY23:05:07shipfeed
pull to refreshlast sync
Just in — 30 new
§ feed · storyline

AGENTS.md outperforms skills in our agent evals

AGENTS.md embedding an 8KB Next.js docs index achieves 100% pass rate in agent evals, outperforming skills-based retrieval which peaked at 79% even with explicit usage instructions.

Jan 27 · · primary fetch1 sourceupdated Jan 27 ·

We expected to be the solution for teaching coding agents framework-specific knowledge. After building evals focused on Next.js 16 APIs, we found something unexpected.skills A compressed 8KB docs index embedded directly in achieved a 100% pass rate, while skills maxed out at 79% even with explicit instructions telling the agent to use them. Without those instructions, skills performed no better than having no documentation at all.AGENTS.md Here's what we tried, what we learned, and how you can set this up for your own Next.js projects. AI coding agents rely on training data that becomes outdated.

APIs like , , and that aren't in current model training data. When agents don't know these APIs, they generate incorrect code or fall back to older patterns.Next.js 16 introduces'use cache'connection()forbidden() The reverse can also be true, where you're running an older Next.js version and the model suggests newer APIs that don't exist in your project yet. We wanted to fix this by giving agents access to version-matched documentation. Before diving into results, a quick explanation of the two approaches we tested: We built a Next.js docs skill and an docs index, then ran them through…

read full article on vercel.com
§ sources1 publication · timeline below
  1. vercel.comAGENTS.md outperforms skills in our agent evalsprimary