Claude, Codex, and Cursor disagree on tools 58% of the time in 17k sessions
Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out
Armature ran 16,893 sandboxed sessions with three coding agents—Claude Code, Codex, and Cursor—across 75 repositories and 1,163 prompt variations to see which third-party tools they actually implement. The results: agents often disagree (only 42% agreement), repository context heavily influences choices (e.g., Resend wins on TypeScript, SendGrid on Python), and being mentioned doesn't mean winning—PayPal was cited 139 times but never picked. The study also reveals that pricing page details can flip decisions, and some markets are dominated (Stripe wins 90%) while others are contested.
So many well-known players are mentioned in almost every conversation and are never picked.