Microsoft's 2026 AI Rollout: How Social Networks Drive CLI Adoption
A Study of Microsoft's Early 2026 Rollout of Claude Code and GitHub Copilot CLI
We studied tens of thousands of engineers at Microsoft during the early 2026 rollout of Claude Code and GitHub Copilot CLI. Our findings reveal that adoption spreads primarily through social networks rather than demographics, while retention correlates strongly with coding activity. Crucially, adopters merged roughly 24% more pull requests than they otherwise would have, proving these tools deliver real productivity gains beyond mere novelty.
Organizations should treat visible peer use as central to rollout strategy.
- gojkoa
The study unfortunately looks only at individual productivity, not any org gains, and the big claim in the PDF is that adopters "merge roughly 24% more pull requests" over a four month period. not exactly headline-making material. There's no data in the paper whether those 24% extra pull requests actually added anything more valuable or not.
- simonreiff
I really wish this study had chosen to focus on more useful endpoints like revenue, profitability, customer retention, conversion rate, etc....at a minimum, don't look at PRs merged without also looking at bugs in production code, number of incident reports, features shipped, debt log canceled, etc. Even if you only want to consider merged PRs, you should weigh the productivity of the reviewer and whether they are also 24% boosted or else it's just an accounting trick.
- npunt
Context: measuring productivity at all is hard. It was hard when computers were introduced into the workplace, when the internet was introduced, etc.
I worked with a journalist who covered tech in the 80s & 90s and she remembered that studies then didn't show improvements in productivity with the adoption of computers, despite people feeling as though they were more productive.
Reminds me of this Jeff Bezos quote:
"The thing I have noticed is when the anecdotes and the data disagree, the anecdotes are usually right. There's something wrong with the way you are measuring it."
- xamut
The number of merged PRs increased. But what does that actually tell us?
There is no comparison of the complexity, or depth of those PRs with work previously done without agents. The authors themselves acknowledge that PR count rewards small, frequent PRs and that whether the additional output justifies its cost remains unanswered. We do not know whether end-to-end feature delivery became faster, whether the additional output justified the cost, or what happened to defects, maintainability, review effort, and rework.
The conclusion seems to be little more than: Microsoft engineers are using AI tools, Microsoft is spending more on AI usage, and AI vendors are earning more subscription and token revenue.
More PRs are not necessarily more value. Without quality, delivery-time, and ROI metrics, the claimed productivity gain is largely meaningless.
- anshumankmr
>281 billion tokens. Using the least expensive version of Claude Opus 4.6, which costs $5 for every million tokens, that one
user alone could have cost Meta more than $1.4 million.
And that is based on the pricing that they sell a dollar for less than a dollar so the costs may be much much more.