LLMs reward expertise

The amplifying mirror analogy works best here. LLMs are ultimately a reflection of your own interactions with its weights, the tone you use, the structure with which you construct your prompt, aspects of an issue you tend to focus on, your breadth of vocabulary and world knowledge and whatnot.

People who (carefully) use it as an extension of their own mind and senses will very likely thrive, and those who use it as a replacement for their minds and their senses will struggle.

One of the Claude skills I made Claude itself generate was the 'learning a concept across tiers' skill -- from ELI5 level to a PhD level, and it triggers whenever I ask it a very general question on a complex topic that isn't my bread-and-butter. The fact that I'm able to choose explanation level from a super smart LLM (that's available 24x7) that can explain any topic under the sun would've been mind-bogglingly sci-fi-ish just 4 years ago in 2022.

This is something that really needs to be formally studied.

I'm inclined to say that this matches my own experience, but I can't rule out confirmation bias on my part.

As a meticulous person generally looking for a very specific code outcome, I prompt in a way intended to get exactly the thing I have in mind, and my results reflect that. But on the other hand, I have coworkers who type ten-word prompts with very limited specificity, and they seem to get results that way as well, and that makes me wonder.

It would certainly be beneficial for my career and financial well-being for the assertion to be true, because it means I don't have to worry about being pushed out of my job by an army of $15/hr vibe coders. But the convenience of that assumption is exactly why I think it's important to be skeptical.

> Of course both are useful, but I’d rather have familiarity with the codebase than a deep general understanding of software systems.

In my experience, getting that familiarity with a particular codebase in a way that isn't surface-level has always been a hands-on process. E.g. just because I know many general things about software, I need to know the particulars of the current codebase I'm in to know what is reasonable to actually apply to it.

This is a chicken and egg problem I find hard to resolve with LLMs. If we're pushed to delegate most work to them, how do you build that expertise? Sure you can ask questions about the codebase, but IMHO that falls under surface-level information, and the devil is often in the deeper details. Hmm.

I do find that "signalling expertise" is important. "I have a significant background in biblical scholarship. You can assume I've read the most important works in NT studies in particular. Do not translate Greek, Latin, Hebrew, or Syriac. Now, I would like to know . . ." That changes things significantly. So does telling it you have 20+ years of experience with C programming, that you have a robust understanding of machine organization, memory layouts, embedded systems, etc.

What's this LLM's generated yapping?

I don’t understand why this is such a revelation. Anyone who has listened to a good/great interview knows the skill of the interviewer plays a big part. To ask good questions, to understand what the other person is saying (AI or human) - that requires skill, expertise and patience.

Someone with less skill or expertise might still get good results, sure. It would just take longer and it wouldn’t be pretty

The breakthroughs are coming from simple prompts, some made by people with no math training:

https://www.newscientist.com/article/2580932-extremely-basic...

The referenced Terence Tao chat did not lead to new breakthroughs.

I consider myself senior engineer. When talking with junior colleagues they often are surprised how little I care about some things and how much I care about others.

These internal "attention weights" are highly influential parameters of how I work with LLM.

E.g. when working with Rust I often hold strict control over structures and lifetimes. But when lately I've been doing token-based bind generation I didn't care about anything outside of high level patterns like RAII and ultimately - API ergonomics which was verified in consumer app.

I've been in position of porting real-code to vibe-code platform and seeing non-technical people prompt-stream (they were shared across accounts) I know why they engaged engineer to run this work.

Their efforts took 6 weeks, I ported app within 4 days and (to be honest with myself) without LLM I that'd be 3M+ work pre-LLM.

In short: I observed same effect as claimed.

Not sure I agree with this. The math guy at anthropic's prompts are essentially:

"suppose you’ve gotta resolve the $CONJECTURE, like absolutely have to, everything depends on it. think really hard, and try to come up with a bunch of ideas to try. but remember to trust yourself and not necessarily in conventional wisdom!!"

https://claude.ai/share/25740bd5-aa97-4bd7-bf58-c4df3793fda7

https://xcancel.com/__alpoge__/status/2083855298239078748

Tao's chat was for him to gain intuition, not to solve the problem from the outset.

What's funny is that every other person gets a different conclusion about who these models reward/empower. I've seen people say that the generalist stands to gain the most and others say that it's the experts. Like all of life, maybe the "winner" is the person who just does stuff.