Ask HN: How to deal with shipping AI-written code you don't understand?
I'm retired, so the only person I answer to about progress is myself. I'm stubborn about choosing quality over speed, but I still like progress. This morning I got preliminary results from a 'C' front-end to my compiler that uses clang to compile to LLVM IR, then translates to our IR and feeds it into the autovectoriser. For the exact same source code, I'm seeing speedups from 2.1× to 1,312× over clang -O2 on benchmarks like saxpy, Mandelbrot, nbody, perlin, and mandel_deep, with checksums matching. Hacker News isn't interested—any post I make linking to the webpages is immediately blocked/dead—so I'll wait until the C++ front-end is ready next week and announce it somewhere else. A compiler that takes standard C/C++ code and speeds it up by automatically finding data-parallel areas and scheduling them on the GPU with no programming overhead is pretty rare, and it also manages the local network so you can run xcd on machines on your LAN or via the TB4/5 bridge between Macs.
Those are some pretty fine speedups, from naive C, with no alterations from the standard "C" code - it automatically finds places that can be made data-parallel, tracks the data-hazards to make sure its safe to "GPU" and writes all the kernels, boilerplate and data-stream-management if necessary.
Obviously the checksums have to match, so I know it did the work correctly too :)
Oh, and it manages the local network as well, so you can run xcd on machines on your lan, or via the TB4/5 bridge between Macs and it'll schedule on other GPUs, just like it manages multiple GPUs on the local machine. It characterises both latency and compute-time to figure out which devices are worth scheduling on.
- bitwizeshift
I’m surprised I haven’t see anyone say this yet, but I don’t let the LLM design the code at all, I let it implement a design I have created.
By that I mean that, rather than let the LLM do the entire structure, I take the time to create the full hierarchy of abstractions and actors that I want my code to do. I decide, at the UML level, how abstractions will sit and interact — and then I make the LLM implement those gaps.
The key thing here is that, by doing this, I DO know how the code works. I may not know how each unit is implemented, but I know each unit’s responsibilities and how they interact with one another. I’ve found that LLMs are HORRIBLE at writing decent abstractions, regardless of how much prompting and skills I’ve given it. It will always get the job done, but it will often be unmaintainable and require bigger and heavier refactors as I introduce more and more requirement changes.
But by flexing my basic design skills, I don’t have those issues. Future refactoring fit into easy abstractions that I’ve designed. And by documenting it up front, I also have what becomes a living document on how this work will turn out.
It takes a few hours up front to plan ahead, but it makes the LLMs job easier and use less tokens, and it makes the code better long term. I don’t care if one unit is suboptimal, as long as the abstraction is right — because at worst I can change an implementation for a better one later.
- ChrisMarshallNY
Testing.
Lots and lots of testing. I usually interactively test with the LLM. Now that I'm spending less time, coding, I get to spend more time, testing.
In my experience, I test more, with an LLM, than I did, with my own code. I don't find coding errors, but I do find implementation errors.
On another note: I ran an LLM on a bunch of really mature code that I wrote, and could swear was "perfect," and it found a number of issues.
Nobody's perfect. Trust; but verify.
- ge96
My Jira tickets are written by AI started just feeding the ticket into AI (shrugs)
No I only do that for unit tests so far
I went through like a mid-life crisis when the thing I learned/spent last 13 years on became meaningless.
- PaulHoule
Ask the AI how it works!
It's totally fair to load up some open source project into an agentic IDE and instead of asking it to write something ask questions like "how does X happen when I use feature Y?" or "can you explain how client code could accomplish Z"
All the time I see an AI use a language feature or library feature that I never used before or I see some code I don't understand and it does much better than a lot of humans do at explaining what it did.
- spacedcowboy
I’m slowly coming around to the idea that if it works, I’m good with that. I never used to personally check all my colleagues work, either, I just trusted that they’d done a good job.
That’s not to say I blindly trust an LLM - my project is an autovectorising compiler. I have it write a bazillion tests, and I have real, non-trivial programs that it compiled. I have it run a ‘gate’ before releases that takes literally hours on 5 different architectures. To say this code is tested is … masterful understatement.
I think what helps is a long planning session, going into details, working out corner cases, designing test-cases, you know work and effort. Once that’s done it’s easier to let it work through the night and do a major production feature.
I still review the code, but hand on heart I couldn’t say I’d reviewed everything, sometimes I miss something in a complicated check-in. I try to keep checkins small and contained, but sometimes that’s not easy to manage alongside the “don’t break stuff” rule as well.
So it comes down to: I’ve been working on this for 6 months or so, and so far this has worked out for me. You know, kind of like getting used to a colleague’s idiosyncrasies over a period of time.
I will say that in my experience the code quality is fairly high. Maybe that’s down to the effort put in ahead of time in the planning stages, I don’t know, but I don’t seem to suffer from the ‘slop’ that people complain of. YMMV.
- skeledrew
Technically it's just a role change. You're now the project manager or something, not the developer. It isn't a project manager's job to understand the code, only to ensure it does what it should and doesn't do what it shouldn't. And to ensure the developers, etc str unblocked and doing well. Embrace the new role.
- 4ndrewl
This is how every product manager feels every day.
If you're an LLM prompter shipping products, you're outputs are more akin to a product managers, not an software engineer's.
- solatic
It's not too dissimilar to life as a Principal level engineer pre-AI. You technically oversee the work of many engineers on their own systems, but you couldn't possibly review all the code yourself. You appreciate that human code review reduces, but does not eliminate, the number and severity of bugs shipped to production. You don't have the low-level details in your head, just the high-level ones. You design how all these engineers interact with each other, without stepping on any toes.
Time will tell whether we were right to move from a system where we trusted humans to understand the low-level system, to one where we trust fancy calculators. But so far, it's working.