> It is pretty clear that initial accuracy issues will become less and less of a problem as these technologies mature.
What do you base this on?
As someone who can both see the amazing things genAI can do, and who sees how utterly flawed most genAI output is, it's not obvious to me.
I'm working with Claude every day, Opus 4.7, and reviewing a steady stream of PRs from coworkers who are all-in, not just using due to corporate mandates like me, and I find an unending stream of stupidity and incomprehension from these bots that just astonishes me.
Claude recently output this to me:
"I've made those changes in three files:
- File 1
- File 2"
That is a vintage hallucination that could've come right out of GPT 2.0.
> That is a vintage hallucination that could've come right out of GPT 2.0.
That's because, despite the many claims to the contrary, the models haven't actually gotten any smarter. They are still just token prediction engines at the end of the day, without any understanding of what they are doing. That's why one should not rely on them.
I think the Linux kernel's standard of disclosure via the "Assisted-By" trailer is the right move.
Makes it clear you used a bullshit machine, without implying it's an author.
...assuming you think using them at all is a good move - I won't deny they have some utility (though I'd argue much lower than many seem to think), but I do presently believe they're a disaster for humanity.
The ruination of the Internet with slop, the massive propagation of propaganda, and the insanely easy-to-wield tools for abuse are in no way worth the ability to accrue tech debt at 10x velocity (though to be clear, accruing tech debt can absolutely be a useful strategy, if one I personally dislike).
I'm not personally sure that's a supernatural event, but if I'd had my eyes deteriorating for years, undergone multiple failed surgeries to stave off blindness, become fully blind, had doctors tell me I was irrepairably blind, and lived without eyesight for years, then had it come back within two days of praying to a Catholic saint for healing...
To me that doesn't follow logically. What if instead of praying to some saint, they found their lucky underwear that they lost when they were a child, and wore that for the first time again? Would that proof that the lucky underwear was somehow instrumental in fixing their eyes?
Apparently the body was able to heal her own eyes, and it would have happened if she prayed to the saint or not.
> Would that proof that the lucky underwear was somehow instrumental in fixing their eyes?
No, obviously not, as there's no reason to believe the underwear are sentient.
Rapidly getting what you ask for, when experts have generally agreed it's impossible, would be very striking.
> Apparently the body was able to heal her own eyes
There's no more evidence for that claim than there is for the claim that a saint did it.
Indeed, the fact that medical personnel say "This does not happen" is arguably evidence against the "it was a natural coincidence" interpretation.
I don't mean to suggest that anyone experiencing such a healing should or would necessarily become religious.
I'm just saying that personally, given my other life experiences this far, I probably would ratchet my probability of there being some unnatural power capable of intervening in the universe from ~55% to 90%, if I experienced sudden, dramatic healing of incurable blindness promptly after praying for it.
And, returning to my original point, there are people out there who asked for something vanishingly rare to the point that some experts label it impossible, then get it quite rapidly after praying for it.
That's a fairly reasonable reason to believe in a personal God of some sort, even if it's not the only plausible explanation.
I have read every post in the series and really appreciated it.
I've had a tremendous amount of respect for you since I first encountered the Jepsen analyses, but your breakdown of the likely impacts of LLMs and ML may impress me more.
You've articulated very well several concerns of mine that I haven't seen anyone else mention, and highlighted other issues I had not previously recognized.
Thank you for publishing this now, when it could still have some influence, rather than polishing and researching and refining until it was thoroughly rigorous and too late to be relevant.
Subversion was (and is) an admirable project, and filled a void by being much better than CVS.
When I discovered git, I couldn't go back to svn - git fit my mind _so_ much better.
It might not have seen the meteoric rise without GitHub, but just like it's weird to find servers running an OS other than Linux these years, I suspect there would have been a steady growth that eventually made it dominant.
I suspect it will be very hard to unseat git at this point - for all its untuitive UI it's good enough for most things, and it's been slowly improving for the use cases where it's weak.
My experience so far is that it's harder and slower for me to understand the genAI code than to write it myself.
Skipping thorough comprehension seems to be the popular choice in my workplace, but it's not one I can justify.
reply