I think it depends on how “before” we’re talking about.
I can remember a time when learning was valued and leaving the camp cleaner than you found it was considered a basic professional standard.
But I can also remember a time when Scrum became all the rage and next thing you know we’re all stuck on the sprinting treadmill, management is obsessing over “velocity”, and it’s generally an everyone-for-themself free-for-all to clear the absolute minimum criteria to get the ticket moved to the “done” column in a semi-desperate effort to keep up with your ever-growing backlog of tickets to which you’ve been over committed. Don’t worry about incomprehensible code or flaky designs; taking your time to do it right the first time looks bad on the KPI dashboard but rework does the opposite because you get to count the second (third, fourth, etc.) times the same task needs to be revisited towards your velocity metrics, too.
I’m not sure most developers younger than maybe 40 realize just how much worse our line of work has become over the past ~15 years.
The first few sections hit me. I think that I used to be a “rockstar”, until I realized it wasn’t a good thing. Perhaps I could get things done 10x faster than my colleagues. But at some point I realized it wasn’t because I was 10x more productive than a normal person; it was that I worked in a way that made the normal people around me 1/10 as productive.
Since then I’ve slowed down. It’s been an overall positive change for my life. Being a team player unfortunately won’t give you the same kind of upward mobility in this crazy industry, but it’s done amazing things for my mental health.
I've definitely shot my own foot off in the past by implementing things in a rockstar manner. In my case, it typically involves me over-abstracting something that really doesn't need it. Or building something for use-cases that never actually happen. So I have to try to keep YAGNI and KISS in mind whenever my mind wanders into over-abstraction territory.
Those companies are certainly writing more code. But It isn’t clear that they are increasing their economic productivity. It could even conceivably have the opposite effect by fueling a race to the bottom.
e.g. an interesting possible canary in this coal mine is that there’s been a 200% increase in the rate of new apps appearing on Apple’s App Store, but it has not been accompanied by a 200% increase in the rate at which people are buying apps.
The AI pundits often seem to apply the logic that code output is directly proportional to revenue and/or profit, and as such it follows that an AI usage increase leads to more code which leads to more revenue.
I don't believe this aligns with the reality of any major company, unless your business is in the literal sense "selling code" your revenue and profit is tangential to the quantity of code you produce. Google is a good example of this: most of their revenue and profit comes from their ad network, which is disconnected from their development productivity and instead heavily reliant on network effects and time in market. If I was a new competitor with infinite AI funds to throw at whatever problem I choose, I can't simply capture their market by developing an exact copy of Google's ad platform. In the same way, Google can't substantially grow their ad network by coding "more" or "better", they still need more customers and consumers to interact with their network to see any increase in revenue.
So it doesn't directly follow that a productivity increase will inherently follow an AI usage increase.
Agreed. I think it’s more likely to expect that most of it is pure waste.
My impression is that most software development work is not profitable. Either the project is abandoned, or it fails, or it gets shipped but doesn’t generate positive ROI. But, like how venture capital works, the minority of projects that are successful make enough money to cover the rest.
Some portion of this is because demand for software projects in general is less than perfectly elastic. So more software does not automatically mean more software sales.
It also seems plausible that, in general, companies tend to fund the projects that are most likely to be profitable. They aren’t perfect at it, but I doubt they’re just rolling dice.
Which would imply that the new work companies can take on thanks to developer productivity gains will tend to be ones that are less likely to generate positive ROI.
meaning AI may only produce a net increase in waste, which only serves to erode profits.
Add to that that it’s been years now and we still don’t have an example of someone army-of-oneing a killer app or anything like that. It’s beginning to feel like another iteration of the amazing blockchain revolution that was always & forever just around the corner.
The unlocked economic activity won’t come from a Google competitor writing code faster. A lot of it will come from “boring” businesses who could benefit from custom software but haven’t had the means to create it themselves. In some cases they may not even know their problem can be solved by software, but some AI they are using for repetitive tasks will notice and offer to build an app for them.
I would go as far as to say writing more Code has almost no impact on their economic productivity. What drives those companies is infrastructure and networks
So far the place where I've seen "more code being written" having a postive effect, has been in paying down tech debt and reduction of overhead. We've rewritten services (bringing multiple microservices back under moduliths) and cut costs. But I'm talking about net-negative code. That's not the point you're making. I agree that puking out 20 new features likely wouldn't gain us more revenue.
If the quality of all apps remains high, but if there is an increase of low quality apps it may not necessarily be great for consumers as it becomes difficult to distinguish which are the good and bad quality apps, making it risky to purchase apps.
It’s not just that. Oftentimes contracts stipulate that the client’s data can’t be transferred across certain boundaries. If you have signed such an agreement, even sending the data to a service on the same cloud provider but in a different region could be a huge compliance violation.
The old saying goes, the market can remain irrational longer than you can remain solvent.
I’m not necessarily expecting a crash any time soon. (But we average a major correction, what? every 8 years? So if you keep predicting one long enough you will eventually have been right all along.) But I do feel comfortable saying OpenAI and Anthropic are overpriced. For more or less the same reason Cisco was overpriced in the late ‘90s. It’s not that what they were making wasn’t valuable; it’s that we got out over our skis a bit over how much of it the world could actually manage to consume in the immediate future.
I can’t imagine them actually being bullish about exponential growth, when both seem instead to be stagnating. I’m more inclined to believe they’re just maintaining a level of hype in public because that’s what you do.
They’ve claimed a big revenue run rate for this quarter. But it’s non-GAAP, so you kind of have to assume shenanigans. Earlier this year they were telling a court their revenue was like 1/4 of what they had told the public. I consider the number they came up with when they had to worry about committing perjury to be more trustworthy (because I’m a pill), so that would also indicate shenanigans. My guess is they are inflating that revenue run rate figure by booking token pre-payments from enterprise contracts now instead of spreading it over time as GAAP would mandate. And at the same time their big enterprise clients are talking about scaling back their usage.
So we’ve got a combination of signs that they’ve been inflating their revenue growth, and signs that their customers are losing their appetite for contributing to that revenue growth. I suppose it’s not a slam dunk, but it feels to me like as strong an indicator as one could hope for a private blitzscaler startup like this.
Oh, to be clear, I'm not saying there is evidence they're all a-okay. I just hadn't seen any evidence that they were stalling out. (I have for OpenAI.)
"Although the company has generated substantial revenue since entering the commercial market—exceeding $5 billion to date—it has nonetheless had to raise more than $60 billion in outside capital to fund its operations".
No new models. Same janky slop but with a bunch of RL and benchmark cheating. A pretend future model which is just the current model but with a longer COT and gated away.
I could have paid cash for my car, but that would have been a bad move. I wouldn’t have had any liquid assets left over for getting me through a rainy day. The interest I paid on the loan was an acceptable price for reducing my overall risk exposure.
Even if Alphabet has $80B sitting in the bank, they could quite reasonably arrive at a comparable decision.
This is a specious argument. I have not studied the case law, but I would guess that the reasons why courts decide in favor of gun manufacturers generally don’t apply to AI. Becauee the guns in question are not able to autonomously shoot people, and because they generally work as advertised.
A more accurate analogy would be Tesla and Autopilot. And they are being held liable in courts. They are being held responsible for autonomous behaviors that are not fully under the control of the operator, and they are being held responsible for misleading operators about the capabilities of the product.
Boeing got in trouble for MCAS, with a comparable legal basis.
I suspect that letting agents spin away unattended for long stretches of time will become less and less popular as more and more companies blow their token budgets and start requiring some answers to difficult questions before agreeing to further loose the purse strings.
I've been enjoying journalist Ed Zitron's recent diatribes about how impossible it is to find a business leader who had a plan for measuring their ROI from adopting AI coding.
What he says he's consistently hearing from them mirrors what I saw at my own employer: they thought they had ROI metrics, but they actually only had usage metrics such as "lines of code committed" or "number of pull requests". The only way those could possibly work as an ROI measure is if your business charges customers by the line of code.
Measuring productivity of developers isn’t really in line with what needs to happen, either. A team can be incredibly productive and still generate negative 100% ROI if what they are building so industriously is stuff that nobody wants to buy.
Which reflects another thing I’ve seen at work. A lot of what AI coding has enabled is diving headfirst into quagmires. Our costs have spiked - not just because of the token spend, also because we gotta pay the cloud platform to run all these new services, operators to operate them, marketers to market them, etc. - but revenue hasn’t budged.
But at least pre AI, most managers presumably subjectively measured devs on relevant performance. Using systems where employees who burn the most tokens ($) per week ‘win’ is crazy - just ask the AI to spin up a subagents to implement every conceivable approach to a task, then spin up n agent judge to pick the winner, and repeat. You've immediately got 50x or whatever your previous usage from that alone.
I can remember a time when learning was valued and leaving the camp cleaner than you found it was considered a basic professional standard.
But I can also remember a time when Scrum became all the rage and next thing you know we’re all stuck on the sprinting treadmill, management is obsessing over “velocity”, and it’s generally an everyone-for-themself free-for-all to clear the absolute minimum criteria to get the ticket moved to the “done” column in a semi-desperate effort to keep up with your ever-growing backlog of tickets to which you’ve been over committed. Don’t worry about incomprehensible code or flaky designs; taking your time to do it right the first time looks bad on the KPI dashboard but rework does the opposite because you get to count the second (third, fourth, etc.) times the same task needs to be revisited towards your velocity metrics, too.
I’m not sure most developers younger than maybe 40 realize just how much worse our line of work has become over the past ~15 years.
reply