As a software engineer, 6 months ago. I was better than AI in essentially every way except raw typing speed.
Now AI has surpassed me in fine-grained problem-solving ability. It can write more reliable code than I can, faster than I can, provided it is given the right guidance.
But one thing that I'm still much better at is identifying technical opportunities and choosing the right tradeoffs.
My feeling is that frontier models are incredibly smart in some ways, but incredibly dumb in other ways
When I chat with Claude using deep technical language about distributed systems issues, the arguments it presents are mind-blowingly good. I worked with many skilled engineers on complex projects but the kinds of arguments Claude makes are on another level. It feels like it can read my mind because it already identified all of the relevant aspects to the current topic and it's incredibly persuasive. I would say it's even better at deep, nuanced technical discussions than I am.
When I ask it to implement a feature using technical language, it does a really good job and the solution usually works out of the box. No bugs at all 95% of the time even after adding 1000 lines!
But the problem it still has is that when it implements solutions, it misses so many low-hanging fruits/opportunities. Especially in terms of performance, maintainability, scalability and UX. It's like it sees and recalls all the relevant parts perfectly but yet somehow misses opportunities which seem extremely obvious to me.
It feels like AI has 0 creativity. It only sees the opportunity once I mention it... And once it sees the opportunity, it demonstrates deep understanding of the technical implications. I think what's surprising is that it understands the suggested solution so well, with such nuance, that I can't understand how it didn't see the opportunity and why it never seems to see it until I mention it.
This is very unhuman-like. There is no way that a human being with that degree of understanding of a topic would be presented with such highly relevant context and not make the connection.
This is exactly how I feel about the recent ChatGPT models. It feels like I'm working with an incredibly enthusiastic junior student who listens to everything I say, checks it against my entire codebase, and thinks through all of the possible connections 1000x faster than I can. But it doesn't spot the ideas first. It actually makes work quite fun, since I can spend so much more time in "idea space" instead of "implementation space".
Thanks for putting it in words. I feel the same about SOTA coding models. I get visibly frustrated by this and often find myself using curse words at Astra.
You're right it's very unhuman to miss obvious opportunities you mentioned. And what's why I think these models would continue sucking as long as they don't have human level general intelligence. They may get marginally better at these tasks but I don't think we can expect them to connect dots like humans before AGI.
This mirrors my understanding when I use Claude code for mathematics. I can have deep discussions with it and it can solve my hairy problems. But whenever we go off the beaten track into design new mathematics, it struggles to make conceptual leaps and find the right definitions. Once I give it my ideas, it is back to its super-human pace and top notch intelligence.
This situation suits me fine, since I am anyways more of an ideas person, than a crunching open problems person. But I understand the desperation of my colleagues who mad solving hard problems their identity.
> Once I give it my ideas, it is back to its super-human pace and top notch intelligence.
That's the real benefit of AI today: you can test your ideas almost as fast as you can generate them, in parallel even. If you have a crazy idea that is unlikely to work but has massive advantages if it does, you just set an agent to explore and prototype it while you focus your main attention elsewhere. And so on.
It's still GIGO, though! An AI can make a bad idea work (and will do so if you don't carefully prompt it to not be so sycophantic!) but it won't work well. Knowing when the idea is bad and should be abandoned is also the part that currently requires human judgment.
That said, given the pace at which models have been improving so far, I can't help but think that this is a transient state of affairs.
Yes very much the same situation in software development.
The people who prided themselves on raw puzzle-solving ability got hit hard. Those who are idea-driven and architecture-oriented feel like they got handed a superpower.
It's quite a big shock at the industry level because the entire software engineering job interview process at essentially all large companies was heavily biased towards well-defined, raw problem-solving under time constraints which is precisely the skill which AI has replaced.
Yeah and companies continue to use leetcode style interviews even today and software engineers keep grinding for them. It's nauseating to contemplate on.
I'm an open source dev and I appreciate AI. I'm glad it exists and I have no problem with people being rewarded for their work on AI.
That said, I also think everyone who contributed content online deserves to be rewarded for their contribution, especially since OpenAI (which made the largest contribution) was a non-profit in the beginning. So there should be some kind of tax. Otherwise I agree, it's not ethical. It's not 'fair use' of copyrights.
Though I'm not surprised by this offering, I feel like I need some time to absorb it. It feels like the stepping stone to the next big thing.
It's going to destroy a lot of startups which were monetizing this exact idea. But clearly it's a low-hanging fruit so it makes sense that OpenAI would do it.
This is literally the recipe for the coming software apocalypse.
Just watch the software engineers who say "Let's be careful" lose their jobs and get replaced by non-technical vibe-coders churning out 10K lines of dirty insecure, unmaintainable code per day... Ticking time bomb.
I swear, good engineers are going to move to North Korea to monetize because of the amount and size of 'opportunities' this will create.
It seems so far-fetched but that's the direction it seems to be heading.
exactly how I feel. I mean let's face it, we were overpaid and underworked because we were considered magicians, but the magic is gone.
What I fear is that it's impossible for the untrained to see the difference between a highly-technical developer and the vibe-coder.
Software is so buggy nowadays (just look at Facebook, or Facebook Ads. Crazy) I wonder what's going to happen in the long run.
I was banking on the opportunities it will create but nah, have you tried debugging AI-generated codes of thousands of lines where the original dev doesn't exist? It's a mess I don't wanna clean up.
>> I was banking on the opportunities it will create but nah
Seems we're on the same wavelength because I had the exact same thought about this.
But actually, I'm probably not so fussy. I wouldn't mind cleaning up the mess if I'm paid hourly... But now my concern is: What if they want me to clean up the mess but demand that I do it in a particular way which makes it impossible?
I'm already seeing signs of this. This was already kind of the case in my last job; I had to fix things and implement new features but we had to keep the same clunky, over-engineered architecture. My current job doesn't have the same degree of architectural legacy baggage but there is a lot of bureaucracy to deal with instead, which is itself restrictive.
I make skills to allow AI to integrate with my platform; it's documentation with cURL commands. It can interact with every aspect of my platform via HTTP and access its full capabilities. I can tweak its token permissions as I like and revoke access if necessary.
My experience is that AI has significantly boosted the value of a quality code-base. A good codebase essentially codes itself.
There are project I've built from scratch that I would feel confident to hand off to a bunch of non-technical vibe coders and I know they would be productive and the product would likely be secure; because the existing codebase already exhibits all the patterns and principles that are required for that kind of project.
It would probably slowly degrade over time if a lot of vibe-coded logic is added on top but I think they could get very far feature-wise whilst keeping the software reliable.
But even though the value of such codebase has increased, people haven't adapted to this new reality. People are generally not good at telling what is good code. Because we don't actually have consensus on a definition. My definition is that good code is code that is easy to extend and maintain.
If implementing a feature requires a huge amount of tokens, then there's a good chance the codebase is not great.
I've worked on a codebase where a small feature requires might require 3k tokens, but on a different codebase, a feature of similar complexity would require 30k tokens minimum... And it's not about the size of the project; it's more about how the logic is divided and the architecture. And importantly; it's not a one-off; it's a clear observable, repeatable pattern.
This article seems like a reaction to DuckDB's surge in popularity. Having multiple DB engines to choose from is good and it often doesn't matter which one you use. One could make the same argument about DuckDB. Many database engines are multi-purpose. Though of course there are specific use cases where a different DB may be more appropriate...
Anyway databases nowadays are a commodity. A sticky commodity but nonetheless they are replaceable; increasingly so in the age of AI where data migrations are easier than ever.
Sandboxed execution is definitely one aspect... But IMO, this is still not secure enough for vibe coders. They will want to have data-driven apps to share among small groups of people, then the security of the sandbox doesn't matter if they expose some external endpoints and if the access control logic which guards data is flawed.
Even if each user gets their own sandbox, they will still want to configure different access rules for different kinds of data which they host.
That said the idea that each user could control and host their own data is interesting and could work. I imagine you could have apps which link data from many different user sandboxes via remote foreign keys.
You could have a centralized data schema controlled by the application owner but the data itself would be held/scattered across a large number of sandboxes.
I enjoyed reading this. It has a "butterfly effect" kind of vibe how such a random trivial-sounding pursuit can lead to something consequential. It's also mind-opening to realize how many strange human activities are happening in the background.
Reading this is especially interesting to me because it's what I've been working towards for the last 14 years or so. Though like this group, it also came together for me piece by piece.
I got interested in this specific idea back in the early days of Firebase I saw someone built a realtime HTML component with PolymerJS called 'collection' and I became consumed by the idea of fully generic realtime self-updating components. My approach is a bit different than OP or that of HTMX though; it's JSON over the wire, not HTML.
I've built a full implementation in Node.js with a set of declarative frontend components.
Now AI has surpassed me in fine-grained problem-solving ability. It can write more reliable code than I can, faster than I can, provided it is given the right guidance.
But one thing that I'm still much better at is identifying technical opportunities and choosing the right tradeoffs.
My feeling is that frontier models are incredibly smart in some ways, but incredibly dumb in other ways
When I chat with Claude using deep technical language about distributed systems issues, the arguments it presents are mind-blowingly good. I worked with many skilled engineers on complex projects but the kinds of arguments Claude makes are on another level. It feels like it can read my mind because it already identified all of the relevant aspects to the current topic and it's incredibly persuasive. I would say it's even better at deep, nuanced technical discussions than I am.
When I ask it to implement a feature using technical language, it does a really good job and the solution usually works out of the box. No bugs at all 95% of the time even after adding 1000 lines!
But the problem it still has is that when it implements solutions, it misses so many low-hanging fruits/opportunities. Especially in terms of performance, maintainability, scalability and UX. It's like it sees and recalls all the relevant parts perfectly but yet somehow misses opportunities which seem extremely obvious to me.
It feels like AI has 0 creativity. It only sees the opportunity once I mention it... And once it sees the opportunity, it demonstrates deep understanding of the technical implications. I think what's surprising is that it understands the suggested solution so well, with such nuance, that I can't understand how it didn't see the opportunity and why it never seems to see it until I mention it.
This is very unhuman-like. There is no way that a human being with that degree of understanding of a topic would be presented with such highly relevant context and not make the connection.
reply