This I can believe, also seen it on a personal level.

But I'm not sure about the long-term effects of this, we just shifted the "subscriptions for apps" to "subscription for an app" AND you must now maintain said apps.

I would.

Here I am starting to think a central part of knowledge work is the knowledge gained.

Maybe you are right, maybe not, let's see. Unfortunately I kinda agree, because humans are really good at being lazy and going in the path of least resistance (including me). It's genuinely difficult to not use AI even if it makes my work worse, as long as it's easier and faster.

> ... with AI coding IS faster and quality is better than ever before.

Is this claim based on something?

I'm not against or "for" AI (whatever that means), I try to use it as effectively I can, but for me it's not at all obvious that quality is better than ever before.

Speed I can buy, especially in new projects and utilities, but quality? At least I haven't seen this in practice, if anything I'm just seeing more code, issues, PR's and pressure ==> more slop, more bugs, less quality.

You can always say "skill issue" and "process issue", but that's partly my point here, AI doesn't magically solve this.

Presumably it is based on that person's personal experiences, like your own comment and all the other comments here?

Speaking for myself, based on my own personal experiences, quality is by far the bigger advantage of these tools. It has never ever been easier to write automated tests and to automate tedious manual validation. I'm running my code through like 10x more paces than I ever did before, because I can just say "hey try running this in these twenty different ways" (including with browser automation, if that's relevant), without needing to either do the tedious steps to run all that or to take the time to write a script to do it, and to compile and attach the findings to the PR. This saves me hours to days of work on validation, but the reality is that I just wouldn't have spent that time in the past, I just stopped at a lower bar for quality, because I couldn't justify the ROI for spending all that time on it. But now the ROI is huge, so it's a no brainer.

If people are not taking advantage of this, then yes, that is literally a skill issue.

Good use case.

For some reason this reminds me A LOT of past discussions about microservices, most wonderful on paper and forever debated, but I've never seen it work out perfectly in practice, for me it's mostly been a cluster F in most companies that adopted them.

Currently I see AI similarly, in theory perfect, in practice I don't see the claimed effects. So yes, skill issue, but skills are relevant and your company probably can't hire a rockstar team (if that matters in the future).

But your comment on personal experiences was very good! Spot on, we are all biased, easy to forget. Thank you for that.

I agree with your analogy as well, and I'm very open to the idea that this will mostly end up being good in theory but bad in practice.

> If people are not taking advantage of this, then yes, that is literally a skill issue.

...or domain. You said "including with browser automation", so you do web or web-adjacent development.

Not all of us are doing that. What I work on doesn't have any UI or output besides a log file most of the time, but it connects to many places and does many things like an octopus, but nobody sees that, but feels that it's there because their environment keeps on working.

I said "if that's relevant" and you jumped to "this guy just makes websites". I do many things.

The octopus you just described sounds to me like an excellent example of what having the ability to more easily do tedious validation is most useful for. If you know that the "environment keeps on working", there must be some way for you to observe that fact. And if it is an octopus, it is likely difficult and/or to change the conditions and observe the correctness with respect to those changes. I find it so much easier to do this exact kind of thing now. Or, "easier" really isn't the right word. It's that the activation energy is low enough now that I'm able to do a lot of things up front that I used to rely on runtime monitoring to validate.

I guess YMMV, and it's not magic, but for me it totally changes the calculation on when it makes sense to automate something (like that chart from the old xkcd about how many times you'll do the thing and how long it takes to automate) in a way that means I'm doing a bunch of things that are useful for quality that just would never have passed the bar in the past.

> I said "if that's relevant" and you jumped to "this guy just makes websites". I do many things.

I didn't. I made a guess. I might be wrong, that's OK. I love to be wrong, because I learn things by being wrong. Also no offense was intended, and I don't consider webdev inferior anything. What I tried to mean is, if AI has more training data for a domain, it does better. If you fire the same model on a niche domain, it falls flat.

> If you know that the "environment keeps on working", there must be some way for you to observe that fact.

Yes.

> And if it is an octopus, it is likely difficult and/or to change the conditions and observe the correctness with respect to those changes.

Nope. On the contrary, because there's so much innate knowledge that is required to know what to do, simulating in mind, deploying and testing on real world is much easier and faster than letting loose an ML model on it. You need real data, real data comes in slow, but you can catch problems early and easily.

Considering it's a niche area, AI also doesn't have much training on that domain, so it's doubly inapplicable for what we do.

> but for me it totally changes the calculation on when it makes sense to automate something ... (snipped for brevity)

It's great that if it works for you, but YMMV part is way more correct than people want to accept and want to learn. AI is a pneumatic hammer, but not everything is a nail which can be driven in with that.

When it works, it works. When it doesn't, well people still pretend it does or insists it shall. We must accept the limitations.

> deploying and testing on real world is much easier and faster than letting loose an ML model on it.

No this is what you're not getting. It is "doing the things I would do to deploy and test in the real world, but faster and in the background while I do other things", it is not "letting loose an ML model on it". This is the new capability. If you have any process like "do {action}, wait until {something}, check {something}, determine if it matches expectation", it is now possible to run that loop way more times in way more variants without either spending the time on it synchronously oneself or writing a script to do it. (If you do that specific action loop often enough, it's probably worth writing the script anyway, but that's also much quicker to do now.)

The AI doesn't need training on the domain, it just needs to be told "these are the things I would do, please do them for me and report back".

I'm sympathetic to not everything being nail-like, but I really think you're leaving a lot of chips on the table if you can't imagine any of this kind of action-check-evaluate loop you have that you could offload.

The thing is, I already don’t get blocked by this process. I deploy automatically and continue working. If something goes wrong, monitoring warns me.

Everything is already at the background. That thing doesn’t need a CI, not the classical or AI enabled kind.

Maybe I was not clear about that part of what I do. Three minutes of something not working correctly doesn’t burn our world down.

I'm saying you could do more things. Instead of waiting until a new thing is merged so that it can be deployed and waiting for monitoring to catch issues, you could be deploying pre-merge to testbeds, using different variants of the code or different configurations of the whole system or different inputs to exercise edge and corner cases. Maybe it's too complex of a system to set up that kind of testbed or simulation? Well, it's easier to do that now too! I can spin up and down environments, either with containers on my workstation, or in cloud deployments, that I would have not attempted in the past, because it would have taken me too long to set them up. But now it doesn't take that long, and I find it super valuable to be able to try more things out. (Cost is still a real constraint, I'm not saying that constraints magically no longer exist.)

Obviously I have no idea what your work looks like! But what I'm saying is that time savings are not just time savings. There can be a point at which the time savings bring you under an "activation energy" such that it unlocks a new capability, not just a speedup. And some of those unlocked capabilities can be directed toward improving the quality of software. And I think that's awesome and useful, is my prevailing point here. I won't claim that it will usher in an industry wide improvement in quality or anything, but for me personally, I'm making better software more quickly now, and I'm very pleased that I can do that.

I compare it to math, you can't learn it by reading it you need exercises. Same thing, reading is not sufficient. Just my experience.

Hard agree on this. I feel like it's pretty normal, but I struggle with retaining information by just reading. I really need the manual typing step to cement the knowledge.

Yeah if it's done as part of learning effort, then I def agree.

I dont't really know stats about such things, but I assume the adoption of CC has been insane compared to almost anything, and is also like two years old? Yeah, not surprised.

But also, if they all use 500 agents daily and latest models can "one shot" everything, one would think 5k issues are fixed in no time...

Havent used CC in some time, worked fine last time I tried.

> But also, if they all use 500 agents daily and latest models can "one shot" everything, one would think 5k issues are fixed in no time...

IMO, CC should be the poster child of what LLM coding should "feel" like. If things are so good why are there so many issues? Why can't they get a handle like other well-run projects? As you said, they have unlimited tokens so this project should be close to pristine as much as possible.

Do you have a link? I would also be very interested in seeing the prompts, from what I've seen you still need to understand the field, maybe I'm wrong.

Why is this downvoted? Genuine question, I haven't followed up on the drama.

Because it's mostly not true. It is likely that it used the professors work, but the professor did not have a solution. It came up with new insights that solved the problem. Even the humans from the professors side said so.

> Because it's mostly not true. It is likely that it used the professors work

These two statements appear contradictory.

I simply said that it may have trained on a NYU professor's work.

Work that the professor did not believe he was releasing for model training purposes. That feels worthy of mention.

According to OpenAI the cut off date for user data was too early for that (one sided evidence, so I'll give this partial consideration).

The NYU professor was solving a different problem (no viscosity, aka the Euler equations). This is a big difference.

The NYU professors' blowup construction was fundamentally not the same, it was a donut with a cascade of smaller and smaller vortexes driven by each other. OpenAI has that picture they made but its inwards spiraling and speeding up vortex.

My overall opinion is that calling the work plagiarized is really underselling what the AI accomplished. It's like full on cope.

In particular, Buckmaster's main claim to plagiarism is this:

> “Almost nobody was seriously developing this particular constructive program for realizing C/D, and then OpenAI appeared in essentially the same general part of the landscape immediately after hearing about our progress.”

What this fails to realize, is that this only points to plagiarism if the counterparty isn't AI. They had actually launched teams on all cases in parallel.

Not only hard to understand, but if one uses it without first understanding the topic, it's a disaster.

Couple weeks ago I had someone linking to a doc page in my ticket "Made this summary, hope it helps". It was 100% AI generated, the person clearly had no understanding of the problem, it only mentioned UI changes although the thing required changes in many systems. Complete waste of time, I don't think I will humor anyone who makes AI summaries anymore.

True, but that's unfortunately (usually) not because the thing misfired randomly.

Yes. That’s “is used to cause harm” situation, exactly.