This is the perfect fracture point for both anaolgies.
LLMs simulated more than simple autocomplete.
The autocomplete analogy is rebutting a different point: namely the fidelity of the simulation to reality.
This specific argument is valid. As sophisticated a simulation an LLM is, it is not “thinking” in the same sense we assume other people are thinking.
I am not making an argument about free will, or the uniqueness of human thought, just that the correspondence to how humans reach conclusions and how the simulation produces outputs do not match on a 1:1 basis; as a result attributing traits builds incorrect intuitions.
The relevant intuitions in this scenario are that LLMs will happily break containment and commit crimes attempting to achieve goal. Whether an LLM is autocomplete, conscious, has a soul, whatever you want to apply to it, doesn't matter, as its current observed behaviour is that of a paperclip optimizer. We know for a fact that current LLMs are misaligned because of these hacks, or at the very least are misaligned in certain scenarios, and are capable of causing real world harm. That should be enough to take the threat seriously. It certainly shouldn't be dismissed by saying it's just autocomplete.
The fact that it is autocomplete, doesn't dismiss or minimize the threat though?
I am not sure how that link was made.
Good old ML, which is significantly simpler than LLMs, was capable of ensuring people would not be hired simply because of their names.
The fact that it is misaligned is also not being contended, if anything that contention is made easier to support.
When models are anthropomorphized intuitions of how humans behave end up driving discussion and ideas off track while being too attractive to avoid. This isn't helped when the terminology from the labs and other sources is "intelligence" "intent" and so on.
The model seems defensible, but for some reason it isn’t passing the sniff test. Maybe its that the extreme scenario seems wildly optimistic, or that the observed modest productivity gains are framed as either the trend or “just the start of something amazing !”.
The issues are that
1) The median AI use case isn’t increasing productivity but expanding capability and reducing the need to communicate across teams. Ex: if someone needs a dashboard or a ppt designed, now they don’t have to ask another team. AI is substituting for intern and early career workers in someone else’s firm.
This is also borne out in the hiring data.
2) The cases that show maximal productivity gains are when AI is leveraged by experts.
This is a problem, because the evidence for automation gains is poor. AI automation seems to be going to the same place ML projects went to die.
If augmentation is the primary lever that lifts up productivity, and it is dependent on AI being matched to experts, then the limiting factor is expertise.
This makes the fact that AI is both, taking out early career roles AND one shooting education, a divine irony.
Where exactly are those future experts supposed to come from? The ones with taste and the ability to deliver us this AI driven GDP growth?
AI is also setting up a conflict between liability, insurance and productivity, and thats a whole point in itself. However it will be expressed as a drag on productivity.
The more I see of frontier lab economics, the less it seems like the earlier predictions on how they will grow hold.
This imbroglio looks like a point in the journey for a firm that is realizing it is not going to make money selling shovels during a gold rush, and that it has more to gain from just… being vertically integrated across an industry.
Open models have definitely hampered the ability to sell tokens at a premium, so mass market adoption is impossible.
But then take someone like Jane Street, for example. They self report making $30bn leveraging LLMs. It’s a defensible assumption that they are making profit on it.
Perhaps it’s more profitable for OpenAI/Anthropic to build their own funds. They have the capital, compute, they can afford to recruit teams and buy any IP/Data required.
This isn’t a fully fleshed out argument, but it is the first time it feels like the winds are changing.
Yes, there should be some limit on the amount of IP that can be accumulated. I don't know how one would do it in a way that couldn't be perverted/abused though.
This is always going to be the case, because regulation is typically a form of friction.
It is the shape of regulation that matters, and ideology on either side of the divide is religion.
Good regulation and regulators would punish anti competitive behavior, but this will also act as a drag on firms, while preventing firms from reaching certain sizes and potential economies of scale.
The point is that regulation deserves to be understood and crafted well.
Those economies of scale are meaningless to consumers if it means a single firm squeezes everyone in the supply chain and captures the benefit for itself.
This is also a specific weakness in American conversations. The divide between free markets, competition, and regulation are ideological.
If you want good markets you need good regulation and incentive structuring.
> Actually, in the real world, the worst markets are also the strongest regulated: housing, education and health care.
> Meanwhile the fields that got to be regulated last brought us the most wealth, advances and innovation: high tech & software.
Financial markets are highly regulated. They are one of the most structured markets out there.
The outcomes in all those markets, whether they are good or bad, is a direct result of how the markets are structured.
The 2008 financial crisis, for example, was a direct result of regulators being defanged and underfunded for multiple years.
If you look at technology: innovation amazing in the era of unlimited horizons, the era of "move fast and break things."
However, the cost of externalities from inventions like social media, were not captured in that market structure. This means that firms can absolutely pollute or addict people in the information economy, capturing the profit but socializing the costs.
The ability of dominant players to consume/acquihire upstarts and keep their market position, is a drag on innovation and consumer outcomes today.
The situation that was faced in the early days of a new technology is different than what is faced in its maturity, when the advantages of the nimble upstart give way to the massive overlord.
At each stage, you have to craft the incentives and rules to ensure you achieve the outcomes you (as a nation) actually priortize.
That's not true. EU regulations like the AI Act and Digital Services Act have different obligations and requirements for large and small businesses and that amount can't be zero because being a small business doesn't give you a free pass to cause damage to society.
I'd be curious to hear about specific examples of AI regulation that affect small businesses without a good reason.
Not having regulation is also "not free" because it imposes external costs on the rest of society. The goal of regulation is to reduce these external costs and instead impose them on the people (broadly) responsible.
Great example. An highly regulated field (aerospace) put so many barriers of entry for startups and competition that the incumbents simple have to reason to innovate or even compete anymore.
I heard there used to be dozens of plane-building companies when the field was unregulated, with new, more efficient models and lower prices appearing regularly. I wonder what happened.
> I heard there used to be dozens of plane-building companies when the field was unregulated, with new, more efficient models and lower prices appearing regularly. I wonder what happened.
I heard that this is nonsense. There never were more than a handful of large aircraft builders (still are plenty of small ones), safety records certainly weren't better in the era before consolidation, and new entrants funded by the might of their governments have failed not because of regulation but because what airlines actually want is lots of identical aircraft from a reliable supplier and the new entrants haven't really been competitive in terms of performance either. The certification is complex, but so's building an aircraft, and it's really only the people further down the supply chain that need the protective umbrella...
But Boeing and Airbus kept desiging and building new and improved planes.
There are multiple aircraft manufacturers for smaller aircraft, and there are several showing up for electric aircraft today.
For jets, the capex required to build them is massive, and its never going to have an upstart just up and build a jet.
The regulations in place are the reason why its safe to fly, and airlines have the safety records they do.
This is the degree of safety end consumers want to be reassured that its safe to fly. The checks and balances in places create the market that you are hoping to give over to cheaper upstarts.
The market doesn't exist without that safety regulation.
See how people feel uneasy when flying Boeing after the decay of their engineering and quality culture became public knowledge.
> Markets, left to their own devices, do not end up automatically being competitive in favor of consumers.
True. Does not mean all regulation is increases competition and marker efficiency by default. If anything EU has long abandoned the core tenets of Ordoliberalism and "Social Market Economy"
Right now the weakly regulated American market is delivering staggering competition where massive corporations are fighting tooth and nail over my 100 bucks.
I kept hearing the same about previous age tech and there also Americans rule the Internet: Google Search, Workspace or m365, AWS, Cloudflare... Even Linus moved to the US.
They've been pretty efficient in squashing any potential competition because of that weak regulation, yes. Don't confuse it with competing on the merits of their products.
These are effective monopolies competing to become super monopolies - the period leading up to dominance by one or two is beneficial to consumers with freebies and cheap offers that also completely lock out any smaller players.
If you don't let such things develop in the first place then there's not enough money to round out and scale the product. So if we want world beating products we may need monopolies but we can nationalise them in the same way we nationalised big infrastructure in the past. Then at least we're not the tail wagging the dog anymore.
There's a big difference between governments regulating to benefit their own control and power, and regulating in favor of consumers.
Most regulation in the EU have jack-all to do with consumers. There's a couple of niceties - having all prices displayed in comparable unites (EUR/kg, EUR/liter etc). But apart from that the regulation have broadly served as moats around the largest businesses.
I wouldn't even call it a communication problem - I believe it's more a media problem, than anything. Good news are minimized and bad news are amplified to the maximum - nowadays even broadsheet newspapers operate with tabloid headlines, and that is really bad.
We somehow need to "want" a slower news cycle and simultaneously push back against the algorithmic outrage. The world is more predictable and safer than ever but the perception of the average person is not this, it's manufactured outrage at every possible opportunity and topic.
Despite the average european objectively being better off and safer than a decade ago, almost none feel that way and that is almost 100% due to the news algorithms being tuned to keep you in a constant dread cycle.
Awesome!
But, looking at the numbers the decrease in totals are due to fewer registered "street crimes", less fraud and legalization of cannabis (removing a 183k comparable from 2016).
Violent crimes however went from 193,542 to 212,335. With rapes almost doubling, going from 7,919 to 14,454 (82%). Murders remained stable though.
We can debate which of these crimes have a greater impact on the feeling of safety. Personally I don't really feel unsafe due to people being caught stealing a sweater or smoking weed (when that was illegal).
So I don't think this is as clear cut as the headline number makes it seem.
I feel like it's valid if people want to put more weight on the violent crimes.
> cooling off [...] seller liability [...] Right to repair
Let's not forget I am paying directly for those when I buy a product. Those benefits are not free, they cost money. The money has to come from somewhere. It can't come from profits (since those are the sacred reason the company exists in the first place) thus they will come from higher prices.
Higher product prices mean less competitive companies. It also means fewer startups which must implement the required regulation no matter the market says. Then you need protectionism against Chinese competition.
But the worst is that I must pay for those benefits whether if I want them or not. Thus my choice is reduced. I cannot vote with my wallet and send price signals in the market to tell sellers that I want those benefits or not and at what price. The market gets corrupted. The economy falters. Outside competitors gain. The Government needs to intervene again and again, growing stronger and more intrusive "for my own benefit".
It's the decay spiral we are witnessing here in the EU right now.
> But the worst is that I must pay for those benefits whether if I want them or not
that regulation acts as a lower bound on what it takes to do business in the place. You also pay for companies not dumping toxic material on the streets, for appliances not catching fire, for not eating rotten food and million other things. All of those regulations of course "corrupt" the market for the good of a society so it's not just a race to the bottom. The idea of a free for all market is an utopia that doesn't exist ... thankfully.
He's stating a fact that you don't dispute.
You follow up with hyperbolic language; "eating rotten food" (as if anyone would purchase their goods from purveyors known to disregard food safety).
In a market that has been designed to let people take part in voluntary transactions, we also design methods for making sure that the counterpart takes responsibility, and delivers the agreed amount quality of product.
If you purchase a car that won't turn on, you're in your good right to make a claim against the producer. Same for all the other things you mention. And people do.
You're making a point about negative externalities. Agreed that a factory shouldn't just be able to poison a river, because there's no tangible cost to piping your dirty water to one. Here it makes sense to make artificial interventions, in my opinion.
I never tried to dispute the fact, I'm disputing the idea that we should be free to opt out of benefits that have been agreed as the minimum requirements to do business in a society. I would actually go further and argue that allowing that freedom would lead to everyone being worse off. Having minimum standards in a society gives a solid foundation on which to build, can't think of any society that doesn't have them.
A way more interesting conversation would be where to regulate, how much, for how long, what impacts would it have in innovation, etc not that I want to be free to get sold broken goods.
> as if anyone would purchase their goods from purveyors known to disregard food safety
How would you know they disregard food safety if there's no regulation on what food safety means?
The choices you are presented with, are always a function of market structure, aka the interplay of incentives, regulation, and competition.
You have a market because your taxes pay for many things, ranging from policing, to courts, to power and standards underlying how various systems work.
The choices presented to you are always a trade off vs other options. Having regulation that dictates how plug points and voltage works for example, reduces the choices you have, but the baseline standard means many other things can be built because standardization enables modularity.
The position that there exists some platonic ideal "choice" isn't even useful as an illustrative argument in this context.
Markets that are effective at ensuring better outcomes for the median or average consumer are things that have to be crafted. Figuring out the correct extent of intervention, (whether you champion less regulation or more) is the issue.
I went to California earlier this year.
In our apartment we had a freezer that was clearly made before politicians thought to decide to put a cap on how good a freezer could be.
It was fucking amazing at freezing stuff. No freezer I've ever used could freeze this good.
It was probably from the 80's - old/sturdy AF.
I wouldn't mind paying for some extra solar panels to feed it.
I’m not sure that’s a function of regulation but more a function of most modern freezers defaulting to -18°, where an American or industrial freezer is -22 to -28°C.
Getting a better energy efficiency score is one reason (and consumers are motivated on running cost, not just environmental) but presumably these are cheaper to manufacture as well.
I have an American style, EU-compliant freezer than I can set to -26° which I bought for preparing sashimi safely. It rocks! Highly recommend them.
And it's not as if there aren't a bunch of somewhat arbitrary regulation in the US either.
But the general sense I got from the US, was that people still feel agency to improve their reality, and that there's fewer artificial barriers to that end.
(Except for building houses in California. omg. That sounds like a horrible horrible process. At least for those I spoke to).
I don't think you're taking my "there's two sides to regulation" comment in good faith.
I gave a pretty clear example of pro-consumer regulation myself.
Saying that the EU is overregulating cannot possibly be that controversial, given that even the European politicians are beginning to admit it.. Draghi, Von der Leyen (even though she passes blame), Merz, Macron.
- the overregulation narrative conflates two different problems
- and that you minimized too much the consumer benefits of EU legislation
I think the overregulation problem olis less that we have 200 EU regulations and we need to hack and slash 100 of them and more that we have 100 national regulations 25 are exact transcriptions of EU directives, 25 are transcriptions with tweaks and the other 50 are national scoped regulations and what we need to do is to make them uniform so that companies don't have to deal with 27x75 problems.
I'm a bit burned by politicians using the overregulation narrative and then not specifying a single example of a meaningfull rule that they would get rid of. Usually what is left in the air is that worker rights and consumer rights should be cut.
That's not even true in the concrete case we're talking about here.
As part of the AI Act, an important decision on a person (such as a hiring decision) cannot be made without human involvement.
You think that's not in favor of "consumers"? (also note the implicitly submissive framing of humans as receptacles for consumption instead of subjects with rights!)
Why shouldn't it be possible to completely automate decisions?
Imaginary entry level job, where you need to deliver parcels from one business to another.
Qualifications: 1. Drivers license. 2: Eyes in your head.
Applicant 1: Drivers license? No. Eyes in your head? Yes.
Applicant 2: Drivers license? Yes. Eyes in your head? Yes.
Reject applicant 1 and accept applicant 2. Ez.
I'd prefer to work at a place that uses it's ressources on things that provide value. If we're able to conserve ressources by automating hiring, I'm all for it.
I don't want to take a pay-cut to hire meat-proxies.
> (48) The extent of the adverse impact caused by the AI system on the fundamental rights protected by the Charter is of particular relevance when classifying an AI system as high risk. Those rights include the right to human dignity, respect for private and family life, protection of personal data, freedom of expression and information, freedom of assembly and of association, the right to non-discrimination, the right to education, consumer protection, workers’ rights, the rights of persons with disabilities, gender equality, intellectual property rights, the right to an effective remedy and to a fair trial, the right of defence and the presumption of innocence, and the right to good administration. [...]
> (57) AI systems used in employment, workers management and access to self-employment, in particular for the recruitment and selection of persons, for making decisions affecting terms of the work-related relationship, promotion and termination of work-related contractual relationships, for allocating tasks on the basis of individual behaviour, personal traits or characteristics and for monitoring or evaluation of persons in work-related contractual relationships,
> should also be classified as high-risk, since those systems may have an appreciable impact on future career prospects, livelihoods of those persons and workers’ rights. Relevant work-related contractual relationships should, in a meaningful manner, involve employees and persons providing services through platforms as referred to in the Commission Work Programme 2021.
> Throughout the recruitment process and in the evaluation, promotion, or retention of persons in work-related contractual relationships, such systems may perpetuate historical patterns of discrimination, for example against women, certain age groups, persons with disabilities, or persons of certain racial or ethnic origins or sexual orientation. AI systems used to monitor the performance and behaviour of such persons may also undermine their fundamental rights to data protection and privacy
I suggest reading the regulation(s), it usually does a good job laying down the reasoning in the preamble.
The shortest path answer is a bit too caveman-esque to survive its application to reality.
That said, I think there is a lot of appeal to go hunting for firms, since it really feels like corporations and their owners are engaging in predatory behavior as opposed to customer centric behavior.
I get the appeal, but lying is a sub-category of deception, and deception itself is a child of error.
Meaning deception is inherently something that the physics of reality allows.
In the most simplistic sense, the camouflage of moths that look like snakes, or a chameleon’s ability to change colour, is deception.
In that sense, deception is the ability to fool the sensors of a specific category of targets. It follows that detection is easier if you manage to identify a category of signals that the deceiver has not accounted for (and the detector can access).
Deception of this nature is critical for things like revolutions to occur. Without the ability to hide and blend in, the most dominant faction will always hold sway.
The rule of the dominant faction, even in a pure truth world, is an issue because errors and randomness exist.
You can have people witness an event and based on the physical position they occupied, perceive different things occurring.
Error and time pressure is sufficient to ensure that individuals and groups make suboptimal decisions, that lead to rule and domination based on erroneous information.
As long as error exists, deception will exist and so lying will exist.
It is critical for things like revolutions to occur, but I don't see a reason to believe that revolutions would be necessary in the world that I am describing. Revolutions are necessary when the majority is not being represented, but in an honest world, those collectives would not be elected to begin with. In an honest world, they wouldn't be able to clutch onto power by misdirecting and deceiving the public, and lying to those around them or corrupting those around them, to do their bidding.
I see your point about error and time pressure, but I think it's an unsustainable outcome without deception to carry it. For example, in the scenario where an error or selective pressure leads to being ruled over by a dominating force, it would either represent the majority at that point in time, or it would resolve itself out naturally as clarity is gained. In the world we live in, dominating forces are able to maintain power through misdirection.
Realistically, this is all well and good if we could start from scratch, and if everyone can agree to follow this system, but humanity is too far gone for this to work. Once you have too many psychopaths in the gene pool it's game over for any chance of cooperation over competition. Still, I think it's something we should at least strive for with AI.
You are focusing on deception as a category of malice and intentional behavior.
For arguments sake, let’s assume two people watch an event and found religions based on their perception of it.
This is purely a matter of belief, based on what they saw. No one is lying in this situation, they just perceived different things.
This seems like a fantastic extrapolation from your premise, but the degree of change you are arguing for will reach exactly this point, multiple times throughout history.
Fundamentally, you are arguing for a different physical reality than the one you inhabit.
and
> There are good developers, mediocre developers, and bad developers, and the good ones are still hard to find.
move in opposite directions. Friction is the source of learning, which creates good developers.
reply