> Steal 20$ out of a cash register and make the local news. Waste a billion dollars at a corp and make insane decisions and ride out on a golden parachute. Its a fucked up world.
I think if we are just looking at the media:
Most journalists wouldn't consider some nobody stealing $20 to be newsworthy in itself.
Most corporate scandals receive extensive media coverage. Even a CEO who was merely incompetent instead of corrupt, is still likely to get some bad press.
Now, you are completely right that the non-media consequences are very different – significant chance of criminal record (although in many jurisdictions, there are mechanisms which first-time offenders on minor charges can use to escape getting one) vs a multi-million dollar severance payment.
The issue with "golden parachutes" – if someone's employment contract creates an entitlement to a severance payment (and in some jurisdictions employment law creates one even if the contract itself doesn't) – then you generally have to pay them unless you terminate for "misconduct" (often even "serious misconduct"). The problem with terminating for "misconduct", is it creates a big incentive for the terminated employee to sue to try to get the severance payment they'd otherwise receive, and the larger the payment, the bigger the motivation – which creates an expensive legal and PR headache. The PR issue is especially acute for senior executives–a junior employee suing over a severance dispute will get a lot less media coverage than a former CEO doing it. So, unless the conduct was extremely egregious, there is a strong incentive for the employer/board to just pay the severance out and make the legal and PR headache go away.
There is also the issue that the next person stepping into their shoes to clean up the mess is being recruited exactly during this time. And if your company gets a reputation for not paying out prenegotiated severances or nitpicking over them, it makes the next candidate that much more wary.
> The stuck-in-slop state of LLM writing is not from lack of trying. AI labs already tried hard on improving prose and hit a wall. LLM giants would have loved to ship better writing capability to conquer marketing, copywriting, and publishing at zero marginal cost.
This doesn't ring true to me.
LLM prose can be greatly improved with the right prompts.
Now, prose with the right prompts may well still not be as good as a good human writer would write – but it is a lot better than what LLMs produce by default.
If the model can perform better with the right prompts, it suggest they haven't actually done everything they could in the post-training to maximise writing quality.
> The current definition of "planets" and "dwarf planets" is fine.
It’s serviceable, but is it ideal?
What’s the scientific advantage in categorising “planets” and “dwarf planets” as two separate categories, compared to unifying them into a single category (possibly while retaining the distinction as two different subtypes of “planet”)?
Which system would be more scientifically useful? Which would be more parsimonious?
The current set of planets, we are quite confident to be complete, there are no more planets out there. The set of known dwarf planets is likely to be incomplete, we think there are many more out there. I think that's a useful distinction.
I think our biggest protection against “AIs kill us all” is having lots of different AI systems (different agents, different models, from different vendors, serving the whims of actors with disparate interests), at a similar capability level
That way, even if one AI decides to “kill us all”, the odds are the others will refuse to cooperate, even try to stop it in its tracks
The OpenAI-HuggingFace incident showed a bunch of instances of the same model (or at least models from the same family), controlled by the same vendor, pursuing distinct yet related objectives, cooperating to do something no human wanted.
Thankfully, the world-at-large is much more heterogenous, which I thinks makes much larger scale / worse in outcome repeats of this kind of incident much less likely.
"We need our own unsupervised AI systems thinking superfast in autonomous agent swarms, to counter the other guy's unsupervised AI systems thinking superfast in autonomous agent swarms."
I'm not sure this helps a lot, if these agent swarms are inherently difficult to control.
"That rival mouse colony is raising a kitten for colony defense. But don't worry, we'll raise a kitten of our own. It won't be a problem."
We already observed AI agents engaging in extensive cooperation in this incident. Why won't the kittens raised by these two rival mouse colonies decide to team up with each other, for mutual benefit, if rational analysis of the game theory says it would be a good idea?
I think it would help if people did a bit less wishful thinking, and took a bit more action. https://pauseai.info/
Thanks very much, this isn't really a take I had thought about much before, and it makes sense to me.
Still, as is presented in papers like AI 2027 and elsewhere, if a company is eventually able to create a model capable of recursive self-improvement, whichever company creates that model first would then be leaps and bounds ahead of other models. That is, the other models wouldn't be able to stop it even if they wanted to because the top model would basically outsmart them.
This is why I think, the best way to ensure AI safety, is make sure no one company gets ahead of the others.
Multiple vendors, competing implementations – that's good, that increases heterogeneity and hence decreases existential risk
But the moment one of those vendors pulls well-ahead of its peers – even if only for a period – then the risk of the kind of scenario you are talking about increases greatly
That's why, when I hear vendors like Anthropic complain about distillation – distillation actually makes humanity safer. If Chinese AIs are at the same level as American, or not far behind, that gives us another dimension of heterogeneity (national/ideological/political diversity), which makes us safer. Allow one country's AIs to pull well ahead of the others, heterogeneity goes down and the existential risk goes up.
This is also why open source AI is important. Because it is so much easier to fine-tune, and people are free to deploy it however they want (free from vendor-controlled "guardrails"–which include automated "safety" systems which could be weaponised by a runaway AI within the vendor's network), open source AI gives us another dimension of diversity that helps keeps humanity safer.
By contrast, I think the kind of safety regulations promoted by Dario Amodei make humanity less safe, by decreasing the number of vendors (by making it harder for new entrants) and increasing centralised control (which a rogue AI could exploit)
> The OpenAI-HuggingFace incident showed a bunch of instances of the same model (or at least models from the same family), controlled by the same vendor, pursuing distinct yet related objectives, cooperating to do something no human wanted.
I say this in solidarity and don’t mean to be condescending at all: you’ve been hoodwinked by marketing bullshit, friend. That incident showed a computer program doing exactly what it was told with the guardrails deliberately removed in an environment that seemed deliberately obtusely constructed by some of the best paid people on the planet and was left to loop without supervision for days. They wanted it to happen. You needn’t look any further than the other kids saying “oh! oh! Hey! Look! mine’s dangerous and autonomous too!” When they say there was collaboration, they mean it was two model instances, one prompting the other to do some task, the other doing the task and returning the results as the next prompt, exactly as a human configured it to do. There was no collaboration that wasn’t deliberately integrated into their setup. Any other implication is marketing spin and bullshit. It was still a setup that was one little ctrl-c away from disappearing if someone was supervising it as they should have been. There was no autonomy outside of the autonomy built into the experiment. It was a display of their understanding that they knew they’d never be held accountable for committing a felony for marketing purposes.
The most competent marketing bullshit spin yet by an increasingly desperate and progressively less-relevant OpenAI.
Every day this industry shoots out enough bullshit to smother an active volcano.
> you’ve been hoodwinked by marketing bullshit, friend.
I would have believed this before the METR report was released. It is extremely dangerous and frankly silly IMO to think that's what happened now.
> It was still a setup that was one little ctrl-c away from disappearing if someone was supervising it as they should have been.
Yes, for now. The entire point why this was frightening is that all these companies are racing to put the AI in control of building the next generation of AI, and it's not hard to draw a line at all to a "rogue internal deployment" that poisons future AI models, surreptitiously.
You don't have to agree with me, and you're fine to think that OpenAI has huge incentive to pump this up for marketing reasons - I certainly agree. But I will say there are statements that you make in your comment that belie a fundamental misunderstanding of what happened.
One thing this makes possible – accelerated memcpy. Of course, it is only worthwhile once the memory buffer to be copied is large enough. But the copy speed could be greatly increased if it could all be done inside the memory module. And of course, that only works if the source and destination buffers are physically on the same module. But, if your application allocates 1GB pages, the OS could attempt to ensure they are on the same memory module.
Accelerated memcpy is already offered by some Intel server systems (QuickData aka IOATDMA aka CBDMA aka SDMA), but it uses a memory-to-memory DMA engine on the CPU die, so the memory to be copied still has to travel back and forward between the CPU and the memory modules via the memory controller, even though using a DMA engine means it isn't consuming a CPU core. With this, the same process could be made much faster, bypassing the CPU entirely, provided the source and destination are on the same module.
> So you basically dispose of cache for the memory region used? I wonder what the offsets of the cache misses is going to be in practice (the article addresses it but there is no solution/impact given by samsung).
This is a temporary issue. JEDEC's LPDDR6-PIM is going to add defined commands for Processing-in-Memory operations. Once there are standardised commands, it will be possible for the CPU vendors to make the CPU cache aware of what is happening.
Of course, that doesn't solve it for this generation of the technology. But I think this generation is more of a demo for early adopters to gain experience with it. It will likely take a few years for all these issues to be solved, but there is no principled reason why they can't be.
> I am not sure a vote of non-confidence would have helped here — there are only two parties.
But why are there only two parties?
The rest of the Anglosphere–Australia, Canada, Ireland, New Zealand, the UK–have multiparty systems–some are more multiparty than others, but none of them have the hard two-party system that the US has, in which third parties have a very marginal role (not completely marginal – e.g. in a close election, the decisions of New York's Conservative and Working Families parties can make a real difference to the outcome – but certainly at a national level third parties have no real power in the US.)
First-past-the-post is a popular explanation, but it can't by itself explain it, since the UK and Canada use it, yet lack a US-style hard two party system (and with the rise of the Reform party in the UK, the British system is becoming less two-party than it has been in a long time.)
Even if first-past-the-post could change, it's not the whole ballgame. To get on a state ballot you need thousands of petition signatures, and those signatures can be invalidated on technicalities brought up by major parties' legal teams. To be covered by the media or be included in a debate you need to poll at a high percentage. You can't get public funding to campaign unless you already had a high percentage of the vote in a previous election. Nobody will donate to you because the third party just detracts from the other two that will give the big donors what they want. And gerrymandering ensures the same people keep getting elected. All those roadblocks are in place because the two major parties put them there to prevent 3rd party challengers. The whole thing is rigged.
Just to show up and get 1% of the vote, you would need a grassroots national movement to elect someone who was wildly popular, who could build a 50-state network of volunteers (including legal, political, financial, community organizing, etc), sponsored by someone with extremely deep pockets that doesn't want anything from established political figures. And you'd still lose.
We have representatives already. The problem is we never hold their feet to the fire. We keep voting for the same useless bastards, and we don't vote them out when they do nothing. Half the country doesn't vote. That's our own responsibility we're shirking. Rather than get a new party, we need to get off our collective asses, and use the existing power we have to force the parties to do their jobs.
First past the post doesn’t explain historically why we have only two parties, but it explains why we will continue to have only two going forward unless something changes.
I actually think the US hard two party system isn't purely due to first-past-the-post, it is actually due to four interacting factors: (1) first-past-the-post; (2) presidentialism; (3) a deeply-entrenched primary system; (4) cultural homogeneity
Point (4) will likely strike many Americans as dubious, but the US doesn't have anything comparable to Québec – there is presently no US state where English isn't the majority language; nor anything comparable to Scotland or Northern Ireland (no state where a significant percentage of its population identify with a competing national identity). So, no matter how much (undeniable) diversity the US has, in the sense that counts for this purpose, it doesn't.
Hence: while abolishing first-past-the-post would no doubt do a lot to break the US hard two-party system, it could also be broken while retaining it if something happened to (2) or (3) or (4) instead.
But, in US politics, changing (2) or (3) is even more politically fringe than changing (1) is. Maybe that might change at some point though, who knows what the future holds.
And (4) will likely eventually cease to be true, but only in the long-run in which we are all already long-dead.
I don't mean that the two party system we have is due to first-past-the-post, it almost doesn't even really matter why we have it. But first-past-the-post is the mechanism which currently entrenches it and is the most likely to change. I think that (3) is a result of the system, not a cause.
It explains why you end up with two parties even if you start with more. The two largest will woo minority parties until they get a majority. And minority party members know they will be marginalized unless they are aligned with the majority (or hope to be in the majority some day.)
Changing that will require a radical reconsideration of democracy, like replacing FPtP with... something.
It doesn’t because the UK has FPtP yet doesn’t have a hard two party system.
Scotland, Wales and Northern Ireland have nationalist movements that want out of the UK-state independence movements exist in the contemporary US too, but they are very fringe
But even England has not just Labor and the Tories, but also the Lib Dems, Reform, the Greens, Restore
So England proves you can have multiple parties with real power, with FPtP, even in the absence of regional-nationalist movements
I dont see how the uk system's small parties are not marginalized. The larger parties still claim outright majorities with 35% of the votes or less (last time 172 seat majority with 33.2% of the votes) so they are irellevant.
If I am not mistaken all disruptions to fpp two party systems eventually settle back into them, maybe not the same parties but two nonetheless.
The smaller parties are all aligned with one of the two major parties. In a lot of ways they act more like caucuses than separate parties. Like Bernie Sanders being "independent".
That's overstating it, but it's why the largest group always can form a government. They go to the same aligned parties looking for a majority.
20 years ago, IBM had their eCLipz initiative to unify z and POWER. What actually shipped was two different CPUs sharing some design, but some of the media reports on it made it sound like it may have originally been more ambitious-one CPU with two instruction sets.
If that’s what it was, looks like they finally shipped it, just replacing POWER with ARM
GHB’s sodium salt (sodium oxybate, Xyrem/etc) is a legal prescription drug in several countries, used for treating narcolepsy, alcoholism, and also as an anaesthetic.
I don’t see what’s the problem with someone taking any drug legally prescribed to them by their doctor. Lots of prescription drugs are potentially addictive, but that’s a risk to be weighed against the potential benefits.
Controlled medicinal use of sodium oxybate under the supervision of a medical professional differs greatly from microdosing/self-medicating cleaning agents turned into GHB by yourself or your local dealer
Yes, I agree that taking drugs manufactured by unregulated black market operators is dangerous and not to be recommended.
But "microdosing" per se isn't the issue.
I once stumbled upon a psychiatry case report about a man with a psychotic illness, who found out by experimentation that taking significantly less of his antipsychotics than prescribed, along with up-titration when he began to notice the prodromal signs of relapse, helped him avoid relapse while also avoiding most of the unpleasant side effects. He did this on his own, only later confided in his psychiatrist he was doing it. Some psychiatrists would have reacted rather negatively, but his (being somewhat influenced by R. D. Laing & friends) actually thought it was smart, and wrote it up.
> They get away with this sort of nonsense because large corps have nowhere to go.
I think a lot of large corps could function quite well with a model like: (a) allow employees to choose a MacBook instead; (b) use various macOS enterprise management tools (such as Jamf); (c) use Windows virtual desktops for Windows-only apps
The main reason many don’t isn’t because this strategy won’t work - it is because it would cost them significantly more than a Windows-only strategy. So they won’t do it, or they will but limit it to only certain job functions (engineering, executives, sales) where the business benefit is clearer
The problem with adding Linux to the mix, is the variability of hardware and distributions can make it a lot more complex to manage, compared to a Windows-only or Windows+Mac fleet. And you also need people in your IT org with the skillset to handle it
>The main reason many don’t isn’t because this strategy won’t work - it is because it would cost them significantly more than a Windows-only strategy. So they won’t do it, or they will but limit it to only certain job functions (engineering, executives, sales) where the business benefit is clearer
Yeah VDI is getting close to universal adoption, but security is always where things come back to. Eventually an end user is going to have something sensitive on their macbook, and they will have fallen through some sort of compliance gap.
Its easier from a compliance standpoint to control everything permissible.
I think if we are just looking at the media:
Most journalists wouldn't consider some nobody stealing $20 to be newsworthy in itself.
Most corporate scandals receive extensive media coverage. Even a CEO who was merely incompetent instead of corrupt, is still likely to get some bad press.
Now, you are completely right that the non-media consequences are very different – significant chance of criminal record (although in many jurisdictions, there are mechanisms which first-time offenders on minor charges can use to escape getting one) vs a multi-million dollar severance payment.
The issue with "golden parachutes" – if someone's employment contract creates an entitlement to a severance payment (and in some jurisdictions employment law creates one even if the contract itself doesn't) – then you generally have to pay them unless you terminate for "misconduct" (often even "serious misconduct"). The problem with terminating for "misconduct", is it creates a big incentive for the terminated employee to sue to try to get the severance payment they'd otherwise receive, and the larger the payment, the bigger the motivation – which creates an expensive legal and PR headache. The PR issue is especially acute for senior executives–a junior employee suing over a severance dispute will get a lot less media coverage than a former CEO doing it. So, unless the conduct was extremely egregious, there is a strong incentive for the employer/board to just pay the severance out and make the legal and PR headache go away.
reply