I have no proof, only personal experience interacting with commenters on HN. No other tech company gets as much positive support when exerting its soft power under the guise of (bullshit) technicalities, e.g. all the issues with EU digital markets, API restrictions, 30% tax. Specifically, people were defending Apple vehemently when they feature gated remote phone control for EU citing concerns about EUs privacy laws (???).
I never stated that I know for a fact that Apple deploys bots, but I would not be surprised. I don’t even need to know if anyone is paid or not - if I cannot distinguish paid posts from fanatic apologism, then I need not care. The resulting posts are worrying all the same.
Similarly, I do not know if Apple was caught off guard by the demand - I wouldn’t trust this article. But Apple does plan production well ahead of time and forecast demand rather precisely. They could’ve been wrong. Them being unable to produce enough 1/2tb mac studios seems like a loss to me.
Interesting that Samsung still pursues PIM. IIRC they had a paper in ISCA21 or 22 where they showed HBM2 module with PIM, which back then impressed me quite a lot.
That being said, I am not sure what’s the killer application for this technology, and without such application adoption is unlikely.
As I understand it, the killer app is llms. You could run MACs directly in RAM, offloading a lot of work from CPU and cutting down on insane (external) memory bandwidth required.
Imagine (this is a fantasy pitch but potentially achievable for some use cases) wanting to run a larger llm and all you have to do is buy more RAM so it fits.
Sure, MACs are nice. However, unless there other, PIM-specific/optimal, algorithms, regular matrix multiplication algorithms like tiling-based won’t work here I think — how would the tile be shared? By doing read/write all the time?
Attention calculations aren't shared across more than one vector during next token prediction (thinking and writing) which this sounds almost perfect for. Per attention layer, for deepseek at 1M context, you want to broadcast a single 1KB vector to 4GB of dot products, and map reduce a 1KB vector back.
Rollie calculation, like the online softmax in FA, implies a centralized computing unit that does the compute and stores the intermediate results in its registers. With PIM you have no centralized compute unit, you have a bunch of memory, and a bunch of MACs all over the place.
How would you do map-reduce across multiple DIMMs w/o extra reads/writes?
PIM implies some sort of distributed compute, which can work for some cases, but I am not sure LLMs are one of them.
Is there a single reason why we can't just "distribute" the online softmax?
Each die-attached PIM accelerator computes online softmax for its own KVs. Then the central unit gathers the softmax intermediates, one intermediate per die, and uses those to compute the final softmax.
The PIM win is that we crater the memory traffic between the central accelerator and the memory dies for attention ops. Most of the attention bandwidth never leaves the memory.
This isn't "run the entire LLM in PIM", no - this is "offload the parts of LLM that benefit from PIM the most to PIM".
> Imagine (this is a fantasy pitch but potentially achievable for some use cases) wanting to run a larger llm and all you have to do is buy more RAM so it fits.
Isn't this how it works today already? Granted you wanted to run it on RAM rather than VRAM.
Yes, but running out of RAM is impractical due to low memory bandwidth.
According to the article/Samsung RAM dies inside can support way higher bandwidth than they expose, they're limited by external interface / bus width:
> Together, they can utilize the chip’s internal bandwidth across all 16 banks, which comes out to 614 GB/s. For comparison, regular DRAM accesses can hit two banks in parallel and max out at 76.8 GB/s.
And that's just for single 64-bit IC. So way faster and more power efficient.
You can scale with more memory channels. Workstation/server platforms go up to 12 or 16 channels if I remember correctly.
Consumer platforms have been stuck at dual channel for decades; most of it I attribute to intentional product segmentation. I'm hoping that LLMs might change eventually for an upcoming consumer platforms; going to 4 channel would be really nice.
We can run Doom on everything. Surely we can run some interesting apps on hardware that's originally made for AI. (One big moment for AI was when people figured out how to run it on hardware originally meant for Doom's successors.)
You have an eight socket server with 96 memory slots, you add 96x PIM memories into the server (optimistic), load all the LLM parameters or KV cache in RAM and exclusively let it perform GEMV and let it rip.
614 GB/s x 96 = 58,944 GB/s.
Alternatively, the memory is used for embedded inference tasks. You can now upgrade from the limited single or two digit MB SRAM accelerators to reasonably fast single digit gigabyte models. Without MoE you could reach 100 tokens per second with an 8B fp8 model on a single channel. With MoE you might break 500 tokens per second.
> I am not sure a vote of non-confidence would have helped here — there are only two parties.
But why are there only two parties?
The rest of the Anglosphere–Australia, Canada, Ireland, New Zealand, the UK–have multiparty systems–some are more multiparty than others, but none of them have the hard two-party system that the US has, in which third parties have a very marginal role (not completely marginal – e.g. in a close election, the decisions of New York's Conservative and Working Families parties can make a real difference to the outcome – but certainly at a national level third parties have no real power in the US.)
First-past-the-post is a popular explanation, but it can't by itself explain it, since the UK and Canada use it, yet lack a US-style hard two party system (and with the rise of the Reform party in the UK, the British system is becoming less two-party than it has been in a long time.)
Even if first-past-the-post could change, it's not the whole ballgame. To get on a state ballot you need thousands of petition signatures, and those signatures can be invalidated on technicalities brought up by major parties' legal teams. To be covered by the media or be included in a debate you need to poll at a high percentage. You can't get public funding to campaign unless you already had a high percentage of the vote in a previous election. Nobody will donate to you because the third party just detracts from the other two that will give the big donors what they want. And gerrymandering ensures the same people keep getting elected. All those roadblocks are in place because the two major parties put them there to prevent 3rd party challengers. The whole thing is rigged.
Just to show up and get 1% of the vote, you would need a grassroots national movement to elect someone who was wildly popular, who could build a 50-state network of volunteers (including legal, political, financial, community organizing, etc), sponsored by someone with extremely deep pockets that doesn't want anything from established political figures. And you'd still lose.
We have representatives already. The problem is we never hold their feet to the fire. We keep voting for the same useless bastards, and we don't vote them out when they do nothing. Half the country doesn't vote. That's our own responsibility we're shirking. Rather than get a new party, we need to get off our collective asses, and use the existing power we have to force the parties to do their jobs.
First past the post doesn’t explain historically why we have only two parties, but it explains why we will continue to have only two going forward unless something changes.
I actually think the US hard two party system isn't purely due to first-past-the-post, it is actually due to four interacting factors: (1) first-past-the-post; (2) presidentialism; (3) a deeply-entrenched primary system; (4) cultural homogeneity
Point (4) will likely strike many Americans as dubious, but the US doesn't have anything comparable to Québec – there is presently no US state where English isn't the majority language; nor anything comparable to Scotland or Northern Ireland (no state where a significant percentage of its population identify with a competing national identity). So, no matter how much (undeniable) diversity the US has, in the sense that counts for this purpose, it doesn't.
Hence: while abolishing first-past-the-post would no doubt do a lot to break the US hard two-party system, it could also be broken while retaining it if something happened to (2) or (3) or (4) instead.
But, in US politics, changing (2) or (3) is even more politically fringe than changing (1) is. Maybe that might change at some point though, who knows what the future holds.
And (4) will likely eventually cease to be true, but only in the long-run in which we are all already long-dead.
I don't mean that the two party system we have is due to first-past-the-post, it almost doesn't even really matter why we have it. But first-past-the-post is the mechanism which currently entrenches it and is the most likely to change. I think that (3) is a result of the system, not a cause.
It explains why you end up with two parties even if you start with more. The two largest will woo minority parties until they get a majority. And minority party members know they will be marginalized unless they are aligned with the majority (or hope to be in the majority some day.)
Changing that will require a radical reconsideration of democracy, like replacing FPtP with... something.
It doesn’t because the UK has FPtP yet doesn’t have a hard two party system.
Scotland, Wales and Northern Ireland have nationalist movements that want out of the UK-state independence movements exist in the contemporary US too, but they are very fringe
But even England has not just Labor and the Tories, but also the Lib Dems, Reform, the Greens, Restore
So England proves you can have multiple parties with real power, with FPtP, even in the absence of regional-nationalist movements
I dont see how the uk system's small parties are not marginalized. The larger parties still claim outright majorities with 35% of the votes or less (last time 172 seat majority with 33.2% of the votes) so they are irellevant.
If I am not mistaken all disruptions to fpp two party systems eventually settle back into them, maybe not the same parties but two nonetheless.
The smaller parties are all aligned with one of the two major parties. In a lot of ways they act more like caucuses than separate parties. Like Bernie Sanders being "independent".
That's overstating it, but it's why the largest group always can form a government. They go to the same aligned parties looking for a majority.
In 2002, Hillary Clinton voted for the Iraq War. In 2008 Obama used that fact successfully to [help] win the Democratic nomination and went on to beat McCain.
> all for profit companies are greedy by their nature, but US companies are on a different level because you can 'buy' the government by lobbying or supporting their election campaigns
How would you call decades of chinese state-sponsored industrial espionage of western IP? Benevolent act?
> How would you call decades of chinese state-sponsored industrial espionage of western IP? Benevolent act?
I want to share 3 things here, to think out loud:
1. lets not go too far to the past, because I will say industrial espionage is less harmful than imperialism. Imperialism costed millions of lives across India, Pakistan, African nations, Indonesia and even in China, we are talking about how the West got extremely rich, which allowed it to live in prosperity, when everyone else were struggling (impact of it is still ongoing for some nations). I am fine with lesser of 2 evils, which is an industrial espionage in this case.
2. I didn't say 'benelovent act'
3. this is not baked by a science, just a personal preference, I prefer if country supports companies to advance the country, rather than allowing companies get too rich and bail them out with taxpayer money if they fail because of their dumb actions. (not saying one is better than other, I like how CCP elevated China from 3rd world country to a level, even the US companies are scared, in very short time, without invading and extracting resources from other countries. But it obviously has its own problems (e.g. how competitive is the life in China for kids and so on), I am not saying China is perfect, but China became superpower in the last 30-40 years, which is an impressive accomplishment.
Sure, but not the way West conducted it, by going and killing generations of people. Look at India, before British colonization they accounted for almost 25% GDP of the world (~1700s), now it is around 3-4%
Or lets look at China itself, before the opium wars and British colonization, it accounted for ~30% of the world GDP, by 1970s it dropped to 4-5%, now climbing back to 20%.
Indonesia would be another example.
You keep defending the West, even though its actions were much worse than "industrial espionage", because you probably live in the West and feel nice about your day to day life and regular things like education feels granted, and not seeing the enormous impact of the West's evil actions to the lives of billions of people around the world in the last 200 years. (US hasn't even stopped yet, see Cuba, Middle Eastern countries)
> Sure, but not the way West conducted it, by going and killing generations of people.
Ridiculous statement. China is doing it today — uighurs are actively erased bit by bit.
> Look at India, before British colonization they accounted for almost 25% GDP of the world (~1700s), now it is around 3-4%
This statement is rooted in “zero sum” logic, and based on the assumption that without colonization India should have stayed with its 25% or more share. Hypotheticals are hypotheticals, and nothing more.
> You keep defending the West,
I’m not defending anyone. I find it ridiculous that you are so high on “the west is bad, China good” horse and completely oblivious to so many things that China does, which are objectively worse.
> I find it ridiculous that you are so high on “the west is bad, China good”
Can you read what I have written? if not, let me copy what I wrote earlier here:
> I am fine with lesser of 2 evils
> But it obviously has its own problems (e.g. how competitive is the life in China for kids and so on), I am not saying China is perfect
I am not saying China is good, I am saying it is lesser of 2 evils with much less negative impact to the lives of human beings around the world, in my opinion.
I am not sure we are arguing here on the merit of the argument itself. You are fine with China’s genocide TODAY because China is the “lesser of two evils”?
I do not understand this. It sounds like a cultist/religious position.
> Sure, but not the way West conducted it, by going and killing generations of people.
But they did. Look at Xinjiang, Tibet, and the history of Manchuria, among others. They did it to themselves as well, like most backwards dictatorship. And "the West” does not really do genocide anymore, either. This is more a consequence of it being a terrible strategy than Chinese benevolence (or anyone’s, really). At some point we realised that direct administration is a burden and that it’s much more cost effective to bribe nominally independent governments without having to deal directly with local populations. China does this a lot, just like evil western imperialist countries. Empires look different in the 21st century than in the 19th, that’s all. And where would you put Russia in a neat west-rest of the world dichotomy?
> You keep defending the West, even though its actions were much worse than "industrial espionage"
“The West” is not a monolith. It’s made of several countries, some of them better than others, and each one of them with its good and bad sides. It is not useful to work with generalisations like that. It obscures any information or lesson we could get and leads to wrong conclusions. It’s also counter-productive because for all its evil, there is nothing “the West” did that is unique. Except in scale, but then it’s more a quirk of history than something being intrinsically evil in “western” people.
I would say that saying about anything “whataboutism” is a cheap way out.
China does industrial espionage for the same end goals as American companies work on new things: market domination. So, I find it funny when someone is mad about American companies but not Chinese, while both pursuit the same goals.
> I wonder just how different their processes really are
Night and day.
Apples has in-house silicon dev groups starting (at least) 2008 w/ acquisition of PA semi for their low power CPUs, followed by the acquisition of Anobit in 2012(?). Apple does their own architecture, microarchitecture, etc, full cycle development.
That is why every subsystem must have means to detect the failures, to allow the replacement of the bad components.
ECC in DIMMs achieves this.
For myself, this has been the greatest advantage of always using ECC, that I have been warned early about the great increase in the frequency of errors in some DIMMs, after many years (over 5 years) of continuous usage, which has allowed me to replace the offending DIMMs and continue to use those systems for some more years.
The big companies consider them a failure in their own computers, so they use appropriate memory, but they consider that "consumers" either must accept them as normal and not a failure, or pay a great overprice in comparison with what a big company pays, in order to have memory devices with error detection.
Plenty of jurisdictions are single consent.
Moreover, what is the difference with using your phone to record?
reply