The code is in _C_ the syntax highlighter says _C++_. _C++_ highlighting works with _C_.
A number of RTOS allow the use of restricted C++ with complex features disabled; often called C with classes. The compiler used will most likely _C/C++_ versus only _C_. Example: µC/OS-II used in military, aerospace, and medical applications. [0]
at first this looked like something one could run on CPU with 64GB RAM with a 2-3 bit quant, at possibly half the speed of 3.6 35B-A3B, however the 50B ngram sidecar makes it impossible. and oddly, unsloth's page lists the ngrams as 50GB even though they say it's in 4 bits. should be 25GB according to my math. anyway, the new ngram architecture makes it pretty much unusable for regular folks who cant afford more than 32-64 GB ram in this RAMocalypse.
I suspect we will see optimizations where the various vectors of the n-gram you actually use are hot in vram, the rest are warm in system memory and then cold storage on nvme. Same with MoE. If your workflow is particularly same-y then you're looking at cache miss below 5% with NTP/MTP turned on and the right harness. Agentic "openclaw" type stuff cache miss might be below 1% in the right local llm setups. There's been zero exploitation of n-gram stuff yet, it will be very interesting as things progress.
that's why they're called "the company" in insider circles. they still do, openly and probably also covertly. their investment arm is called in-q-tel https://www.iqt.org/portfolio , funding things like gitlab and docker. they probably run half of silicon valley. the dead giveaway is a funding story about 2 guys in a garage or dorm.
reply