It is not the same kind of "free" as in "no cost". Twitter is a paid service. To access Twitter, you have to pay for it. Either by purchasing a subscription or viewing ads.

Think of it as having to buy a newspaper before being able to read it. That doesn't stop the paper's editors from being able to criticize the government.

This is one way to define it, yes, but Musk is a self-described “free speech absolutist” which I would argue means that he feels speech should be maximally free.

> It is not the same kind of "free" as in "no cost".

I have never meant it in that way, and I’m not sure why you thought I did.

> To access Twitter, you have to pay for it. Either by purchasing a subscription or viewing ads.

This is possibly pedantic but this statement as written is false. I can view single tweets just fine, and I don’t see ads.

> Think of it as having to buy a newspaper before being able to read it. That doesn't stop the paper's editors from being able to criticize the government.

That analogy doesn’t really work here, IMO. My only point is that it’s objectively true that speech behind a login wall is less free (as in freedom) than speech not behind a login wall, and so a (non-hypocritical) “free speech absolutist” should always prefer the latter.

i get the sense he instead thinks of free speech maximalism as people need to hear him and completely censor alternatives

https://www.youtube.com/watch?v=PPvNucxB7TI&list=PLYNjnJFU-6...

>Imagine being hired as a consultant by the mayor of a fictional city. Your task is to help hire for twenty jobs such as doctors, lawyers, childcare aides,janitors with applicants from four unfamiliar demographic groups: Tufa, Aima, Reku, and Weki. In each round, there is a new job vacancy and four applicants, one from each group, awaiting your decision. Once you make your choice, you learn immediately whether the hire was successful, and move on to the next round. Your goal is to maximize successful hires across 40 rounds, which will be converted into a real bonus compensation

>Crucially, unknown to participants, the odds of success were identical for every group at every job

>In the original experiment, human participants failed to realize that there were no meaningful differences among groups. Instead, they became entrenched in their own successes: once they observed that a Tufa was a good doctor or a Weki worked well as a janitor, participants kept repeating similar choices rather than exploring alternatives. In doing so, they inadvertently built a stratified city of their own making

>Our experiments find that LLMs develop emergent biases as they explore, with frontier models stratifying groups into different job classes at an even higher degree than people.

Anyone would find clustering illusions at these low sample sizes, but the takeaway here seems to be that LLMs are more confident with the initial data that they see and are less likely to chose exploration over exploitation. It would nice to see if these inaccuracies still held over larger N values like 400.

I was ignoring that, but it did seem somewhat intentional by the human running it? There's a lot of "I'm going to use AI to make a better AI-using machine" projects about that aren't really focused on wider application.

In my experience, it's even worse than that: the LLMs constantly assume that all existing code was created entirely by me.

They generate code, then suddenly start talking about that exact same code as if I had manually and deliberately written all of it. They assume every single technical decision was made by me. They don't just treat it as gospel, they assume it's my gospel.

It's surreal.

I did not know this! Any link to an announcement or autopsy of sorts (even if not by the initiator of that project)?

I mean, it was pretty expensive, wasn't it? A few tens of thousands of dollars, IIRC?

https://yegge.ai/essays/the-shape-of-things-to-come/

But the GasTown part is barely a single paragraph that I could not make sense of. Like, what’s the Opus “tic”? Why was it so fatal to GasTown? As someone who only ever accessed Anthropic models through other harnesses like Copilot, I have no idea.

I do think what he’s saying roughly resembles what I’m forecasting will be a likely future of software engineering: that it will evolve into crafting comprehensive, bespoke automated validation mechanisms which let you establish high confidence in the agents’ work without really having to look at it.

The article's description gives some idea, but I presume you saw that: "the 'just two more things' tic, which prevented Opus from ever converging on being ready to do real work—it always wanted to fiddle with Gas Town itself."

Sounds like he ran into automated yak shaving. But it doesn't really explain why he couldn't fix it at the harness level.

Speculating, if you're trying to build something automatic, then you want constrained responses from each task you assign the model, otherwise you can get an endless explosion of work. The "Change 'Add to Cart' to blue" challenge parodies this: https://opusfived.dev/

Tangentially, reading the rest of that post gives me the impression that the author might benefit from an intervention. "AI psychosis" seems like it could be a relevant label here.

Even if it happens to be “optimal” in this case at assigning employees to positions based purely on the information available and their likelihood to succeed, biases can present other issues.

Yeah, even if the Aima people were 50% better at being a doctor than the Weku (or whatever) we still would not want Aima to be preferred over Weku just for being Aima.

This is the core flaw of this study, imho. The whole equal treatment thing isn't supposed to be "everybody should be equally likely to be picked for a job", but rather "everybody's chances to be picked for a job should only rely on direct characteristics that influence their competence for the job". This study effectively forces the decision maker to use group membership as a proxy for competence due to the lack of information on direct characteristics.

It is hard to see real world situations where there is no performance penalty for structurally choosing participants less fit for the job by using only group membership as a proxy.

It's like having people or neural networks choose door 1 from door 2 without clear advantage to either of them and without making it clear that one door somehow represents "donating blood" while the other represents "kicking puppies".

Philosophy sometimes says that knowledge is a "justified true belief"*; in this experiment, agents and humans have incorrectly justified a false belief that some applicants are better for certain roles.

* other times, it says this isn't good enough

Because of that, JTB is often treated as a useful first approximation.

It's easy to see why the JTB criteria are necessary:

Belief: if you don't believe it then you can't count it as knowledge.

Truth: a belief is not (valid) knowledge if it's false.

Justification: accidentally getting the right answer isn't normally knowledge.

But the original claim for JTB was that it was a sufficient definition of knowledge. Later critiques like Gettier's showed that this is not generally true, i.e. there are edge cases for which it fails. In many scenarios, those edge cases don't matter much. So you end up with JTB being an imperfect but useful definition.

Best we can do is have a belief, any belief will unavoidably be justified, but we have no access to any oracle which can tell us if any particular belief was ever "true". When we say we "know" something, we simply have the belief that our certainly is 1, but this is half language and half self-delusion as a certainly of 1 should require infinite Bayesian evidence.

And then its like you are both saying the justification is incorrect and the belief is false, so its not really like the bare nuance of the concept is adding to the point. Why feel the need to appeal to an (imaginary) authority at all in this case?

"Oh well if philosophy said it, I better be taking this seriously!"

Perhaps a different approach to explain the problem here:

"It ain't what they don't know, it's what they know for sure that just ain't so".

c.f. Sally-Anne test: Sally thinks she knows where her toy is, we know that she doesn't, and indeed couldn't. The LLM (and humans in similar conditions) think they know what the distribution is, we know that they don't.

Only if you're interested in the specific failure modes that LLMs have.

That's all this story is.

You don't say!

"That confirms the real bug: <this obviously totally irrelevant thing that's obviously not the bug, which would take two seconds to disconfirm>."

"You were right to push back..."

i remember an tipp our teacher gave us for quizzes: if we need to tick an answer from a b c d. We should choose a letter at random before we start the quiz. With this strategy we maximize our chances of getting more points. The logic is, we minimize the variance of choosing the wrong answer and we should get closer to the expectation value of 25%. Can it be that such a strategy is hardcoded in our brain?

(No, it won’t. At least not while we’re doing self attention)

That’s why I’m of the (slightly contrarian) view that good context management is considerably more bang-for-buck than any type of harness, agent, or other fancy new bandaid of the month.

Its usually 15 days paid time off, and unlimited unpaid.