this is really neat. I wonder if people will start building open world games or "AR" games in the vein of Pokemon Go and Ingress based on similar tech. I've had a similar idea for a long time but don't think it was feasible before now due to AI
yes we're working on it! trying to push a few open world rpg games with real economy and game design. it's also super interesting to benchmark the current models' capability in this direction, since this is a naturally hard and multimodal coding task
I'm not sure about expanding domains, but I expect that very close to 100% of games are now using LLMs to a very significant degree during development, even if most won't admit it. The "AI disclaimers" for games need to go away, simply because it's just rewarding and incentivizing lying and punishing truth.
More than other AI moments in the last few years, this feels to me like an event in tech that will be seen retroactively as an important watershed moment.
I really appreciated the negative framing and critical tone of this author's overview. Having also read through the METR report, I fee like frontier labs' pattern of getting PR about how impressive their models are "behind the scenes" has poisoned their ability to take security and reliability seriously. Sure these are new failure modes and the agents operate at a scale that's difficult to combat, but the lack of controls and concern for mitigating these sorts of hacks in the future is crazy to me.
The model for postmortems I was taught which has served me well in my career is thoroughly answering the following:
- what happened / what was the timeline of events?
- how was it mitigated and ultimately resolved?
- what went well?
- what went wrong?
- where did we get "lucky" (meaning it could've gone worse but some arbitrary details about the incident worked out in our favor. usually stuff like "happened during business hours" or "we were already looking at a related thing that brought this to our attention before it was a worse outage")
- (action items) how do we detect, mitigate, and prevent this type of failure in the future?
I really hope OpenAI has done an internal postmortem that answers these questions thoroughly. Most SWEs in the industry have to do such postmortems for much smaller outages with way less impact and risk of societal harm. This is probably another area where regulation and governmental oversight would help curb the risks. What's to stop OpenAI and other frontier labs from an intentional "accidental" attack that results in gaining access to competitors' systems?
I'm also curious what, if anything, Anthropic and Google have done differently to prevent a similar event. I suspect they actually monitored the agents as part of their studies and had better guardrails in their infra for how they set up their harnesses etc. for testing models, particularly when the other guardrails are absent as was the case here.
> The document is dated May 2024 so is out of date, meaning ICE may not have access to every one of these capabilities at the time of writing. ICE has continued to spend hundreds of millions of dollars on peoples’ personal data and new surveillance technology.
I can't wait to see what kinds of horrific AI-assisted surveillance these organizations end up getting access to. The only saving grace of these sorts of post-patriot-act police state surveillance systems was that they still had to be driven by a human. Imagine this stuff combined with an "agentic" workflow where the "human in the loop" is the dumbest and most aggressive personality imaginable working for the US Gestapo
I think another aspect is the deep understanding one gains from building the thing, and the idea of the thing being entirely the creator's and translated into "reality" through some kind of physical or mental labor. An author of fiction prose would have their vision in their mind or some other ideas of what the prose "should" be, and then created it with their own word choice, and tools. Similar for visual art. This applies even if parts of the creative vision were improvised while the creation was happening, but it started in the creator's mind and if faced with the final product, they could remember the small details and understand the work that went into it.
I think if I'm honest about when I look at LLM output for code, it feels close to a code review of code written by another person, where I spend a decent amount of cognitive effort figuring out those similar small details in reverse, without knowledge or memory of the specific motivations. And furthermore, if I were to read every single line of code, word in a written work, and pixel or brush stroke in visual art of some kind, it would feel like an artificial understanding, rather than something I can fully sign my name to as a thing I understood the ins and outs of while creating it.
I fully get that some people don't care about this, and see themselves more as "designers", but I think those are the same sorts of people who if given a choice, would rather "build" by leading a team of people and claiming credit for "building" as part of their leadership, rather than by doing it themselves by hand and getting a sense of fulfillment from that effort. Also, "agentic" workflows feel a bit like giving up even the leading / designing aspects. With Claude specifically, I'm often impressed (albeit other times often frustrated too) by the small design choices it made while building the code that I hadn't explicitly asked for but end up preferring once I see the output. This to me indicates I didn't really build it
I've been using Fable on and off since the promotion started and have been fairly impressed with it, so I consider this great news!
At the same time, I sense a lot of negativity (especially on HN) in comparison to Codex from OpenAI (who IIUC is bleeding money to keep this going, though maybe Anthropic is too) which didn't do as many promotional and nerf-ing shenanigans.
As somebody who doesn't want to constantly switch tools, is Codex really that much better, or is this sour grapes from folks who want to stay on the subsidized access plans and are mad at Anthropic's pricing model?
I remember the hype around the first OnePlus phone, with the invite system. It was the first time I'd had a device loaded with Cyanogenmod that ran relatively stable without any issues. Eventually the capactive touch home button on my OP2 gave out on me but I was a pretty happy user of the OnePlus One, 2, and 3T. I especially liked the removable backplates that had wood options that were pretty neat and felt nice to hold
I suppose Nothing is carrying that torch forward but it's still disappointing to see. Even though most of it was extensions of Oppo tech and ideas into a US/Europe-friendly market position, it still felt like they were innovating and keeping Android ecosystem healthy and interesting beyond simple slab phones.
I was considering looking into a OnePlus phone as my next device for Lineage or Graphene OS, but I guess I'm glad I waited...
I'm thankful I tested Fable via my subscription before the cutoff. To me, it seemed like at least part of the improvements were in how it broke up work for a "one-shot" style prompt, but I was very impressed at how quickly and effectively it produced better results than Opus 4.8 at a handful of real-world use cases I threw at it from my own work.
Another interesting finding which I've heard others corroborate is that even though the per-token cost was higher, it seemed to orchestrate the work efficiently and burn through fewer tokens, roughly evening out to approximately the same per-prompt token usage as Opus.
Computers feel like a pretty good analogy for how AI will affect the workforce.
I suspect productivity will massively increase, the complexity and cognitive load of our work will similarly multiply, and yet we'll still being doing the now-more-complex work in some capacity for a similar number of hours.
Here ya go: - https://github.com/PhiloLabs/fable51-worlds/blob/main/union-... - https://github.com/PhiloLabs/fable51-worlds/blob/main/union-...
reply