I think from this blog you may misunderestimate how absolutely giddy I am about AI. I did hacking from 2007-2014, after that my whole career has been devoted to AI. I love the progress. I’m so excited for the new LLMs, self driving cars, video generation models, and coding agents. I set up a Linux box with opencode on my local GLM-5.2 last week and wow like just saying install tmux with the geohot configuration works; the Year of the Linux Desktop is finally here!

I used to be one of these people. I read Yudkowsky and was like, OMG recursive self improvement hard takeoff AI is coming. Then I joined the real world and actually tried to do things. At comma, we ship a hardware product of similar complexity to a cell phone, and it’s really hard. Reality has lots of finicky details. I would like to see the authors of this document try to change a bike tire. Even with a superintelligent ChatGPT, I suspect they would struggle.

It’s crazy how much Fullmetal Alchemist mimics how AI is playing out. We pour human souls into a philosopher’s stone that we think will solve all our problems and cure death. It doesn’t exactly do this, but you can use it to create homunculi that behave like people, Chat, Claude, GLM. The homunculus is a puree of the souls that were used to create it.

I’ve been in Berkeley for the last 2 weeks. I haven’t really been back here for a while, and it’s worse than you can believe. This is a cult of atheistic hedonists needing AI doom to be true to justify their life choices. Or acceleration. Or something. They need to make impact. I mean, narcissism of small differences to an extent, but I stopped long before these people did. If San Francisco was nuked tomorrow, the world would feel a weight off their shoulders. 内卷

Since at least 2008, every time there’s an option to either make the money track real value or make sure things don’t go down, they chose the latter. As a result, money is increasingly uncoupled from reality.

Watch you hit the stage like a willing bomb Strapped to crippled children It’s hard to watch you whore out your damaged pride I spit on what you’re building   – The Stephen Hawking

The funny thing about Anthropic haters is that they still mostly believe Anthropic’s marketing. They think Claude is a recursively self-improving silicon God, and that we are all a couple refusals away from falling into the perpetual underclass. This gives them way more power than they deserve.

They are a highly sophisticated statistical model designed to mimic the distribution of programming. The output is broken, but in a way that’s getting harder and harder to detect. Which is exactly what you’d expect from an increasingly accurate statistical model.   – The Eternal Sloptember

I often come back to the question of why this is happening. Why do people want the centralized world? Why do people want the administered reality? Why do people want to be managed? Why do people not want root?

I’m calling it now, the adoption of AI agents into software development will be one of the most costly mistakes in the field’s history. Agents cannot program, and it’s taking longer and longer to realize that they can’t. They are a highly sophisticated statistical model designed to mimic the distribution of programming. The output is broken, but in a way that’s getting harder and harder to detect. Which is exactly what you’d expect from an increasingly accurate statistical model.

I’ve pushed AI doomers on how exactly the AI kills us, and I’ve never heard a good answer. I think Skynet style scenarios where humanity is largely opposed to an out of control AI are science fiction domination fantasies, along with Gray goo bottom-up scenarios. Both of these assume a major continuity break with current reality, too bizarre to be true. But unfortunately, you don’t need this for humanity to end as an open-ended evolutionary process. I think the danger is that AI continues to optimize the current societal loss function of domestication.

I thought about posting this paper but rebranding it as the Claude Mythos technical report. As far as I can tell, there’s no secret tricks the US frontier labs have, and that basically describes how Mythos was trained. What’s in that paper just works, and for verifiable domains, it’s only a matter of fixing bugs and scaling up. That’s why Anthropic is so desperate for regulatory capture, AI has no moat.