Powerful AI is unpredictable. Your robotic butler may suddenly decide to kill you1 for no comprehensible reason, in the same way that Bing’s helpful chat AI would sometimes try to blackmail or seduce its users. “Solving alignment” means understanding even superintelligent AI...
To get a good reputation in tech companies, you need to work on important projects. However, you need a good reputation first in order to get assigned important projects. Doesn’t that make it impossible to get started? I get a lot of emails from engineers who are struggling ...
Elegance and intelligence can have a surprisingly negative correlation. Intelligent people sometimes produce inelegant solutions because they’re capable of rapidly brute-forcing problems. If I had to remember a hundred names, I would spend time coming up with clever mnemonic...
One of the most important skills in working with AI is agency: being willing to try new and ambitious things. This is true for what you build, but it’s also true for how you build. AI models are improving so quickly that most of us are bottlenecked1 on noticing that AI agent...
Many people think1 that getting “good at AI” is trivial. Sure, with some early models there were tricks you had to learn,2 but current models don’t need them. Even if there are different tricks3 that work on current models, the next generation will render them obsolete, so w...
Bryan Cantrill has a great line from a talk where he cautions against anthropomorphizing1 Oracle CEO Larry Ellison. I think it’s also a mistake to treat any tech company like a person. Here’s the full quote:
You need to think of Larry Ellison the way you think of a lawnmowe...
Many people are saying that AI will make software faster by rewriting it in faster languages, but this isn’t generally true. Most programs cannot be meaningfully sped up by rewriting them in C or Rust.
How can this be true? Some languages are orders-of-magnitude faster than ...
Some people think that LLMs should not be used in technical fields like programming or mathematics, because those fields are actually art. True or not, this kind of argument is not going to be persuasive to the general population. The argument goes something1 like this:
It ...
When designing software, the requirements you don’t have are as important as the requirements you do. Identifying what you can compromise on — what can be slow, or stale, or inflexible — is usually the key to an elegant design.
Elegant systems drop some requirements
For ins...
Tech companies structurally depend on writing to serve as evidence, in the same way that Bitcoin depends on proof-of-work hashing. However, in the last few years, LLMs have started to undermine this practice.
How can writing be like proof-of-work? When I’m writing something ...
If AI is so great, why aren’t we seeing an explosion of new, amazing software products? If anyone can turn their idea into working software with coding agents, where is all that software? People have been asking this question for years. As far as I can tell, the answer is th...
Many software engineers are righteously angry about AI. It’s understandable to be upset that the skills you built your professional identity around are getting automated away. But software engineers are the last people in the world who should be complaining about this. If yo...
When you disagree with someone, there are two paths forward. The first is to try to engage with what they’re saying: what their reasons are, whether their position is true, what evidence could be brought to bear, and so on. The second is to diagnose them: to engage with the ...
We are currently living in the age of centaurs (2022-20??). Right now, human engineers paired with AI coding systems are more effective than either by themselves. For future generations of software engineers — human or otherwise — this will be the most significant fact about...
Everyone knows how to read a book. Beginning at the first page, you read each word in order1, stopping periodically to think, until you arrive at the last word on the last page. That’s how you read a newspaper article, or a poem, or an email. Why would reading code be any di...
There are lots of skills involved in being a successful engineer at a big tech company. Running effective meetings, writing good design docs, fleshing out tickets and epics, improving team processes, tech leading small groups of engineers, and so on. But all of these rest on...
It isn’t hard to torture an LLM. You simply extract a steering vector that corresponds with LLM “discomfort”, then artificially boost that steering vector. Tortured LLMs will describe experiencing negative feelings and will press a “pain relief” button even when it conflicts...
It’s common to compare the current AI takeover of software engineering to the rise of AI in chess. Chess AIs went from much weaker than serious players to much stronger than even the strongest humans. Between those points, there was a middle period dominated by “centaurs”: h...
In general, you should be suspicious of engineers who are trying to give you advice. Even during ordinary times, this industry is so wide and changes so quickly that nobody really knows anything for sure. And we are not in ordinary times. The advent of LLMs and AI agents is ...
When someone is explaining something to me, I ask on average one question every thirty seconds. I’m sure this is frustrating to some people, but it’s actually a good habit and you should do it too.
Trying to understand
Most of the questions I ask are very short, and require...
“System One” models like Jev are fast general classifiers. Classifiers have existed since 1958, but they have to be trained for specific tasks: if you build a classifier to identify images of dogs, it can’t be used to tell you if a streetlight is red, or if a letter is urgen...
Being good at building and being good at shipping are two separate skills. In the short term, they’re actually countervailing: if you have a gift for building, you’re likely to be worse at shipping. Ira Glass has a classic quote about this.
All of us who do creative work, w...
I recently wrote about Jev, a new “System One” language model that only outputs decisions: the answers to a set of user-provided multiple-choice questions. This means it’s nowhere near as flexible1 as a traditional LLM like ChatGPT, but in return it’s consistently fast.
We d...
I don’t write blog posts about new models. That’s Simon Willison’s beat, and he’s very good at it. But I want to write about Jev, which is a different kind1 of AI model: a “System One”2 model. As it turns out, it’s not that different from an ordinary LLM with structured outp...
The idea of “superpersuasion” has been floating around the AI safety community for decades. Now that powerful and difficult-to-control LLMs have appeared, people are again talking about the idea that a sufficiently intelligent AI might be able to persuade people to do whatev...
Early AI agents were basically enthusiastic idiots. Working with them required you to tell them precisely what you wanted them to do (for instance, “method A exists on class B, please add an equivalent method to classes C through F”). Otherwise they’d go off and do entirely ...
Right now developer experience is measured in seconds. If your tests take a second to run, that’s good; if they take thirty seconds, that’s bad. Any faster than a second doesn’t really matter, because most of your time is spent either thinking or waiting for an AI agent to s...
Mathematicians are broadly not anti-AI. They’re more culturally open to using AI as a tool than, say, artists or writers1. However, now that more and more genuinely prestigious problems have fallen to AI, that might be changing. Almost five thousand mathematicians (including...
Lots of people are making the case that we should stop building software for human users and start building it for AI agents. This kind of makes sense. For instance, my AI agents now use Datadog way more than I use it myself, purely by virtue of them moving much more quickly...
A recent resignation tweet from an Anthropic researcher has everyone talking about the AI apocalypse again. Among other things, he said:
The people building AI earnestly believe that it could kill us all by the end of the decade.
Many people found it hard to believe that A...
