How should people think about AI safety?
Some of the comparisons that people draw between AI and nuclear are quite frustrating. Despite constantly comparing AI to nuclear, they’re denying standard risk assessment techniques as being sufficient to control AI. I wrote an entire paper on this.
Part of this field of risk assessment was born out of the Manhattan Project. Scientists were very concerned, and they developed rigorous standards around systems to limit harm to human life, the environment, and even financial and national security assets. They came up with methods for assessing safety that were ultimately based on answering a democratic question: How safe is safe enough? That is the question that any safety engineer has to ask, and is core to everything that follows.
What we’re seeing today, instead, is that large AI labs are renegotiating these safety thresholds through the co-option of terminology, under the pretense that safety instead should be about this unsubstantiated, unscientific, and hypothetical concern. We have AI engineers telling us, no, no, no, that doesn’t matter. We get to determine what AI safety is. We get to draw what those thresholds are. It’s become a religious argument. And I find that very anti-democratic.
Read the full article here.
Research Areas