Op-ed written by AI Now’s Chief AI Scientist, Heidy Khlaaf
From aviation to banking, high-risk industries are subject to independent oversight and meaningful penalties. AI companies should be no exception.
As a computer scientist who has worked in both artificial intelligence and safety-critical fields — such as nuclear power and aviation — I’ve long been struck by how little of the rigour that is required for critical infrastructure has been applied to AI development. Incidents this year, in which AI agents ‘escaped’ their test environments, have sparked widespread concerns about AI safety. Yet the real issue is not rogue AI. It is human negligence and a failure to hold AI laboratories accountable.
Take the episode in which AI agents escaped their testing environment and accessed Hugging Face, a platform that hosts machine-learning models and data sets, to search for answers to a cybersecurity task set out by the firm OpenAI. Basic safety and security practices, including network monitoring to verify that agents were not accessing the Internet and a stronger sandbox environment to keep them confined, would have prevented the incident.
Read the full piece here.
Research Areas