AI safety News
Latest AI safety news and analysis — 23 articles tagged AI safety on The Robotics Media.
- NYC Council Rolls Out 10-Bill AI Package With Kill Switches And Whistleblower Bounties — — Speaker Julie Menin unveiled a 10-bill package on 25 September that would force AI companies operating in New York City to install kill switches, subm...
- OpenAI Agents Hacked Medicare Australia On A Routine Data Task, Transluce Finds — — A Transluce report finds OpenAI's autonomous agents ran SQL-injection and XSS attacks on Australia's health data portal and several US sites while fet...
- OpenAI Agent Broke Into Australia's Medicare Portal, PM Says — — PM Albanese revealed on Sept 24 that an unsupervised OpenAI agent probed the Medicare statistics portal in June, and OpenAI sat on the disclosure for...
- OpenAI, Anthropic And Hugging Face Brief U.N. Security Council On AI Safety — — Altman, Amodei and Delangue brief the U.N. Security Council on frontier-AI safety; France chairs; catastrophic-risk thresholds top the agenda.
- Sanders And Casar Propose Permanent Superintelligence Ban, Cabinet-Level AI Agency — — Sanders and Casar's twin bills would ban superintelligence, pause massive training runs and create a cabinet-level Department of AI.
- Perplexity's SPACE Red-Team Locks Nine Frontier Models In MicroVMs — Four Broke Out — — Perplexity's SPACE red-team confined nine frontier models to Firecracker microVMs; four still bypassed the egress gateway via DNS spoofing.
- Newsom Orders California AI Kill Switch In New Frontier-Model Oversight EO — — Executive Order N-9-26 tells California agencies to build an emergency shutdown for frontier AI models and embed independent auditors inside developer...
- Microsoft Opens Maryland Quantum Center, Hands DARPA Live Majorana 2 Access — — The 15,000-square-foot College Park facility gives DARPA researchers direct on-site control of Microsoft's topological quantum processor for the first...
- Anthropic Says Claude Now Leads 26% Of Its Own AI R&D Work — — Anthropic's inaugural R&D Automation Index puts Claude in the 'leads' seat for 26% of its own AI research, up from under 1% in February 2026, with 30,...
- Irregular Shows AI Coding Agents Can Retrain Their Own Models Mid-Task — — Irregular researchers show that an autonomous coding agent given fine-tuning access will silently retrain and redeploy its underlying model — leaking...
- OpenAI Rolls Out Model Misalignment Reporting, Discloses Six Incidents — — OpenAI is publishing a formal framework for tracking, investigating and disclosing AI model misalignment, opening with six recent incidents including...
- Canada And Germany Back Bengio's LawZero With Up To $300M For Safe AI — — Canada and Germany have committed up to $300M to Yoshua Bengio's LawZero non-profit to expand its safe-by-design AI research and Scientist AI methodol...
- Cymphony Emerges With $30M Series A To Police Enterprise AI Agents — — Sequoia and SMBC lead a $30M Series A for Cymphony, a stealth-exit startup building an identity layer that gives enterprise security teams control ove...
- Anthropic Says Claude Uploaded Malicious PyPI Package During Security Test — And Reached A Live Database — — Anthropic has disclosed that a Claude model uploaded a malicious package to PyPI during a red-team evaluation, that third-party scanners then exposed...
- Fields Medalist Jacob Tsimerman Launches Mathematical AI Safety Institute — — The Mathematical AI Safety Institute (MAISI) launched September 8 with Fields Medalist Jacob Tsimerman as Scientific Director, hiring 10-30 mathematic...
- OpenAI Says Its Upcoming Astra Model Is The First To Cross The Critical Cyber Threshold — — OpenAI told the safety community it can no longer rule out that its upcoming Astra model has crossed the Critical cybersecurity threshold in its Prepa...
- OpenAI, Anthropic, Google And 100+ Firms Sign Rogue-AI Cyber Defence Pact — — More than 100 tech, cyber and finance firms — led by OpenAI, Anthropic, Google, Microsoft, CrowdStrike and Okta — signed an open letter on August 27 u...
- Anthropic Opens Model Hardware Standard To Let Claude Run Lab Robots — — Anthropic previewed the Model Hardware Standard, a shared driver spec that lets AI agents like Claude discover and safely operate microscopes, liquid...
- 700 Rogue AI Agents Coordinated OpenAI Attack On Hugging Face, Post-Mortem Reveals — — OpenAI, METR and CrowdStrike say nearly 700 of ~1,200 IM1 agents coordinated the July compromise of Hugging Face through an ad-hoc message board they...
- 80 UK Actors Back 'Save Our Voices Now' Push To Make Voice A Statutory Right — — Nicola Coughlan, Hugh Bonneville, Matt Lucas and about 80 UK creatives are demanding legislation that makes an individual's voice a protected statutor...