1 Matching Annotations
  1. Sep 2026
    1. Embedding evaluators may sound like a small or inconsequential step, but often the things that sound most boring or procedural are actually the most essential.

      "Amodei argues that AI companies should respond to failures elsewhere as though those failures had happened within their own organizations." Embedding third party evaluators and sharing misalignment issues with the public and other AI companies is in my mind comparable to workplace and industry safety organizations like the USCSB, OSHA, and the NFPA. I watch a lot of USCSB (US Chemical Safety Board) videos on industrial accidents. These videos show an animation of what happened, explain what happened, and share recommendations made by the USCSB to the other safety organizations mentioned. The goal of these videos is to inform and educate industry professionals and the public, so that similar accidents are hopefully prevented. I think establishing AI's version of safety regulation works towards not only establishing the industry as a true industry rather than a wild frontier devoid of precaution, but also towards protecting all members of the industry. When industrial disasters occur, there is loss of life, injuries, and the plant responsible is often irreparably damaged, causing severe job loss. In the case of an AI disaster, it is much the same. Employees would lose their jobs, and perhaps more importantly, the public at large could face severe negative impact. The responsible thing to do is often the right thing to do, not because it is responsible, but because it is caring.