Investor discussion: AI risk management
Summary notes from an investor expert discussion with Malcolm Murray the research lead at SaferAI on AI risk managment.
Malcolm leads research at SaferAI which is a non-profit producing research, tools, and assessments on AI risk management. SaferAI produces one of the only assessments of AI model developers on risk management – SaferAI Ratings. The session highlighted that AI risk management is falling behind model capabilities, but investors can help by pushing companies to adopt existing best practices.
- The gap between predicted AI risks and current real-world harm is still wide, but it will probably close soon. Cyber could be first for a major AI incident (as highlighted by Mythos recently) and labour market disruptions are steadily increasing.
- Current AI risk management practices are lacking in key areas such as novel risk identification (for example, manipulation and multi-agent risks) and board oversight. Frontier safety frameworks have a high degree of discretion and approaches are weakening over time.
- On the positive side, if companies adopted best practices across SaferAI’s 65 sub-criteria there would be a significant improvement from a median score of 18% to 51%. Hence, investors can play an important role in encouraging companies to adopt these.