Investor discussion: Multi-agent risks
Summary notes from an investor expert discussion with Marta Bienkiewicz from Cooperative AI on multi-agent risks from advanced AI.
Key takeaways for investors:
- Encourage companies to be cautious deploying multi-agent systems as we cannot fully control and oversee these systems yet – the root cause of failures cannot be identified by human annotators in up to 30% of simulated scenarios and model drift means a compliant system today may not be tomorrow. And regulatory compliance and legal liability are unclear.
- A key part of the solution is developing an ecosystem of AI auditors to provide external assurance. Investors can support this by encouraging companies to commission third-party audits rather than relying on developers’ self-certification.
- Encourage AI developers to build cooperative intelligence into agent design, train agents to understand legal and social norms, and develop circuit-breaker capabilities to halt network-level failures.