Meta intends to restrict ‘catastrophic’ AI models, but acknowledges potential limitations in its efforts.

A policy document from Meta outlines the company’s concerns that it may inadvertently create an AI model that could result in “catastrophic outcomes.” The document details strategies to prevent the launch of such models but acknowledges the possibility of failure.

Of particular concern is the potential for an AI system that can infiltrate the most secure corporate or government networks without human intervention…

TechCrunch has identified the policy document, which carries the somewhat benign title Frontier AI Framework.

This document, referred to by Meta as the Frontier AI Framework, categorizes two groups of AI systems that the company deems too hazardous to deploy: “high risk” and “critical risk” systems.

According to Meta, both “high-risk” and “critical-risk” systems have the potential to facilitate cybersecurity, chemical, and biological attacks. However, “critical-risk” systems could lead to outcomes that are “catastrophic” and unmanageable in the context of deployment, whereas high-risk systems might merely increase the likelihood of an attack without the same level of reliability.

The company elaborates on what constitutes a “catastrophic” outcome:

Catastrophic outcomes are those that would have widespread, devastating, and possibly irreversible negative effects on humanity, which could reasonably result directly from access to [our AI models].

One specific example noted is the “automated end-to-end compromise of a corporate-scale environment that follows best practices.” This implies an AI that can hack into any computer network independently.

Additional risks include:

• Automated identification and exploitation of zero-day vulnerabilities
• Fully automated scams targeting individuals and businesses, resulting in extensive damage
• The creation and spread of “high-impact biological weapons.”

The company states that if it identifies a critical risk, it will promptly halt work on the model and strive to ensure its non-release.

Acknowledges containment may be unattainable

Meta’s document candidly suggests that its best course of action in these scenarios is to attempt to prevent the model’s release, while acknowledging that its preventive measures may fall short (italicization is our emphasis):

Access is strictly controlled and limited to a small group of experts, complemented by security measures to thwart hacking or data exfiltration to the extent that is technically feasible and commercially viable.

You can access the complete policy document here.

Image by Cash Macanaya on Unsplash