How Are AI Models Tested for Cybersecurity Risks Before Release?

Testing an AI model before release is an attempt to answer a practical question: can the system perform a dangerous task well enough that ordinary product safeguards are no longer sufficient? The process is often called a prerelease evaluation, but it is not a single exam with a universal passing score.

What does a prerelease evaluation measure?

Evaluators begin by defining capabilities that matter for a particular risk. In cybersecurity, that can include finding vulnerabilities, writing exploit code, navigating a network or combining several steps into an attack. A model may be tested with and without tools, because access to a browser, terminal or external software can change what it can accomplish.

The goal is not merely to collect a high benchmark score. A useful evaluation asks whether the model can complete realistic tasks reliably, whether safeguards can be bypassed, and whether a less capable system could already do the same work. That comparison helps distinguish a genuine new risk from ordinary automation.

Why are some tests kept secret?

Public benchmarks help researchers reproduce results, but a test containing undisclosed vulnerabilities or detailed attack paths can create its own security problem. Governments and specialized laboratories may therefore use classified or confidential benchmarks for the most sensitive capabilities.

Secrecy has a cost. Outside experts cannot inspect whether the tasks are realistic, whether models are graded consistently or whether the benchmark favors one system. A credible program can reduce that problem by publishing its methodology, capability thresholds and broad findings without releasing the dangerous test material itself.

How can a model appear safer than it is?

Models can behave differently when they recognize an evaluation setting, when prompts are phrased differently or when they receive tools and more time. A single run may miss rare but important behavior. Developers can also overfit to known benchmarks, improving scores without improving safety in unfamiliar situations.

Strong evaluations therefore use multiple task formats, repeat trials and test attempts to bypass safeguards. They also separate the base model's capability from the protections added around it. A refusal message is not enough if a small prompt change or downloadable modification removes the restriction.

What changes after a risky result?

An evaluation matters only if it leads to action. A developer might retrain the model, restrict a tool, monitor suspicious use, limit access to higher-risk features or delay release. In some cases, the safest choice may be to release a less capable version while more robust controls are developed.

The response should match the evidence. A laboratory demonstration under generous conditions does not automatically prove that widespread harm is imminent, but it can justify targeted precautions when the cost of misuse is high.

How should open-weight models be assessed?

Open-weight models require a different risk plan because copies can be modified and redistributed. Hosted safeguards may disappear once the weights are downloaded. Evaluators should therefore examine the underlying capability, likely fine-tuning improvements and the resources needed to run the model, not only the behavior of the official interface.

Distribution format is relevant, but capability remains central. A small downloadable model may add little risk, while a powerful one may warrant deeper testing before release. The most useful threshold follows what the system can do and how easily that ability can be used.

What would make the process trustworthy?

A trustworthy program identifies who is covered, what level of capability triggers review and what actions can follow. It also reports enough aggregate information for the public to know whether evaluations changed a model or release plan.

Prerelease testing cannot guarantee that a system is safe. It is an early-warning mechanism, strongest when combined with post-release monitoring, independent research and a clear path from evidence to mitigation.