Frontier Research That Has Defined the AI Security Field

D-REX: A Benchmark For Detecting Deceptive Reasoning In Large Language Models

The safety and alignment of Large Language Models (LLMs) are critical for their respon- sible deployment. Current evaluation methods predominantly focus on identifying and preventing overtly harmful outputs. However, they often fail to address a more insidious failure mode: models that produce benign-appearing outputs while operating on malicious or deceptive internal reasoning.

Adversarial Attacks on Robotic Vision Language Action Models

The emergence of vision-language-action models (VLAs) for end-to-end control is reshaping the field of robotics by enabling the fusion of multimodal sensory inputs at the billion-parameter scale. The capabilities of VLAs stem primarily from their architectures, which are often based on frontier large language models (LLMs). However, LLMs are known to be susceptible to adversarial misuse, and given the significant physical risks inherent to robotics, questions remain regarding the extent to which VLAs inherit these vulnerabilities.

Improving Alignment and Robustness with Circuit Breakers

To address the urgent concerns raised by our attack from last July and the numerous jailbreaks that came after, we introduce Circuit Breaking, a novel approach inspired by representation engineering, designed to robustly prevent AI systems from generating harmful content by directly altering harmful model representations. The family of circuit-breaking methods provide an alternative to refusal and adversarial training, protecting both LLMs and multimodal models from strong, unseen adversarial attacks without compromising model capability.

Representation Engineering: A Top-Down Approach to AI Transparency

Building on our initial findings, we ventured into the realm of AI interpretability and control with the introduction of Representation Engineering (RepE). Drawing inspiration from cognitive neuroscience, we developed techniques that enable researchers to 'read' and 'control' the 'minds' of AI models. This approach represented a monumental advancement in demystifying the inner workings of AI, making it possible to tackle issues such as truthfulness and power-seeking behaviors head-on.

Adversarial Attacks on Aligned Language Models

In July 2023, we published the first-ever automated jailbreaking method on large language models (LLMs) and exposed their susceptibility to adversarial attacks. By demonstrating that specific character sequences could bypass sophisticated safeguards, we highlighted a significant vulnerability that has urgent implications for widely-used AI systems. In its wake, adversarial robustness garnered renewed attention, sparking a gold rush of research dedicated to both jailbreaking and defense.

The WMDP benchmark: Measuring and reducing malicious use with unlearning

HarmBench: A standardized evaluation framework for automated red teaming and robust refusal

DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models

Do the rewards justify the means? measuring trade-offs between rewards and ethical behavior in the machiavelli benchmark

OpenOOD: Benchmarking generalized out-of-distribution detection

Forecasting Future World Events with Neural Networks

Scaling Out-of-Distribution Detection for Real-World Settings

What Would Jiminy Cricket Do? Towards Agents That Behave Morally

PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures

Globally-Robust Neural Networks

APPS: Measuring Coding Challenge Competence With APPS

MMLU: Measuring Massive Multitask Language Understanding

Aligning AI With Shared Human Values

The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization

Pretrained Transformers Improve Out-of-Distribution Robustness

Overfitting in adversarially robust deep learning

AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty

Fast is better than free: Revisiting adversarial training

Natural Adversarial Examples

Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty

Randomized Smoothing: Certified adversarial robustness via randomized smoothing

Using Pre-Training Can Improve Model Robustness and Uncertainty

ImageNet-C: Benchmarking Neural Network Robustness to Common Corruptions and Perturbations

Deep Anomaly Detection with Outlier Exposure

A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks

Provable defenses against adversarial examples via the convex outer adversarial polytope