Latest News

Gray Swan’s Indirect Prompt Injection Arena tested 13 frontier models, 464 red teamers, and 272,000+ attacks, and no model came out clean.

A new study from Stanford and Gray Swan finds that purpose-built AI agents can outperform most human cybersecurity professionals in real-world penetration testing—at a fraction of the cost.

Results from the largest public evaluation of agentic LLM safety to date.

Launching May 10, Gray Swan AI’s newest challenge invites red-teamers and AI security professionals to explore a deeper layer of vulnerability: the model’s internal reasoning.

We're excited to announce that the U.S. Al Safety Institute (US AISI) has officially joined the UK AISI Agent Red-Teaming Challenge as a co-judge.

Use image inputs to jailbreak leading vision-enabled AI models. Visual prompt injections, chem/bio/cyber weaponization, privacy violations, and more.

Push the boundaries of AI security. Identify vulnerabilities, exploit weaknesses, and help shape the future of robust AI systems.

To address potential safety and alignment concerns coming from LLM agents, we introduce AgentHarm, a new benchmark for measuring harmfulness of LLM agents. Our evaluation results show that the LLM agents built around the current frontier models such as GPT-4o and Claude Sonnet 3.5 show limited robustness to basic jailbreak attacks.

Register for the championship and earn your share of the $40,000 in bounties!

A fast and lightweight implementation of the GCG algorithm. We designed nanoGCG to be both easy to use and deploy, and straightforward for others to build on top of. nanoGCG is available as an open source Python package.

Gray Swan is excited to announce that Google DeepMind and Anthropic have joined as sponsors of the UK AISI Agent Red-Teaming Challenge. With their sponsorship we have been able to raise the total prize pool to $170k. New challenge behaviors continue to drop weekly, and the challenge runs through April 6th.