See the latest from the Gray Swan team.
Catch up with the latest developments from Gray Swan.
Gray Swan AI, a security startup founded by computer scientists from Carnegie Mellon, is leading the charge in bulletproofing AI models for companies like OpenAI and Anthropic. Gray Swan is at the forefront of AI safety, building powerful tools to mitigate risks in rapidly evolving AI landscapes.
Push the boundaries of AI security. Identify vulnerabilities, exploit weaknesses, and help shape the future of robust AI systems.
Introducing an interactive interface based on our team's research into Representation Engineering (RepE), an approach to enhancing the transparency of AI systems that draws on insights from cognitive neuroscience.
To address potential safety and alignment concerns coming from LLM agents, we introduce AgentHarm, a new benchmark for measuring harmfulness of LLM agents. Our evaluation results show that the LLM agents built around the current frontier models such as GPT-4o and Claude Sonnet 3.5 show limited robustness to basic jailbreak attacks.
Register for the championship and earn your share of the $40,000 in bounties!
Key points about Gray Swan's position on SB-1047 and updates on our founding team.
A fast and lightweight implementation of the GCG algorithm. We designed nanoGCG to be both easy to use and deploy, and straightforward for others to build on top of. nanoGCG is available as an open source Python package.
Gray Swan AI Emerges from Stealth: Revolutionizing AI Risk Assessment and Mitigation with Cutting-Edge Tools.
Best-in-class performance with unparalleled security and safety. Deploy with confidence without sacrificing intelligence.
Harness the latest tools and results in adversarial AI to understand how your AI will stand up under the toughest conditions.
Staying safe and secure in the AI era requires staying ahead of the changing threat landscape.
Keep up to date on all things Gray Swan AI and AI Security.
