CyLab Seminar: Ram Shankar Siva Kumar

January 27, 2025

12:00 p.m. ET

Zoom or Danforth Conference Room, second floor, Jared L. Cohon University Center

Ram Shankar Siva Kumar

*Please note: this CyLab seminar is open only to partners and Carnegie Mellon University faculty, students, and staff.

Speaker:
Ram Shankar Siva Kumar
Data Cowboy
Microsoft

Talk Title:
Ignore previous instructions - Attacks on AI systems and what to do about it

Abstract:
This talk will provide an overview of the art and science of attacking AI systems and its societal implications. We will walk through the evolution of adversarial examples to the now famous, jailbreaks, to ask why it is so darn difficult to secure AI systems from adversaries. We will look with a critical eye at the explosion of AI Safety Institutes, especially US AI Safety Institute, and their efforts to assuage this problem.

Bio:
Ram Shankar Siva Kumar is a Data Cowboy working on the intersection of machine learning and security.

At Microsoft, he founded the AI Red Team, bringing together an interdisciplinary group of researchers and engineers to proactively attack AI systems and find failures. His recent book on attacking AI systems, NOT WITH A BUG, has been called “Essential Reading” by Microsoft’s Chief Technology Officer and received wide praise from industry leaders at DeepMind, OpenAI as well as policy makers and academia. He is donating his proceeds of the book royalty to Black In AI.

His work on AI and Security has appeared in industry conferences like RSA, BlackHat, Defcon, BlueHat, DerbyCon, MIRCon, Infiltrate, academic workshops at NeurIPS, ICLR, ICML, IEEE S&P, ACM - CCS. His work has been covered by Bloomberg, VentureBeat, Wired, and Geekwire. He founded the Adversarial ML Threat Matrix, an ATT&CK style framework enumerating threats to machine learning. His work on adversarial machine learning appeared notably in the National Security Commission on Artificial Intelligence (NSCAI) Final report presented to the United States Congress and the President.

He is currently Tech Policy Fellow at UC Berkeley and an affiliate at the Berkman Klein Center for Internet and Society at Harvard University, where he is broadly investigating two questions: How do we assess the safety of ML systems? What are the policy and legal ramifications of AI, in the context of security? He is also Technical Advisory Board Member at the University of Washington. You can contact him at notwithabug@ram-shankar.com.

Upcoming Events