Academic Publications

  • Threat of Offensive AI To Organizations

    Computers and Security, 2022
    TL;DR: There are 32 ways an adversary can use AI to automate their tools, tactics and techniques so as to attain their objectives
    Yisroel Mirsky, Ambra Demontis, Jaidip Kotak, Ram Shankar Siva Kumar, Deng Gelei, Liu Yang, Xiangyu Zhang, Maura Pintor, Wenke Lee, Yuval Elovici, Battista Biggio

  • Adversarial for Good? How the Adversarial ML Community's Values Impede Socially Beneficial Uses of Attacks

    ICML Workshop on Adversarial Machine Learning, 2021
    TL;DR: Adversarial ML researchers believe attacks are “bad” and defenses are “good” without the context. They believe their work is neutral when it is not.
    Kendra Albert, Maggie Delano, Bogdan Kulynych, Ram Shankar Siva Kumar

  • Ethical Testing in the Real World: Evaluating Physical Testing of Adversarial Machine Learning

    NeurIPS Workshop on Dataset Curation, 2020
    NeurIPS Workshop on Navigating the Broader Impacts of AI Research, 2020
    TL;DR: Papers on physical adversarial attacks like adversarial cap, adversarial tshirt, adversarial glassess characterize themselves as "real world." Despite this framing, however, we found the physical or real-world testing conducted was minimal, provided few details about testing subjects and was often conducted as an afterthought or demonstration.
    Kendra Albert, Maggie Delano, Jonathon Penney, Afsaneh Rigot, Ram Shankar Siva Kumar

  • Reflecting on Paradise Lost via Reinforcement Learning and Resistance AI Literature

    NeurIPS Workshop on Resistance AI, 2020
    TL;DR: Anachronistic parallels between John Milton’s 350 year old poem and Reinforcement Learning
    Ram Shankar Siva Kumar

  • Legal Risks of Adversarial Machine Learning Research

    ICML Workshop on Law and Machine Learning, 2020
    TL;DR: We use the flagship federal anti-hacking law, the Computer Fraud And Abuse Act (CFAA), to analyze attacks on AI systems.
    Ram Shankar Siva Kumar, Jonathon Penney, Bruce Schneier, Kendra Albert

  • Adversarial machine learning-industry perspectives

    IEEE Security and Privacy Workshop, 2020
    TL;DR: 25 out of 28 organizations we surveyed do not know about attacks on AI systems
    Ram Shankar Siva Kumar, Magnus Nyström, John Lambert, Andrew Marshall, Mario Goertzel, Andi Comissoneru, Matt Swann, Sharon Xia

  • Politics of adversarial machine learning

    ICLR Workshop on Trustworthy ML, 2020
    TL;DR: Attacks on AI systems have political dimensions (in the Langdon Winner sense; not republican/democractic). That is, attacks on AI systems enable or foreclose certain options for both the subjects of the machine learning systems and for those who deploy them, creating risks for civil liberties and human rights
    Kendra Albert, Jon Penney, Bruce Schneier, Ram Shankar Siva Kumar

  • Hardening quantum machine learning against adversaries

    New Journal of Physics, 2018
    TL;DR: Quantum machine learning systems are also fallible to adversarial examples and can be disrupted by an adversary
    Nathan Wiebe, Ram Shankar Siva Kumar

  • Law and Adversarial Machine Learning

    NeurIPS Workshop on Security in Machine Learning, 2018
    TL;DR: We explore how some aspects of computer crime, copyright, and tort law interface with adversarial ML
    Ram Shankar Siva Kumar, David R O'Brien, Kendra Albert, Salome Vilojen

  • Practical machine learning for cloud intrusion detection: challenges and the way forward

    ACM Workshop on Artificial Intelligence and Security, 2017
    TL;DR: Operationalizing machine learning based security detections is extremely challenging, especially in a continuously evolving cloud environment.
    Ram Shankar, Andrew Wicker, Matt Swann