Large Language Models (LLMs) such as OpenAI Codex are increasingly being used as AI-based coding assistants. We conducted a security-driven user study (N=58) to assess code written by student programmers when assisted by LLMs. Our results indicate that in low-level C programming with pointer and array manipulations, the security impact is small: AI-assisted users produce critical security bugs at a rate no greater than 10% more than the control, indicating the use of LLMs does not introduce new security risks.
Recent Work
An exploration of pedagogical approaches to teaching machine learning security concepts to large undergraduate and graduate classes. This work presents curriculum design strategies, hands-on exercises, and assessment techniques that effectively prepare students for security challenges in deployed ML systems.
This research investigates novel defense mechanisms against adversarial examples targeting vision transformer architectures. We demonstrate that certain architectural modifications can significantly improve robustness while maintaining performance on clean inputs.
Recent News
- August 2023: Presented our paper on LLM code assistants at USENIX Security ‘23
- May 2023: Received teaching award for CS-GY 6033 Design and Analysis of Algorithms
- January 2023: New course on Machine Learning Security launched
Recent Blog Posts
🛡️ Securing Large Language Models: Adversarial Fine-Tuning Against Prompt Injection **By Gustavo Sandoval, Denys Fenchenko, and Junyao Chen NYU** As powerful as large language models (LLMs) like GPT-3 are, they’re not invincible. A critical weakness has been exposed in the form of prompt injection attacks — a form of adversarial...
This is a sample blog post. Lorem ipsum I can’t remember the rest of lorem ipsum and don’t have an internet connection right now. Testing testing testing this blog post. Blog posts are cool. Headings are cool You can have many headings Aren’t headings cool?
This is a sample blog post. Lorem ipsum I can’t remember the rest of lorem ipsum and don’t have an internet connection right now. Testing testing testing this blog post. Blog posts are cool. Headings are cool You can have many headings Aren’t headings cool?