We introduce two challenging datasets that reliably cause machine learning model performance
to substantially degrade. The datasets are collected with a simple adversarial filtration …

While neural networks have achieved high accuracy on standard image classification
benchmarks, their accuracy drops to nearly zero in the presence of small adversarial perturbations …

Machine learning (ML) systems are rapidly increasing in size, are acquiring new capabilities,
and are increasingly deployed in high-stakes settings. As with other powerful technologies, …

This report surveys the landscape of potential security threats from malicious uses of AI, and
proposes ways to better forecast, prevent, and mitigate these threats. After analyzing the …

Rapid progress in machine learning and artificial intelligence (AI) has brought increasing
attention to the potential impacts of AI technologies on society. In this paper we discuss one …

To understand neural network behavior, recent works quantitatively compare different
networks' learned representations using canonical correlation analysis (CCA), centered kernel …

We develop efficient algorithms for estimating low-degree moments of unknown distributions
in the presence of adversarial outliers and design a new family of convex relaxations for k-…

We propose a new test to measure a text model's multitask accuracy. The test covers 57 tasks
including elementary mathematics, US history, computer science, law, and more. To attain …

Abstract Machine learning systems trained on user-provided data are susceptible to data
poisoning attacks, whereby malicious users inject false training data with the aim of corrupting …

We introduce four new real-world distribution shift datasets consisting of changes in image
style, image blurriness, geographic location, camera operation, and more. With our new …