For a popular audience

We think these are great resources to share with others who might be interested in learning about what AI safety is in a low-cost way. We think these are great to look at before applying to our programme, to get a feel for what the content is going to be about.

• Intro to AI Safety (18 minute video by Robert Miles, a YouTuber who makes content about AI safety)

• The case for taking AI seriously as a threat to humanity – Kelsey Piper (an author for Vox’s Future Perfect column reporting on ‘Finding the best ways to do good’)

• Benefits & Risks of Artificial Intelligence – Future of Life Institute (an institute dedicated to ensuring tomorrow’s technologies are beneficial for humanity)

• Without specific countermeasures, the easiest path to transformative AI likely leads to AI takeover – Ajeya Cotra. We think this post explains the alignment problem in the context of deep learning, which makes a relevant introduction for modern machine learning enthusiasts.

• An Introduction to the AI Alignment Landscape – Neel Nanda.

• Human Compatible by Stuart Russell (UC Berkeley) is our default recommendation for people who haven’t already read books about AI safety. We think this book provides the best and most focused introduction to the alignment problem, though it doesn’t cover a wide breadth of solutions.

• The Coming Wave by Mustafa Suleyman is a more recently published book. It mostly discusses misuse risks from AI (like bioweapons, cyberattacks, political unrest), with a small amount on misalignment risk.

• Chip War introduces the history of the semiconductor supply chain, and geopolitical tensions around AI chips. It mostly focuses on Taiwan and relations between US and China (the two most important powers in the semiconductor industry).

More academic introductions

These resources are useful for people who want a more comprehensive introduction to AI safety than the previous section. These resources take a variety of different approaches. Some are specifically targeted at machine learning practitioners, which we feel is useful for connecting elements of AI safety with the forefronts of machine learning research.

• Bibliography of research areas that need further attention according to the Centre for Human Compatible AI, UC Berkeley. On that site, you can set a priority threshold for which materials to show.

• Unsolved problems in ML Safety – Dan Hendrycks. This paper frames many problems in AI safety in the context of modern machine learning systems. We think it’s a good introduction for machine learning academics hoping to learn more about machine learning problems they could help work on.

• ML safety scholars’ course – Dan Hendrycks. This course takes a Machine Learning-first approach to AI safety, which forms the basis of a closely-related field Hendrycks terms ‘ML safety’. It may be a good way to learn safety-relevant ML techniques and concepts, at the same time as ML safety concepts.

• Alignment forum curated sequences

Basic ML introductions:

If you’re interested in working on technical AI safety, the current AI paradigm dictates that Machine Learning knowledge is essential. The below resources are not safety-relevant, but could be a good place to start to learn more about machine learning.

• What is a neural network – 3Blue1Brown Deep Learning Series for a very high-level understanding of what ML is.

• fast.ai courses (most recommended). Focuses on various state of the art techniques and applied deep learning. Emphasises a code-first approach, where models are applied and prompts you to think on a lower level later.

• Deep learning with Pytorch crash course

• Deep learning for computer vision (Justin Johnson)

• Google’s machine learning crash course

More advanced:

• Jacob Hilton’s deep learning curriculum (Open AI researcher) – aimed mostly at skilling up in what’s needed for working with Large Language Models.

• Spinning up in deep RL (Open AI) – somewhere between a technical blog post and a text book. Nails the important relevant equations for reinforcement learning.

• NYU’s Deep Learning course