How to avoid the 2 mistakes behind 89% of rejected AI alignment applications

We’ve recently opened applications for our October 2024 AI alignment course. To help applicants understand how to maximise their chances of success, I’ve reviewed the data for our June 2024 round.[1] I found that 89% of rejected AI alignment applicants either:

• stated an overly vague path to impact (54%); or
• misunderstood the course’s focus (35%).

This article demonstrates what these mean, and what a strong application might look like. It ends with some general advice to further boost your application.

A few months ago we did a similar analysis for our April 2024 AI Governance course. People told us they found this analysis useful. It found 4 common mistakes, which overlapped with the ones we found in AI alignment course applications.

54%: Overly vague path to impact

The most common pitfall we see is applicants not clearly articulating how they plan to use the course to pursue a high-impact career. Ultimately, this is why we exist to the point that it’s baked into our legal documents.

This is a made-up example of the kind of application we think is too vague for us to accurately assess:

Application too vague: Over the next 5 years, AI is likely to radically transform our society. I want to be at the forefront of making AI systems safer, and think the knowledge and skills I could gain on the course will help me achieve this. I want to use this course to learn more about AI alignment, so I can use this knowledge in my career to make better decisions about AI. I’m particularly excited to cover a wide range of topics to get me a broad understanding of AI alignment and help me in my career. Lastly, it’s been a long term goal of mine to become a leader in my field, and I think this course will be a useful stepping stone towards that.

Strong applications specify:

• the concrete steps they will take after the course;
• how the course will help them achieve those steps; and
• how these steps could contribute to AI safety.

It’s fine not to have everything planned out. You might be uncertain about what AI alignment work you might do! However, we expect you to explain what you’re considering and why, and how the course will help you decide.

If you’re planning a significant career change, it’s helpful to demonstrate you’ve thought it through. You might explain what’s motivating the change, and why you might be suited to this kind of work. 80,000 Hours’s career guide is a great starting point to evaluate your fit.

There are many plausible and strong paths to impact. Made-up examples of strong applicants:

Working in a relevant role already: I’ve recently changed role at Google, and am now working on AI safety. I come from a background in ML engineering but mainly in the context of narrow AI systems. My new team is particularly focused on mechanistic interpretability, but I think a wider understanding of the AI safety field would help me understand the problem better, as well as what others are working on. This would allow for better prioritising work and collaborating with others in AI safety, so I could do better safety research: ultimately contributing to making powerful AI systems safer. I also would be interested in using the connections I gain from the course to help me develop my network, so I have people I can ask questions to when I don’t know the answers!

Planning a relevant career change: I’m working as a product manager at a healthtech startup, but want to change career to support AI alignment research. I saw the AI Safety Institute was hiring, and it sounds like I could be a good fit as a programme manager there: I’m used to having to understand technologies, and coordinate technical workstreams. To test this further, I spoke to a recruiter about the role and it did sound like something I’d like: although they suggested it would be helpful for me to upskill on AI safety first. Taking the course would therefore better prepare me for this role (and even if I don’t get this role, I could apply to similar jobs elsewhere), and help me develop my network so I am exposed to more opportunities in the field. I’d expect that working at the AI Safety Institute could help unblock some of their programmes, so governments can do better research.

Finishing higher education: I’ll soon be finishing my master’s degree in data science. I’m excited to use this to solve global challenges, and I’m not sure between working on tackling global poverty directly or working on preventing a future AI catastrophe. I have some understanding of AI by reading through the first weeks of the curriculum, but I’m still a bit fuzzy. I’d like to take the facilitated course for the accountability it provides, as well as the opportunity to discuss some of my confusions with a group. After the course, if I do decide to pursue AI safety I’d appreciate the later sessions on what opportunities are available - I could see myself working on safety research at an AI company or AI Safety Institute.

Conducting further study: I’m part way through my master’s degree in computer science. I’m thinking about applying for PhD programmes in AI. I’m passionate about AI safety and am involved in running a local AI safety group, but I’m not certain whether I want to do technical alignment research or do more on the governance side of things. I have spent a bit of time looking at some AI alignment papers which seem interesting, but it’s hard to know exactly what working in the field would look like. I’d like to take this course so I can decide between a PhD in alignment or governance - plus be able to better evaluate how useful different topics within an AI alignment PhD might be.

35%: Misunderstanding the course’s focus

Some applicants misunderstand what our AI alignment course aims to deliver. It aims to help people contribute to technical AI safety research - i.e. developing and improving methods to make AI systems safe. The course also focuses on methods that might work to mitigate catastrophic risks from AI.[2] This might lead to work as a research scientist, research engineer, or programme manager, among other roles.

It does NOT prepare professionals to work in general AI roles, or to implement existing well-known AI safety methods. We think implementing well-known safety techniques is incredibly important, and would be very keen for a course on this to be developed. However, this is not what our course currently covers.

Similarly, this program is not intended to get people into AI governance roles. Those looking for the latter are much better served by our separate AI governance course!

Not a good fit, general AI engineer: I work as a machine learning engineer at a tech startup developing recommendation systems for clothing brands. I'll be responsible for implementing AI safety measures in our products, in particular ensuring our models are robust and unbiased. I'd like to take this course to understand the latest AI safety techniques and how we can implement them within our organisation. This would help improve AI safety by ensuring our AI systems are reliable and fair to users.

Moderate fit,[3] doing AI research focused on present harms: I'm a data scientist working on financial models, and I'm passionate about addressing bias in AI systems. My goal is to develop novel research methods to debias credit models to ensure decisions are independent of someone’s race, gender, or socioeconomic background. I want to take this course to learn cutting-edge techniques in AI safety that I can apply to my work on fair lending practices.

Good fit, doing AI research on catastrophic risks: I work as a research scientist at DeepMind, focusing on reinforcement learning algorithms. I'll be contributing to projects related to AI alignment, in particular exploring methods for scalable oversight of AI systems to mitigate catastrophic risks. I'd like to take this course to deepen my understanding of the core AI alignment problems, as well as learn about other technical approaches being developed in the field. This would help me contribute more effectively to foundational research on making advanced AI systems aligned with human intentions.

General application tips

Beyond addressing the above issues, here are some additional tips for submitting a strong application:

Highlight impressive or relevant experience, even if it’s not a ‘formal’ qualification. People often miss:

• Research projects, personal blogs or internships in policy or AI safety;
• Running professional or university groups related to AI safety;
• Completing other independent upskilling, especially other online courses; and
• Projects or voluntary work outside your default education or job path, including those that aren’t specifically related to AI.[4]

Make your application easy to understand. Avoid unexplained acronyms, numerical scores, or inside references that require contextual understanding. As an example: ‘I scored at HB level on the YM371 module I took at university, and published a paper in IJTLA. I also did well in a high school debate competition.’ This is hard to evaluate because:

• We probably don’t know the specific module codes or grading schemes for your university. It’s also hard to judge how meaningful publishing a paper is in an unknown journal. We do try to look up these kinds of things, but we often can’t find what people are referring to.
• We don’t have enough context to judge how impressive the debate competition claim is. For example, is this at their local school’s debating club one afternoon? Or was this at a national championship with thousands of the top students around the country? And what is ‘well’ - winning or placing highly, or just scoring a couple of points?

Write in plain English. Many applicants think they need to make what they’ve done sound fancy, or it seems more professional to use big words or complicated sentences. You should not do this. Instead, write in plain English to help us understand and make a positive case for your application. The Plain English Campaign’s guide is a good starting point for learning how to write clearly.

Strike a balance with length. One-line responses usually don’t provide enough context to evaluate your application effectively. Overly long answers hide key points among unnecessary detail, and make it harder for us to identify the relevant parts of your application. The application form provides guidance on how long we expect answers to be.

Read the questions carefully, and make sure you’re answering what they’re asking for. The descriptions are there to help you!

Put yourself in our shoes. Does your application make a good case that your participation in our course would result in improved AI alignment research, that meaningfully reduces AI risks?

Applying to our course

The last common mistake is not applying at all, or forgetting to do so by the deadline! Now you know how to put your best foot forward, apply to our AI alignment course today.

Appendix: Full results

54%: Vague path to impact

35%: Misunderstanding the course focus

5%: Other - mainly these were when people didn’t present as having relevant skills, or impressive past achievements

3%: Unclear achievements, for example ‘I scored at H34 level on my TLA exam, which demonstrated my achievements in the YM371 module I took at university.’

2%: Not stating any achievements, for example ‘I haven’t done anything relevant.’

<1%: Overly complex or indirect path to impact

<1%: Stating contradictory goals

Footnotes

1. We evaluate applications to our courses based on a number of factors and try to make positive cases for all applications. The mistakes listed in this article were only used for analysis after all decisions were finalised, and not used as criteria for accepting or rejecting people.

   We got the data by classifying rejected applications using a large language model into 7 buckets (including a ‘none of the above’) bucket, based on our experience reviewing the applications, then spot-checked random samples of these to ensure the numbers were accurate.

   The legal basis under the UK GDPR for processing this personal data was our legitimate interests. This processing was for statistical purposes, to improve the user experience of applying to our courses and to promote our courses.

   All the example applications are made up to protect applicants’ privacy, but aim to be representative of the class of applications we have seen.

2. We focus on catastrophic risks because we think their impact could be huge. We’ve discussed some of them like authoritarian lock-in, automated warfare, and loss of control in our article on AI risks. Even though most of them are unlikely to materialise, further reducing their likelihood and impact is highly neglected, with incredibly few people working on this. We think at the moment these are more neglected relative to their scale than present harms.

   We acknowledge that AI systems also pose a lot of present harms, and think more people should work on tackling these. These are briefly covered in our courses, but in limited detail. This is because we think they are less neglected, and due to the time constraints of a short part-time introductory course. There are also several courses which go into detail on present risks, while there are almost no other AI courses that explore catastrophic risks.

   There’s sometimes conflict between folk working on present and catastrophic risks. However, most methods for reducing catastrophic risks also work for reducing present risks, and the other way around. For example, evaluations of AI systems can both provide us information about the propensity of a model to be biased in its decisions today, and whether it might take dangerous agentic actions in future. We think both groups would benefit from learning from each other (and other fields!), and we’re keen to build bridges between the communities so we can effectively mitigate both the present and future harms.

3. By moderate fit, we mean that we’d likely accept this person if they displayed strong technical skills. However, given our limited capacity we’d prioritise places for people focusing on catastrophic risks first.

4. There’s a balance to be struck here: if you already have impressive achievements in AI, focus on those. Otherwise, consider mentioning an impressive achievement elsewhere.

   We prefer you explaining one achievement in detail, rather than listing many smaller things. This will help us better understand your application, as well as help you keep to the answer length guidelines.