Safety Cases for Procurement of AI

This project won the "Best Policy Design" prize for our AI Governance (August 2024) course. The text below is an excerpt from the final project.

Governments are under pressure to innovate while simultaneously being expected to deliver more public services. Government agencies, notoriously known for lengthy procedures, are prime candidates for optimization solutions. The increasing cost of living has created additional demands on authorities to deliver childcare benefits, unemployment support, and other services. Under such competing pressures, it makes sense for public authorities to turn to the private sector for optimization tools. After all, the e-governance trend has been ongoing, and governments have benefited from introduced data and digital tools.

However, as will be highlighted in Section 1.1, AI is not merely a continuation of e-governance but a disruptive force of its own. This requires a more tailored approach. Unfortunately, the procurement framework has not been updated to reflect and respond to AI. I will take a closer look at the US and UK procurement frameworks,
given that these are the jurisdictions where procurement guidelines were most recently updated with the intent of aligning them with technology innovation. These developments are accompanied by emergent research and calls from public voices on how the procurement process should be adjusted. The Ada Lovelace Institute called
for assessment processes.

In this paper, I argue that safety cases can be a useful solution to reimagining the methodology of procurement and updating it for AI risks, without altering existing frameworks. A safety case is an efficient method for helping the developing company focus on the simple but important question: "How do you know that your system is safe enough?" Safety-critical tools transported into procurement for AI reflect the safety-critical nature of AI as a socio-technical tool that, when deployed in morally-loaded, highly value-laden contexts such as the public sector, may have life-altering consequences.

To read the full project submission, click here.