Ethics & Algorithms Toolkit

A risk management framework for governments (and other people too!)

The question isn’t whether you should, but when will you start?



Government leaders and staff who leverage algorithms are facing increasing pressure from the public, the media, and academic institutions to be more transparent and accountable about their use. Every day, stories come out describing the unintended or undesirable consequences of algorithms. Governments have not had the tools they need to understand and manage this new class of risk.

GovEx, the City and County of San Francisco, Harvard DataSmart, and Data Community DC have collaborated on a practical toolkit for cities to use to help them understand the implications of using an algorithm, clearly articulate the potential risks, and identify ways to mitigate them.

We developed this because:

  • We saw a gap. There are many calls to arms and lots of policy papers, one of which was a DataSF research paper, but nothing practitioner-facing with a repeatable, manageable process.
  • We wanted an approach which governments are already familiar with: risk management. By identifing and quantifying levels of risk, we can recommend specific mitigations.

Our goals for the toolkit are to:

  • Elicit conversation.
  • Encourage risk evaluation as a team.
  • Catalyze proactive mitigation strategy planning.

We assumed:

  • Algorithm use in government is inevitable.
  • Data collection is typically a separate effort with different intentions from the analysis and use of it.
  • All data has bias.
  • All algorithms have bias.
  • All people have bias. (Thanks #D4GX!)

David Anderson

Advisory Board Member, Data Community DC

Joy Bonaguro

Former Chief Data Officer, City and County of San Francisco

Miriam McKinney

Analyst, Center for Government Excellence @ Johns Hopkins University

Andrew Nicklin

Director of Data Practices, Center for Government Excellence @ Johns Hopkins University

Jane Wiseman

Senior Fellow, Ash Center for Democratic Governance and Innovation @ Harvard

The Toolkit

To use this toolkit, we assume you:

  • Have some knowledge of data science concepts or experience with algorithms
  • Largely understand your data

Overview and Introduction

Overview and introduction linked here

Part 1: Assess Algorithm Risk

Part 1 linked here

Worksheet for Part 1

Worksheet linked here

Part 2: Manage Algorithm Risk

Part 2 linked here


Appendices linked here


Got feedback for us?


We are grateful for the media stories covering the toolkit. If you’d like to write an article or otherwise help spread the work, please contact the Center for Government Excellence.

Related Content

Media Stories


The contents of this site and the Ethics & Algorithms Toolkit are licensed under a Creative Commons Attribution 4.0 International License.

The site code is licensed under an MIT license.