Evaluating Cyber Security Evidence for Policy Advice (ECSEPA)

Dates: October 2017 – October 2019

Lead researchers: Professor Madeline Carr, UCL and Professor Siraj Shaikh, Coventry University

Overview 

Cybersecurity is considered a Tier One risk to National Security. Civil servants across Government are working on policy advice for cybersecurity – but what evidence do these policymakers rely on? What is the quality of that evidence? How effective are the judgements about threats, risks, mitigation and consequences based on that evidence?

ECSEPA was designed to provide support for the cybersecurity policy community. A mapping exercise identified exactly where cybersecurity policy development is taking place across Government, and what evidence is used in their decision-making processes. Policy crisis games brought together cybersecurity policymakers from across Government to build a picture of how cybersecurity policy recommendations are made. Finally, research on how civil servants working on cybersecurity policy use evidence led to the development of an evidence quality assessment model (EQAM) – designed as a first step towards a tool for policymakers to assess the effectiveness of the evidence they use in cyber security decision making.

Policy implications

Recognising that policymakers often use a limited range of evidence, the EQAM is a proposed method for policymakers, specifically civil servants who provide short- and long-term policy advice on cybersecurity, to measure the quality of evidence they use. The model is a simple two-dimensional map that positions evidence samples relative to each other based on two dimensions of evidence quality: source and credibility. The vertical axis captures the split in evidence sources between data and human sources. The horizontal axis expresses credibility based on the methodology and provider. For example, the vertical axis could place the value of data sources over the value of human sources in establishing the quality of evidence. Credibility is judged on a case-by-case basis (such as methods used and the evidence provider). The use of such a model could improve the quality of cybersecurity decision making, through the discussions around the use and quality of evidence it could prompt. The researchers are seeking input from senior policy stakeholders for further validation of the model.

The map provides a visualization of the complex, rapidly developing UK cybersecurity policy landscape. It is available for policymakers to download and edit to keep the information up to date.

Methods 

Policy crisis games, design and testing of an Evidence Quality Assessment Model, mapping exercise.

Find out more:

Funders: EPSRC.

External collaborators: This project was supported by GCHQ/NCSC throughout, includingthrough participation in the policy crisis games.Follow on work: Findings from ECSEPA around the quality of evidence used in relation to cybersecurity and the experience of the policy crisis games have been taken forwards to the ongoing Cyber Readiness for Boards project.

Skills

Posted on

December 9, 2021