Informing Policies on Autonomous Weapons

Impact case study (REF3b)

Institution: University of Sheffield

Unit of Assessment: 11 - Computer Science and Informatics

Title of case study: Shaping international policy and stimulating international public debate on Autonomous Weapons Systems (Ethics)

1. Summary of the impact

Prof. Noel Sharkey’s research into the legal, ethical and technological basis for the use of autonomous weapon systems in warfare has had significant impact on public debate and policy worldwide. Impact on the policy debate includes briefings and reports to the UN Special Rapporteur and the EU Parliament, briefings to UK, German and French decision makers, as well as talks to senior military in 26 countries, of which 4 have incorporated the research findings into their officer training. The research has influenced the work of NGOs and charities, providing key evidence for a report commissioned by Human Rights Watch (HRW). The findings of Sharkey’s research led to a new NGO, International Committee for Robot Arms Control (ICRAC), made up of legal, technological and political experts whose aim is to stimulate the debate on robots in warfare. Together with a steering committee of 8 other NGOs including HRW and the Nobel Women’s peace initiative, ICRAC has spearheaded an international campaign for a new internationally legally binding treaty to prohibit the use and development of autonomous weapons. Over 340 articles/appearances by Sharkey in national and international media have fuelled public debate.

2. Underpinning research

Two strands of Sharkey’s research underpin the impact, one on robotics and the other on the ethics and legality of the use of robots in autonomous weapon systems (AWS). Sharkey’s work on robot programming, learning, and construction over 15 years, has produced a body of research work in robotics, particularly on the use of neural networks for learning robot behaviours. This work includes, for example: (1) work on robot localisation – the problem of determining a robot’s location, quickly, reliably and accurately – to which he and collaborators proposed a novel solution exploiting self-organizing maps and ensemble techniques [R1]; (2) work on robotic arm control, specifically the problem of how to rapidly adapt a neural net controller for a robotic arm given the new geometric space that arises when sensor position is changed or sensors replaced, to which he proposed a new solution based on using a genetic algorithm to learn initial parameters that allowed the neural net controller to adapt [R2]; (3) work on robot navigation and obstacle avoidance, in which Sharkey showed that a robot architecture where a multi-layer perceptron learns from the behaviour of embedded hardwired reactive controllers outperforms an architecture using prewired controllers alone [R3]. This body of work gave Sharkey deep insights into the capabilities and limitations of autonomous robots and established his technical credibility as a robotics expert.

In 2005 he began to investigate the ethical issues surrounding the use of robots in various applications areas, particularly military applications. This led him, in the next phase of his research, to explore whether the use of robots in AWS could be deemed either morally defensible or legal, according to international laws and conventions governing weapons and warfare. Supported by a Leverhulme Trust Senior Research Fellowship (“Robots on the Battlefield: an ethical and technical appraisal”, 2008), this work has led to 11 publications in journals pertaining to military ethics, e.g. [R4], and to the law and technology, e.g. [R5], as well as to pieces in high impact science and engineering journals, such as Science, IEEE Computer and IEEE Intelligent Systems [R6]. Through this work Sharkey has made a number of contributions. He has argued that autonomous robot weapons fail to meet two key principles adopted by laws governing warfare (International Humanitarian Law), such as the Geneva and Hague Conventions, specifically the principles of discrimination (that it must be possible for an attacker to distinguish combatants from non- combatants) and proportionality (the anticipated loss of life and damage to property incidental to attacks must not be excessive in relation to the concrete and direct military advantage expected to be gained). His papers analyse the perceptual and cognitive capabilities required to make such judgments and argue that current robot technologies fall far short of possessing these capabilities, despite the claims made about them by arms developers and military organizations. He illustrates his arguments with reference to specific robot technologies and military robots. His work also explores the concept of autonomy, distinguishes degrees of autonomy military robots can exhibit and discusses how failures with current non-autonomous, “man-in-the-loop” remote military weapons, such as drones, are even more likely to occur in more AWS. Since 2007 this work has attracted more than 185 academic citations and has made a significant contribution to shaping the academic debate about the ethics and legality of AWS.

3. References to the research

(** are outputs which best demonstrate underpinning research)

R1 Gerecke, U., Sharkey, N.E., and Sharkey, A.J.C. (2003) Common Evidence Vectors for self- organized ensemble localization, Neurocomputing, 55/3-4, 499-519. doi: 10.1016/S0925- 2312(03)00391-6

R2 **Rathbone, K. and Sharkey, N.E. (2001) Evolving Lifelong Learners for a Visually Guided Arm, Integrated Computer-Aided Engineering, 9(1), 1-23.

R3 **Sharkey, N.E. (1998) Learning from innate behaviors: a quantitative evaluation of neural network controllers, Machine Learning, 31, 115-139.

R4 Sharkey, N.E. (2010) Saying No! to Lethal Autonomous Targeting, Journal of Military Ethics, Vol. 9, No. 4, 299-313. doi: 10.1080/15027570.2010.537903

R5 Sharkey, N.E. (2011) The automation and proliferation of military drones and the protection of civilians. Journal of Law, Innovation and Technology, 3(2) 229–240. doi: 10.5235/175799611798204914

R6 **Sharkey, N.E. (2008) Cassandra or False Prophet of Doom: AI Robots and War. IEEE Intelligent Systems, 23(4), 14-17. doi: 10.1109/MIS.2008.60

4. Details of the impact

“Prof. Sharkey’s work stands out not only because of its high quality technical analysis but also because it engages with some of the most problematic and controversial ethical issues raised by these developments. He has clearly established himself as the leading critic of the assumptions on which the leading military contractors are now proceeding in the development of new lethal technologies. Because of the depth of his technical and scientific expertise, his work is an indispensable reference point for those working on these issues.” (Prof. Philip Alston, Former UN Special Rapporteur on Extrajudicial Executions).

Stimulating and informing public debate

Sharkey used the skills learned during his EPSRC Public Engagement fellowship to bring the issue of military robots to the attention of a worldwide audience through widespread and intensive media coverage, and to call for an urgent and informed international debate on the development of unmanned weapons. “[His] passion for tackling the complex issues [ ] has brought the scientific and ethical issues related to the development of lethal robotic weapons to the public in a way that would not have happened without him” [S1] (Nobel Peace Laureate and Chair of Nobel Women’s Initiative).

His concerted press campaign (see Figure 1) was launched with an opinion piece in The Guardian in August 2007, but it was his scholarly article “Cassandra or False Prophet of Doom” [R6] in July 2008 that established him as the foremost expert advocating caution. This led to Sharkey being in great demand throughout 2009, especially in national and international press. As awareness of and expertise in the issues became more widespread, Sharkey was no longer the only media voice and his appearances fell in 2010, rising gradually to a new peak in 2013 following the launch of the International Committee for Robot Arms Control (ICRAC)[S3] and the Campaign to Stop Killer Robots (CSKR)[S7], see section on NGOs below.

From 2011 onwards, Sharkey’s presence on major international broadcast and online media in particular has increased steadily, with appearances on Al Jazeera English (broadcast to over 220m homes in over 100 countries); Sky News; CNN (638k viewers and 75m unique online users in April 2013); and Voice of America (sold to national stations worldwide with a reach of 90m viewers). In the REF period, Sharkey has featured in a total of over 340 print, radio, television and online appearances/articles in over 50 countries. “His dissemination activities have been crucial to the public understanding of his work” [S6] (current UN Special Rapporteur on Extrajudicial Executions).

Figure 1: Number of media appearances by year Influencing the work of NGOs and Charities

Following Sharkey’s efforts to raise awareness of the issue, a number of international organisations have used his research to inform and shape their policy. “He has worked extensively with NGOs and other organizations to influence policy on LARs [Lethal Autonomous Robots]” [S1].

The Director of Human Rights Watch’s Arms Division confirms: “[Sharkey’s] work on fully autonomous weapons was hugely influential in our report “Losing Humanity: The case against killer robots” [19/09/2012 – S2], which makes a number of recommendations that reflect Noel’s research and analysis. This report is widely viewed as a cornerstone of the new global NGO Campaign to Stop Killer Robots, which is coordinated by Human Rights Watch” [S4].

In September 2009, Sharkey was led by his concerns over the findings from his research to co- found an NGO, the International Committee for Robot Arms Control (ICRAC) [S3]. It brings together 26 experts in robotics technology, robot ethics, international relations, international security, arms control, international humanitarian law, and human rights law who foster global efforts toward effective limitation of military robotics.

Sharkey’s work with NGOs culminated in April 2013 in the launch of the international civil society Campaign to Stop Killer Robots (CSKR) calling for an internationally legally binding treaty to prohibit the use and development of autonomous weapons. Sharkey was “a key driver in the creation of the Campaign [ ]. Noel has emerged as both a preeminent technical expert for the

Campaign and one of its most effective spokespersons. His work is admired and utilized across this growing coalition of more than 40 NGOs in more than 20 countries” [S4] (including Amnesty International, Article 36, Handicap International, the International Peace Initiative and the Nobel Women’s Initiative, many of whom were instrumental in the successful International Campaign to Ban Landmines [S7]). “It is accurate to say that Noel’s work was key in laying the basis for the Campaign, and continues to be invaluable in carrying it forward” [S4].

Stimulating Policy Debate

“His work on fully autonomous weapons has led not only to a political campaign by NGOs such as Human Rights Watch, but has had influence on those who make policy” [S6] (UN Special Rapporteur on Extrajudicial Executions).

United Nations: “Prof. Sharkey’s work has been highly influential in all of the major efforts that have been undertaken in recent years to influence public policy in this area, especially at the international level. This includes the report that I prepared in 2010 [and] a follow-up report by the current UN SR on extrajudicial executions in 2013 [S5]. [   ] The issue has now been placed on the agenda of the principal UN body responsible for arms issues [...] On 26 June 2013 the UN High Representative for Disarmament Affairs urged the Panel “to study potential developments in this field and generate an in-depth analysis of the legal, technical, political, military and moral aspects and impact of the militarization [...] of the [  ] use of fully autonomous weapon systems”. This new direction also owes much to the pioneering work of Prof. Sharkey” (Prof.Philip Alston, Former UN Special Rapporteur on Extrajudicial Executions). Recently this has resulted in the UN's Convention on Conventional Weapons agreeing to convene to discuss lethal autonomous weapons. The group will report back in 2014 to start the process in 2015 with a view to producing a new Protocol VI.

European Union: Sharkey’s research has influenced the national policies of three key EU member States, as well as the policy of the European Parliament. He has provided briefings to two UK All- Party Parliamentary Groups (APPGs). A Vice-Chair of the APPG on Drones, Baroness Stern, confirms: “[Sharkey] was known as a key voice providing decision makers at national and international level with sound evidence on which to base their policy and was he was therefore one of the first speakers to address the newly formed parliamentary group [....] [His] knowledge and expertise from his research have provided a detailed and scientific evidence base that has been invaluable to those speaking in parliament. In the past year there have been four debates on drones and related matters.” The Chair of the APPG on Weapons and the Protection of Civilians confirms: “Noel has addressed UK parliamentarians on several occasions in the past year. In December 2012 he addressed the APPG on Drones, as well as the APPG on Weapons and the Protection of Civilians. In April 2013, our Group arranged a cross-party parliamentary briefing at which Noel addressed an audience of approximately 20 MPs and Peers. [...]The concerns raised by Noel’s research motivated a Vice-Chair of our APPG [...] to call a parliamentary debate on this subject in the House of Commons on 17^th June 2013 to question and clarify the Government’s policy on autonomous weapons. Evidence from Noel played a significant part in informing this debate, and continues to motivate our group within parliament to push for the government to introduce a moratorium on the development of autonomous weapons” [S8]. He delivered briefings to the German Foreign Policy Conference, Parliamentary Groups, Foreign Office and Defence Ministry in the last year. In France, “Professor Sharkey has regularly addressed a range of gatherings of military and government personnel and his expertise in the ethics and technology of roboticised weapons systems has contributed to the ongoing process to inform and shape policy. [He] may be praised for his crucial contribution to set the ethical issues raised by roboticised weapons systems on the French and European defence policy agenda. [He] is today a highly influential voice on this issue among both national and international military and civilian decision makers and his activities have been instrumental in driving forward the global debate on this subject, which would not have enjoyed the high profile it currently does without his input”. (Policy Advisor, French Ministry of Defence [S10]). The 2013 European Parliament Policy Document on The Human Rights Implications of the Usage of Drones and Unmanned Robots in Warfare cites his research extensively, concluding that “the EU should make the promotion of the rule of law in relation to the development, proliferation and use of unmanned weapons systems a declared priority of European foreign policy” [S9].

Armed forces: Sharkey has addressed senior military in 26 countries (including Mitre Corp) and elements of his research are incorporated in officer training in the US, UK, France [S10], the Netherlands and an Academy for officers of all former Soviet states.

5. Sources to corroborate the impact

S1. Letter from Nobel Peace Laureate and Chair of Nobel Women’s Initiative confirming impact on informing public debate and on NGOs.

S2. HRW and Harvard Law School Int. Human Rights Clinic report www.hrw.org/reports/ 2012/11/19/losing-humanity-0 (NS cited pp2, 3,8,12,15,16,18,20,23,24,29,31,32,38-40,49)

S3. ICRAC website confirming Sharkey as co-founder: http://icrac.net/who/

S4. Letter from Director of Human Rights Watch’s Arms Division confirming impact on NGOs

S5. Report A/HRC/23/47 to UN General Assembly, April 2013 (NS cited pp7,13). See www.ohchr.org/EN/HRBodies/HRC/RegularSessions/Session23/Pages/ListReports.aspx

S6. Letter from UN Special Rapporteur on Extrajudicial Executions confirming influence on policy at international and national level.

S7. CSKR website confirming membership and ICRAC involvement www.stopkillerrobots.org/coalition/

S8. Letter from Chair of All Party Parliamentary Group on Weapons and the Protection of Civilians confirming impact on UK parliament.

S9. European Parliament 2013 Policy Document on The Human Rights Implications of the Usage of Drones and Unmanned Robots in Warfare (NS cited on pp11, 14, 28, 39, 47) Available from: www.europarl.europa.eu/committees.

S10. Letter from Policy Advisor at the Directorate for Strategic Affairs, French Ministry of Defence confirming impact on French and EU policy.