New artificial intelligence tools could help tackle online abuse

  • Researchers from the University of Sheffield are developing new tools which could be used to automatically detect and counter abuse and hate speech online
  • Project is developing novel artificial intelligence and natural language processing methods to responsibly tackle this issue
  • Researchers are looking at current AI methods used to detect online abuse in two areas: the gaming industry and messages directed at politicians on social media
  • Findings will help create systems that are effective, fair, unbiased, context-aware and respectful of language differences within communities

Twitter's logo

New tools which could be used to automatically detect and counter abuse on social media, are being developed by researchers at the University of Sheffield.

The research, led by Professor Kalina Bontcheva from the University’s Department of Computer Science, is developing new artificial intelligence (AI) and natural language processing (NLP) methods that could be used to responsibly tackle abuse and hate speech online.

Launched in collaboration with Wendy Hui Kyong Chun of Simon Fraser University in Canada, the project is looking at AI methods that are currently being used to detect online abuse and hate speech within two areas; the gaming industry and messages directed at politicians on social media.

Researchers in the study intend to use their findings to develop new AI algorithms that are effective, fair and unbiased. The systems will be context-aware and respectful of language differences within communities based on race, ethnicity, gender and sexuality.

Researchers will examine the biases embedded within current content moderation systems that often use rigid definitions or determinations of abusive language. These current systems often paradoxically create new forms of discrimination or bias based on identity, including sex, gender, ethnicity, culture, religion, political affiliation or others.

The research team is aiming to address these effects by producing more context-aware, dynamic systems of detection.

A close up of someone holding a video game controller

Furthermore, the project is looking to empower users by making the new tools open source, so they can be embedded within new strategies to democratically tackle abuse and hate speech online. They could also be used as part of community-based care and response measures.

Professor Kalina Bontcheva said: “There has been a huge increase in the level of abuse and hate speech online in recent years and this has left governments and social media platforms struggling to deal with the consequences.

“This large rise in abuse and hate speech online has sparked public outrage with people demanding governments and social media companies do more to tackle the problem, but there are currently no effective or technical processes that can tackle the problem in a responsible or democratic manner.

“We are developing novel AI and NLP methods to address the problem while also developing a substantial programme of training for academics and early career researchers to build capacity and expertise in this key area of research.”

The project, Responsible AI for Inclusive, Democratic Societies: A cross-disciplinary approach to detecting and countering abusive language online, is led by the University of Sheffield in collaboration with Simon Fraser University. It is funded by UK Research and Innovation (UKRI) as one of 10 UK-Canada projects to support the responsible development of AI, including ensuring all members of society trust AI and benefit from it.

Research by Professor Kalina Bontcheva and her colleagues from Sheffield’s Department of Computer Science also recently found that politicians were targeted by increasing levels of abuse on social media ahead of the General Election in 2019. For more details on this, read the team's blog or access their research paper:

Additional information

Computer Science at the University of Sheffield

The University of Sheffield

With almost 29,000 of the brightest students from over 140 countries, learning alongside over 1,200 of the best academics from across the globe, the University of Sheffield is one of the world’s leading universities.

A member of the UK’s prestigious Russell Group of leading research-led institutions, Sheffield offers world-class teaching and research excellence across a wide range of disciplines.

Unified by the power of discovery and understanding, staff and students at the university are committed to finding new ways to transform the world we live in.

Sheffield is the only university to feature in The Sunday Times 100 Best Not-For-Profit Organisations to Work For 2018 and for the last eight years has been ranked in the top five UK universities for Student Satisfaction by Times Higher Education.

Sheffield has six Nobel Prize winners among former staff and students and its alumni go on to hold positions of great responsibility and influence all over the world, making significant contributions in their chosen fields.

Global research partners and clients include Boeing, Rolls-Royce, Unilever, AstraZeneca, Glaxo SmithKline, Siemens and Airbus, as well as many UK and overseas government agencies and charitable foundations.


For further information please contact:

Sean Barton
Media Relations Officer
University of Sheffield
0114 222 9852