How to make sense of long comment threads on newspaper websites

An eight-member team led by Professor Rob Gaizauskas and Dr Mark Hepple have collaborated with the Guardian to create a system which is able to transform a large thread into a coherent and concise account of the commenters’ opinions.

It uses a computer imitation of the skills traditionally used by newspaper reporters, the commenters are treated as if they are attending a virtual town hall meeting. The comments are then grouped and summarised so that the writer or a newly arrived reader can quickly gain a sense of the issues raised by commenters.

View the video which illustrates how this is done.

The system, SENSEI, has been developed in collaboration with the Guardian through testing its viability on the comment-is-free section.

Professor Gaizauskas says: “Reader comments are a mix of insight and inanity that few of us have the time or energy to digest. SENSEI has the very practical aim of helping readers to make sense of this mass of comment. We use advanced natural language processing technologies to automatically summarise social media conversations.”

Find out more about the SENSEI project.