The Giving Voice to Digital Democracies is part of the Centre for the Humanities and Social Change, Cambridge, funded by the Humanities and Social Change International Foundation, and we started 2020 with a Fact-checking Hackathon on 10-12 January. The event took place at the Cambridge University Engineering Department.
The project manager Marcus Tomalin welcomed attendees to the event before Mevan Babkar, head of automated fact checking at FullFact, gave an insightful talk about human-based fact-checking. She discussed the various ways in which information can be used and abused, and she explained FullFact’s fact-checking processes. It was particularly fascinating to hear about their work during the recent general election.
James Thorne, a PhD student at the Department of Computer Science and Technology, talked about fact extraction and verification, and how approaches from Natural Language Processing can help. He also discussed the Fact Extraction and VERification (FEVER) shared-task (http://fever.ai/).
Jonty Page, a current 4th-year engineering student, gave an overview of an open source fact-checking system the participants could develop during the Hackathon, and he highlighted some potential challenges and topics they could explore. Given a claim to be fact-checked, the baseline system (i) retrieves Wikipedia pages relevant to the claim, (ii) selects particular sentences from those pages which relate to the claim, and (iii) classifies those sentences either as supporting or refuting the original claim, or else as providing too little information to either support or refute it.
Creating an Interdisciplinary Environment
The task of dealing with false claims automatically is necessarily an interdisciplinary task. The Hackathon created a collaborative environment for researchers from a variety of backgrounds. The weekend brought together people with expertise in areas including linguistics, psychology, sociology, education, criminology, mathematics, philosophy, critical thinking, natural language processing, computer science, and software engineering. Therefore, it was a profoundly interdisciplinary event. On the second day of the Hackathon, Dr Shauna Concannon ran some introductory sessions on Python for participants who wanted to learn more about coding, and especially using Python to analyse natural language.
“This is my first hackathon and I’ve really enjoyed its interdisciplinary nature, it’s really welcoming, it’s really engaging, it’s open to newcomers.”
Ideas & Projects
The teams worked on different aspects of the fact-checking task, including developing new methods for retrieving relevant sentences and documents by integrating information contained in hyperlinks, identifying claims that required multiple pieces of evidence in order to be correctly classified; identifying problematical linguistic patterns (such as claims that required comparisons or which included temporal assertions or quotations), and developing new methods for evaluating conflicting evidence using a confidence scoring metric.
“I came to the fact checking hackathon because I think it is a very important problem to work on. I learnt that automated fact checking is a very hard task that involves a number of different components.”
The interdisciplinary interest that this event generated confirms the urgent need for inclusive and collaborative events that bridge the divide between technology, the humanities, and the social sciences.
“It was a great opportunity to come together with people from different backgrounds, people who are doing mathematics, engineering, computer science, linguistics, criminology.”