BACKGROUND: The large and growing number of research publications, coupled with poor search precision, can make identifying all studies eligible for inclusion in a systematic review both challenging and time consuming. Machine learning and text mining technologies have great potential, but may best be considered as aids to human effort, rather than replacements. Emerging approaches to finding research are not limited to technological solutions though, and new human processes – including ‘crowdsourcing’ - are showing that it is possible to make the study identification process more efficient.
AIMS: To present, and for participants to have hands-on experience with, some of the latest automation and crowdsourcing tools to support study identification in systematic reviews. To consider critically the evidence base that supports the use of the tools. To discuss their use as a group, and how users might contribute to their further development and evaluation
CONTENT: We will outline and experience the ways in which new technologies are being applied to searching and study selection in systematic reviews. We will provide overviews of: Current applications for searching, including approaches that aim to improve sensitivity and/or precision, or to aid database translation; Current applications for study selection, including approaches that aim to reduce the number needed to screen or expedite quality assurance; Living systematic reviews: how we can utilise new technologies to maintain the currency of a given review – or suite of reviews; How some study identification tasks can be carried out at scale – outside the scope of individual reviews – making study identification much more efficient, and reducing duplication of effort on a global scale.
We will also summarise and discuss the current evidence base to consider as a group how mature particular technologies are, whether they are ready for use, or what additional development and evaluation is necessary.
Learning outcomes : Participants should be able to: Differentiate some ways that new technologies and processes – including machine learning, text mining and crowdsourcing - help with study identification; Be familiar – and have interacted – with some of the latest tools which utilise these new technologies and processes; Be developing a critical awareness of the evidence base and the issues that need to be borne in mind when using these tools; Have an introductory understanding of how some of the new technologies work.
Type of interactivity : Most of the time will be devoted to hands-on experience with tools, and discussion about their use. Please bring a laptop / tablet with you to try the online tools for yourself. We will adopt the following pattern of activity for each technology we cover:
- Introductory presentation to include: how the technology works, how it can be used, and what evidence is available to support its use;
- Individual and paired hands-on experience with using the tool;
- Group discussion (with feedback) on the strengths and weaknesses, acceptability and usability of the tool.
For those who attended our EAHIL workshop in 2018, this year’s workshop will additionally cover crowdsourcing as well as providing up-to-the-minute overviews of the latest technologies and their evaluations. A new theme will be a focus on human-machine interaction: rather than thinking that the machine will be able to do all the work, we consider how the human and machine together are able to achieve more than either operating alone.
Level : Intermediate
Target audience : Information specialists, librarians, and review authors; also of relevance for commissioners and users of reviews
Preparation for the session : No