PhD defense
Supporting post-publication peer review: Empirical investigations, computational methods and guidelines
- PhD candidate
- Yagmur Ozturk (LIG (GETALP, SIGMA))
- Date
- Room
- Maison des Langues et des Cultures, Jacques Cartier room (ground floor), 1141 rue des Universités, 38400 Saint-Martin-d'Hères · online
Thesis conducted at the Laboratoire d’Informatique de Grenoble (GETALP and SIGMA teams), supervised by François Portet, Cyril Labbé and Frédérique Bordignon. The defense will take place in English; the presentation will take 45 minutes. It will be followed by a reception in the Magellan room, in the same building.
Jury#
- François Portet, Professor, Université Grenoble Alpes, Thesis Director
- Cyril Labbé, Professor, Université Grenoble Alpes, Thesis Co-director
- Konstantin Todorov, Associate Professor, Université de Montpellier, Reviewer
- Iana Atanassova, Professor, Université Marie et Louis Pasteur, Reviewer
- Jodi Schneider, Associate Professor, University of Wisconsin–Madison iSchool, Examiner
- Boris Barbour, Research Director, CNRS, Délégation Paris Centre, Examiner
- Didier Schwab, Professor, Université Grenoble Alpes, Examiner
Guests:
- Jennifer Wright, PhD, Cambridge University Press
- Frédérique Bordignon, Research Engineer, École nationale des ponts et chaussées, Thesis Co-supervisor
Abstract#
The scientific record is dynamic, constantly growing and expected to be corrected when errors are found. However, the volume of problematic scientific publications is growing faster than formal correction mechanisms (e.g. corrections and retractions) or community-driven efforts, such as post-publication peer review (PPPR), can keep pace with. We address this problem by assisting PPPR with natural language processing (NLP), investigations of the scientific record and standardized guidelines. We pursue these fronts through three contributions. First, we investigate how problematic content enters the record in the first place. One such example of this is paper mills, businesses that sell authorship of scientific papers, often advertised through social media. Through a systematic investigation, we trace these offers to their resulting publications in conference proceedings and document recurring red flags in the literature. In doing so, we advance the knowledge on paper mill activity in conference proceedings, which is a venue that remains under-studied compared to journals. Second, we address a gap in existing PPPR tools, which largely rely on bibliographic metadata or pre-defined problematic textual content alone when re-assessing publications. We present CiteScreener, a decision support system that combines bibliographic and retraction metadata with NLP-based citation-context classification to help PPPR practitioners (regardless of their expertise level) identify problematic citations. CiteScreener’s web interface has been evaluated by PPPR practitioners with varying levels of experience, who found it useful not only for surfacing problematic citations but also for identifying issues beyond its intended scope. Third, motivated by a recurring finding that practitioners need more guidance in PPPR, we present the Collection of Open Science Integrity Guides (COSIG). COSIG is an open-source project that curates and standardizes expert PPPR knowledge into an accessible source. Its aim is to reduce the barriers to entry that currently keep effective PPPR participation concentrated among a small number of experienced practitioners. Together, these contributions address the scientific record’s correction problem from three complementary angles: understanding the problematic content, equipping practitioners with tools to identify such content, and curating the knowledge needed to act on it.
