Speaker:

Nihar Shah, Associate Professor, Machine Learning and Computer Science Departments, Carnegie Mellon University

Title:

AI meets Peer Review: The Ace, the Cheat, the Bluff, and the Tell

Abstract:

We will discuss four facets of AI in Peer Review:
(1) The Ace: Evaluating AI in peer review, and where it surpasses human reviewers;
(2) The Cheat: How fraudsters can game vulnerabilities in the use of AI in the review process;
(3) The Bluff: Autonomous AI scientists have great promise, but also suffer from critical methodological pitfalls;
(4) The Tell: Detecting LLM generated reviews with mathematical guarantees on the family-wise error rates.

The talk will build on this blog post.

Speaker Bio:

Nihar B. Shah is an Associate Professor in the Machine Learning and Computer Science departments at CMU. His research is on the evaluation of science and the science of evaluation. His group develops computational tools with strong theoretical guarantees, and also designs and conducts controlled experiments for evidence-based policy design. His work has been used in the review of several hundred thousand papers and thousands of proposals, in over two hundred venues. He is a recipient of the 2017 David J. Sakrison memorial prize from EECS Berkeley for a "truly outstanding and innovative PhD thesis", the Young Alumnus Medal from the Indian Institute of Science, a JP Morgan faculty research award, Google Research Scholar Award, an NSF CAREER Award 2020-25, and several Best Paper Awards.

Location:

In person: Chambers Hall, 600 Foster Street, Lower Level
Remote option: https://northwestern.zoom.us/j/99003479328

About the Speaker Series:

Wednesdays@NICO is a vibrant weekly seminar series focusing broadly on the topics of complex systems, networks, and artificial intelligence. It brings together attendees ranging from graduate students to senior faculty who span all of the schools across Northwestern, from applied math to sociology to biology and every discipline in-between. Please visit: https://bit.ly/WedatNICO for information on future speakers.

Related Info