Automatically Correcting Large Language Models: Surveying the Landscape of Diverse Automated Correction Strategies

Abstract

While large language models (LLMs) have shown remarkable effectiveness in various NLP tasks, they are still prone to issues such as hallucination, unfaithful reasoning, and toxicity. A promising approach to rectify these flaws is correcting LLMs with feedback, where the LLM itself is prompted or guided with feedback to fix problems in its own output. Techniques leveraging automated feedback—either produced by the LLM itself (self-correction) or some external system—are of particular interest as they make LLM-based solutions more practical and deployable with minimal human intervention. This paper provides an exhaustive review of the recent advances in correcting LLMs with automated feedback, categorizing them into training-time, generation-time, and post-hoc approaches. We also identify potential challenges and future directions in this emerging field. 

Author Biography

Liangming Pan

Liangming Pan is a Postdoctoral Scholar at the Natural Language Processing Group, University of California, Santa Barbara (UCSB), working with Prof. William Yang Wang. He obtained his Ph.D. from the National University of Singapore in Jan 2022, jointly advised by Prof. Min-Yen Kan and Prof. Tat-Seng Chua. His broad research interests include knowledge bases, natural language processing, and data mining. To be specific, his research topics include Question Answering, Question Generation, and Automated Fact Checking.

Michael Saxon

Michael Saxon is currently a third-year Ph.D. student at the University of California Santa Barbara. 

Wenda Xu

Wenda Xu is a third-year PhD student at UCSB NLP group, co-advised by Prof. William Wang and Prof. Lei Li. I obtained my B.S. in Computer Science at the University of California at Davis. 

Deepak Nathani

Deepak Nathani is currently a 1st year PhD student at UC, Santa Barbara advised by Prof. William Wang in the UCSB NLP lab. My research interests lie broadly in the areas of Natural Language Generation, Commonsense Reasoning, and Dialog Systems. 

Xinyi Wang

Xinyi Wang is a third-year PhD student in the computer science department at the University of California, Santa Barbara (UCSB). She is advised by Professor William Yang Wang. Her research interests are in understanding deep learning models, especially pre-trained large language models, using principled causality-based/probabilistic approaches. 

William Yang Wang

William Wang is the Director of UC Santa Barbara's Natural Language Processing group and Center for Responsible Machine Learning. He is the Duncan and Suzanne Mellichamp Professor of Artificial Intelligence and Designs in the Department of Computer Science at the University of California, Santa Barbara. He received his Ph.D. from the School of Computer Science, Carnegie Mellon University. He has broad interests in Artificial Intelligence, including statistical relational learning, information extraction, computational social science, dialog & generation, and vision.