"Reviewer Feedback Should Be Evaluated in Return and Incentives Provided to Reviewers"
...Collaboration Proposals Continue
The research team led by Professor Seulgi Lee at the Graduate School of Artificial Intelligence at UNIST has received the Outstanding Paper Award in the Position Track at ICML, one of the world's top three artificial intelligence conferences.
This marks the first time that a paper from Korea has won the top paper award at ICML.
Research team, (from left) Professor Seulgi Lee, Researcher Jaeho Kim, Researcher Yunseok Lee. Provided by UNIST
Professor Lee's team was recognized for their research diagnosing problems in the peer review system of AI conferences and proposing solutions.
Peer review is the process by which researchers in the same field anonymously review papers submitted to journals or conferences to determine whether they should be accepted. While it is a crucial procedure to ensure research quality and filter out subpar results, a recent surge in submissions?especially at AI conferences?has led to both a shortage of reviewers and a decline in review reliability. Some reviewers rely entirely on generative AI or submit reviews without thoroughly reading the papers, and such cases are not uncommon.
To address these issues, the research team proposed a system in which paper authors evaluate reviewer feedback in return, and also suggested providing incentives to reviewers.
Jaeho Kim, the first author, stated, "Our proposal received high marks as a practical alternative that gives reviewers both responsibility and motivation. Open Philanthropy, an international charity, has already reached out to suggest collaboration."
Although AI technology is evolving rapidly, there has been ongoing criticism both within and outside academia that the conference systems, research culture, and social norms supporting it remain outdated. This is also the background for major AI conferences, including ICML, establishing the Position Track. ICML introduced this track for the first time in 2024, and NeurIPS has also confirmed that it will launch a similar track starting from its conference in December.
Unlike technical tracks that focus on individual algorithms or AI models, the Position Track addresses policy, ethical, and social issues triggered by AI technology. However, papers must demonstrate logical consistency, objective evidence, and academic persuasiveness comparable to technical papers to be accepted.
Yunseok Lee, co-first author, said, "The peer review improvement method we proposed can be applied not only in AI but also in other academic fields. It is applicable in all areas where peer review is used to evaluate papers, including medicine, biology, and chemistry."
Professor Seulgi Lee commented, "As AI technology advances, it is important to support not only technical research but also research on AI policy and systems. This study is a good example of such a balanced approach."
The award ceremony was held on July 15, 2025 (local time) in Vancouver, Canada, during the annual ICML conference.
ICML (International Conference on Machine Learning) is an international academic conference in the field of machine learning that originated as a workshop under the American AI Society (AAAI) in the 1980s and became independent in 1988. Along with NeurIPS and ICLR, it is regarded as one of the world's top three AI conferences. Each year, over 10,000 papers are submitted, but only about 20% are accepted, making it highly competitive. This year, a total of 12,107 papers were submitted, and only 3,260 were accepted.
This research was supported by the National Research Foundation of Korea (RS-2023-00277383) under the Ministry of Science and ICT, the AI Graduate School Program of the Institute of Information & Communications Technology Planning & Evaluation (RS-2020-II201336), and the Google Cloud Research Credit Program (Gemma 2 Academic Program).
© The Asia Business Daily(www.asiae.co.kr). All rights reserved.
