o1-Preview, Only 1 Mistake Out of 45 Questions
"The Time to Surpass Humans Is Not Far"
Artificial intelligence (AI) achieved a near-perfect score in the Korean language section of the College Scholastic Ability Test (CSAT), earning a top grade.
Domestic AI startup MarkerAI utilized OpenAI's AI model to take the 2025 CSAT Korean language exam. The 'o1-Preview' model missed only one question out of 45, receiving a raw score of 97 points, corresponding to the highest grade. Although the exam duration is 80 minutes, o1-Preview spent only 35 minutes to achieve this top grade.
The only question that 'o1-Preview' got wrong was question number 8, which involved reading two nonfiction passages about modernization and evaluating logical thinking by applying given examples from the choices. This question recorded the highest incorrect answer rate of 81.5% among the 2025 CSAT Korean language section, making it the most difficult question for test takers. MarkerAI explained that o1-Preview made an error in understanding the context of the passages and choices and grasping the hidden intent of the question. It fell into the 'attractive wrong answer' trap set by the examiners.
MarkerAI has been evaluating AI models' performance on the Korean language section of the CSAT over the past decade. The 'o1-Mini' model, which took the 2025 CSAT Korean language exam, scored 78 points, while 'gpt-4o' scored 75 points, placing them in the 4th grade range.
The development speed of o1-Preview is steep. Last year, it scored 88 points on the CSAT Korean language section, but within one year, it raised its score to near perfect. GPT4o scored 65 points last year, placing it in the 4th grade. Generative AIs from Meta, Google, and others have recently scored between 3rd and 9th grade levels on the Korean language section over the past ten years.
MarkerAI researcher Jin Man-sang wrote on his blog, "The near-perfect score of 97 points achieved in the 2025 CSAT demonstrates that the Korean language proficiency of LLMs (Large Language Models) is approaching the point where it will surpass human performance."
© The Asia Business Daily(www.asiae.co.kr). All rights reserved.
