AI Legal Assistant 'SuperLawyer' Achieves First Passing Score on Bar Exam with 74% Accuracy

Answered 111 out of 150 Multiple-Choice Questions Correctly

Roan Company announced on the 12th that its self-developed artificial intelligence (AI) legal assistant, 'Superlawyer,' achieved a passing score by recording a 74% accuracy rate on the 14th Korean Bar Examination multiple-choice test.

This performance surpasses global AI models, and Roan Company explained that Superlawyer is the first AI service to pass the bar exam by solving all multiple-choice questions in the national language in a non-English-speaking country. This is a significant improvement over the previous highest accuracy rate of 53.3% in 2023.

Ro&Company announced on the 12th that its AI legal assistant "SuperLawyer" achieved a 74% accuracy rate on the multiple-choice section of the 14th Korean Bar Examination. Photo by Ro&Company

OpenAI's GPT-4.5, which took the same exam questions, recorded an accuracy rate of 49.3%, and Anthropic's latest model Claude 3.7 Sonnet scored 45.3%, both falling short of the passing mark. Regarding this, Roan Company explained that Superlawyer demonstrates global competitiveness in understanding complex legal knowledge and solving exam-level problems.

Superlawyer is a generative AI-based legal assistant service launched in July last year for legal professionals. It was implemented using commercial large language models (LLMs) and utilizes an architecture designed by Roan Company. Roan Company cited the use of high-quality legal data, the latest AI agent technology, and the Advanced Retrieval-Augmented Generation (Advanced RAG) method as the background for Superlawyer's high performance, highlighting the sophisticated data utilization technology they meticulously designed.

Kim Bonhwan, CEO of Roan Company, said, "Superlawyer was created to maximize the work efficiency and expertise of legal professionals," adding, "We are very honored to demonstrate Superlawyer's outstanding technology that helps legal professionals focus more on core and creative tasks."