Financial Optimization AI Model Selected by LG CNS... Customized Evaluation Tool Developed

Development of Evaluation Metrics for Financial AI Service LLM Suitability
Utilizing 29 Evaluation Indicators and 1,200 Datasets

LG CNS announced on the 24th that it has developed a finance-tailored artificial intelligence (AI) evaluation tool for financial companies looking to adopt generative AI.

This evaluation tool assesses dozens of publicly available open large language models (LLMs) using pre-built datasets to identify the AI model best suited for each financial service, such as banking, insurance, and securities. Open LLMs have their source code and algorithms publicly available, allowing anyone to utilize them. Examples of open LLMs include EXAONE 3.5 from LG AI Research, Llama from Meta, and Qwen 2.5 from Alibaba.

An LG CNS employee is introducing the 'Financial Customized LLM Evaluation Tool.' Provided by LG CNS.

For security reasons such as data leakage, financial companies train open LLMs with their own data to build proprietary models when adopting AI. These customized models are specialized for their intended purposes. Closed LLMs like OpenAI's ChatGPT and Google's Gemini do not have publicly available source code, so companies cannot build and utilize their own AI models based on them.

LG CNS’s finance-specialized evaluation tool consists of 29 evaluation indicators and approximately 1,200 datasets. The main evaluation criteria comprehensively test abilities such as reasoning based on financial knowledge, mathematical reasoning, understanding complex questions, document summarization, comprehension of financial terminology, and AI agent usage skills. In particular, 29 evaluation indicators include financial reasoning problems that are difficult for AI to answer, enabling performance verification.

LG CNS enhanced the tool’s completeness by collaborating with financial experts to verify that the evaluation data reflects service information and specialized knowledge used in actual financial fieldwork. This allows for determining whether AI provides accurate answers even under financial regulatory requirements or complex service structures.

Hyun Shin-kyun, CEO of LG CNS, said, "The biggest concern for companies wanting to introduce generative AI into financial services is identifying which AI model is most suitable for their service. LG CNS’s finance-specialized AI evaluation tool will be the optimal solution to quickly resolve these concerns for financial companies."

Meanwhile, LG CNS recently launched a generative AI platform construction project with NH Nonghyup Bank and is building a generative AI-based counselor response system with Shinhan Card. Last year, it developed the LM-based future branch called 'AI Branch' with Shinhan Bank and is operating a future customer center (FCC) incorporating AI, cloud, and other digital technologies for KB Financial Group.