"Up to 11 Times Faster" KAIST Achieves Dramatic Improvement in 'PIM' Semiconductor Communication

KAIST announced on the 19th that Professor Dongjoo Kim's research team from the Department of Electrical Engineering and Computer Science, in collaboration with research teams from Northeastern University and Boston University in the United States, and the University of Murcia in Spain, has developed a technique that improves the communication performance of Processing-in-Memory (PIM) semiconductors by 11 times compared to existing methods.

(From left) Hyojun Son, Ph.D. candidate, Department of Electrical Engineering, KAIST; Dongjun Kim, Professor, KAIST. Provided by KAIST

Recently, as the proportion of memory bandwidth used in research fields such as artificial intelligence, big data, and life sciences has increased, research and development of PIM semiconductors, which place computing units inside memory, have also become active.

The joint research team solved the bottleneck problem where existing PIM semiconductors must communicate through the CPU, which is necessarily connected outside the PIM semiconductor when utilizing internal units. Through this technique, the communication performance of PIM semiconductors can be accelerated up to 11 times compared to existing methods, the team emphasized.

To achieve this, Professor Dongjun Kim's research team identified the limitations of the communication structure between internal computing units in existing PIM semiconductors and proposed a method that applies an 'interconnection network' structure, which directly connects computing units by maximizing the use of the bus structure for data movement inside the memory.

The joint research team minimized the CPU's involvement in communication processing during computation for PIM semiconductors and developed an interconnection network structure specialized for PIM semiconductors.

Conceptual diagram of communication between PIM processing units applied with PIM-specialized interconnect proposed by Professor Dongjun Kim's research team. Provided by KAIST

An interconnection network is a connection structure between computing units used in large-scale system designs that include multiple computing units, and it is considered an essential element in system design involving multiple computing units. The importance of interconnection networks increases as the system scale grows.

Additionally, memory processes have the problem of difficulty in adding complex logic. However, the network structure developed by Professor Dongjun Kim's team implemented an efficient interconnect in PIM.

This structure is specialized for collective communication patterns widely used in parallel computing and machine learning fields, and it minimizes the main components that cause costs in existing networks by utilizing the determinism of collective communication, which allows pre-identification of communication volume and data movement paths for each computing unit.

For example, existing PIM semiconductors suffer significant performance loss because communication must go through the CPU. However, by applying the PIM-specialized interconnection network developed by the joint research team, application performance can be dramatically improved compared to existing systems. This is due to maximizing the utilization of internal memory bandwidth in PIM semiconductors and the scalability of communication performance increasing as the scale of the PIM memory system grows.

Professor Dongjun Kim explained, "Reducing data movement is a key factor in all system semiconductors, including PIM. While PIM can improve the performance and efficiency of computing systems, data movement between PIM computing units can limit performance scalability, restricting application areas. The PIM interconnect developed by the joint research team can be a solution to these problems."

Meanwhile, this research was conducted with support from the National Research Foundation of Korea, Samsung Electronics, and the Next-Generation Intelligent Semiconductor Technology Development Project of the Institute for Information & Communications Technology Planning & Evaluation.