Samsung and SK Focus on 'Computing Memory'... In the AI Era, "No Alternative to PIM"

Increase in On-Device AI-Based Smartphones and PCs
‘PIM’ Needed to Solve Memory Bottleneck

Academic and Industry Efforts Required for PIM Commercialization
PIM Attractiveness Must Be Enhanced from a Business Perspective

Samsung Electronics and SK Hynix have identified 'Processing in Memory (PIM)' as the key to overcoming the limitations of memory semiconductors in the artificial intelligence (AI) era. They also proposed that efforts should be made to commercialize PIM in related fields as on-device AI (AI running inside devices) based smartphones and PCs increase.

A panel discussion is underway at the '2024 2nd AI-PIM Semiconductor Workshop' held on the 30th at El Tower in Yangjae, Seoul. From the left: Professor Min Kyung-sik of Kookmin University, Professor Park Jong-seon of Korea University, Kim Dong-gyun, Fellow at SK Hynix, Son Gyo-min, Master at Samsung Electronics / Photo by Kim Pyeong-hwa

Kim Dong-gyun, Fellow at SK Hynix, said at the '2024 2nd AI-PIM Semiconductor Workshop' held at El Tower in Yangjae on the 30th that memory challenges are growing as AI services are increasingly applied to smartphones. He stated, "Even with AI translation alone, after about 3 to 4 hours, memory usage peaks, causing significant battery consumption."

He added, "In the case of mobile devices, hardware limitations are particularly clear, so PIM is the most suitable solution," explaining, "If a personal assistant is to handle everything on a smartphone, I believe there is no alternative to PIM."

PIM is a next-generation memory concept that includes some processor functions within memory semiconductors to enable computation. Its purpose is to resolve the memory bottleneck caused by the increasing amount of data exchanged between the processor and memory. The main computations are handled by the processor, while simple calculations are processed within the memory. This approach has advantages in speeding up tasks and reducing power consumption.

Currently, both Samsung Electronics and SK Hynix are developing PIM technology and related products. However, due to the high technical difficulty and the various changes involved in the adoption process, such as altering architectures, active implementation is challenging. Opinions also differ regarding the commercialization potential. This was the background for the discussion of various measures for PIM adoption at the event.

Son Kyo-min, Master at Samsung Electronics, explained, "PIM is effective in solving the memory bottleneck, but the most difficult part (in adoption) is that the related ecosystem must be changed." He added, "When thinking about the PIM concept, each level may differ, but at the lowest level, Samsung Electronics and SK Hynix share the view that the processor should only offload to DRAM when it faces difficulties due to the memory bottleneck."

He also said, "When PIM is mentioned, people talk about challenges, but change is difficult and trials are inevitable. If we believe this is the only way, then despite difficulties, moving forward has always been the development of the IT industry, and PIM also carries that much momentum." He continued, "It is a solution that is absolutely necessary within at least five years," and emphasized, "Both academia and industry must work together."

Professors Park Jong-sun of Korea University (Department of Electrical and Electronic Engineering) and Min Kyung-sik of Kookmin University (Department of Electronic Engineering), who participated in the discussion, advised that the attractiveness of PIM should be increased from a business perspective. They suggested that ways to enhance cooperation with processor development companies should be considered for PIM adoption.

Professor Park said, "Processor manufacturers might be concerned about the idea of performing computing within memory. To improve energy efficiency, the architecture must be changed, and they might worry about purchasing more expensive memory (PIM)." He proposed, "It will be important to create a business model that benefits companies holding computing governance so that PIM adoption can be advantageous to them as well."

Meanwhile, Jeong Ki-tae, Vice President of Samsung Electronics’ Foundry Business Division, who attended as a speaker at the event, revealed that they are focusing on the development of embedded MRAM (eMRAM) technology, which has become increasingly important. According to Vice President Jeong, they have completed the development of products based on the 14nm (nanometer; 1nm = one billionth of a meter) process and are nearing completion of the 8nm process development.