[Kim Pyeonghwa's Peace and Chips] Samsung Electronics and SK Hynix Point to 'PIM' as an Alternative to Memory Limits... What Is It?

Rapidly Increasing Data in the AI Era
Memory Technology Can't Keep Up with Data Growth Speed
Can Processing-in-Memory 'PIM' Provide a Breakthrough?
Standardization Following Commercialization Remains a Challenge

Editor's NoteSemiconductors. We hear about them every day, but when it comes to explaining the concept, it’s hard to find the words. They are called the rice of modern industry, so it feels like something you must know about, yet articles and books on semiconductors are so difficult that people tend to avoid them. But did you know that semiconductors are actually quite interesting? Even a single semiconductor component hides the mechanisms of the entire industry and the interests of nations. Peace and Chips will spoon-feed you the hidden contexts and meanings of the somewhat uncomfortable semiconductor field. All you have to do is just hold out your spoon.

[Asia Economy Reporter Kim Pyeonghwa] "Improving performance solely by enhancing memory is something that must be done, but if we hit a limit there, another approach is needed." This was said by Kwak Nojeong, President of SK Hynix, during his keynote speech at the Semiconductor Exhibition (SEDEX) 2022 held on the 5th at COEX, Samsung-dong, Seoul.

At this event, President Kwak pointed to 'intelligent semiconductors (PIM)' as one way to overcome the limitations of memory. South Korea’s memory technology is said to be among the world’s top tier, so why did he mention limitations? What exactly is PIM, a term unfamiliar to many, and why is it considered an alternative? This week, we aim to answer these questions.

'PIM' to Solve Data Bottleneck Issues

Semiconductors are broadly divided into memory and non-memory (system) semiconductors. Memory semiconductors are used for storing information, while system semiconductors focus on computation. DRAM and NAND flash are representative memory products, whereas central processing units (CPU) and graphics processing units (GPU) are examples of system semiconductors.

PIM differs from conventional semiconductors in that it is a memory semiconductor but resembles system semiconductors. PIM stands for 'Processing In Memory,' meaning that computation is performed within the memory itself. Some processor functions are embedded in the memory so that simple calculations can be done directly in memory, while essential computations are handled by the CPU or GPU. The goal is to speed up processing by reducing the delay caused by the increasing volume of data exchanged between the CPU and memory.

Image of HBM-PIM developed by Samsung Electronics / Provided by Samsung Electronics

So why is PIM being discussed now? It is because the era of artificial intelligence (AI) and big data has arrived, causing a rapid increase in the amount of data to be processed. Global data demand, which reached 64 zettabytes (ZB) in 2020, is expected to grow to 180 ZB by 2025 and 540 ZB by 2030 due to the advent of AI and related services. To put it in perspective, 1 ZB is equivalent to 100 billion 10-gigabyte (GB) movies, making it difficult to grasp the scale.

Lee Jung-bae, President of Samsung Electronics’ Memory Business Division, said at a semiconductor event held in Silicon Valley, USA, on the 5th (local time), "The total storage capacity of memory produced by Samsung Electronics over about 40 years has exceeded 10 trillion GB, with half of that produced in the last three years, reflecting the rapid digital transformation."

Samsung Electronics and SK Hynix Achieve PIM Milestones... Commercialization and Standardization Remain Challenges

The semiconductor industry anticipates the arrival of a hyper-connected era where all devices are integrated through AI technology. In this context, memory semiconductors are expected to face performance limits, and the keyword 'convergence' is proposed as a solution. As seen in PIM, this means the fusion of memory and system semiconductors. The semiconductor industry explains that unprecedented challenges require new innovations that overturn existing frameworks.

Samsung Electronics’ development of the world’s first high-bandwidth memory (HBM)-PIM, which combines memory semiconductors and AI processors into one, in February last year is a representative example. By integrating PIM technology into HBM used for high-performance computing (HPC) and machine learning, performance was improved more than twofold compared to conventional HBM. Samsung Electronics reportedly devoted 10 years to technology verification (PoC, proof of concept) to showcase PIM technology.

Image of GDDR6-AiM developed by SK Hynix / Provided by SK Hynix

Another leading player in the memory semiconductor market, SK Hynix, introduced a PIM-based graphics DRAM called 'GDDR6-AiM' in February this year. This product can be used for HPC, machine learning, big data computation, and storage. Compared to regular DRAM, it is up to 16 times faster in certain computations and reduces energy consumption by up to 80%.

The government is also making efforts to promote PIM. It views PIM as a key stepping stone toward becoming a comprehensive semiconductor powerhouse beyond just a memory powerhouse. The Ministry of Science and ICT and the Ministry of Trade, Industry and Energy have decided to invest a total budget of 402.7 billion KRW from this year through 2028 to carry out the 'PIM AI Semiconductor Core Technology Development Project.' Last month, the two ministries launched the PIM AI Semiconductor Project Group as part of this effort.

Of course, just because PIM is gaining attention as an alternative to memory limitations does not mean the path ahead is smooth. Even if PIM technology is developed, compatibility challenges must be overcome during commercialization. This means multiple companies need to accept PIM technology as a standard. With ongoing government support and private sector efforts, we hope the upcoming news related to PIM will bring positive outcomes.