The Next Big Thing Identified by the 'Father of HBM'? [Paek Jongmin's Shockwave]

Kim Joungho's KAIST Lab, Driving the Future of HBM with Vibrant Energy
Fostering Practical Engineers Through Discussion, Communication, and Industry Collaboration
Presenting a Blueprint for the Future of HBM
"After HBM, HBF Will Rise"

On September 8, I visited TeraLab, the laboratory of Professor Kim Joungho from the Department of Electrical Engineering at KAIST in Daejeon. My goal was to find out how memory semiconductors, once considered low-value, were transformed into high-priced High Bandwidth Memory (HBM).

This laboratory has been at the forefront of HBM development, which opened a new chapter for the Korean semiconductor industry. The HBM that has shaken the semiconductor industry, including companies like Samsung Electronics, SK Hynix, and Nvidia, was researched in earnest here. Now, HBM is etched in the minds of the public as a symbol of Korea’s semiconductor industry.

Professor Kim Joungho of KAIST is explaining the plaques symbolizing collaborations with various companies posted in front of his laboratory. Photo by Paek Jongmin, Tech Specialist

At the entrance to the lab, plaques announcing collaborative research with leading companies such as Samsung, Intel, and Naver greeted visitors. It was clear that Professor Kim viewed collaboration with industry as a given. He enjoys driving change in real-world industrial settings.

Since its establishment by Professor Kim Joungho in the early 2000s, TeraLab has become a global research hub focused on memory and packaging research, with HBM at its core. TeraLab gained recognition as the laboratory that turned HBM from a concept into reality, which is why Professor Kim is called the “father of HBM.” Professor Kim and his research team did not remain confined to the lab?they went directly to the industrial field. Since the early 2010s, they have participated in the commercialization of HBM with Samsung Electronics and SK Hynix, building a bridge between industry and academia. The packaging simulation technology and power and signal integrity analysis methods developed in the lab became the foundation for the world’s first HBM product development.

When I entered the lab, students were continuing their research at their desks. They frequently gathered in small groups to discuss and freely talked about research directions and thesis topics with their advisor, Professor Kim Joungho. Any preconceptions about a rigid semiconductor lab were completely dispelled. It was an open environment.

A student who had just returned from a collaborative project at Samsung Electronics emphasized the strong sense of unity, saying, “Our lab collaborates more, discusses more freely, and shares materials openly.” They are all united toward a single goal: advancing HBM. While there is fierce competition, the atmosphere of sharing data and ideas to improve overall results serves as a great example for the future of the Korean semiconductor industry, extending beyond the lab itself.

In June, TeraLab announced an HBM roadmap spanning from 2025 to 2040. For a university lab, this announcement was highly significant. The roadmap is a blueprint outlining 15 years of technological progress from HBM4 to HBM8.

Professor Kim predicted, “In HBM4, some GPU functions will be integrated into the base die, and LPDDR memory will be used together to alleviate data bottlenecks.” For HBM5, the number of TSVs (Through-Silicon Vias) connecting the DRAMs will increase to over 4,000, an SRAM cache will be built in, and immersion cooling?submerging the package in coolant?will become mainstream. In HBM6, multiple HBM stacks will be placed on the base die like twin buildings, and a hybrid interposer combining glass and silicon will be introduced. The real transformation begins here. HBM7 will feature embedded cooling technology, allowing coolant to flow directly between memory dies.

At this stage, a new concept called High Bandwidth Flash (HBF), which uses NAND, will emerge. Finally, HBM8 is expected to evolve into a full 3D structure with HBM placed both above and below the GPU, achieving a bandwidth of up to 64 terabytes per second. Professor Kim stated, “It is not the GPU, but the bandwidth and connectivity of HBM that will determine AI performance.” Creating the theoretical foundation to realize this blueprint is the mission of Professor Kim’s team.

During this visit, Professor Kim placed particular emphasis on High Bandwidth Flash (HBF). He explained, “HBM will be responsible for speed, but HBF will take care of capacity.” The idea is to stack hundreds of layers of NAND flash and reconfigure it into a high-bandwidth structure similar to HBM. He predicts that HBF could become another pillar of the memory market within the next ten years. Given that this prediction comes from the father of HBM, it cannot be taken lightly. Professor Kim added that overseas companies have already approached him to collaborate on this research.

Despite early doubts about who would use such expensive and complex memory, Professor Kim’s team continued their research out of academic curiosity, and their achievements exploded with the advent of ChatGPT. The HBM developed by SK Hynix and Samsung Electronics incorporates Professor Kim’s research. The lab’s results have been presented at international conferences such as ISSCC and VLSI, and they provided key ideas for the HBM standardization process.

TeraLab’s achievements have been published in dozens of papers at top international conferences like ISSCC, VLSI, and ECTC. Their papers on packaging and memory interfaces have been cited hundreds of times in IEEE journals. The “hybrid bonding-based TSV structure” proposed by the lab was directly applied to the design of HBM3E and next-generation HBM.

Professor Kim Joungho of KAIST is discussing the research direction of HBM with students. TerraLab.

HBM is not the only area of expertise. The lab’s 2.5D and 3D packaging technologies are also world-class. The microchannel cooling design and interposer power optimization analysis presented by the lab have become basic models referenced by global companies in next-generation GPU and AI chip design. Professor Kim has represented Korea in the International Technology Roadmap for Semiconductors (ITRS) and the IEEE Heterogeneous Integration Roadmap (HIR), expanding TeraLab’s research achievements to the global standardization stage.

These research achievements have also led to talent development. More than 200 people have earned master’s and doctoral degrees from TeraLab, and many now work as key technical personnel at global semiconductor companies such as Samsung Electronics, SK Hynix, Nvidia, Google, and Apple. Big tech companies are eager to recruit students who have gained experience in semiconductor research from diverse backgrounds. In addition to major semiconductor companies, students have gone on to work at Meta, Tesla, and Groq. Professor Kim said, “These days, even Apple is eager to recruit students from our lab.”

TeraLab’s operational model is unconventional. Professor Kim stated firmly, “You can’t hit the jackpot with short-term government projects.” He believes that government-driven research support actually restricts creativity. Instead, the lab covers its operating expenses through corporate research projects, while master’s and doctoral students focus exclusively on HBM for their degree research. In front of Professor Kim’s office, you can see the logos of many companies that have collaborated on projects. They even worked with Hyundai Motor Company. The lab secures funding through corporate projects and dedicates academic research to a specific long-term topic. Through this process, students gain hands-on experience in corporate research environments and develop into immediately employable talent.

Professor Kim also took a different approach to research outcomes. He said, “The reason HBM could succeed is because we didn’t obsess over patents.” He believes that when obtaining patents becomes the goal, technology becomes restricted. “It’s more important to let many people freely use and develop ideas,” he said. Chenming Hu, a professor at UC Berkeley and the inventor of FinFET technology, which has led semiconductor miniaturization since 2010, also did not patent the technology for similar reasons.

Recently, Professor Kim has been deeply immersed in agentic AI. Even though his retirement is approaching, he spares no effort in embracing new fields. He and his students are researching HBM automatic design using agentic AI. Even though the AI era has opened thanks to HBM, his research continues without end.