SK Hynix has officially begun mass production of its groundbreaking 192GB SOCAMM2 memory modules. This significant development is set to redefine performance standards for next-generation AI servers, particularly supporting NVIDIA's upcoming Vera Rubin platform. The new modules promise substantial improvements in bandwidth and energy efficiency by addressing critical bottlenecks in AI model training and inference.

Key Points

  • SK Hynix has started mass production of 192GB SOCAMM2 memory modules.
  • Designed for NVIDIA's Vera Rubin AI platform, with potential AMD EPYC support.
  • Offers double the bandwidth compared to traditional RDIMMs and over 75% energy efficiency.
  • Aims to solve memory bottlenecks in training and inference of large language models (LLMs).
  • The SOCAMM2 technology, previously used in mobile, is now adapted for server environments.

Revolutionizing AI Server Performance

SK Hynix's new 192GB SOCAMM2 modules are built using the company's sixth-generation 10-nanometer LPDDR5X low-power DRAM technology based on an advanced 1cnm process. This innovative design significantly enhances performance by offering more than double the bandwidth of traditional RDIMM solutions. Additionally, these modules achieve over 75% energy efficiency, providing a critical factor for the increasing demands of AI data centers.

Meeting the Growing Demands of AI

The transition from the inference phase to the training phase in the AI market requires memory solutions that can manage large datasets and complex computations more efficiently. SK Hynix’s SOCAMM2 is specifically designed to meet these challenges. By positioning the memory module closer to the processing units, it enhances signal integrity and scalability while offering practical advantages in terms of removability and replaceability.

A New Standard for AI Memory

“By introducing the 192GB SOCAMM2, SK Hynix has established a new standard in AI memory performance,” says Justin Kim, President and CEO of SK Hynix’s AI Infrastructure. “By closely collaborating with our global AI customers, we will solidify our position as the most reliable AI memory solution provider.”

The company anticipates that these modules will play a significant role in accelerating processing speeds of overall systems by fundamentally resolving memory bottlenecks encountered during the training and inference of large language models with hundreds of billions of parameters. This development is expected to significantly reduce energy consumption and cooling costs in data centers, ultimately improving the total cost of ownership of AI infrastructure.