Samsung Electronics introduced its new zHBM 3D-stacked memory architecture on September 16. A company executive announced the technology at the AI Infrastructure Summit in Santa Clara, California. The architecture aims to increase AI response speeds from 100 to 1,000 tokens per second. This performance jump specifically targets next-generation agentic AI applications.
The zHBM design vertically stacks High Bandwidth Memory directly on top of AI accelerator chips. This 3D approach shortens data paths to eliminate bottlenecks found in current 2.5D packaging. Samsung claims the technology delivers eight times the performance of HBM5. The new architecture also provides three times the power efficiency of HBM5. The company will co-design hardware with accelerator customers to manage thermal challenges.