Nvidia NVHBM is the newest addition to the company’s NVLink Fusion program, a custom implementation of the high-bandwidth memory that powers nearly every AI accelerator running today. The technology promises higher bandwidth, lower power draw, and a smaller on-die footprint than traditional HBM4e, and it will be offered exclusively to Nvidia’s custom silicon partners.
NVLink Fusion gives partners the building blocks needed to connect custom chips into the NVLink scale-up domain, the interconnect that joins many separate processors into a single coherent system such as the Vera Rubin NVL72 rack-scale accelerator. NVHBM extends that toolkit with a custom HBM base die that Nvidia says was designed and validated alongside leading memory vendors, giving developers faster time-to-market than building commodity HBM from scratch. It is not a replacement for HBM, but a new component reserved for custom silicon designs.
How NVHBM Boosts Bandwidth and Compute
Memory bandwidth is critical for AI accelerators, and NVHBM promises up to 30% higher bandwidth per stack than standard HBM4e. For memory-bandwidth-bound workloads, that increase translates into greater throughput, including a higher tokens-per-second rate for AI inference.
The custom base die also changes where memory circuitry lives. Traditionally, the HBM memory controller sits on the primary silicon die within the package. NVHBM instead moves the controller into the base die of the memory stack and supplies a smaller custom PHY that NVLink Fusion customers integrate into their own designs. According to Nvidia, this frees up valuable package real estate that can support up to 30% more compute area on the primary die, while simplifying the interposer routing used to link multiple chips in advanced packaging.
Power Efficiency for Large-Scale AI
NVHBM also delivers efficiency gains, using 15% less power than commodity HBM4e stacks. In the Vera Rubin rollout, Nvidia has repeatedly emphasized that every watt not going into token production is a watt wasted. Those savings can be redirected toward better performance-per-watt, additional functional units in a custom accelerator, or higher sustained performance within the same power budget.
Lower power at higher bandwidth is significant for accelerators that constantly move large data structures such as model weights and KV caches. When multiplied across thousands of chips, the energy saved on data movement can be reinvested into greater accelerator performance or used to support more accelerators within a fixed power envelope.
For now, these advantages remain benefits aimed at Nvidia’s prospective custom silicon partners, with the custom base die and PHY made available through the NVLink Fusion program.
Source
Image: tomshardware.com
