The excitement around the chat bot ChatGPT and other generative neural networks did not go unnoticed by Nvidia. Especially for launching large language models, like the GPT family from OpenAI, the “green” chipmaker prepared the H100 NVL accelerator. This is a dual modification of the H100 with increased memory capacity and bandwidth.
The Nvidia H100 NVL accelerator actually consists of two PCI Express 5.0 x16 expansion cards combined using the NVLink interface (600 GB / s). In the server, such a device will occupy four slots. Each of the halves is made on a 4-nm GH100 processor, is characterized by a maximum TDP of 350-400 W and provides computing power at the level of the SXM version of the H100.
Processor Nvidia GH100
However, the main “feature” of the accelerator lies in the memory subsystem. The new GH100 processor variant has six HBM3 multi-layer memory stacks active, while earlier versions were limited to five. This not only increased the size of the buffer, but also increased its throughput. The specifications of the Nvidia H100 NVL accelerator, as well as the SXM and PCIe modifications, are summarized in the table.
The release of Nvidia H100 NVL is scheduled for the second half of this year. Regarding the price of the device information has not been reported.
