Home/NVIDIA GPUs and Accelerators

NVIDIA H100 NVL GPU

Optimized for large language model inference, the NVIDIA H100 NVL GPU combines Hopper architecture with 94GB of HBM3 memory in a PCIe form factor. This dual-slot, 10.5-inch PCIe Gen5 card supports up to 3x NVIDIA NVLink bridges, allowing a pair of cards to connect with 600 GB/s of bidirectional bandwidth and deliver a combined 188GB of shared memory. Designed for demanding AI workloads rather than gaming, the H100 NVL does not include display connectivity.
Check Availability
NVIDIA H100 NVL GPU side view
NVIDIA Elite Partner

NVIDIA H100 NVL GPU - System Overview

h100 nvl gpu in systemAdditional Information

Featuring a PCIe 5.0 x16 interface, the full-height, full-length, double-width NVIDIA H100 NVL utilizes a passive heat sink for cooling and operates at a maximum thermal design power (TDP) of up to 400W through a single 16-pin power connector. In addition, it incorporates second-generation Multi-Instance GPU (MIG) technology, which partitions the GPU into up to 7x isolated instances of 12GB each. NVIDIA AI Enterprise is also included.

For NVIDIA-Certified systems, the H100 NVL operates at a base clock speed of 1,080 MHz and a boost clock speed of 1,785 MHz. Its powerful Tensor Cores deliver peak performance of up to 3,341 TFLOPS for FP8 operations and 1,671 TFLOPS for FP16/BFLOAT16 operations, making it a powerful solution for AI inference and other computationally intensive workloads.

NVIDIA H100 NVL GPU - Specifications

Architecture

- Hopper architecture

GPU Memory

- 94GB

GPU Memory Bandwidth

- 3.9TB/s

Form Factor

- PCIe dual-slot air-cooled, full-height, full-length (FHFL) 10.5”

Max Thermal Design Power (TDP)

- 350-400W (configurable)

Interconnect

- NVIDIA NVLink: 600GB/s
- PCIe Gen5: 128GB/s

Multi-Instance GPUs

- Up to 7 MIGS @ 12GB each

Performance

- FP64: 30 teraFLOPS
- FP64 Tensor Core: 60 teraFLOPS
- FP32: 60 teraFLOPS
- TF32 Tensor Core: 835 teraFLOPS
- BFLOAT16 Tensor Core: 1,671 teraFLOPS
- FP16 Tensor Core: 1,671 teraFLOPS
- FP8 Tensor Core: 3,341 teraFLOPS
- INT8 Tensor Core: 3,341 TOPS

Decoders

- 7 NVDEC
- 7 JPEG

Server Options

- Partner and NVIDIA-Certified Systems with 1-8 GPUs

NVIDIA Enterprise

- Included

NVIDIA H100 NVL GPU - Documentation

NVIDIA H100 NVL GPU Data Sheet NVIDIA H100 NVL GPU Product Brief
Consult
the Experts! Talk to one of our trained professionals to create a system to your exact specifications.
Contact Us