Reference

A plain-language glossary of the hardware, specs, and terms we work with every day.

3DS RDIMM: A 3DS RDIMM is a server memory module whose DRAM packages stack several dies, joined by through-silicon vias, to reach 256 GB per module.
AMD Instinct: AMD Instinct is AMD's line of data center GPU accelerators for AI and HPC, built on the CDNA architecture and programmed through ROCm.
Arm vs x86 server: Arm servers run CPUs built on the Arm instruction set, such as NVIDIA Grace. x86 servers run Intel Xeon or AMD EPYC. Each needs its own builds.
BF16: BF16 is a 16-bit floating point format with 8 exponent and 7 mantissa bits, matching FP32's range at lower precision.
BMC: A BMC (baseboard management controller) is the small dedicated chip on a server motherboard that runs IPMI or Redfish for remote management.
Barebone server: A server chassis shipped with the motherboard, backplane, power supplies, and cooling installed, but no CPU, memory, storage, or GPUs.
Blower vs open-air GPU cooler: A blower cooler exhausts a GPU's heat out the card's rear bracket. An open-air cooler vents it into the case for the chassis fans to remove.
Burn-in testing: Burn-in testing runs new hardware at sustained full load before deployment to catch early failures before a customer sees them.
CUDA compute capability: CUDA compute capability is the major.minor version that sets which hardware features an NVIDIA GPU supports, such as 8.0 for the A100.
CUDA cores vs Tensor cores: CUDA cores run general parallel math one operation at a time. Tensor cores complete a small matrix multiply per clock for AI math.
CUDA vs ROCm: CUDA is NVIDIA's parallel computing platform for its GPUs. ROCm is AMD's open source software platform for GPU computing on AMD hardware.
CXL (Compute Express Link): Compute Express Link (CXL) is an open interconnect on the PCIe physical layer that gives CPUs coherent access to device and expander memory.
Cluster computing: Cluster computing links multiple servers together so they operate as one system for a single workload.
Colocation: Colocation is renting rack space, power, and network connectivity in a third-party data center for hardware you own and manage yourself.
Context window: A context window is the maximum number of tokens a language model can hold at once, covering the prompt, the history, and its own reply.
Continuous batching: Continuous batching is an LLM serving method that adds and drops requests at every decode step instead of waiting for the whole batch to finish.
Coolant distribution unit (CDU): A coolant distribution unit (CDU) moves heat from a liquid-cooled rack's coolant loop into facility water while keeping the two fluids apart.
DDR4 vs DDR5: DDR4 puts one 64-bit channel on each module, up to 3,200 MT/s. DDR5 splits each module into two 32-bit subchannels, from 4,800 MT/s.
DGX: DGX is NVIDIA's line of complete, factory-built AI servers and workstations, sold with GPUs, CPU, memory, and software preinstalled as one unit.
DGX Spark: DGX Spark is NVIDIA's compact desktop AI computer, built on the GB10 Grace Blackwell Superchip with 128 GB of unified memory.
DGX SuperPOD: DGX SuperPOD is NVIDIA's turnkey reference architecture for clusters of DGX systems, built from repeatable blocks called scalable units.
Deep learning: Deep learning is machine learning using neural networks with many stacked layers, trained on data to approximate a function.
DisplayPort vs HDMI: DisplayPort and HDMI are the two video outputs on workstation GPUs, differing mainly in bandwidth per version and licensing.
Drive form factors (U.2, M.2, EDSFF): U.2, M.2, and EDSFF are the three physical shapes an SSD ships in. None of them interchange without an adapter.
ECC memory: ECC memory detects and corrects single-bit errors in RAM automatically, using an extra chip per module to store parity data.
EPYC vs Xeon: EPYC is AMD's x86 server CPU line. Xeon is Intel's. They differ in cores per socket, memory speed, PCIe lanes and maximum socket count.
FP16: FP16 (half precision) is a 16-bit floating point format, 1 sign bit, 5 exponent bits, 10 mantissa bits, used in mixed precision training.
FP32: FP32 (single precision) is a 32-bit floating point format, 1 sign bit, 8 exponent bits, 23 mantissa bits, the default type for GPU compute.
FP4: FP4 is a 4-bit floating point format used in NVIDIA Blackwell GPUs for AI inference, trading precision for lower memory use and higher throughput.
FP64: FP64 (double precision) is a 64-bit floating point format, 1 sign bit, 11 exponent bits, 52 mantissa bits, used in HPC and simulation.
FP8: FP8 is an 8-bit floating point format used for neural network training and inference, storing each value in 1 byte instead of 2 or 4.
Fine-tuning: Fine-tuning continues training a pretrained model on a smaller, focused dataset so it adopts a specific style, domain, or task.
GDDR: GDDR is graphics-oriented DDR memory built for bandwidth over capacity. GDDR6 and GDDR7 are the current versions.
GGUF: GGUF is a single-file format that packs a language model's weights, tokenizer, and settings together, built for llama.cpp.
GPU passthrough: GPU passthrough assigns one physical GPU directly to a single virtual machine, bypassing the hypervisor for near-native performance.
GPU server: A GPU server is a rack-mounted computer built to hold multiple GPUs, with the power and cooling to run them all at full load.
Grace Blackwell superchip: A Grace Blackwell superchip is an NVIDIA processor that pairs an Arm CPU with Blackwell GPUs so both share one coherent memory pool.
Gray market hardware: Gray market hardware is a genuine product sold outside a manufacturer's authorized channel, usually without transferable warranty or support.
HBM: HBM is memory stacked in vertical layers beside a processor and connected through a silicon interposer for very high bandwidth.
HGX: HGX is NVIDIA's SXM GPU baseboard that server makers build into their own systems, not a complete server on its own.
Hopper vs Blackwell: Hopper is NVIDIA's single-die H100 and H200 GPU generation. Blackwell is the dual-die B200 and B300 generation, and it adds FP4.
Hybrid storage: Hybrid storage combines fast NVMe or SSD drives with high capacity HDDs in one system, using each tier for what it does best.
INT4: INT4 is a 4-bit integer format holding 16 values, used mainly to quantize neural network weights for inference.
INT8: INT8 is an 8-bit integer format that stores whole numbers from -128 to 127, paired with a scale factor for quantized inference.
IPMI: IPMI is the standard protocol for monitoring and controlling a server remotely, independent of its operating system or power state.
ISV certification: ISV certification is a software vendor's formal validation that a specific GPU, driver version, and OS combination runs its application correctly.
Immersion cooling: Immersion cooling is a method that submerges servers in a dielectric liquid, which carries heat away without conducting electricity.
Inference: Inference is the stage where a trained model answers a prompt, producing output from fixed weights rather than adjusting them.
InfiniBand vs Ethernet: InfiniBand is a purpose-built lossless fabric for GPU clusters. Ethernet is the general-purpose standard that now competes through RoCE.
KV cache: The KV cache stores each previous token's key and value tensors so a transformer doesn't recompute them at every generation step.
LPDDR5X: LPDDR5X is low-power DRAM, usually soldered to the board, with speed grades up to 10.7 Gbps per pin. It trades upgradability for efficiency.
Lead time: Lead time is the span from placing an order to the goods arriving. It runs from days for in-stock parts to months for allocated ones.
Liquid cooling vs air cooling: Liquid cooling carries chip heat away in a coolant loop instead of airflow, handling far higher power density than air at higher cost.
LoRA: LoRA (low-rank adaptation) fine-tunes a model by training small adapter matrices instead of all its weights.
Local LLM: A local LLM runs inference on hardware you own, keeping data on premises instead of sending prompts to a cloud API.
MGX: MGX is NVIDIA's modular server reference architecture, letting OEMs mix CPU, GPU, and DPU trays on shared chassis designs.
MOQ: MOQ, or minimum order quantity, is the smallest number of units a supplier will sell on a single purchase order for a given part number.
MRDIMM: MRDIMM is a DDR5 server module that runs two ranks at once through a data buffer to reach 8,800 MT/s or more. Only CPUs that list it can use it.
Memory bandwidth: Memory bandwidth is the rate data moves between memory and a processor, set by bus width times data rate and measured in GB/s or TB/s.
Memory channels: A memory channel is an independent data path between a CPU's memory controller and its DIMM slots. Bandwidth scales with populated channels.
Memory-bound vs compute-bound: A memory-bound workload waits on data from memory. A compute-bound workload waits on the processor's math units.
Mixed precision: Mixed precision runs most neural network math in 16-bit formats while keeping FP32 for sensitive steps like weight updates.
Mixture of experts (MoE): Mixture of experts splits a model into parallel sub-networks and routes each token through only a few of them, not the whole model.
Multi-GPU workstation: A multi-GPU workstation runs two or more GPUs in one machine, usually on separate parallel jobs rather than one combined render.
Multi-Instance GPU (MIG): Multi-Instance GPU (MIG) is NVIDIA's hardware partitioning of one GPU into up to seven isolated instances with dedicated memory and compute.
NPU: An NPU is a dedicated chip block that runs AI inference at low power, rated in TOPS, separate from the CPU and the GPU.
NVIDIA AI Enterprise: NVIDIA AI Enterprise is NVIDIA's supported software platform for production AI, licensed per GPU and released in branches with fixed support windows.
NVIDIA GPU architectures: NVIDIA GPU architectures are the chip design generations, such as Ampere, Hopper, Blackwell, and Rubin, that define a GPU's era and features.
NVIDIA vGPU: NVIDIA vGPU is software that shares one physical GPU among several virtual machines using time-sliced or MIG-backed virtual GPUs.
NVIDIA-Certified Systems: NVIDIA-Certified Systems is NVIDIA's program that tests partner servers, workstations and edge systems with specific GPUs and network adapters.
NVL72: NVL72 is NVIDIA's liquid-cooled rack design that joins 72 GPUs and 36 CPUs in one NVLink domain, as in the GB200 NVL72 and GB300 NVL72.
NVLink and NVSwitch: NVLink is Nvidia's direct GPU-to-GPU interconnect. NVSwitch is the crossbar chip that lets many GPUs share it as one any-to-any fabric.
OAM vs SXM: OAM is the Open Compute Project's open module format for AI accelerators. SXM is NVIDIA's own module format for its data center GPUs.
OEM vs ODM: ODM designs and builds a product from scratch. OEM puts its own brand on that product and sells it.
Open Compute Project (OCP): The Open Compute Project (OCP) is a foundation, started by Facebook in 2011, that publishes open data center hardware specs such as Open Rack V3.
Open weights: Open weights are a model's trained parameters released for anyone to download and run, instead of access only through a hosted API.
PCIe 4.0 vs PCIe 5.0 vs PCIe 6.0: PCIe 4.0 runs 16 GT/s per lane. PCIe 5.0 doubles that to 32 GT/s. PCIe 6.0 doubles it again to 64 GT/s with PAM4 signaling.
PCIe lanes: PCIe lanes are the individual data paths linking the CPU to GPUs, drives, and cards. Lane count sets the bandwidth ceiling on a slot.
PCIe power connectors: PCIe power connectors are supplemental cables, 6-pin, 8-pin, and 12V-2x6, that deliver power to a GPU beyond the 75 W the slot itself supplies.
PSU efficiency (80 PLUS): PSU efficiency is the share of AC input a supply delivers as DC output. 80 PLUS certifies it in tiers from Bronze to Titanium.
Passive vs active GPU cooling: A passive GPU cooler is a fanless heatsink cooled by the server's fans. An active GPU cooler carries its own fan on the card.
Prefill vs decode: Prefill reads an LLM prompt in parallel and is compute-bound. Decode emits one output token per step and is memory-bound.
Quantization: Quantization reduces a model's numerical precision, from 16-bit to 8-bit or 4-bit, shrinking memory use with some loss of accuracy.
RAID: RAID combines multiple physical drives into one logical volume for redundancy, performance, or both, depending on the level used.
RDIMM vs UDIMM: RDIMM adds a register chip that buffers command and address signals for higher capacity. UDIMM skips that chip for lower cost and latency.
RMA: RMA (return merchandise authorization) is the number a vendor issues before it accepts a defective part back for repair, replacement, or credit.
RTX PRO: RTX PRO is Nvidia's brand for workstation and server GPUs, successor to Quadro, with certified drivers and ECC memory.
RTX PRO 6000 Workstation vs Max-Q vs Server Edition: The Workstation Edition runs at 600 W. The Max-Q Edition caps at 300 W. The passive Server Edition is configurable from 400 W to 600 W.
Rack PDU: A rack PDU is the rack-mounted power strip that feeds every server and switch in one cabinet from one or two upstream circuits.
Rack unit: A rack unit (U) is 1.75 inches of vertical rack height. Server chassis are sized in whole U, most commonly 1U, 2U, and 4U.
Redundant power supply: Two or more power supplies in one server, each able to carry the full load, so the system stays up when one fails or loses input power.
Retrieval-augmented generation (RAG): RAG pairs a language model with a document search step, so it answers using retrieved passages instead of only what it learned in training.
SAS vs SATA vs NVMe: SAS, SATA, and NVMe are the three interfaces a drive connects through, and they differ in protocol, speed, and cost.
SKU: SKU (stock keeping unit) is the code identifying one specific, orderable version of a product, distinct from others sharing its name.
SXM vs PCIe: SXM is a socketed GPU module built for full NVLink bandwidth. PCIe is the standard add-in card that fits an ordinary server slot.
Single-slot vs dual-slot GPU: A single-slot GPU fills one expansion slot opening. A dual-slot GPU fills two, which limits how many cards fit in one chassis.
Single-socket vs dual-socket server: A single-socket server has one CPU. A dual-socket server has two, joined by a coherent link, for more cores and memory channels.
Speculative decoding: Speculative decoding is an LLM speedup where a draft proposes tokens and the full model verifies them in parallel, keeping its output distribution.
Storage server: A storage server is built to hold and serve data, spending its budget on drive bays and network throughput instead of CPU or GPU power.
Strix Halo: Strix Halo is AMD's codename for the Ryzen AI Max 300 series, such as the Max+ 395 with a 40-unit GPU and up to 128 GB of shared memory.
Structured sparsity: Structured sparsity is a pruning method, such as NVIDIA's 2:4 pattern, that lets tensor hardware skip zeroed weights for up to 2x matrix throughput.
Studio vs Game Ready driver: Game Ready drivers target day-one game support. Studio drivers are tested against creative apps and ship on a slower cadence.
TAA compliance: TAA compliance means a product was made or substantially transformed in the U.S. or a country designated under the Trade Agreements Act.
TDP: TDP is the heat, in watts, a chip's cooling system must remove at sustained rated clocks. It's a cooling target, not a power draw cap.
TF32: TF32 is an NVIDIA tensor core math mode that runs FP32 matrix multiplies with a 10-bit mantissa for faster AI training.
TFLOPS vs TOPS: TFLOPS counts trillions of floating-point operations per second. TOPS counts trillions of operations per second, usually INT8 or FP4.
Tensor parallelism: Tensor parallelism splits individual weight matrices across multiple GPUs, so each one computes a slice of every layer in a model.
Thermal throttling: Thermal throttling is a GPU or CPU lowering its own clock speed when it hits a temperature or power limit, so work runs slower.
Threadripper PRO vs Xeon W: Threadripper PRO and Xeon W are AMD's and Intel's single-socket workstation CPU platforms, differing in cores, PCIe lanes, and ISV support.
Threadripper vs Threadripper PRO: Threadripper runs on TRX50 boards with four memory channels and 80 PCIe 5.0 lanes. Threadripper PRO runs on WRX90 with eight channels and 128 lanes.
Time to first token (TTFT): Time to first token (TTFT) is the delay from sending a request to receiving the first output token, covering queueing and prompt prefill.
Tokens per second: Tokens per second measures how fast a model generates output during inference. It's the standard throughput metric for LLM serving.
Tower vs rack server: A tower server stands upright on a floor or desk. A rack server mounts flat in a 19-inch rack and is sized in rack units.
Transformer: A transformer is the attention-based neural network architecture behind most large language models, including GPT and BERT.
Unified memory: Unified memory is a single memory pool shared by the CPU and GPU, instead of separate system RAM and dedicated VRAM on a graphics card.
VLA model: A VLA (vision-language-action) model takes camera images and a language instruction and outputs robot actions directly.
VRAM: VRAM is the dedicated memory on a graphics card, separate from system RAM, holding textures, frame buffers, or model weights.
Workstation: A workstation is a desktop or tower built for sustained professional compute work such as CAD, rendering, or AI development.
Workstation GPU vs gaming GPU: Workstation GPUs use the same silicon as gaming cards but add ECC memory, more VRAM, and drivers certified against professional software.
vLLM vs llama.cpp: vLLM is a Python engine for high-throughput, multi-user LLM serving. llama.cpp is a C/C++ engine for running quantized GGUF models locally.