Reference

A plain-language glossary of the hardware, specs, and terms we work with every day.

BMC: A BMC (baseboard management controller) is the small dedicated chip on a server motherboard that runs IPMI or Redfish for remote management.
Barebone server: A server chassis shipped with the motherboard, backplane, power supplies, and cooling installed, but no CPU, memory, storage, or GPUs.
Blower vs open-air GPU cooler: A blower cooler exhausts a GPU's heat out the card's rear bracket. An open-air cooler vents it into the case for the chassis fans to remove.
Burn-in testing: Burn-in testing runs new hardware at sustained full load before deployment to catch early failures before a customer sees them.
CUDA cores vs Tensor cores: CUDA cores run general parallel math one operation at a time. Tensor cores complete a small matrix multiply per clock for AI math.
Cluster computing: Cluster computing links multiple servers together so they operate as one system for a single workload.
Colocation: Colocation is renting rack space, power, and network connectivity in a third-party data center for hardware you own and manage yourself.
Context window: A context window is the maximum number of tokens a language model can hold at once, covering the prompt, the history, and its own reply.
DGX: DGX is NVIDIA's line of complete, factory-built AI servers and workstations, sold with GPUs, CPU, memory, and software preinstalled as one unit.
Deep learning: Deep learning is machine learning using neural networks with many stacked layers, trained on data to approximate a function.
DisplayPort vs HDMI: DisplayPort and HDMI are the two video outputs on workstation GPUs, differing mainly in bandwidth per version and licensing.
Drive form factors (U.2, M.2, EDSFF): U.2, M.2, and EDSFF are the three physical shapes an SSD ships in. None of them interchange without an adapter.
ECC memory: ECC memory detects and corrects single-bit errors in RAM automatically, using an extra chip per module to store parity data.
FP16: FP16 (half precision) is a 16-bit floating point format, 1 sign bit, 5 exponent bits, 10 mantissa bits, used in mixed precision training.
FP32: FP32 (single precision) is a 32-bit floating point format, 1 sign bit, 8 exponent bits, 23 mantissa bits, the default type for GPU compute.
FP4: FP4 is a 4-bit floating point format used in NVIDIA Blackwell GPUs for AI inference, trading precision for lower memory use and higher throughput.
FP64: FP64 (double precision) is a 64-bit floating point format, 1 sign bit, 11 exponent bits, 52 mantissa bits, used in HPC and simulation.
FP8: FP8 is an 8-bit floating point format used for neural network training and inference, storing each value in 1 byte instead of 2 or 4.
Fine-tuning: Fine-tuning continues training a pretrained model on a smaller, focused dataset so it adopts a specific style, domain, or task.
GDDR: GDDR is graphics-oriented DDR memory built for bandwidth over capacity. GDDR6 and GDDR7 are the current versions.
GGUF: GGUF is a single-file format that packs a language model's weights, tokenizer, and settings together, built for llama.cpp.
GPU passthrough: GPU passthrough assigns one physical GPU directly to a single virtual machine, bypassing the hypervisor for near-native performance.
GPU server: A GPU server is a rack-mounted computer built to hold multiple GPUs, with the power and cooling to run them all at full load.
Gray market hardware: Gray market hardware is a genuine product sold outside a manufacturer's authorized channel, usually without transferable warranty or support.
HBM: HBM is memory stacked in vertical layers beside a processor and connected through a silicon interposer for very high bandwidth.
HGX: HGX is NVIDIA's SXM GPU baseboard that server makers build into their own systems, not a complete server on its own.
Hybrid storage: Hybrid storage combines fast NVMe or SSD drives with high capacity HDDs in one system, using each tier for what it does best.
IPMI: IPMI is the standard protocol for monitoring and controlling a server remotely, independent of its operating system or power state.
ISV certification: ISV certification is a software vendor's formal validation that a specific GPU, driver version, and OS combination runs its application correctly.
Inference: Inference is the stage where a trained model answers a prompt, producing output from fixed weights rather than adjusting them.
InfiniBand vs Ethernet: InfiniBand is a purpose-built lossless fabric for GPU clusters. Ethernet is the general-purpose standard that now competes through RoCE.
KV cache: The KV cache stores each previous token's key and value tensors so a transformer doesn't recompute them at every generation step.
Lead time: Lead time is the span from placing an order to the goods arriving. It runs from days for in-stock parts to months for allocated ones.
Liquid cooling vs air cooling: Liquid cooling carries chip heat away in a coolant loop instead of airflow, handling far higher power density than air at higher cost.
LoRA: LoRA (low-rank adaptation) fine-tunes a model by training small adapter matrices instead of all its weights.
Local LLM: A local LLM runs inference on hardware you own, keeping data on premises instead of sending prompts to a cloud API.
MGX: MGX is NVIDIA's modular server reference architecture, letting OEMs mix CPU, GPU, and DPU trays on shared chassis designs.
MOQ: MOQ, or minimum order quantity, is the smallest number of units a supplier will sell on a single purchase order for a given part number.
Mixture of experts (MoE): Mixture of experts splits a model into parallel sub-networks and routes each token through only a few of them, not the whole model.
Multi-GPU workstation: A multi-GPU workstation runs two or more GPUs in one machine, usually on separate parallel jobs rather than one combined render.
NPU: An NPU is a dedicated chip block that runs AI inference at low power, rated in TOPS, separate from the CPU and the GPU.
NVIDIA GPU architectures: NVIDIA GPU architectures are the chip design generations, such as Ampere, Hopper, and Blackwell, that define a GPU's era and features.
NVLink and NVSwitch: NVLink is Nvidia's direct GPU-to-GPU interconnect. NVSwitch is the crossbar chip that lets many GPUs share it as one any-to-any fabric.
OEM vs ODM: ODM designs and builds a product from scratch. OEM puts its own brand on that product and sells it.
Open weights: Open weights are a model's trained parameters released for anyone to download and run, instead of access only through a hosted API.
PCIe lanes: PCIe lanes are the individual data paths linking the CPU to GPUs, drives, and cards. Lane count sets the bandwidth ceiling on a slot.
PCIe power connectors: PCIe power connectors are supplemental cables, 6-pin, 8-pin, and 12V-2x6, that deliver power to a GPU beyond the 75 W the slot itself supplies.
PSU efficiency (80 PLUS): PSU efficiency is the share of AC input a supply delivers as DC output. 80 PLUS certifies it in tiers from Bronze to Titanium.
Quantization: Quantization reduces a model's numerical precision, from 16-bit to 8-bit or 4-bit, shrinking memory use with some loss of accuracy.
RAID: RAID combines multiple physical drives into one logical volume for redundancy, performance, or both, depending on the level used.
RDIMM vs UDIMM: RDIMM adds a register chip that buffers command and address signals for higher capacity. UDIMM skips that chip for lower cost and latency.
RMA: RMA (return merchandise authorization) is the number a vendor issues before it accepts a defective part back for repair, replacement, or credit.
RTX PRO: RTX PRO is Nvidia's brand for workstation and server GPUs, successor to Quadro, with certified drivers and ECC memory.
Rack PDU: A rack PDU is the rack-mounted power strip that feeds every server and switch in one cabinet from one or two upstream circuits.
Rack unit: A rack unit (U) is 1.75 inches of vertical rack height. Server chassis are sized in whole U, most commonly 1U, 2U, and 4U.
Redundant power supply: Two or more power supplies in one server, each able to carry the full load, so the system stays up when one fails or loses input power.
Retrieval-augmented generation (RAG): RAG pairs a language model with a document search step, so it answers using retrieved passages instead of only what it learned in training.
SAS vs SATA vs NVMe: SAS, SATA, and NVMe are the three interfaces a drive connects through, and they differ in protocol, speed, and cost.
SKU: SKU (stock keeping unit) is the code identifying one specific, orderable version of a product, distinct from others sharing its name.
SXM vs PCIe: SXM is a socketed GPU module built for full NVLink bandwidth. PCIe is the standard add-in card that fits an ordinary server slot.
Storage server: A storage server is built to hold and serve data, spending its budget on drive bays and network throughput instead of CPU or GPU power.
TDP: TDP is the heat, in watts, a chip's cooling system must remove at sustained rated clocks. It's a cooling target, not a power draw cap.
Tensor parallelism: Tensor parallelism splits individual weight matrices across multiple GPUs, so each one computes a slice of every layer in a model.
Threadripper PRO vs Xeon W: Threadripper PRO and Xeon W are AMD's and Intel's single-socket workstation CPU platforms, differing in cores, PCIe lanes, and ISV support.
Tokens per second: Tokens per second measures how fast a model generates output during inference. It's the standard throughput metric for LLM serving.
Transformer: A transformer is the attention-based neural network architecture behind most large language models, including GPT and BERT.
VLA model: A VLA (vision-language-action) model takes camera images and a language instruction and outputs robot actions directly.
VRAM: VRAM is the dedicated memory on a graphics card, separate from system RAM, holding textures, frame buffers, or model weights.
Workstation: A workstation is a desktop or tower built for sustained professional compute work such as CAD, rendering, or AI development.
Workstation GPU vs gaming GPU: Workstation GPUs use the same silicon as gaming cards but add ECC memory, more VRAM, and drivers certified against professional software.