Azure GPU VMs for AI: the ND, NC, and how to choose.
Everyone writes about the GPUs. Almost nobody explains how you get one, pay for it, and run it in production — which is the part the job is about.
New to cloud? CAMPUX is a free, build-first course. Start here →
The short version: use the NC-series for inference and smaller training runs, and the ND-series, such as ND H100 v5, for large-scale distributed training, where nodes are wired together with NVIDIA InfiniBand so many VMs behave as one cluster. In practice you rarely hand-provision either; you set a quota, then pool them behind AKS or Azure Machine Learning and let a scheduler place work. Treat the exact SKUs and prices here as a snapshot that will drift; treat the family logic as settled.
That single paragraph answers most of the search traffic that lands on this page. The rest of the article is for the person who has to make it real: pick a family, get capacity approved, keep the bill sane, and not leave a forty-dollar-an-hour GPU idling overnight. That is a cloud-engineering problem far more than a machine-learning one, and it is where the money and the mistakes live.
The N-series, decoded
Azure puts every GPU VM under the letter N. Within that there are a handful of families, and the letters after the N tell you the intent. NC is compute. ND is deep-learning-at-scale. NV is visualization. There are a couple of niche ones (NG for cloud gaming, NP for FPGA) that you can ignore for AI. If you remember only the first three, you can place almost any workload correctly.
NC-series is the workhorse. Current sizes run on NVIDIA H100 (NCads H100 v5), A100 (NC A100 v4), the older V100 (NCv3), and the T4 for lighter inference (NCasT4 v3). You get one GPU or a few in a single VM. That covers most of what people mean when they say "I need a GPU": serving a model, fine-tuning, training something that fits on one machine. It is the family you will touch first and most often.
ND-series exists for one thing: training that is too big for a single node. A current ND H100 v5 VM carries eight H100 GPUs joined internally by NVLink, and the part that matters is that each GPU gets its own 400 Gb/s NVIDIA Quantum-2 InfiniBand link so many of these VMs can be stitched into one training fabric with very low latency between them. There is no single-GPU ND size; the family only makes sense when you are running across many GPUs at once. If you are not doing distributed training of a large model, you do not need ND, and you should not pay for it.
NV-series is the one people put on the list and then misuse. It is built for virtual desktops and 3D visualization (CAD, rendering, remote graphics workstations), not for training neural networks. It has GPUs, so it looks tempting, but reaching for NV to train a model is using the wrong tool because it was on the shelf.
If you want the broader tour of Azure VM naming beyond GPUs — the B, D, E, and F families and how the version suffixes work — the Azure VM sizes and series note covers it. This page stays on the GPU families and how AI teams run them. It is also one spoke of the larger Azure AI infrastructure picture, which ties the compute layer to storage, networking, and the managed AI services.
The comparison, on one screen
The whole decision fits on one screen. The cost column is deliberately vague, see the honest note below it, but the "best for" column is the one to trust.
| VM family | Best for | GPU / interconnect | Rough cost posture |
|---|---|---|---|
| NC-series | Inference, fine-tuning, single-node training | NVIDIA H100 / A100 / V100 / T4; one or few GPUs per VM, no cross-node InfiniBand fabric | Lowest sane entry point; T4 sizes ~$1–2/hr, A100/H100 sizes into low tens/hr |
| ND-series | Large-scale distributed training of big models | 8× NVIDIA H100 (or A100) per node, NVLink inside, 400 Gb/s InfiniBand per GPU across nodes | Highest; a full 8-GPU node runs into the tens of dollars/hr and up |
| NV-series | Virtual desktops, 3D visualization, remote graphics — not training | Visualization-class NVIDIA GPUs; no training-grade multi-node interconnect | Mid; priced for graphics workstations, not for AI math |
| Serverless API (no VM) | Calling an existing model from an app; most inference in production | None you manage — Azure OpenAI / AI model catalog runs the GPUs | Per-token, zero idle cost; often cheapest until volume is very high |
One honest caveat on that last column: GPU pricing is the most volatile number on Azure. It changes by region, by generation, by how you buy, and by whatever the current supply crunch looks like. Every figure above is a rough posture as of 2026, meant to convey order of magnitude — a T4 costs single-digit dollars an hour, an ND node costs an order of magnitude more. Price your exact size in your exact region in the Azure pricing calculator before you commit to anything, and distrust round numbers, including mine.
Do you even need a GPU VM?
This is the question the GPU-spec blogs never ask, and it is the one that saves the most money. If your task is "call a large language model from my application," you almost certainly do not want a GPU VM at all. Azure OpenAI and the Azure AI model catalog serve models behind an API and bill per token. You provision nothing, you patch nothing, and you pay zero when no requests are in flight. A GPU VM, by contrast, bills every hour it exists whether or not it is doing anything — and a GPU that idles between requests is the most expensive paperweight in the building.
You want your own GPU VM when the managed API cannot do the job: you are training or fine-tuning a model, you need a custom or open-weight model the catalog does not host, you have data-residency rules that keep the weights in your own subscription, or you need control over the runtime that a hosted endpoint will not give you. Those are real and common reasons. But "I'm building an AI feature" is not automatically one of them. Start by asking whether an API call does it, and only rent silicon when the answer is a specific, defensible no.
A GPU that idles between requests is the most expensive paperweight in the building.
The part the GPU blogs skip: operating them
Say you have decided you genuinely need GPU VMs. Almost every guide stops here, at the moment you would start working. The interesting problems are all downstream of choosing a SKU, and they are ordinary cloud-engineering problems.
Quota is the first wall. A fresh subscription has a GPU vCPU quota of roughly zero. You do not get to deploy an H100 node because you feel like it; you file a quota-increase request, justify it, and wait — and for the newest GPUs in the busiest regions, capacity is genuinely scarce and the request can be denied or take days. Plan for this. The engineer who assumes an ND H100 v5 is one az vm create away has not shipped GPU work in production.
Cost control is a scheduling problem. The way you keep a GPU bill from detonating is to never let a GPU sit idle on demand. That means Spot capacity for interruptible training — the same mechanism covered in Azure Spot virtual machines, at a large discount in exchange for possible eviction — plus autoscale-to-zero so pools shrink to nothing when no jobs are queued, plus reservations only for the baseline you know you will run around the clock. Getting this wrong is how a team wakes up to a five-figure surprise.
You pool them; you do not pet them. In a real environment nobody hand-creates named GPU VMs. You put a GPU node pool behind Azure Kubernetes Service, or you let Azure Machine Learning manage a compute cluster, and jobs request GPUs from the pool. The scheduler places the work, scales the pool up when the queue grows, and scales it back down, ideally to zero, when it drains. That is the difference between a demo and a system: the demo has a GPU VM someone SSHes into, and the system has a pool that expands and contracts on its own and bills only for what ran.
Anyone can recite that H100 beats A100. The signal that you have actually run this is talking about the operating envelope: "I'd file the quota request first because capacity is the bottleneck, run training on Spot ND nodes behind an AKS pool that scales to zero, and keep inference on a serverless endpoint so we're not paying for idle GPUs." That sentence is worth more than a memorized spec sheet, because it is the part of the job that a spec sheet cannot teach.
Why this is a good thing to know right now
The market for people who can wire up and run AI infrastructure is the least crowded corner of cloud work. Plenty of people can pass a fundamentals exam; far fewer can get an H100 quota approved, keep a training bill under control, and pool GPUs behind AKS without leaving money on the meter. That gap is exactly where junior engineers can stand out, because the field is new enough that the certified-versus-hired distance is short — the skills are scarce and the demand is real. If you want the evidence for that demand, the state of Azure cloud-engineer hiring note lays it out, and the Azure cloud engineer roadmap shows where the compute and AI-infra skills sit in a full learning path.
None of it requires renting an H100 to practice. You learn the operating model on cheap hardware — a small T4 NC size, an AKS pool with a spot node, autoscale set to zero — and the same patterns scale straight up to the expensive silicon. The knowledge transfers; the bill does not have to.
Questions people also ask
What is the difference between Azure NC and ND series?
Both are N-series GPU VMs, but they aim at different jobs. NC-series is the general compute-and-ML family — single or few GPUs, good for inference, fine-tuning, and smaller training runs, with current sizes on NVIDIA H100, A100, or T4. ND-series is the distributed-training family — multi-GPU nodes wired together with NVIDIA InfiniBand so many VMs act as one large training cluster. If one node holds your model, reach for NC. If training has to span many nodes at once, that is what ND exists for.
Which Azure VM is best for AI training?
For large-scale training of big models, the ND-series is Azure's purpose-built family — current-generation ND H100 v5 nodes carry eight NVIDIA H100 GPUs joined by NVLink and 400 Gb/s InfiniBand per GPU so training can scale across many nodes. For fine-tuning or training a model that fits on one machine, an NC-series size such as NCads H100 v5 or NC A100 v4 is usually the cheaper, saner choice. Match the family to whether the job fits on one node or must span many.
Do I need a GPU VM to run AI on Azure?
Often no. If you only need to call a large language model, Azure OpenAI and the Azure AI model catalog serve models over an API and bill per token, so you never provision or pay for an idle GPU. You want your own GPU VM when you are training or fine-tuning models, running a custom model the managed services do not offer, or need full control of the runtime. For most application work, the serverless API is faster to ship and cheaper than renting a GPU that sits idle between requests.
How much does an Azure GPU VM cost?
Treat any single number with suspicion — GPU VM pricing moves, varies by region, and depends heavily on how you buy. As a rough posture as of 2026: a small T4-based NC size runs on the order of one to two dollars per hour on demand, an A100 or H100 NC size runs into the low tens of dollars per hour, and a full eight-GPU ND H100 v5 node runs into the tens of dollars per hour and up. Spot pricing and reservations cut those figures substantially. Always price your exact size in your region in the Azure pricing calculator before committing.
What GPUs does Azure offer for AI?
Across the current N-series families Azure offers NVIDIA data-center GPUs including the H100 (NCads H100 v5, ND H100 v5), the A100 (NC A100 v4, ND A100 v4), the older-generation V100 (NCv3), and the T4 for lighter inference (NCasT4 v3). NV-series and NG-series carry GPUs aimed at visualization and cloud gaming rather than training. Exact SKUs are added and retired regularly, so confirm what is available in your target region on Microsoft Learn before you design around a specific chip.