Azure AI Foundry, explained — the platform layer above the GPUs.
Everyone wants to build with AI on Azure and nobody can quite say what Foundry is versus Azure OpenAI versus Azure Machine Learning. This is the plain version, plus the part the AI tutorials leave out.
New to cloud? CAMPUX is a free, build-first course. Start here →
Azure AI Foundry is Microsoft's platform for building, deploying, and operating AI apps and agents on Azure. It bundles a model catalog, two deployment modes (serverless pay-per-token and managed compute pay-per-hour), plus evaluation, observability, and an agent runtime, all under one Azure resource with shared identity, networking, and policy. Treat the exact feature names as fluid; the shape is settled.
That last hedge matters more than usual here, because Microsoft is in the middle of renaming the whole thing. As of 2026 the documentation leads with Microsoft Foundry, folding the old Azure AI Foundry, Azure AI Studio, and Azure AI Services brands into one resource and one portal. I verified the current wording on Microsoft Learn before publishing this, and the older hub-based experience is now labelled "Foundry (classic)." The name you see in a given blog post depends mostly on when it was written. I will use "Azure AI Foundry" through this piece because that is still what most people are searching for, and note where the new name changes things.
What Foundry really is
Strip away the marketing and Foundry is a control plane for AI workloads. You point it at a model, choose how that model runs, wire your application to the endpoint, and then use the surrounding tooling to test outputs, watch cost and latency, and, increasingly, orchestrate agents that call tools on your behalf. It is the layer that sits above the raw GPUs and below your application code.
The reason it exists is that the old way was fragmented. If you wanted OpenAI models you provisioned an Azure OpenAI resource. If you wanted an open-weight model you went somewhere else. If you wanted to fine-tune or train, that was Azure Machine Learning, a different portal with a different mental model. Foundry's pitch is one resource, one project, one endpoint, with a catalog behind it that spans vendors. Whether that consolidation is finished or still half-built depends on the week you check, so hold the particulars loosely even while the overall direction looks locked in.
Foundry versus Azure OpenAI versus Azure Machine Learning
These three get conflated constantly, and the confusion is understandable because they overlap and share plumbing. The cleanest way to hold them in your head is by the job each one is built for.
| Layer | What it is | When you reach for it |
|---|---|---|
| Azure AI Foundry | The platform. A single Azure resource that unifies a model catalog, deployment, evaluation, observability, and an agent runtime under shared identity, networking, and policy. | You are building an application or agent on top of foundation models and want one place to deploy, monitor, and govern it — especially across more than one model or vendor. |
| Azure OpenAI / Foundry Models | The model access layer. Azure OpenAI serves OpenAI's models via API; the wider Foundry Models catalog adds models from Microsoft, Meta, Anthropic, and others behind the same platform. | You need to call a specific model — a GPT deployment for chat, an embedding model for search. If that is genuinely all you need, the endpoint alone is enough. |
| Azure Machine Learning | The model-building platform. Datasets, experiments, training pipelines, and MLOps for models you create and own end to end. | You are training or heavily fine-tuning your own models and need experiment tracking, pipelines, and lifecycle control — the classic data-science workflow. |
| Raw GPU VMs | The bare compute. N-series virtual machines with GPUs that you patch, scale, and babysit yourself. | You have an unusual runtime, a licensing constraint, or a cost model that only works when you own the hardware. Rare, and you take on all the operational weight. |
Read the table top to bottom and you are moving from most managed to least managed. Foundry hands you the most abstraction and the fastest path to something working; raw GPU VMs hand you total control and total responsibility. Most people building AI features should live in the top two rows and touch the bottom two only when they have a concrete reason. If you are already comfortable with how Azure hosts managed services, the same instincts apply — Foundry is a managed platform like App Service is for web apps, just aimed at models and agents.
Foundry is the layer above the GPUs and below your code — the part that turns a model endpoint into something a company will run in production.
Foundry Models: the catalog
The catalog is the piece people underestimate. Foundry Models is a browsable library of foundation models — the OpenAI family you would have reached through Azure OpenAI, plus open-weight and partner models from Microsoft, Meta, and others. Microsoft's own numbers put it well past a thousand models, and you should read that figure the way you read any vendor count: the order of magnitude is the point, not the exact number, which will be stale by the time you finish this sentence.
What the catalog buys you is optionality without re-plumbing. You can start on a large frontier model to prove a feature works, then swap in a smaller, cheaper model for the requests that do not need the big one, without leaving the platform or rebuilding your auth and networking. That swap-ability is a real engineering advantage, and it is the main reason to build on Foundry rather than wiring a single model endpoint directly into your app.
Deployment: serverless versus managed compute
This is the decision that shows up on your bill, so it is worth getting right. Foundry gives you two broad ways to run a model, and they price on completely different axes.
Serverless, pay-per-token. You call an API, you pay for the tokens in and out, and you never see the infrastructure. There is no capacity to provision and nothing idling when traffic is quiet. For spiky, unpredictable, or early-stage workloads this is almost always the right start — your cost tracks your usage and a slow week costs you almost nothing.
Managed compute, pay-per-hour. You reserve dedicated capacity, GPU-backed instances that Foundry manages, and you pay for that capacity by the hour whether or not it is busy. This wins when you have steady, high-volume traffic, need predictable latency, or want a model that is not offered serverless. The trap is leaving it running idle; dedicated compute bills the same at 3am with zero requests as it does at peak.
Serverless feels free until a runaway agent loops and generates millions of tokens overnight; managed compute feels safe until you leave an endpoint provisioned over a quiet weekend. Both failure modes are ordinary cloud cost hygiene, not AI problems — budgets, alerts, and someone watching the bill. The teams that get burned are the ones who treated "it's just an API call" as though it had no cost surface. It always does.
Agent Service, briefly
The newest layer is the agent runtime, Foundry Agent Service. An agent, in this context, is a model wrapped with instructions, memory, and a set of tools it can call: search, code execution, a function in your own system. Instead of your application driving every step, the agent decides which tool to use and when, then reports back. Foundry hosts these agents, versions them, and gives you the tracing to see what the agent did on each run.
It is genuinely useful and genuinely early. The APIs and terminology are shifting — Microsoft has already moved from an Assistants-style API to a newer Responses-style one, renaming the core concepts along the way. If you are learning this now, learn the shape (a model, tools, memory, a runtime that orchestrates them) and expect the specific method names to move under you. That is the honest state of the whole agent space in 2026, not just Foundry.
Where the cloud engineer fits
Now the part the AI tutorials skip, and it is the part that pays. Every Foundry demo you will watch ends at "look, it answered." Every Foundry workload a company runs in production needs a whole layer underneath that the demo never shows: the boring cloud-engineering work.
Identity, first. A production AI app authenticates with Microsoft Entra ID and a managed identity, not a key pasted into an environment variable. Networking, second: you do not want your model traffic crossing the public internet, so you put the Foundry endpoint behind a private endpoint and lock the public route off. Then cost — token budgets and alerts so a looping agent does not become a five-figure surprise. Then governance: role-based access so not everyone can deploy a model, and policy so deployments land in approved regions. None of that is AI work. All of it is what separates a notebook demo from something that survives a security review.
This is also, quietly, the best career news in the field. The market is flooded with people who can call a model API and thin on people who can stand up the identity, networking, cost, and governance around it safely. AI infrastructure is the least-saturated corner of a crowded market precisely because it demands the unglamorous cloud skills on top of the shiny model ones. If you already know how to run Azure well, Foundry is not a new discipline — it is your existing discipline pointed at a new workload. We pull the whole picture together in the Azure AI infrastructure guide, and if search is your entry point, Azure AI Search is the natural next read.
The takeaway
Azure AI Foundry, or Microsoft Foundry as the labels catch up, is the platform layer for AI on Azure: catalog, deployment, evaluation, observability, and agents in one resource. It sits above Azure OpenAI, which is now one model source inside it, and beside Azure Machine Learning, which is still where you go to build models rather than build with them. Learn the shape and you can ignore the naming churn. And remember the quiet truth underneath all of it: the model is the easy part. The identity, the private networking, the cost controls, and the governance are the job, and they are the reason a company will pay you to run this instead of watching a demo.
Questions people also ask
What is Azure AI Foundry?
Azure AI Foundry is Microsoft's platform for building, deploying, and operating AI applications and agents on Azure. It gives you a catalog of models, two ways to deploy them (serverless pay-per-token or managed compute pay-per-hour), tools for evaluation and observability, and an agent runtime — all under one Azure resource with unified identity, networking, and policy. In 2026 Microsoft began renaming it to Microsoft Foundry, so you will see both names in the wild.
What is the difference between Azure AI Foundry and Azure OpenAI?
Azure OpenAI is a service that gives you API access to OpenAI's models, such as the GPT family. Azure AI Foundry is the broader platform that Azure OpenAI now sits inside. Foundry adds a multi-vendor model catalog, agent tooling, evaluation, and observability on top, and it exposes OpenAI models alongside models from Microsoft, Meta, and others. If you only ever call one GPT deployment, the OpenAI endpoint is enough; once you need multiple models, agents, or production monitoring, you are effectively using Foundry.
Is Azure AI Foundry the same as Azure Machine Learning?
No. Azure Machine Learning is the platform for training, tracking, and deploying your own custom models — the classic MLOps workflow with datasets, experiments, and pipelines. Azure AI Foundry is aimed at building applications and agents on top of existing foundation models you mostly consume rather than train from scratch. They overlap and share plumbing, but the intent differs: Machine Learning is for people building models, Foundry is for people building apps with models.
Is Azure AI Foundry being renamed?
Yes. In 2026 Microsoft began consolidating Azure AI Foundry, Azure AI Studio, and Azure AI Services under a single brand, Microsoft Foundry, with one Azure resource type and portal at ai.azure.com. Documentation now leads with the Microsoft Foundry name and treats the older hub-based experience as Foundry (classic). The names in this space churn often, so check Microsoft Learn for the current wording before you quote it.
Do you need to know Azure to use AI Foundry?
For a demo, no — you can click through the portal and call a model in minutes. For anything real, yes. A production Foundry workload still needs the ordinary cloud-engineering layer underneath: Microsoft Entra identity and role-based access, private endpoints so traffic does not cross the public internet, cost controls on token and compute spend, and governance. That underneath layer is exactly the skill set most AI tutorials skip, and it is what turns a demo into something a company will actually run.