<![CDATA[Ravi's Substack]]>https://blog.proxperity.aihttps://substackcdn.com/image/fetch/$s_!4PKo!,w_256,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fecc4bf4f-539c-4b8c-af4e-780243c75d8b_460x460.jpegRavi's Substackhttps://blog.proxperity.aiSubstackSat, 01 Aug 2026 19:08:30 GMT<![CDATA[Microsoft AI Foundry 101: The AI Infrastructure of This Era]]>https://blog.proxperity.ai/p/microsoft-ai-foundry-101-the-ai-infrastructurehttps://blog.proxperity.ai/p/microsoft-ai-foundry-101-the-ai-infrastructureSat, 01 Aug 2026 17:34:08 GMT

Every era of computing has had an infrastructure layer that quietly carried everything built on top of it. The PC era had the operating system. The internet era had the web server and the database. The cloud era had virtualized compute and storage. The AI era needs something new: a layer that lets organizations pick models, ground them in their own data, turn them into agents, and run all of it with enterprise security and governance.

Subscribe now

Share

Subscribe now

Microsoft’s answer is Azure AI Foundry, now called Microsoft Foundry. This article is a plain-language introduction for people who keep hearing the name and want to understand what it actually is, why it matters, and what the evidence says about where it is headed.

Thanks for reading Ravi's Substack! Subscribe for free to receive new posts and support my work.

What is it, in one paragraph

Foundry is Microsoft’s unified platform for building AI applications and agents. Microsoft describes it as “the enterprise AI platform to build, ground, and govern AI apps and agents at scale.” Instead of stitching together a model provider here, a vector database there, and a monitoring tool somewhere else, Foundry puts model access, agent building, knowledge grounding, evaluation, and governance behind a single platform with one management plane. It launched as Azure AI Foundry at Microsoft Ignite in November 2024, evolving out of Azure AI Studio, and was renamed Microsoft Foundry at Ignite 2025.

The building blocks

You can think of Foundry as six pieces that snap together.

Foundry Models. A catalog of more than 11,000 models, including frontier models from OpenAI and Anthropic, plus Meta, Mistral, xAI, Cohere, DeepSeek, Black Forest Labs, Microsoft’s own Phi and MAI families, and thousands of open-source models from Hugging Face. Since Anthropic’s Claude arrived in November 2025, both Claude and GPT frontier models have run side by side on the same platform, and Claude in Foundry reached general availability in June 2026. A built-in Model Router can even select the best model for each request automatically, balancing quality against cost.

Foundry Agent Service. A managed service for building and running AI agents, meaning applications that use a model to reason about a request, call tools, access data, and take multi-step actions. It reached general availability at Build 2025 with support for over 1,400 enterprise data connectors, and now supports multi-agent workflows, persistent memory, and open protocols such as MCP and A2A, so agents can interoperate across vendors and even across clouds. Agents can be published straight into Microsoft Teams and Microsoft 365 Copilot.

Foundry IQ. The knowledge layer. It turns enterprise data in places like SharePoint, Azure Blob Storage, and OneLake into reusable, permission-aware knowledge bases that any agent can query. Retrieval respects the access controls your organization already has, which is the difference between a demo and something a bank can deploy.

Foundry Observability. Tooling to evaluate, trace, and monitor AI systems: quality and safety evaluators, red teaming before release, and OpenTelemetry-based tracing in production. This is the part most homegrown AI stacks skip, and the part enterprises need most.

Foundry Control Plane. A single place for security, governance, and oversight of agents, anchored by Microsoft Entra Agent ID, which gives every agent its own managed identity. If agents are going to act on behalf of your company, they need the same identity discipline as employees.

Foundry Local. The same stack running on devices and at the edge, including offline and air-gapped scenarios. Microsoft reported that Foundry Local reached 560 million devices within six months of launch.

A tour of the portal: Discover, Build, Operate

Article content
Image generated with MAI Image 2.5 Pro, one of the Microsoft AI models available in the Foundry catalog.

The components above describe what Foundry is made of. The portal itself is organized around what you do with it, in three phases that mirror the lifecycle of any AI application: Discover, Build, and Operate.

Discover is where model selection stops being guesswork. You can browse the catalog by provider, from Azure OpenAI and Anthropic to Microsoft, Meta, Mistral, and DeepSeek, explore prebuilt services for speech, language, translation, and content understanding, and compare collections such as models sold directly by Azure, Foundry Labs research models, and Foundry Local for on-device use. The standout feature is the model leaderboard. It ranks models across four dimensions: a quality index covering reasoning, coding, math, and knowledge, safety measured by attack success rates where lower is better, throughput in tokens per second, and benchmark cost. Microsoft’s guidance is practical: start with the scenario leaderboard that matches your use case, open the detailed benchmarking results for a shortlist, then deploy, try the model in the playground, or evaluate it on your own data. Being able to weigh a frontier model against a cheaper alternative with published quality, safety, and cost numbers, side by side and before writing any code, is worth a lot of engineering hours.

Build is the workspace where a project comes together: agents, model deployments, fine-tuning, tools, connected services, knowledge, memory, data, evaluations, and guardrails all live in one place. The significance is less any single feature and more the adjacency. Evaluations and guardrails sit next to the agent you are building, not in a separate product you bolt on later.

Operate is the control plane view for everything running in production: active agents, agent success rates, run volume over time, token usage, estimated cost, alerts, compliance posture, and quota, filterable by subscription and project. It even embeds AI assistance for questions like analyzing performance trends or spotting anomalies in agent metrics. This is the unglamorous phase where most AI initiatives historically stall, and having day-two operations built into the same platform that shipped the agent is a large part of the infrastructure argument this article is making.

More than models: context is the real engineering problem

Here is a lesson the industry has learned the hard way: the model is rarely the reason an AI project fails. What separates a demo from a production system is context, meaning the enterprise knowledge, permissions, memory, and tools you supply to the model. Microsoft has leaned into this idea explicitly. Asha Sharma, President of CoreAI Product, put it simply at Ignite 2025: “The more context an agent has, the more grounded, productive, and reliable it’s likely to be.”

This discipline now has a name, context engineering, and Microsoft has built products around it. Foundry IQ is described on its own product page as “a context engineering platform” for Microsoft’s intelligence layer. Rather than treating retrieval-augmented generation as a one-time lookup, its agentic retrieval engine plans queries, searches multiple sources in parallel, and iterates on the results. The outcomes are measurable: at the Foundry IQ general availability announcement in June 2026, Microsoft reported that knowledge bases improved recall by up to 54 percent compared with single-shot RAG, and that retrieval enhancements lifted answer quality benchmarks by up to 20 percent.

Context also spans more than documents. Work IQ brings signals from how people work in Microsoft 365, Fabric IQ brings business data and ontologies, and Foundry IQ brings organizational knowledge, together forming what Microsoft calls its intelligence layer. Add agent memory, which reached production at Build 2026 with procedural memory that Microsoft says delivers 7 to 14 point absolute gains in task success rates, and you can see the pattern: the durable engineering work in AI is happening around the model, not inside it.

The overlooked layer: sandboxes for agent code

Agents do more than answer questions. They write code, execute it, work with files, and act on live systems. The moment an agent runs code it generated itself, or code shaped by content it read somewhere, you are executing untrusted code, and untrusted code demands hard isolation. Microsoft’s own security benchmark says it plainly: use a sandboxed execution environment to isolate function and plugin runtimes, and design agent systems with the expectation that some attacks will succeed.

The service that answers this on Azure is worth knowing even though it sits outside Foundry: Azure Container Apps dynamic sessions. Dynamic sessions provide sandboxes that are fully isolated by a Hyper-V boundary and designed specifically for running untrusted code, drawn from pools of prewarmed environments so new sessions are allocated in milliseconds. It is a general Azure capability, generally available since late 2024, with official integrations for frameworks such as LangChain, LlamaIndex, and Semantic Kernel, so you can use it in your own applications directly.

This layer is still evolving fast: at Build 2026 Microsoft introduced Azure Container Apps Sandboxes, now in preview, a new resource type that gives agents persistent, isolated workspaces with suspend and resume, state snapshots, and controlled network egress. Where dynamic sessions are ephemeral and pool-managed, Sandboxes are stateful and developer-controlled, a sign that sandboxed compute is becoming a first-class primitive for the agent era.

The tight integration with Foundry is where it gets interesting. When an agent built on Foundry Agent Service uses the code interpreter tool, that Python code runs in exactly this kind of Microsoft-managed sandbox: Hyper-V isolated, blocked from outbound network requests, with a separate session per conversation. The same philosophy now extends to the agents themselves. Hosted agents in Foundry Agent Service run in per-session, VM-isolated sandboxes, each with dedicated compute, memory, and a persistent filesystem. Microsoft’s engineering team described the alternative, multiple customers sharing an instance where an agent writes files and touches credentials, as “a security and isolation nightmare.” Sandboxing is not a nice-to-have in the agent era. It is a load-bearing part of the infrastructure.

Bring your own code: Microsoft Agent Framework

Foundry is the platform, but developers also need an open way to write agents in code. That is Microsoft Agent Framework, an open source SDK and runtime that merges two earlier Microsoft projects, AutoGen from Microsoft Research and Semantic Kernel, into a single programming model. It was announced in October 2025 and reached its production-ready 1.0 release in April 2026 under an MIT license, with stable support for .NET and Python and Go in preview.

The framework gives you the pieces production agents actually need: a graph-based workflow engine with checkpointing for multi-agent orchestration, pluggable memory, middleware, built-in OpenTelemetry observability, and native support for open protocols such as MCP and A2A. It is deliberately not locked to one model provider, with connectors for Foundry, Azure OpenAI, OpenAI, Anthropic, Amazon Bedrock, Google Gemini, and local models through Ollama.

Its relationship with Foundry completes the story: build and test your agent locally with the framework, then deploy it to Foundry’s hosted agents with a few lines of code, where it inherits the sandboxed runtime, identity, and observability described above. Local open source flexibility on one end, governed enterprise infrastructure on the other, with the same code in between.

My own career coach agent, Twin, runs on this framework. More on that in a future post.

Why “infrastructure of the era” is not just a slogan

Bold framing deserves evidence, so here are the numbers, all from Microsoft’s own earnings calls, investor materials, and official blogs.

Adoption has compounded quarter after quarter. At Build in May 2025, Microsoft cited more than 70,000 organizations using Foundry. By October 2025 that figure was 80,000. On the fiscal year 2026 fourth quarter earnings call in July 2026, Satya stated: “We now have 100,000 Foundry customers, and revenue more than doubled year-over-year.” On the same call he noted that nearly 90 percent of the Fortune 500 ground their agents in enterprise context using Foundry, Fabric, and Work IQ.

Usage is growing faster than adoption. Microsoft reported over 500 trillion tokens processed through Foundry APIs in fiscal 2025, up seven times year over year. By July 2026, the number of Foundry customers running at a one trillion token annualized rate had increased four times year over year. In January 2026, Microsoft disclosed that more than 1,500 customers were already using both Anthropic and OpenAI models on the platform, which shows the multi-model bet is being validated by real customer behavior.

The underlying platform is scaling with it. On that July 2026 call, Satya also announced that Azure’s annual revenue surpassed 100 billion dollars for the first time, up 41 percent, with AI demand as the primary driver. Foundry sits at the center of that motion.

Independent recognition backs this up. In December 2025, Microsoft announced it was named a Leader in the Gartner Magic Quadrant for AI Application Development Platforms, positioned furthest for completeness of vision.

What this means if you are just getting started

If you are a developer, the entry point is simple: create a Foundry project, pick a model from the catalog, and you get a single endpoint and SDK that works across models, so you can swap models later without rewriting your application. From there you can attach tools and knowledge, evaluate before shipping, and publish an agent to Teams or your own app.

If you are a technology leader, the more important idea is optionality. Model rankings change every few months. A platform that treats models as interchangeable components, keeps your data grounding and governance constant, and lets you route work to whichever model is best today is a hedge against betting on the wrong horse.

The AI era will be defined less by any single model and more by the infrastructure that turns models into dependable systems. The evidence above suggests Foundry has become one of the strongest candidates for that role. Understanding it now is a good investment, whichever platform you ultimately choose.

A note on this article: I wrote this in a personal capacity and it reflects my own views, not an official Microsoft communication. All details are drawn from Microsoft’s public documentation, blogs, and investor materials as of July 2026. If anything becomes outdated as the platform evolves, tell me in the comments and I will keep it corrected.

Sources

Thanks for reading Ravi's Substack! Subscribe for free to receive new posts and support my work.

]]>