Model Migration Process on Microsoft Foundry and Azure OpenAI

Every app built on an LLM will eventually move to a new model. The model you shipped may be retired, or a newer model may offer better quality, cost, or performance. Changing a model name in code from a retiring model such as gpt-4o to a newer one such as gpt-5.1 may take one line. That line hides a much larger migration. 

Model failures are often silent to the systems using the model and loud to users. Nothing crashes. Error rates stay flat. Every dashboard says the migration went fine. Meanwhile, responses change shape, summaries become longer and more hedged, JSON fields disappear, and tool calls fire in a different order. Users notice. Support queues grow. Downstream code that depended on the old behavior starts to break.

A successful migration preserves the application’s behavior or improves it in measurable ways. That requires a repeatable process to detect drift, adapt safely, and prove quality before broad rollout.

The model migration process has six phases: Discover → Assess → Adapt → Validate → Roll out → Retire. This article explains what each phase looks like, which Microsoft Foundry tools support it today—including Azure OpenAI capabilities—and where teams still need to build around the platform. It then applies the process to a retail shopping assistant and points to additional resources in Go deeper at the end of this post!

Why migrate now

Every model has a retirement date. On Microsoft Foundry, generally available models typically ship with a retirement date about 18 months out, and older model families are actively replaced. For example, the model lifecycle and retirement schedule lists gpt-4o (2024-05-13) as retiring on October 1, 2026, with gpt-5.1 as its replacement.

What happens at retirement depends on how you buy capacity:

• Standard, Global Standard, and Data Zone Standard (pay-as-you-go) deployments are auto-upgraded on a rolling, region-by-region schedule. You control the timing with versionUpgradeOption set to one of: OnceNewDefaultVersionAvailable, OnceCurrentVersionExpired, or NoAutoUpgrade. NoAutoUpgrade means the deployment stops working at retirement. Priority Processing follows the same path.


• Provisioned (PTU) deployments are not auto-upgraded. You migrate them yourself, either in-place (traffic moves over a 20–30 minute window with no downtime) or side-by-side (stand up the new deployment, test, shift traffic, delete the old one).


• Batch deployments follow the side-by-side path: deploy the new model, resubmit jobs, retire the old deployment.

The developer problem is the same in every case: traffic eventually reaches a different model, but the platform cannot tell you whether the application still behaves as it did before. A responding endpoint does not prove that the app behaves correctly. A new model can change formatting, tone, tool-calling behavior, or JSON shape in ways that quietly break downstream code.

When should I migrate?

Start before the retirement date. Automatic upgrade handles the traffic transition for eligible deployments, but the team still owns behavioral validation. Provisioned deployments also require a manual migration.

Microsoft typically makes a replacement available in Global Standard about 90 days before retirement, in provisioned regions about 30 days before retirement, and in standard regions about two weeks before retirement. That gives you time to evaluate the new model on your own terms. Retirement dates cannot be extended.

You also do not need a deprecation notice to begin. If a newer model may improve quality, speed, or cost, run it through the process now. Waiting turns the switchover into a slow train wreck: responses drift, parsing becomes brittle, support tickets accumulate, and the team ends up debugging a model it did not choose on a date it did not pick. A deliberate migration makes the retirement date a formality and creates a process the team can reuse.

Who this is for

This process fits teams that own an LLM-powered feature inside a larger application and run migrations deliberately. It also applies to AI-native platform teams that centrally manage models for other application teams. The phases remain the same, though platform teams may run them faster and in parallel rather than in sequence. Fine-tuned workloads are out of scope here because they cannot be upgraded automatically, have separate training and deployment retirement schedules, and turn the Adapt phase into a distillation or retraining exercise rather than primarily prompt work.

The six phases

Foundry tools at a glance

Microsoft Foundry provide tools for each phase of the Model Migration Process. 

Breakdown of each phase

0. Prepare the test dataset

Before starting the six phases, build a set of representative inputs, expected outputs, and agreed success criteria. This dataset gates the middle of the lifecycle: Adapt needs inputs for replay, and Validate needs ground truth and scoring criteria.

Step 0 describes the workload rather than the candidate model, so it can begin during Discover, before the team selects a target. Build the dataset from captured production traffic or domain examples in .csv or .jsonl. If representative data is not available, use the simulator to generate synthetic inputs.

Two practices determine whether this work pays off:

• Instrument capture before you need it. Production content capture is opt-in and never retroactive. Log prompts, responses, latency, and token counts now so the team has traffic to evaluate later.


• Freeze the dataset. Keep inputs, ground truths, and success criteria fixed throughout the migration. If they change, source and target results are no longer comparable.

You also need an inventory of the model deployments your workload uses, including their deployment types (Standard, Provisioned, or Batch). For each source model, note its retirement date and suggested replacement from the Model retirement schedule.

1. Discover

Discover begins when something forces the team to consider a model change: a deprecation notice, a new generally available model, a cost or latency problem, or a capability gap. The phase ends with a decision to begin migration or stay on the current model if it remains stable, performs well, and is not approaching retirement.

Foundry tools. The model lifecycle and retirement schedule publishes retirement dates and suggested replacements. The Azure OpenAI model retirements documentation explains notification timing, including at least 60 days for generally available model retirements and at least 30 days for preview model retirements. It also explains how to configure Azure Service Health advisories and use the Models API for programmatic lifecycleStatus and deprecation checks. Those APIs provide the foundation for an internal discovery system.

Where it breaks. Customers may learn about a retirement through email, a service health alert, or a production error. By the time the right team sees the signal, it may already be deep into the deprecation window and heading toward retirement.

What your team provides. The schedule and Models API expose the data through a stable contract. Mature enterprises may add a thin notification layer that routes it to the right owners.

2. Assess

The team chooses a candidate target model and confirms that it is usable: the correct region and SKU, enough quota, and availability alongside the current model so rollback remains possible.

Assess also includes projecting monthly cost against historical traffic. Pricing structures change between model generations through reasoning tokens, cached input, structured-output overhead, and other factors. Those changes can move unit economics by 2x or more. For regulated workloads, compliance requirements such as BAA, FedRAMP, and regional Standard versus Global Standard availability may narrow the candidate list before quality testing begins.

Foundry tools. Start with the replacement suggested in the retirement schedule, then build a shortlist with model benchmarks, which compare quality, safety, cost, throughput, and latency. Use trade-off charts such as quality versus cost and the side-by-side model comparison for up to three models. Compare context windows, feature support such as function calling, structured output, and vision, and available endpoints. Confirm SKU, region, quota, and upgrade mechanics in the model retirements documentation.

Where it breaks. Teams face several plausible candidates, such as gpt-5.1, gpt-5.2, and a nano variant, without clear positioning between them. A selected model may be unavailable in the required region or SKU, a constraint that sometimes appears only after planning is underway. Historical traffic may also show that the new model costs substantially more, forcing an unplanned budget decision.

What your team provides. Public benchmarks should filter the candidate list, not make the final decision. Confirm the shortlist against the team’s own workload. Build the monthly cost view from token logs and current pricing.

3. Adapt

Adapt is often the most time-consuming phase for embedded and product-facing workloads. Validate may take longer for regulated workloads.

First, replay the existing workload on the new model without changing it. This isolates changes caused by the model. Diagnose shifts in verbosity, reasoning depth, structured-output adherence, tool-call shape, and latency. Then update the application until it recovers or improves on the previous behavior.

Prompt editing is only one part of Adapt. A migration often changes four other surfaces:

• Parameters. temperature, top_p, max_tokens, and reasoning-effort controls may not map directly between generations. Some are unsupported by newer model families.


• Tool definitions. Argument names, descriptions, and required fields that reliably guided the old model may need clearer wording or tighter constraints.


• Output schemas. Structured-output behavior changes between models. A schema the old model followed loosely may need explicit constraints, or the new model may finally enforce it.


• Calling code. API and SDK differences, including Chat Completions versus Responses, streaming formats, and new or renamed request fields, can require code changes. Downstream parsers may also assume the old response shape.

For agentic and workflow workloads, schema and tool-call changes can outweigh prompt changes.

Foundry tools. Prompt Optimizer is available through the Optimize button below the system instructions field in the Agent playground. It restructures instructions, explains each change by paragraph, and supports iteration. For example, a team can add a constraint such as “keep the JSON schema exactly” and optimize again. It is a fast first pass for a prompt that would otherwise be rewritten by hand.

For agent workloads, agent optimization tunes instructions, tools, and model selection together. Prompt Optimizer and agent optimization are available in Microsoft Foundry, not Azure OpenAI. When production data is unavailable, the simulator can generate synthetic and adversarial inputs.

Where it breaks. Most migration time is spent in a manual diagnosis loop. Teams rerun prompts by hand, compare outputs by eye, and rarely record what changed or why. For agent builders, chat benchmarks may miss tool-call regressions such as extra fields, renamed arguments, or changed call sequences. Those problems appear only when the team replays real agent traces.

Plan for three constraints:

• Start with the optimizers, then verify their output. They apply general practices in a single pass rather than fitting changes to the team’s dataset. They tune instruction text, not tool definitions or output schemas. Copy the original prompt first because there is no version history, then evaluate the optimized prompt against the frozen dataset.


• Expect more manual work when moving between providers or model families. There is no “optimize for target model X” flow. Moving from one family to another, such as OpenAI to Claude, still requires deliberate prompt and schema translation.


• Record traffic before you need it. Replay is only as useful as the captured data. Existing traces are available as an evaluation source for agents today, while content capture is opt-in and never retroactive. Log prompts, responses, latency, and tokens now to prepare for the next Adapt phase.

4. Validate

Run the adapted workload against a quality rubric on the frozen dataset. The rubric may combine rules, LLM-as-judge evaluation, human review, existing user-feedback signals, or a domain-specific scoring framework. Examples include a clinical summarization rubric for healthcare or a tool-call sequencing assertion for agents.

Validation produces a pass-or-fail decision for production exposure. AI-native teams may run the same signal continuously on every commit rather than treating it as a one-time gate.

The dataset is a dependency for both Adapt and Validate. Build and freeze it early, around Assess, even though its primary purpose belongs to this phase. Validation then has two touchpoints:

1. Before Adapt, freeze the dataset and success criteria, then run the current model to establish the source baseline.


2. After Adapt, run the target model against the same dataset and evaluators, compare it with the source baseline, and make the release decision.

Prepare the evaluation runner early and apply the gate after Adapt. Both steps belong to Validate.

Foundry tools. The Azure AI Evaluation SDK, installed with pip install azure-ai-evaluation, includes more than 30 evaluators. They cover grounding, relevance, retrieval, coherence, fluency, question answering, reference-based similarity, F1, BLEU, ROUGE, safety, agent behavior, and Azure OpenAI graders. Teams can also build custom LLM-as-judge evaluators for task-specific rubrics.

The portal evaluation flow runs the same evaluators against model, agent, dataset, and trace targets. Run identical evaluators against source and target outputs on the frozen dataset so the results remain comparable.

Measure the three dimensions used for sign-off:

• Quality: evaluator results


• Latency: leaderboard time to first token and throughput, plus operational latency from the workload


• Cost: (input tokens × input price) + (output tokens × output price)

Where it breaks. Most teams do not have an evaluation suite. Teams that do often built it themselves and may not use platform evaluation tools. Regulated workloads add mandatory human review, which can become the bottleneck. For those teams, migrations often stall in Validate rather than Adapt.

What your team provides. The evaluators are ready to run, but model workloads still require teams to curate a domain-relevant test set from production traffic. That is why Phase 0 pays for itself.

5. Roll out

Promote the validated configuration in stages: non-production, then a canary or weighted percentage of production traffic, followed by broader exposure. Compare live latency, errors, and quality signals with the pre-migration baseline, then commit or roll back.

Some workloads cannot expose a new model to customer traffic during testing, including flows involving protected health information or financial transactions. Use shadow or mirror mode instead: run the new model offline against production inputs and compare its outputs with the old model without affecting users.

Foundry tools. Migration mechanics depend on the deployment SKU:

• Standard, Global Standard, and Data Zone Standard deployments upgrade automatically on a rolling schedule. Control timing with versionUpgradeOption: OnceNewDefaultVersionAvailable, OnceCurrentVersionExpired, or NoAutoUpgrade. Priority Processing follows the same path.


• Provisioned, Global Provisioned, and Data Zone Provisioned deployments migrate manually, either in place during a 20-to-30-minute Azure-managed traffic transition or through side-by-side deployments.


• Batch deployments migrate side by side. Deploy the new model, resubmit jobs, then retire the old deployment.


• Fine-tuned deployments do not upgrade automatically. They follow separate training and deployment retirement schedules, so plan retraining or distillation early.

See the model retirements documentation for deployment-specific guidance.

Use continuous evaluation to score a sample of production traffic in the Foundry Observability dashboard. Connect evaluation results to traces for root-cause analysis and configure Azure Monitor alerts for quality regressions.

Where it breaks. Offline evaluation can miss production quality and latency regressions. Rollback decisions may also be forced by deprecation deadlines rather than evidence.

What your team provides. Teams implement weighted routing between deployments in their application or gateway layer. They must also choose how long to keep the old deployment warm for rollback. Embedded copilot teams often target about 30 days. Design both mechanisms once and reuse them for future migrations.

6. Retire

Retire is easy to forget. Decommission the old deployment, free its capacity, archive evaluation artifacts, update internal documentation, and communicate the change to downstream owners. That may include customer-facing documentation, marketing pages, support runbooks, and audit logs. Regulated workloads may need to retain artifacts for years.

Retirement is also a governance step.

Foundry tools. Use the Models API to confirm that the old version is retired through lifecycleStatus or 410 Gone. Use the observability dashboard to confirm that the active deployment count falls. Add useful production traces to the golden dataset so the next migration starts with better evidence.

Where it breaks. Teams skip the phase. Zombie deployments accumulate, leaving teams with structural debris from migrations they never finished.

What your team provides. The observability dashboard shows deployment count, but the team must decide which deployments still carry traffic. Create an explicit retirement ticket rather than relying on someone to remember. Embedded copilot teams also need to update public claims such as “powered by gpt-4o” after the model changes.

Worked example: Zava’s Shopping Assistant migrates from gpt-4o mini to gpt-5.x

Zava is a fictional retailer used as a stand-in for a real customer story. The example reflects patterns observed in customer-facing embedded AI workloads.

Zava’s Shopping Assistant is one of the company’s largest LLM workloads. It has two LLM stages:

1. Per-review insight extraction identifies sentiment, attribute mentions, and defect signals across thousands of product reviews.


2. Product-level summaries present those findings to shoppers on the product page.

Together, the two stages account for a meaningful share of Zava’s token volume.

Discover

Zava’s central AI Platform team made gpt-5.x models available internally and notified feature teams. The Shopping Assistant team learned about the models through that channel and received a target migration window before gpt-4o mini’s deprecation.

Zava has an internal discovery layer built on the retirement schedule and Models API. Microsoft provides the underlying data, while Zava routes the signal to application owners.

Assess

The team compared gpt-5.4 nano, which offered lower latency and cost, with gpt-5.1 and gpt-5.2 using leaderboard trade-off charts. Selection remained difficult because of the rapid release cadence, unclear positioning between variants, and the lack of a behavioral benchmark for product question-and-answer workloads.

Capacity planning required coordination with the AI Platform team. Both gpt-4o mini and the gpt-5.x candidate needed to remain available in the same regions so the team could roll back.

Adapt

The team ran its existing Shopping Assistant prompts against gpt-5.4 nano using a sanitized traffic sample. Customer queries were scrubbed of personally identifiable information before replay.

The behavioral comparison found three problems:

• Summaries used more hedged language and sometimes contradicted the underlying review evidence, creating a shopper-trust risk.


• Insight counts varied across runs. The model sometimes extracted substantially more or fewer attribute mentions than gpt-4o mini, affecting downstream filtering.


• Latency varied more than expected on the synchronous product-page path.

Prompt Optimizer helped restructure the summary prompt, but the team still diagnosed the differences manually. It built its own replay system and behavioral comparison on top of captured traffic. Reengineering took weeks and extended beyond prompts: parameters and downstream parsing for the insight-extraction output also changed.

Validate

The team scored outputs with Zava’s Product Answer Quality (PAQ) rubric. Its nine criteria cover factual grounding, attribute accuracy, tone, and refusal behavior for questions outside the catalog. Zava implemented the rubric as custom evaluators in the Azure AI Evaluation SDK.

Initial evaluations used unchanged prompts to isolate model behavior. The team reran them after each prompt change. Zava’s QA team also completed a manual review, which the company requires for every new model used in a customer-facing workflow.

The per-review insight extraction stage still has no automated evaluation, a gap the team has accepted for now.

Roll out

The validated configuration moved to non-production and then through staged exposure: employees first, followed by a small percentage of shoppers.

The canary exposed latency regressions that offline evaluation had missed. The team rolled back the latency-sensitive synchronous product-page path while keeping the offline pregeneration path on the new model.

Retire

Retirement is not complete. The gpt-4o mini deployment remains warm for rollback. The team must resolve the synchronous-path latency regression before retiring it, which sends that code path back to Adapt.

This split state is common. Retire can lag Roll out by weeks or months, and the old deployment remains visible in deployment-sprawl data.

Lessons from the example

• Adapt consumed most of the schedule. Replay tooling, behavioral comparison, and prompt reengineering are the clearest opportunities for Microsoft to shorten migrations.


• Validate worked because Zava had invested in it. Most customers do not have an equivalent to PAQ. Making domain-specific evaluations cheaper to build would improve confidence in this phase.


• The migration is partially live and partially rolled back. The process must support split-state workloads rather than assuming a binary switch from old model to new.

How to use this process

The six phases can serve as a checklist and an interview script for teams creating or auditing a migration process.

• Documentation and process: Lead with the six phases. Most teams recognize them immediately.


• Investment priorities: Start with Adapt and Validate. Across customer stories, those phases consume the most time and confidence.


• Interviews and postmortems: For each phase, ask whether it happens, who owns it, which tool the team uses, where it failed last time, and what evidence would increase confidence.


• Metrics: Discover and Retire are the easiest phases to instrument through measures such as announcement reach and active deployment count. Adapt and Validate require purpose-built telemetry that most teams do not yet have.

Go deeper

Two companion resources turn the process into concrete implementation steps:

• Microsoft Learn guide: This article follows the six phases through identifying affected deployments, preparing a test dataset, adapting prompts, evaluating source and target models, and rolling out by deployment type.


• Foundry Models Accelerator: This community toolkit includes a deployment-inventory scanner, a feasibility and assessment playbook, code and API migration audit scripts, an A/B evaluation runner with golden datasets, and rollout guidance. It follows the same six-phase process. The Foundry Models Accelerator is a community-built toolkit provided as-is under the MIT License. It falls outside Microsoft Support. Always verify model availability and retirement dates against official documentation. 

Additionally, check out the Foundry Forgebook which hosts a plethora of recipes that walk through the required code changes to migrate from different source to target models, even across model families. 

The goal of a model migration is to change the model without changing your application’s behavior, or to change it measurably for the better. Lead with the six phases, invest first in Adapt and Validate phases, and treat the Retire phase as a governance step.