As AI systems evolve from single-turn assistants into tool-using agents, evaluation has to evolve with them.
For a traditional LLM application, teams often focus on whether the final response is relevant, coherent, or grounded. For an agent, that is only part of the problem.
An agent may produce a plausible answer while still selecting the wrong tool, sending incorrect tool parameters, ignoring tool output, violating user constraints, making unnecessary calls, or failing to complete the task.
This is why Microsoft Foundry separates agent evaluation into multiple layers and supports both system-level and process-level evaluation. Microsoft describes agent evaluators as a way to systematically assess quality, safety, and performance across agent workflows—not only the final response.
https://learn.microsoft.com/en-us/azure/foundry/observability/how-to/evaluate-agent
Start with the evaluation target
Foundry allows evaluations against different targets:
- Agent — run the selected agent against test inputs and evaluate newly generated behavior
- Model — evaluate a model directly
- Dataset — score outputs that already exist in a database
This distinction matters because the evaluation workflow changes depending on the target. When the target is an Agent, Foundry generates a fresh response for each input and evaluates that result. When the target is a Dataset, Foundry evaluates the responses already present in the dataset.
https://learn.microsoft.com/en-us/azure/foundry/how-to/evaluate-generative-ai-app
For agentic systems, I find it useful to evaluate in two scopes:
Individual turns for detailed debugging of tool use and response behavior.
Full conversations for multi-turn task completion, conversational coherence, and user satisfaction. Microsoft currently recommends starting with full conversations and simulated data for controlled testing, then using real conversations in production.
System evaluation: did the agent actually accomplish the task?
The first layer is the end-to-end outcome.
Important evaluators include:
Task Completion
Did the agent actually complete the user’s task?
For example:
“Find black running shoes under $120 in size 9 in Dallas.”
An agent may correctly search for products but stop before checking inventory. The response may look reasonable, but the task is still incomplete.
Task Adherence
Did the agent follow its instructions, policies, and explicit user constraints?
If the customer says the budget is $120 and the agent recommends a $129 product without clearly identifying the budget violation, adherence should suffer.
Intent Resolution
Did the agent correctly understand and address what the user wanted?
A tool call can be technically valid but semantically wrong. Searching generic sneakers when the user explicitly requested running shoes is a good example.
Customer Satisfaction
This looks beyond technical correctness and asks whether the overall interaction is likely to satisfy the user.
These metrics answer a simple but important question:
Did the system actually solve the user’s problem?
Microsoft groups these kinds of measurements under system evaluation for agent workflows.
Process evaluation: did the agent take the right path?
This is where agent evaluation becomes much more interesting.
Foundry includes dedicated process evaluators for tool-using agents.
Tool Selection
This checks whether the agent chose the appropriate tool.
Suppose an agent has:
- search_products
- check_inventory
- get_promotions
- web search
If the user asks:
“Is DailyRun X available in size 9 in Dallas?”
The right enterprise action is to use the inventory tool.
If the agent uses web search instead, the answer may still sound plausible, but the process is wrong.
Tool Input Accuracy
This checks whether the tool was called with the correct inputs.
For example:
check_inventory( product_id=”DEMO-SHOE-002″, size=”9″, location=”Dallas-TX” )
The agent may choose the right tool but pass the wrong size or location.
That means Tool Selection can pass while Tool Input Accuracy fails.
Tool Call Success
This evaluates whether the invocation itself succeeded.
For example:
- the API returned successfully
- there was no timeout
- there was no execution failure
But a successful tool call does not mean it was the correct call.
An API may return HTTP 200 for the wrong SKU or wrong location.
So:
Tool Call Success measures execution reliability, not semantic correctness.
Tool Output Utilization
This checks whether the agent correctly used what the tool returned.
Imagine the inventory tool returns:
{ “available”: false }
but the agent replies:
“Yes, size 9 is available.”
The tool worked. The input may even have been correct. The failure is in how the agent used the result.
Tool Call Accuracy
This gives a broader signal about whether tool invocation behavior was correct overall.
Together, these evaluators answer:
Did the agent execute the right workflow, not just produce a convincing answer?
Response quality still matters
Even when the process is correct, the final response can still be poor.
Foundry supports quality evaluators such as:
- Relevance
- Groundedness
- Completeness
- Coherence
- Fluency
Groundedness
Groundedness asks whether claims are supported by the available context or evidence.
For enterprise applications, this is especially important because answers should be based on authoritative sources rather than unsupported model memory.
Completeness
Did the agent answer all important parts of the request?
If the user asks:
“Is this available and is there a member discount?”
an answer that checks inventory but ignores the promotion question is incomplete.
Coherence and Fluency
These evaluate communication quality.
Microsoft defines coherence around logical and orderly presentation of ideas, while fluency focuses on readability, grammar, vocabulary, and clarity.
https://learn.microsoft.com/en-us/azure/foundry/concepts/evaluation-evaluators/general-purpose-evaluators
Built-in evaluators are not enough for every enterprise agent
A retail agent, finance agent, claims agent, or operations agent will always have business-specific requirements that generic evaluators cannot fully capture.
Microsoft’s current guidance recommends using a rubric evaluator as a primary measure when teams need to express application-specific criteria such as policy enforcement, tool-use accuracy, or communication standards. Built-in evaluators can then be layered on top for broader coverage.
A retail-agent rubric might include:
- never claim availability without inventory verification
- never invent discounts
- never silently violate a stated budget
- prefer enterprise catalog tools over public search
- avoid duplicate or unnecessary tool calls
- clearly identify compromises when no exact match exists
This is where evaluation starts to look much more like business acceptance testing.
https://learn.microsoft.com/en-us/azure/foundry/concepts/evaluation-evaluators/rubric-evaluators
The dataset becomes a regression suite
Foundry evaluation datasets are reusable test collections.
They are useful when comparing:
- Prompt V1 vs V2
- one model vs another
- tool-definition changes
- orchestration changes
- release candidate vs production version
When Foundry evaluates an agent against a dataset, it generates a new response for each input and scores it. When the target is the live agent, Foundry ignores existing responses in the dataset.
This changes prompt engineering from:
“This prompt looks better.”
to:
“This version performs better on the same regression suite.”
That is a much more defensible engineering practice.
Golden datasets, synthetic data, and production traces all serve different purposes
A mature evaluation strategy uses multiple data sources.
Golden datasets: Best for critical known scenarios and deterministic regression testing.
Synthetic data: Useful for expanding coverage and generating edge cases before enough production traffic exists.
Full-conversation simulation: Useful for testing multi-turn user journeys and end-to-end task behavior.
Production traces: Useful for evaluating what real users actually experienced.
Foundry supports evaluation directly from Application Insights traces and can evaluate deployed interactions without replaying the original request.
https://learn.microsoft.com/en-us/azure/foundry/observability/how-to/cloud-evaluation-deployed-interactions
Production evaluation should operate on traces
Production failures are often different from development failures.
Real users introduce:
- unexpected phrasing
- missing information
- contradictory constraints
- unusual tool sequences
- edge cases that synthetic test data may not cover
Foundry can evaluate traces already captured in Application Insights. The trace-evaluation workflow supports selecting traces by trace ID or by agent filter, and also supports intelligent sampling to select a representative subset rather than evaluating every interaction.
That creates a strong operational loop:
Run → Trace → Evaluate → Identify failures → Add to regression suite → Fix → Re-evaluate
Microsoft Foundry’s evaluation stack becomes especially useful as teams can evaluate not only what an agent said, but also how it behaved operationally, which tools it used, whether it completed the task, and how that behavior changes over time.