Microsoft proposes open evaluation and control tools for AI agents

Microsoft announced ASSERT and the Agent Control Specification as part of an effort to make AI agents easier to evaluate and control across different development frameworks.

What Microsoft announced

In a Microsoft Foundry Blog post titled “Build agents you can trust across any framework with open evals and a control standard,” Microsoft said ASSERT is an open-source, policy-driven evaluation framework for AI agents. The company also introduced the Agent Control Specification, described as a portable runtime control standard intended to govern agents across frameworks such as LangChain, CrewAI, LiteLLM, and OpenAI.

The announcement addresses a practical problem for organizations building agentic systems: agents may use tools, call external services, and make multi-step decisions, but evaluation and governance mechanisms are often tied to a specific framework or implementation. Microsoft’s stated approach is to separate evaluation and runtime control from any one agent framework.

ASSERT focuses on trace-aware evaluations

The GitHub repository for responsibleai/ASSERT describes ASSERT as a “framework-agnostic, trace-aware evaluation harness” for AI agents and LLM applications. According to the repository description, ASSERT generates behavior-specific test cases from requirements.

That framing is important because many agent failures are behavioral rather than simply textual. An agent may produce a plausible final answer while taking an unsafe action, using the wrong tool, or violating an operational requirement along the way. A trace-aware evaluation system is designed to inspect more than just final output, using execution traces to assess whether the agent followed expected behavior.

Microsoft’s blog post characterizes ASSERT as policy-driven, which suggests that teams can express requirements as policies and then use those policies to test agent behavior. The public repository description does not by itself prove how well ASSERT works in production, but it does indicate Microsoft is positioning the project as a reusable evaluation harness rather than a benchmark tied to one model or one application.

Agent Control Specification targets runtime governance

Microsoft also pointed to the Agent Control Specification as a portable runtime control standard. The Microsoft Agent Governance Toolkit repository describes the toolkit as including policy enforcement for autonomous AI agents and lists the Agent Control Specification as its deterministic policy decision runtime.

In practical terms, runtime control is different from pre-deployment evaluation. Evaluation frameworks help developers test whether an agent is likely to behave as expected. Runtime policy enforcement is meant to make decisions while the agent is operating, such as whether a proposed action should be allowed, denied, or constrained.

The Agent Governance Toolkit repository’s description of a deterministic policy decision runtime suggests Microsoft is trying to make those decisions predictable and auditable. That may matter for organizations that need to show how agent actions are governed, especially when agents can interact with business systems, developer tools, or external APIs.

Why framework-agnostic controls matter

The Microsoft Foundry Blog specifically mentions frameworks and providers including LangChain, CrewAI, LiteLLM, and OpenAI. By naming those ecosystems, Microsoft is presenting ASSERT and the Agent Control Specification as tools for mixed environments rather than a Microsoft-only stack.

That is a notable design goal because many teams do not standardize on a single agent framework. A prototype might begin in one framework, move to another for orchestration, and rely on different model providers through an abstraction layer. If evaluation rules and runtime controls are coupled too tightly to one framework, governance can become inconsistent as applications evolve.

A portable control standard could help teams apply the same policy logic across agent implementations, provided the relevant frameworks and applications adopt or integrate the specification. The sources provided do not establish broad industry adoption, so the practical impact will depend on implementation quality, maintainer support, and uptake by developers.

The broader significance

Microsoft’s announcement reflects a broader shift in AI development from model-only evaluation toward system-level evaluation. Agentic applications combine model output with tools, memory, retrieval, orchestration, and external actions. As a result, testing only the final text response may miss important risks.

ASSERT, as described by the responsibleai/ASSERT repository, is aimed at generating test cases from requirements and evaluating agent traces. The Agent Governance Toolkit, according to Microsoft’s GitHub repository, adds policy enforcement for autonomous AI agents using the Agent Control Specification as a deterministic runtime decision mechanism.

Together, the projects suggest a two-part approach: test agent behavior before deployment, then enforce policy decisions while the agent runs. Microsoft’s sources present this as an open and framework-agnostic approach, but the announcement should be read as an early tooling and standards proposal rather than proof that agent governance problems are solved.

For developers and organizations experimenting with AI agents, the immediate takeaway is that Microsoft is making agent evaluation and runtime policy enforcement more explicit parts of the development process. Whether ASSERT and the Agent Control Specification become widely used will depend on how easily teams can integrate them into existing agent applications and how clearly they support real-world governance requirements.