Adobe Research has presented SAGE-Agent and ClarifyBench, a research framework for evaluating how LLM agents decide when to ask clarification questions before making tool calls. The work, also listed by ACL Anthology and described on arXiv, focuses on ambiguity in multi-turn, tool-augmented tasks.
Adobe Research has presented SAGE-Agent and ClarifyBench, a research effort aimed at evaluating how large language model agents handle ambiguous or incomplete user instructions before calling external tools.
According to Adobe Research’s publication page, the project focuses on “structured uncertainty guided clarification” for LLM agents. The central problem is familiar in tool-using AI systems: a user may ask for an action, but leave out details that affect which tool should be used or what parameters should be passed. Instead of guessing, an agent may need to ask a clarification question.
The arXiv abstract describes SAGE-Agent as a method that represents uncertainty over tool-call parameters in a structured way. It frames the clarification process using a partially observable Markov decision process, or POMDP, and uses an expected value of perfect information, or EVPI, objective to decide when asking a question is worthwhile.
The work also introduces ClarifyBench, which Adobe Research describes as a benchmark for multi-turn, dynamic tool-calling disambiguation. In this setting, an agent must decide whether it has enough information to act, whether it should ask the user for more details, and how to use the answer in subsequent tool calls.
The arXiv listing characterizes ClarifyBench as a benchmark for multi-turn tool-augmented disambiguation. This is narrower than general chatbot evaluation: it is specifically concerned with cases where the agent must resolve missing or ambiguous tool parameters across an interaction.
That distinction matters because tool use can turn a vague response into a concrete action. If a user request leaves out a location, time, file, preference, or other parameter, the agent’s decision to infer, ask, or proceed can materially change the outcome. The Adobe Research page positions SAGE-Agent and ClarifyBench around this decision point.
ACL Anthology lists the paper in Findings of ACL 2026. Its entry states that SAGE-Agent achieves 7–39% higher coverage while reducing clarification questions by 1.5–2.7x. Those figures, as presented by ACL Anthology, suggest the authors are measuring both task completion breadth and the cost of asking users additional questions.
The combination is important for clarification systems. Asking too few questions can lead to incorrect tool calls; asking too many can burden the user and slow task completion. The reported results indicate that the proposed approach is intended to improve coverage while being more selective about when to ask for more information.
The available source excerpts do not provide all experimental details, such as the full set of baselines, datasets, or evaluation protocols. The strongest supported claim is that Adobe Research, ACL Anthology, and arXiv describe the work as a benchmark and method for structured uncertainty and clarification in tool-using LLM agents, with ACL Anthology reporting the stated coverage and question-reduction results.
LLM agents are increasingly evaluated not only on whether they can produce fluent language, but also on whether they can make reliable decisions while using tools. In a tool-calling workflow, ambiguity is not just a conversational inconvenience; it can determine the parameters of an external action.
SAGE-Agent’s emphasis on uncertainty over tool-call parameters reflects a broader research concern: agents should know when they lack enough information. The EVPI framing described in the arXiv abstract offers one formal way to weigh the value of asking a clarification question against the expected benefit of acting immediately.
ClarifyBench’s contribution, as described by Adobe Research and arXiv, is to provide a setting where this behavior can be tested over multiple turns rather than in a single static prompt. That makes it more aligned with realistic interactions, where the agent’s first question and the user’s answer change what should happen next.
For developers and researchers building tool-using assistants, the paper points to a practical evaluation question: not simply whether a model can call a tool, but whether it can recognize when the tool call is under-specified and ask only the clarifications that are likely to matter.
According to Adobe Research’s publication page, the project focuses on “structured uncertainty guided clarification” for LLM agents.
The central problem is familiar in tool using AI systems: a user may ask for an action, but leave out details that affect which tool should be used or what parameters should be passed.
Instead of guessing, an agent may need to ask a clarification question.
Continue reading
