Common Mistakes and Troubleshooting
Most problems people run into while building their first few agents in AI Agent Studio come down to a handful of recurring patterns. This page walks through the ones that show up most often, and how to fix them.
The goal is too broad
If you give an agent a goal like "handle customer support," it has no real way to know what "handle" means in any specific case, so its behavior ends up inconsistent from one run to the next. This is the single most common cause of an agent that seems to work sometimes and fail other times for no obvious reason.
The fix is almost always to narrow the goal until it describes one decision, made well, rather than a whole department's worth of responsibility. "Resolve level-1 billing disputes under $200" is a goal you can actually test against. "Handle customer support" isn't.
The skill's prompt is vague about categories
A skill that has to choose between a few outcomes (refund, replacement, escalate; or approve, deny, request more info) needs the criteria for each outcome spelled out, not implied. If your prompt says "decide whether to approve the request," but never defines what makes a request approvable, you're relying on the model to guess your policy, and it will guess inconsistently.
Write out the actual decision rules as if you were training a new employee: what specifically qualifies for each category, and what to do when a case doesn't clearly fit any of them.
No guardrail on the risky action
It's tempting to skip guardrails on a test agent since it feels like extra setup for something you're "just trying out." The problem is that agents that start as tests often quietly become the thing people actually rely on, and by then nobody remembers to go back and add the guardrail. Add at least one guardrail on any action with real consequences (issuing a refund, sending an external email, modifying a record) from the very first version, even while you're still testing.
Testing only the easy cases
It's natural to test an agent with the same handful of clean examples you used while designing it. Those cases will almost always pass, because you designed the prompt around them. The cases that actually reveal problems are the messy ones: a complaint with no order number, a request that's ambiguous between two categories, an angry customer whose complaint buries the actual issue in three paragraphs of frustration. Build a small test set that deliberately includes edge cases before you trust an agent with real traffic.
Skipping data grounding, then wondering why answers feel generic
An agent without a connected data source can only reason from the complaint or request text itself, plus whatever general knowledge the underlying model has. That's fine for an early prototype, but if the agent's answers feel disconnected from your actual policies or records, the fix usually isn't a better prompt. It's connecting a real knowledge or data source so the agent has something concrete to ground its reasoning in.
Treating a prompt change like a minor tweak
Because editing a prompt feels as easy as editing text, it's easy to underestimate how much a small wording change can shift an agent's behavior. Before pushing a prompt change to a live agent, rerun it against your existing test cases, not just the one case that prompted the change. It's common for a fix to one case to quietly break a different case that was working fine before.
The agent and a human disagree, and it's not obvious why
When an agent's decision doesn't match what a human reviewer would have chosen, resist the urge to just tweak the prompt until it agrees with that one case. First check whether the disagreement is really a case of the agent reasoning badly, or whether the human reviewer applied judgment or context the agent was never given access to in the first place. Sometimes the real fix is adding a data source or a rule to the prompt, not making the prompt vaguer or more complicated in an attempt to cover every possible exception.
Where to go from here
If you're consistently running into one of these issues, it's usually worth going back to Core Concepts and re-checking that each piece (goal, skill, guardrail, PRE) is doing the job it's meant to do, rather than layering fixes on top of a foundation that needs to be rebuilt.