15 Ways an AI Agent Can Go Wrong at Work

ai-harm-assessment-methodology
Overview of the six-step methodology, from framework definition through risk scenario generation and validation to taxonomy construction and evaluation. Step 1 defines the multi-layer framework, which organizes sociotechnical risk into three nested layers: technical capability (the AI agent, the goals it pursues, and the environment it acts in), human interaction (how workers engage with those components), and systemic impact (consequences that propagate beyond the individual task). Step 2 selects 2,078 computer-based O*NET job tasks (209 roles, 20 industries). Step 3 applies the framework to each task through a structured prompt, anticipating 8,356 risk scenarios labeled with severity and deployment mode. Step 4 validates the scenarios with 45 workers from the selected roles and with an LLM-as-a-judge, on validity, utility, and discovery metrics. Step 5 builds the taxonomy from 8,589 scenarios: the 8,356 anticipated ones plus 233 describing risks that have already occurred, drawn from workplace AI incidents and documented agent failure cases; scenarios that do not map onto the Generative AI Risk Taxonomy produce new categories and sub-categories, extending it into the 15-category Workplace AI Agent Risk Taxonomy. Step 6 evaluates the taxonomy on structural integrity, contextual coverage, and practical usability.

A clinician gets an AI-generated triage score. It looks right. It's confident. She signs off, and stops doing the mental check she used to do on her own. Do that a few thousand times across an organization, and you've built the most common AI risk in the modern workplace. Not a rogue robot. A quiet handoff.

"Keep a human in the loop" is the phrase everyone uses to make an AI rollout feel responsible. This study suggests it's exactly where the danger lives. It analyzed 8,356 AI risk scenarios pulled from 2,078 real jobs, then validated them with 45 workers and an independent AI judge from a different model family. The expectation was that automation, taking the human out, would be the scary part. The opposite showed up. The two biggest risk categories both cluster around the "safe" setup where a human stays in the loop:

  • (1) The agent is wrong, confidently. Erroneous actions were 30.6% of all risks, 88% rated high or critical. And it's not hallucination. Fabrication was under 1%. It's answers that fit the inputs but not the situation, delivered with certainty, and approved by a human who trusts them.
  • (2) The human quietly stops checking. Skill and oversight erosion was 21.3%, and 97% of it came from augmentation. It's mostly rated "limited" severity, which is the trap. It's invisible until it's irreversible.

Lisanne Bainbridge named this back in 1983. She called it the "irony of automation": the more reliable a system becomes, the less people practice the skills they'll need when it fails. AI just made the system reliable enough, and proactive enough, to accelerate the fade.

The takeaway that should reset your roadmap: a more accurate model won't fix this. The failure isn't inside the model. It's at the seam between the agent and the person. Automation's risks are loud: financial, operational, reputational. Augmentation's risks are silent, and they land on the worker, not the balance sheet. Two different problems. They need two different playbooks.

So before you ask "is the model good enough?" ask the question that actually predicts harm: What happens to the person standing next to it? Can your people genuinely override the agent, or have they quietly started to just approve it?


Publications

  • Unaccountable Delegation, Fading Skills: Mapping the Risks of Workplace AI Agents. AIES 2026 PDF