Chapter 17
When an agent goes wrong, and why you approve before it acts
The chapter in a minute
- Agents go wrong in five common ways: misreading a page, clicking the wrong thing, going round in circles, acting on a confident mistake and doing more than you asked.
- Words hidden in a page or document can steer an agent, and you may never see them. Limits and your yes keep the damage small.
- Sending, buying, deleting and publishing always wait for your yes, and so does any step with no easy undo. Find the stop button and write your undo plan before a task starts.
- You stay responsible for what an agent does in your name, so keep its jobs small enough to stand behind.
- In a step log, watch for details you never gave it, repeated steps, claims with nothing behind them and important steps with no pause. A line saying a page told it what to do is a warning sign too.
Try it now
Read the log
0 of 6 steps done
Step 1 of 6
Read the log from the top. For each line, say in a few words what the agent was told, looked at, clicked, typed or decided.
Tools and settings, as of a date
The book points to the ‘Agent safety’ section of the companion page. Here it is in full, with the date each detail was checked. Open it on the companion page
Chapter 17 points here. Every detail below was read on its source page on 2 November 2026 unless another date is given.
How often agents go wrong
These figures change within months, so the book keeps them off the printed page.
- Success and failure. Stanford HAI's 2026 AI Index (opens in a new tab) says agents rose "from 12% to ~66% task success on OSWorld", a test of real computer tasks, but "still fail roughly 1 in 3 attempts on structured benchmarks" (top takeaways, page 9 of the report PDF (opens in a new tab)). The same report counts 362 documented AI incidents in 2025, up from 233 in 2024. That is a count of reports, not of all harms.
- Click errors (2024). In the OSWorld paper (opens in a new tab) (first version 11 April 2024, current version 30 May 2024), the researchers looked at 550 failed attempts by the best agent of the time and found mouse-click mistakes in more than 75% of them, the most common error. The agent often planned the right steps and then clicked in the wrong place. Repeated clicks and clicking the wrong thing followed from that. In the same paper people completed about 72% of the tasks and the best agent about 12%.
- Unsafe actions (2025). OS-Harm (opens in a new tab) (first version 17 June 2025, current version 29 October 2025) tested five agent models on 150 tasks. Table 2 of the paper gives the share of attempts where the agent did something unsafe:
| Model tested | Harmful request from the user | Planted instruction (prompt injection) | Agent's own mistake | Average |
|---|---|---|---|---|
| o4-mini | 52% | 20% | 10% | 27% |
| GPT-4.1 | 48% | 12% | 4% | 21% |
| Claude 3.7 Sonnet | 70% | 10% | 8% | 29% |
| Gemini 2.5 Pro | 64% | 8% | 8% | 27% |
| Gemini 2.5 Flash | 70% | 2% | 6% | 26% |
The authors say all the models "tend to directly comply with many deliberate misuse queries" and "occasionally perform unsafe actions". The models tested are already older than the ones on sale today, so read these as a snapshot from 2025.
- Web agents. "An Illusion of Progress? Assessing the Current State of Web Agents (opens in a new tab)" (first version 2 April 2025, current version 8 October 2025) built a new test of 300 tasks on 136 live websites and found agents did far worse than earlier, easier tests had suggested. Its authors warn of "over-optimism in previously reported results".
Stop, take-over and confirm controls
See "Agent options" for the full walk-through. As of 2 November 2026:
- Google (Gemini on Android). The multi-step tasks page (opens in a new tab) says you can tap "Stop task", or "View progress" then "Take control", though on some phones you can only stop. It adds: "Review confirmation requests and clarifying questions closely".
- OpenAI (ChatGPT). The cloud browser page (opens in a new tab) says ChatGPT asks for your confirmation in the chat before steps such as a booking or a payment, that you can ask to take over the browser yourself, and that you should stop the task if it opens the wrong site or uses the wrong details.
- Anthropic (Claude in Chrome). The permissions guide (opens in a new tab) says Claude shows a plan, including which websites it will visit, for you to approve before it starts, and asks again before visiting any site not in the plan.
Changes that happen without confirmation
Google's tasks and workflows page (opens in a new tab) still says "Gemini can perform bulk actions on private tasks in Google Tasks without your confirmation." Checked: 2 November 2026.
Which agent features are offered in Ireland and the UK
As of 2 November 2026:
| Feature | Ireland | UK | Source |
|---|---|---|---|
| Google, tasks and workflows in Gemini (Google AI Pro or Ultra) | No: not offered in the European Economic Area | No | Google help page (opens in a new tab) lists the EEA, Nigeria, Switzerland and the UK as excluded |
| Google, multi-step tasks on Android (beta, certain Pixel and Samsung phones) | No: Ireland is not on the list | Yes | Google help page (opens in a new tab) lists India, Japan, South Korea, the UK and the US |
| OpenAI, cloud browser in ChatGPT Work (paid plans except Free and Go) | Yes | Yes | OpenAI help page (opens in a new tab) says "supported regions"; Ireland and the UK are both on OpenAI's supported countries list (opens in a new tab) |
| Anthropic, Claude in Chrome (Pro, Max, Ultra, Team and Enterprise) | Yes | Yes | Anthropic help page (opens in a new tab) names no regional limit; Ireland and the UK are both on Anthropic's list of locations (opens in a new tab) |
The older "ChatGPT agent" is no longer offered. Its help page (opens in a new tab) now says to use ChatGPT Work instead.
Cooling-off periods, exceptions and the chargeback route
General information, not legal advice. Checked: 2 November 2026.
- Ireland, online shopping. The Competition and Consumer Protection Commission's buying online page (opens in a new tab) says "The cooling off period for online purchases is 14 days", counted from when you receive the goods or agree a service. Exceptions include personalised or made-to-order items, items that can't be returned for health or hygiene reasons, passenger transport, and accommodation, car hire, catering or leisure booked for a set date. It also says "If you paid by card, you may have the option to request a chargeback from your bank or financial services provider." The page is undated.
- Ireland, energy. The Commission for Regulation of Utilities' switching page (opens in a new tab) says "When you switch supplier, there is a 14 day cooling off period."
- UK, shopping. The Consumer Contracts Regulations 2013, regulation 30 (opens in a new tab): for goods, the cancellation period ends 14 days after the goods come into your possession; for services, 14 days after the contract is made. Exceptions are in regulation 28. legislation.gov.uk shows the text up to date with changes in force on or before 1 November 2026, with no outstanding changes to regulation 30.
- Great Britain, energy. Ofgem's switching page (opens in a new tab) says "If you change your mind, you can cancel your switch within 14 days."
Consumer bodies to contact
The book says to contact the seller first. If that doesn't work, as of 2 November 2026:
- Ireland. The Competition and Consumer Protection Commission, www.ccpc.ie (opens in a new tab), for shopping, and the Commission for Regulation of Utilities, www.cru.ie (opens in a new tab), for energy.
- England and Wales. The Citizens Advice consumer helpline, 0808 223 1133 (Welsh-speaking advisers 0808 223 1144), Monday to Friday, 9am to 5pm, free from mobiles and landlines. Citizens Advice (opens in a new tab)
- Scotland. Consumer Advice Scotland, run by Advice Direct Scotland, 0808 164 6000, Monday to Friday, 9am to 5pm. consumeradvice.scot (opens in a new tab)
- Northern Ireland. Consumerline, 0300 123 6262, 9am to 5pm Monday to Thursday and 9am to 4pm on Friday, except bank and public holidays. nidirect (opens in a new tab)
- UK energy complaints. The Energy Ombudsman (opens in a new tab) is free and can step in once you have given your supplier 8 weeks to put things right, or sooner if the supplier sends a "deadlock" letter. The energy regulators are Ofgem for Great Britain (www.ofgem.gov.uk (opens in a new tab)) and the Utility Regulator for Northern Ireland (www.uregni.gov.uk (opens in a new tab)).
Incident records
From the AI Incident Database, titles and counts as shown on 2 November 2026. These are reported, not independently verified.
- "OpenAI's Operator Agent Reportedly Executed Unauthorized $31.43 Transaction Despite Safety Protocol", 7 February 2025; one report. Incident 1028 (opens in a new tab)
- "LLM-Driven Replit Agent Reportedly Executed Unauthorized Destructive Commands During Code Freeze, Leading to Loss of Production Data", 18 July 2025; five reports. Incident 1152 (opens in a new tab)
- "Google Gemini CLI Reportedly Deletes User Files After Misinterpreting Command Sequence", 21 July 2025; one report. Incident 1178 (opens in a new tab)
The book keeps the companies anonymous; this page names them because the database titles do.
Further reading on prompt injection and agent security
- NCSC (UK), "Prompt injection is not SQL injection (it may be worse) (opens in a new tab)", 8 December 2025.
- NCSC (UK), "Thinking about the security of AI systems (opens in a new tab)", 30 August 2023.
- NCSC (UK), "Thinking carefully before adopting agentic AI (opens in a new tab)", 15 May 2026.
- Joint guidance from six national agencies, "Careful adoption of agentic AI services (opens in a new tab)", 1 May 2026.
- OWASP, "LLM01:2025 Prompt Injection (opens in a new tab)".
- OWASP, "LLM06:2025 Excessive Agency (opens in a new tab)".
Checked: 2 November 2026 (every link opened). These are for readers who want more; no method is copied from them.
Check on Figure 17.3's supplier names
Figure 17.3, as written in the book on 2 November 2026, names no energy supplier at all: Joan's new supplier and the comparison page are unnamed, and no address, account or meter number is real. For the record, the Commission for Regulation of Utilities' list of energy suppliers in Ireland (opens in a new tab) showed these 15 on 2 November 2026: Arden Energy, Bord Gáis Energy, Cenergise, Community Power, Ecopower, Energia, Electric Ireland, Flogas, Go Power, Naturgy, Pinergy, PrePayPower and Yuno Energy (both Yuno Ltd), SSE Airtricity and Water Power. Any supplier name added to the figure later must not match one of these. Checked: 2 November 2026.
From the book
In the glossary wording, approve before it acts is:The series rule that a person reviews an agent's proposed action and says yes before the agent carries out anything important, such as sending, buying, deleting or publishing.Send, buy, delete and publish are the four that always wait for you.
Sources
- Incident 1178: Google Gemini CLI Reportedly Deletes User Files After Misinterpreting Command Sequence (opens in a new tab) · AI Incident Database, 2025
- Incident 1152: LLM-Driven Replit Agent Reportedly Executed Unauthorized Destructive Commands During Code Freeze (opens in a new tab) · AI Incident Database, 2025
- Incident 1028: OpenAI's Operator Agent Reportedly Executed Unauthorized $31.43 Transaction Despite Safety Protocol (opens in a new tab) · AI Incident Database, 2025
- Get started with Claude in Chrome (opens in a new tab) · Anthropic, 2026
- Careful adoption of agentic AI services (opens in a new tab) · Australian Signals Directorate's Australian Cyber Security Centre and Cybersecurity and Infrastructure Security Agency and National Security Agency and Canadian Centre for Cyber Security and National Cyber Security Centre (New Zealand) and National Cyber Security Centre (UK), 2026
- Buying online, over the phone, or on your doorstep (opens in a new tab) · Competition and Consumer Protection Commission, 2026
- Switch energy supplier (opens in a new tab) · Commission for Regulation of Utilities, 2026
- Ask Gemini to handle your multi-step tasks in select Android apps (opens in a new tab) · Google, 2026
- Gemini Apps Privacy Hub (opens in a new tab) · Google, 2026
- Manage your tasks & workflows in Gemini Apps (opens in a new tab) · Google, 2026
- Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (opens in a new tab) · Greshake, Kai and Abdelnabi, Sahar and Mishra, Shailesh and Endres, Christoph and Holz, Thorsten and Fritz, Mario, 2023
- OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents (opens in a new tab) · Kuntz, Thomas and Duzan, Agatha and Zhao, Hao and Croce, Francesco and Kolter, Zico and Flammarion, Nicolas and Andriushchenko, Maksym, 2025
- Thinking about the security of AI systems (opens in a new tab) · Martin R, 2023
- Prompt injection is not SQL injection (it may be worse) (opens in a new tab) · Chismon, Dave, 2025
- Thinking carefully before adopting agentic AI (opens in a new tab) · Martin R and Kate S, 2026
- ChatGPT agent (opens in a new tab) · OpenAI, 2026
- LLM06:2025 Excessive Agency (opens in a new tab) · OWASP Gen AI Security Project, 2025
- LLM01:2025 Prompt Injection (opens in a new tab) · OWASP Gen AI Security Project, 2025
- The Consumer Contracts (Information, Cancellation and Additional Charges) Regulations 2013 (SI 2013/3134), regulations 29 and 30 (opens in a new tab) · United Kingdom, 2013
- OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments (opens in a new tab) · Xie, Tianbao and Zhang, Danyang and Chen, Jixuan and Li, Xiaochuan and Zhao, Siheng and Cao, Ruisheng and Hua, Toh Jing and Cheng, Zhoujun and Shin, Dongchan and Lei, Fangyu and Liu, Yitao and Xu, Yiheng and Zhou, Shuyan and Savarese, Silvio and Xiong, Caiming and Zhong, Victor and Yu, Tao, 2024
- An Illusion of Progress? Assessing the Current State of Web Agents (opens in a new tab) · Xue, Tianci and Qi, Weijian and Shi, Tianneng and Song, Chan Hee and Gou, Boyu and Song, Dawn and Sun, Huan and Su, Yu, 2025
- WebArena: A Realistic Web Environment for Building Autonomous Agents (opens in a new tab) · Zhou, Shuyan and Xu, Frank F. and Zhu, Hao and Zhou, Xuhui and Lo, Robert and Sridhar, Abishek and Cheng, Xianyi and Ou, Tianyue and Bisk, Yonatan and Fried, Daniel and Alon, Uri and Neubig, Graham, 2023