01Set staff rules

Something Went Wrong With AI at Work: What to Do in the First Hour

First-hour steps when client data goes into the wrong AI account or an AI tool acts on its own: contain it, report it, write it down, then decide.

Written by
AI Ground Rules
Last reviewed

When something goes wrong with AI at work, the first hour has one job: stop it getting worse and tell the right person. Stop the task. Pause or disconnect the tool if you can. Do not forward or re-paste the sensitive content. Tell your firm’s named contact now. Write down the tool, the account, the time and what happened, without copying the sensitive data. Deciding whether it is a reportable breach comes later the same day, and it is the owner’s call, not yours.

This guide is for staff and owners of small professional-services firms. It supports check 06, “Incident reporting”, in the free AI Use Checklist.

What counts as an AI incident

Report any of these. You do not need to be sure it is serious.

  • Wrong account. Client or staff information went into a personal AI account, or a tool that is not approved for it.
  • AI acted on its own. An AI tool or agent sent an email, changed or deleted a record, booked something or published something it should not have.
  • A wrong answer got out. An AI answer with a wrong figure, date, name or reference reached a client, or informed a decision before anyone checked it.
  • A connected app can read too much. A note-taker, inbox assistant, browser extension or “sign in with your work account” app has access to email, files or meetings it should not see.

The first hour: six steps

Do these in order.

  1. Stop the task. Do not carry on, and do not send, publish or share the output. If an agent or automation is still running, press stop. Then check it really stopped: look for scheduled or queued actions, such as emails waiting to send.
  2. Pause or disconnect the tool, if you can do it straight away. Turn off the agent, end the meeting recording, or disconnect the app. If you are not sure how, leave it and go to step 4. Do not spend the hour searching settings.
  3. Do not spread the content. Do not forward the email, re-paste the text into another AI tool to “check” it, or screenshot it for a colleague. If a password or access code went in, change it now.
  4. Tell the named contact now. That is the data decision owner in your AI acceptable use policy, or their backup. Use the channel the policy names. Do not wait until you are certain.
  5. Write it down. Use the note below. Record the tool, the account, the time and what happened. Describe the data (“one client’s bank statement”); do not copy it.
  6. Keep the evidence. Do not delete the chat, the email or the log yet. Looking at it is not keeping it, and deleting in ChatGPT, for example, cannot be undone. The data decision owner records the facts first: what went in, when, which tool and account, and what kind of data. They describe the data rather than copy it, and keep that record where only they and the people handling the incident can see it. Then they decide, with the lawyer or insurer where relevant, whether to keep or delete the original. The FTC’s guide says plainly: do not destroy evidence.

The incident note

Copy this into a message to the data decision owner. Leave blanks rather than guess.

AI incident note

Reported by: [name]. Reported to: [name]. Date and time reported: [date, time].

When it happened: [date, approximate time].

Tool and account: [product and plan, for example “ChatGPT, personal Plus account” or “Microsoft Copilot Chat, work account”].

Type of incident: [wrong account / AI acted on its own / wrong answer got out / connected app with too much access / other].

What happened: [two or three plain sentences].

What kind of information: [for example “one client’s invoice list, about 40 rows, names and amounts”. Do not paste the data itself.]

Whose information: [which client, staff or other people, if known].

Did it leave the firm? [yes / no / not sure. Who received it?]

Was anything lost, changed or deleted? [yes / no / not sure. Which records?]

What I have done so far: [stopped the task, disconnected the app, changed a password…].

Still running or still connected? [yes / no / not sure].

How to pause, disconnect or delete in common tools

Do not delete anything until the data decision owner has recorded the facts and decided, with advice where needed, that it can go. When they do, these are the vendor’s own steps, checked on 14 November 2026. Menus change, so check the linked page if a label does not match your screen.

ChatGPT

  • Delete one chat. In the sidebar, open the menu next to the chat’s title and choose Delete. Archiving is not deleting. OpenAI’s help article says deletion cannot be undone, and that OpenAI can take up to 30 days to remove a deleted chat from its systems, longer where it must keep data for security or legal reasons. Files saved separately may need deleting on their own.
  • Disconnect a connected app. Open Settings, then Plugins, and select the app. Open the menu next to the connected account, or the app’s own menu, and choose Disconnect. This is per account: other accounts connected to the same app can stay active, so check each one. Disconnecting does not delete past chats or files. See OpenAI’s app accounts article.

Microsoft Copilot and Microsoft 365

  • Delete your Copilot history (work account). Sign in to My Account, go to Data and Privacy, then Data options, open Copilot activity history and select Delete. Microsoft’s support page says this clears history across all the Microsoft 365 apps at once, not one app, and that full deletion can take some time. Files you saved with Copilot’s help stay.
  • Block an agent (admin). In the Microsoft 365 admin center, go to Agents, then All agents, select the agent and choose Block. Microsoft’s agent page says this blocks agents built in Agent Builder or Copilot Studio across Copilot, Outlook, Teams and the other Microsoft 365 apps. For agents built in SharePoint or Microsoft Foundry, it only removes them from Copilot Chat. So also stop the flow or automation behind the agent, and check that scheduled or queued actions have really stopped. Deleting an agent keeps it recoverable for 30 days.
  • Remove an app’s access (admin). In the Microsoft Entra admin center, go to Entra ID, Enterprise apps, All applications, select the app, then Permissions. Admin-granted permissions can be revoked there. Microsoft’s permissions page notes that permissions a single user granted cannot be revoked in the portal, so this one is for your IT provider.

Gemini and Google Workspace

  • Delete a Gemini chat. In Gemini, open Settings and help, then Activity, and delete the item. Google’s help page says that for work accounts, these settings are controlled by your Workspace administrator, so you may not be able to delete or change retention yourself. Ask your admin.
  • Remove an app from a Google Account. Open your Google Account’s third-party connections, choose the app, select See details, then Remove access (Google help). Removing access stops future access. It does not delete what the app already collected; ask the app’s vendor about that.
  • Block an app for the firm (admin). In the Admin console, go to Security, Access and data control, API controls, Manage App Access. Find the app, choose Change access and set it to Blocked (Workspace Admin Help).

Within the day: is it a reportable breach?

The data decision owner decides this, with advice. This section is not legal advice. It points to the rules that most often apply, so you know who to ask.

A breach is not only data going somewhere it should not. The ICO’s breach guide also counts data that is lost, destroyed or changed. An agent can delete or corrupt records without sending anything anywhere. Start with four questions:

  1. Was personal information involved? Names, contact details, financial, health or ID details about real people.
  2. Did it go somewhere it should not? A personal account, an unapproved vendor, the wrong recipient. (Who could see it.)
  3. Was information lost, changed, deleted or made unavailable? A record overwritten, a file deleted, figures changed. (Whether it is still correct, and still there when you need it.)
  4. Could it cause harm? Fraud, embarrassment, discrimination, loss of trust, a breach of a client’s confidence, a decision made on wrong records.

Then check each of these.

  • EU and UK data protection law. Under GDPR Article 33, a controller must tell the regulator about a personal data breach without undue delay and, where feasible, within 72 hours of becoming aware of it, unless it is unlikely to result in a risk to people. The UK ICO’s breach guide says the same for UK GDPR. If the risk to people is high, they must be told too.
  • If you hold the data for a client. Under Article 33, a processor handling data on a client’s behalf must tell that client without undue delay.
  • Keep a record either way. Article 33 also requires controllers to document every personal data breach, including ones not reported to the regulator. Your incident note is the start of that record.
  • US state laws. The FTC’s breach response guide says every state, plus D.C., Puerto Rico and the Virgin Islands, has a breach notification law. What counts as a breach, and the deadline, varies by state and by the kind of data. Talk to a lawyer about which apply.
  • Client contracts and engagement letters. Many set their own notice terms, sometimes shorter than the law. Check the contract for the client whose data was involved.
  • Your cyber insurance policy. Insurers often require prompt notice and may want you to use their breach response line before you take other steps. Check the policy wording.
  • Professional rules. Regulated professions may have their own duties to report or tell clients. Check with your professional body.

If the answers are “no personal data, nothing left the firm, nothing lost or changed”, record that and move on to fixing the cause.

Who to call

Make this list before you need it. Put names and numbers in your policy.

Who When to call
Data decision owner, or their backup Always, first.
Firm owner or managing partner When client data left the firm, or a client might need to be told.
IT provider To disconnect apps, block agents, revoke access, check logs or change passwords across the firm.
Cyber insurer As soon as personal or client data may be exposed. Check whether they want to be called before anything else.
Lawyer To decide on regulator, client or individual notice, and deadlines.
The client Only when the owner decides, using agreed wording.

Fixing the cause afterwards

Once it is contained, deal with the work itself, then the cause.

  • Fix affected work. If a wrong AI answer or an agent’s action reached a client or a record, correct it, or withdraw it and send a corrected version. Put back any records that were changed or deleted. The owner decides what to tell the client.
  • Decide who allows the tool back. The data decision owner decides whether the tool, app or agent comes back on, and on what terms. Until they do, it stays off. Write the decision in the approved tools register.

Then find out why it happened and change one thing. Usually it is one of these:

  • The rule was unclear. Staff did not know the personal account was off limits. Make it plainer and add this case as an example.
  • The approved tool was hard to reach. Staff used a personal account because the work one was slow or missing. Fix access.
  • The tool had too much access. Cut its permissions to what the job needs. For agents, use the human approval checklist.
  • No one checked the output. Name a reviewer for that kind of work.

Then go back to the AI Use Checklist worksheet. Write the gap, the first action, an owner and a due date. Check that check 06 now reads “yes”: staff know where to report and what to write down.

A fictional worked example

This scenario uses an invented firm and people.

At 10:15 on a Tuesday, Priya, a junior accountant at Fernhill Accounts, wants help with a reconciliation. She pastes 30 rows of a client’s bank transactions into ChatGPT. At 10:20 she notices the account menu shows her personal email. It is her own Plus account, not the firm’s Business workspace.

  • 10:21. She stops. She does not paste anything else, and she does not delete the chat.
  • 10:23. She messages Sam, the data decision owner, on the channel the policy names: “AI incident. I pasted 30 rows of one client’s bank transactions into my personal ChatGPT at about 10:15. No passwords. Nothing sent outside the chat. Full note follows.”
  • 10:30. She sends the incident note. It names the client and describes the data as “30 rows: dates, payees, amounts, last four digits of the account”. It contains none of the rows.
  • 11:00. Sam calls the managing partner. They check the client’s engagement letter, which asks for notice of any data incident within 48 hours. Sam looks at the chat on Priya’s screen and records what went in, the time and the account in the incident note, without copying the rows. He asks her not to delete it yet.
  • Afternoon. The partner calls the firm’s lawyer and the insurer’s helpline. The lawyer advises on whether notice to the regulator or the client is needed, and confirms the record is enough. Only then does Sam ask Priya to delete the chat. The partner tells the client that day.
  • Next week. The cause was simple: Priya’s work ChatGPT login had expired, and the personal one was already open. The firm adds “check the account menu” to onboarding and asks staff to keep personal AI accounts in a separate browser profile. Sam writes it on the checklist worksheet with a due date.

What this guide does not cover

This guide gives practical first steps. It does not decide your legal obligations under privacy law, professional rules, client contracts or insurance. Breach rules depend on where you and the affected people are. Get advice from a qualified lawyer before deciding whether to notify a regulator, a client or individuals.

04Next step

The kit's 06 Incident Report form walks staff through the report, then helps the owner coordinate the response and the restart.