Executive Summary:

The Bottom Line

Complete privacy of workplace communication is in jeopardy. Your safest bet is to behave as if everything you say, type, or have on your screen could be easily retrieved by the wrong person. Use words and sentences that will reflect well on you and others in case someone with enough permissions asks AI what you said. Protect your privacy before it is too late.

Key Risks

• AI Assistants That Watch Your Screen: To do tasks for you, AI must see everything on your screen, and the wrong person could find out what it saw.
• Meeting Content: There is no such thing as ‘off-the-record’ in meetings where AI transcription tools are active
• Draft Documents: AI tools may index and analyze everything you type, including emotion-filled documents while “venting,” even if you change the text before you save
• Email Messages: AI may respond with information stored in your deleted items folder unless permanently removed
• Third-Party Tools: Meeting assistant tools may access your file systems and mailboxes and ignore your security controls, creating privacy nightmares
• Unauthorized AI Connections: Employees may link outside AI tools to your data via API keys or MCP servers, often without IT’s knowledge or approval

 

Essential Protections

Your IT team should implement:

• Data Loss Prevention (DLP) tools with proper configuration and upkeep
• Data Sensitivity Labeling to restrict who can see what data
• Retention Limits on meeting notes, files, and email after mandatory holding periods
• Administrator approval requirements before third-party apps can access company data
• DNS Filtering to help restrict users from giving sensitive information to unapproved chatbots

 

Full Article follows:

From now on, if you want to write something you expect to stay private, it’s a good idea to use a pen and paper or something other than your computer. What you say in online meetings can now be transcribed, stored, and retrieved. Even more concerning, anything you type into a document draft you save, including angry drafts, can be accessed by AI systems and potentially disclose what you believed to be private information. The same goes for email messages, sent and received. Deleting files, messages, and meeting information and preventing unauthorized copies are more important than ever.

Some executives at my keynote presentations say, “I wish AI would give me answers based on what is happening in our company. I would get so much better results than my generic answers now!”

Their wish is granted. Retrieval-augmented generation (RAG) means AI can retrieve your organization’s information to provide relevant responses, including updates on what’s happening within it. The process is designed to keep information within your company and prevent it from leaking to other companies or third parties.

Some newer workplace AI assistants, like the one you may use today, review a user’s permissions and then access documents, meeting transcripts, and email messages the user can access, all in real time. If you remove a file, the data is usually no longer available for AI retrieval within a day, though it can take longer. The rest of this article will refer to this newer type of retrieval.

But the dark side of this fantastic feature is reduced privacy. AI tools with document or email access permissions are designed to enhance AI responses by gathering information from meetings, emails you send and receive, and files you’ve saved. The AI tools examine all information, including files saved in your online storage that have accumulated over many years. If someone with the right privileges asks AI a question about a topic or person, unless you deleted all instances of the old meeting notes, email messages, files, and other sources of information, what you said in a meeting or typed into an email or a saved document might appear in the results. Angry messages, failed plans, and long-forgotten mistakes can be resurrected even though you’ve put them behind you. Undeleted inappropriate jokes a friend emailed you, or private conversations with your loved ones through company email, could be exposed, too.

Before going any further, let’s explain what this article covers. When people talk about AI privacy, they are often concerned that what they type into an AI chat tool will leave their organization and show up somewhere else in the world. That’s not what we’re covering here. We’re covering the situation where, although the data stays within your organization, other people in your organization might find out more than they need to know, even without trying. Given a request, AI can quickly return data based on the user’s privileges without the user needing to find a specific file, message, or meeting. Unfortunately, they might see content they never expected or intended to see, perhaps private or sensitive information they shouldn’t have access to, a phenomenon dubbed AI “oversharing.”

This article focuses on companies with multiple users sharing data, rather than on a single user or a tiny office with users who do not use shared storage. However, everyone, including single-computer organizations, should read the section below entitled “Potentially Dangerous Third-Party AI Assistants.”

Using AI assistants, information stored within your organization may be available to anyone else in your organization with the appropriate access privileges. People no longer need to invest energy in searching; as long as they have access rights, they can ask a simple natural-language question using AI and find the data in the blink of an eye.

It’s becoming apparent that humans will be forced to accept this reality. Humans must be very cautious about what they say in a meeting, type into a file they save, or in an email. Of course, you have no control over what information someone could send you in an email, making the situation worse.

The scenario is now real that you can ask AI to submit an expense report, register you for a course, and do anything else you do now using your mouse and keyboard as it watches your screen. AI assistants like this have been a dream of ours for many years. Your computer can just imitate you because whatever program you’re using or website you’re visiting thinks it’s you moving the mouse and pressing the keys on the keyboard. Once your AI assistant learns what it needs to do and what you want it to do, life is going to be so much easier and so much better.

The big privacy risk people are missing in this wonderful scenario is that AI needs to read your screen to accomplish all of this. It needs to know where the buttons are, where the checkboxes are, where the places are to enter data. And it can only get that information by literally watching your screen all the time and paying attention to everything you do.

Imagine the privacy risk if you start typing a sentence on your screen and type something you think is funny, but then you realize it’s not funny at all, so you backspace over it. In the past, that was no big deal, because no one was seeing what you type on a screen. They only see something when you email it, print it, or save it. You could pretty much do whatever you wanted to while you were typing. It makes it easy to compose. Ever since word processors replaced typewriters, you’re able to edit, copy, paste and delete until you get your document just the way you want it to look. However, as more people adopt AI assistants that can read their screen, the privacy that used to come with using your backspace key is gone. Whatever you typed before you thought better of it, wherever you had that paragraph before you moved it, all that information may be being gathered.

The good part is that your AI assistant will learn your processes and how you work very clearly, and it can emulate you to be the best assistant you could possibly have. The downside is that the wrong person who has enough permission could ask questions of your AI assistant and quickly find out things you never expected anyone to know.

Software and operating systems that support gathering your and your organization’s data to provide more relevant answers (RAG) usually include multiple privacy safeguards. However, protections can be bypassed in certain circumstances, such as an official e-discovery.

The way it typically works is for AI tools to verify the user’s permissions for the data before considering augmenting the response with additional information. When a user asks for information, the system is designed to provide information that the user has permission to see, a process called trimming.

For example, workplace AI assistants integrated with your organization’s email applications have access to your messages. When you ask for information, the AI tools are designed only to give you information based on the contents of your email. Unless you’ve delegated email access to someone else, random people in your organization should be unable to receive answers augmented with information from your sent and received email messages.

However, a technology leader at a leading provider told me that their AI tool does not respect a user’s email privacy when someone with elevated rights wants to harm the user and asks the AI for sensitive private messages. He explained that all user email content is available to other users with enough privileges. He explained the trade-off between data access and privacy with this metaphor: Before AI augmentation, he said, finding sensitive data in a company was “like looking for a needle in a haystack” – scattered across random files and email messages. Now, he explained, with AI-powered tools, “you find the needle immediately just by asking a question.” He reminisced about asking one of his technical pros, “Show me email messages where anyone praised our competitors.” He said the results appeared instantly, with sender information fully visible. “The AI tool doesn’t give you a haystack,” he concluded. “It gives you a stack of needles.”

A member of my team and I eagerly visited with AI technology leaders, hoping to persuade them to make conversations completely private for sensitive meetings such as conversations related to an M&A, personnel matters that require confidentiality, trade secrets, and new competitive products or services that would harm a company if the details were discovered prematurely.  The most senior person we visited, who influences AI privacy at a large, well-known software company, was surprised to hear me suggest that executives sometimes want online meeting discussions to remain private forever.

He is not alone in believing that all executive communications should be discoverable. Executives’ knowing that their conversations could be disclosed helps ensure corporate accountability and serves as a strong deterrent to executive misconduct. Transparency is required by some regulations and, in certain circumstances, by law. Some people feel it is unfair for executives to enjoy privileged communications with immunity from e-discovery.

The senior executive with the power to determine AI privacy behaviors emphasized that the whole point of AI ingesting meeting conversations and other data is to make information available for AI processing; any restrictions reduce the tool’s functionality. He explained that this reaffirms the position that productivity outweighs privacy. He acknowledged concerning incidents of oversharing sensitive data with users, and he accurately pointed out that these are often due to their customers not properly preparing, deploying, or maintaining AI tools and data governance and privacy controls.

He retorted that executives who want to hold private meetings with undiscoverable content should use encrypted messaging apps like Signal rather than his company’s online meeting platform. He also told me he appreciated my feedback about leadership sometimes needing absolute privacy, and that they’ll consider it.

Yet their position is firm, and companies that use workplace AI assistant tools that access company information must now accept the sometimes limited privacy controls of those tools, which may include a significant risk of compromising the privacy of sensitive company information within their organizations. While I acknowledge that many application providers build in protective controls, complete privacy of workplace communication is in jeopardy.

There are many examples of data augmentation across the industry. One is Microsoft 365 Copilot, which can use RAG to augment responses with information from email, meetings, and files. Microsoft offers tools that let administrators restrict what data AI can search across your organization, such as specific files, emails, calendar events, and meetings. However, restricting this can have trade-offs, such as breaking traditional searches for content in email, documents, and Teams. Ask your IT team which restrictions are currently configured for your organization. Some more advanced protections, such as automatic data sensitivity labeling, are only available in more expensive Microsoft 365 licensing tiers. The good news is that Microsoft sometimes makes advanced features available in less expensive tiers over time. Separately, ask your IT team what’s currently available at your licensing level and used in your organization. If your license allows data sensitivity labeling but not automatic labeling, you will need to assign labels manually. Or your license might not support data sensitivity labeling at all.

To clear up any Copilot confusion: Microsoft also offers the free “Microsoft 365 Copilot Chat,” and it does not offer RAG features to automatically access your organization’s data. Users can only upload files manually for tasks like summarization. It also provides web searches and typical chat interactions. Users who are not subscribed to the paid version will see the free Copilot in a chat pane in Word, Excel, PowerPoint and Outlook, in the Microsoft Copilot app and on the web. Important: When logging in to the free Copilot with a personal email address, there are fewer protections and potential privacy issues by default. Ask your IT team to confirm your team is logging in with their business accounts and whether the free Copilot is configured to protect your data.

Back to AI with RAG features: Google Gemini is now integrated with Google Workspace and can review and consider information within it as it responds to user prompts. Google does not release information to the world by training Gemini on your data, and they provide strong security measures to help keep private data private. But even with the provided settings, a qualified person in your organization must configure and keep those measures current. Sometimes the default settings favor functionality over privacy, so your team must be familiar with them and keep up with changes. At the time of this writing, Google provides a feature called Workspace Intelligence that continuously gathers information from Gmail, Calendar, and other parts of your workspace so that Gemini already knows details about your current data. Although this is enabled by default, your IT Administrators can disable it at the domain, OU, or group level. Even with Workspace Intelligence disabled, if a user references a specific email or file, Gemini can still retrieve information from it. Google launched a centralized AI control center that allows your team to configure governance and auditing for Gemini and other AI agents across Google Workspace.

From now on, you must carefully choose your words in online meetings and never say anything you don’t want discovered. Content discussed in meetings may be captured in AI-generated transcripts, summaries, or recordings, making even previously casual conversations potentially discoverable in legal proceedings. By default, permissions for AI to return results from the transcript are typically given to all meeting attendees. If someone is invited but is late or a no-show at the meeting, avoid the temptation to make a joking remark or an offhand comment about them. That person could later want to know if they’d missed anything important and ask AI, “Did anyone say anything about me?” Your comment will be disclosed. Depending on what you said and their level of sensitivity, you might find yourself in an HR nightmare. There is no such thing as ‘off-the-record’ in meetings where AI transcription or summarization tools are active. With some commonly used operating systems and tools, this recording is always enabled and difficult to block.

Distributing AI-generated meeting summaries to participants without first having a human review them for accuracy is dangerous. AI is prone to hallucinations and transcription errors, especially when audio quality is poor. AI also makes errors when people use ambiguous language, such as “They said it was approved.” Who is “they,” and what did they approve? AI will try to decide but could get it wrong. Other examples are “We need to address the issue” or “Send it to them.” AI must make a guess, based on the context of the conversation, what “we,” “they,” “issue,” and “it” refer to. Sometimes AI, understandably, guesses wrong, and meeting summaries can include inaccurate information and topics never discussed.

After Abraham Lincoln died, historians discovered in archives that he had written scathing letters to his generals but never sent them. If you sometimes type emotion-filled documents while “venting,” even if you never intend to share the information, the AI tools may index and analyze everything you type in the draft by reading the file you saved or reading directly off your screen. In an e-discovery situation, or if someone with elevated privileges asks a question, the AI tool could reveal what you never intended to share.

One major application provider automatically saves a version history of previous content. If you don’t want AI to pull data from previous versions, don’t keep old versions. If you update a file for tone or accuracy, do so in the current file or delete old versions to keep previous content from showing up in AI answers.

Preventing content from old, deleted email messages from appearing in responses can be tricky, since AI may respond with information stored in your deleted items folder. You must remember to empty your deleted items folder, or your IT team can set up specific retention policies that permanently delete email messages after a set date or message age. Of course, as with files, if email messages are backed up and later restored, the restored versions may appear in responses to AI prompts. And this also assumes that your workplace AI assistant tool does not save old messages elsewhere for retrieval.

And, if your AI assistant is reading your screen or recording what you type, none of these strategies will help you unless you also have a way to tell the assistant to erase what it learned and be positive it didn’t already share that information.

This article’s goal is to make you aware and encourage you to accept the new reality that workplace privacy is evaporating quickly in the age of AI. Your safest bet is to behave as if everything you say, type, or have on your screen could be easily retrieved by the wrong person.

Let’s cover some things you can do to help protect yourself.

Be sure your IT team uses governance and privacy protections such as:

DLP: Major enterprise software providers offer highly effective data loss prevention (DLP) tools that help keep sensitive information private by detecting and blocking attempts to share, copy, or send it outside approved channels. DLP is a policy-enforcement layer that inspects and blocks data movement based on rules your organization defines. It doesn’t grant or restrict who can view data. IT professionals, compliance officers, and other administrators with elevated access can still often see data that DLP was meant to protect.

Data Sensitivity Labeling: Most enterprise AI assistant providers explain that their tools respect file permissions and features like Data Sensitivity Labeling. You and your users can specify data labels for your content, such as “private” or “confidential,” to further restrict who can see what data. However, if someone opens an e-discovery, all undeleted data is potentially available. Thus, nothing you say or type is wholly protected if the data still exists.

Retention Limits: A representative from a major tech company suggested that executives can avoid e-discovery exposure of what they say in sensitive topic meetings by setting retention limits on meeting notes, files, and email. The system will erase the data after a mandatory holding period.  He pointed out the risk that if a meeting attendee puts notes or a summary in the meeting chat, that chat information will not be purged.

Why Deletion May Not Be Enough: As mentioned throughout this article, remember that one of your best protections is deleting files, chats, messages, meetings and backups you don’t want AI to use in responses. However, the effectiveness of this strategy depends on whether the tool’s RAG features save information elsewhere even after you’ve deleted it.

DNS Filtering: If users upload sensitive information to an unapproved chatbot, that information could be exposed or otherwise compromised. DNS filtering is a technology that helps your IT Team limit users’ access to specific websites, including unapproved AI chatbots. Often it is most effective to block the entire AI category and allow-list approved chatbots and AI websites. Of course, DNS filtering doesn’t always work. If you only perform DNS filtering at your gateway firewall, then if users take laptops home, to a coffee shop, or anywhere not connecting through your firewall, there is no restriction. That’s why your team might ask to use DNS filtering that runs on the computer itself, so wherever the user connects, the filtering remains in effect. DNS filtering isn’t complete protection because savvy users can bypass it by using a VPN, proxy, or other methods to connect, but it is still an essential tool to help protect your organization’s data.

Potentially Dangerous Third-Party AI Assistants: An IT Professional at one of our best customers called me last week in alarm because he noticed a new app on their system had rights to scour their email messages and file storage. What used to be a third-party meeting assistant tool has “upgraded” its feature set to include an AI-driven search system across documents, notes, and email messages. Questions to ask:

• When a third-party meeting tool accesses your file systems and mailboxes, does it save any snippets of your information on its company’s servers?
• If so, do they encrypt the data and automatically erase the data from their systems when you delete a sensitive file or remove an email from your account?
• Can they provide a log or audit trail of who accessed your data?
• Do they train their tool on your data, potentially exposing it to their other customers?
• What happens to your data if you stop using their product?
• How do they define what data is yours vs. their data?

The tools may also offer to gather information from other third-party note-taking tools, CRMs, and users using other operating systems. From a functionality perspective, there is great allure to having an AI assistant so familiar with everything in your work life. However, it is also a privacy nightmare if the system ever over-shares sensitive information, if the third party gets compromised by threat actors, or if your organization loses visibility into where your sensitive data is stored and who can access it. Before enabling tools like this, you must thoroughly vet the third party to determine if they have the necessary security controls in place and will maintain the security of your data. Remember the saying, “your organization’s security is only as good as your third party’s security.” To help prevent employees from unknowingly granting outside apps access to your company’s emails, files, and other sensitive data, ask your IT team to change the “Allow User Consent” Setting from the default to require administrator approval before any third-party app can access company data.

Employees Linking AI Tools Without Authorization: Another risk is that your workers connect to unapproved AI tools without anyone in IT knowing. As they work, some of your users may be bombarded with tempting requests to add an AI feature or skill that supposedly will make them more productive or provide other attractive features. There’s a good chance that at least some of your users will be tempted to accept that offer. When they do, they’re allowing some third-party tool whose servers are who knows where and whose security is who knows how good. And now those strangers have some level of access to your organization’s data. You and your IT team don’t even know about it. These connections are often made via API keys or MCP servers, but those terms may never appear when the AI tool is connected. These unapproved links are one of the major security risks organizations face. Your IT Team can often monitor and even block some connections, but some programs do not offer security controls or visibility for your IT team when your users make connections. A determined user could run an MCP server on their laptop that reads files directly from their disk, making it invisible to your IT team and requiring no permissions.

Outside Parties: Another risk is that if any of your workers sent the data or made it available to an external party, it might end up in their system and be exposed by their AI someday.

AI Incident Response Plan: Develop a thorough plan for AI incidents. Plan now how you will manage AI-related crises, such as unauthorized data leakage, undetected hallucinations, discrimination (bias), and security issues like prompt injection and insider misuse. Include your legal and regulatory advisors during planning, as they can address their appropriate obligations.

Security Considerations for Incident Response, HR Investigations and more: Many organizations use ticketing or helpdesk systems that weren’t originally designed to handle sensitive issues, including cybersecurity incidents, HR complaints, and insider threats. Examples include Jira, ServiceNow, or Teams/Outlook. Those systems are integrating AI features. If you allow AI tools to automatically index your primary helpdesk system, they may unexpectedly augment responses and disclose sensitive investigation content to unauthorized users. This creates risks, including exposing privileged communications with legal counsel, compromising the integrity of confidential evidence, and disclosing sensitive employee information. Instead, use a completely separate access-controlled case management system for incident response, HR investigations, and other sensitive matters. Ensure this system is excluded from AI indexing and augmentation. Work with your legal and compliance teams to isolate systems, enforce strict access policies, and implement appropriate retention and audit-log controls.

Know that your IT team is already very busy, and adding AI governance to their responsibilities may require removing something else or outsourcing.

As time passes, AI will gather more information from your existing documents and data (this gathering is called RAG), what it thinks was said in all meetings, and what it sees on your screen. People will become more aware of the new normal in privacy. Use words and sentences that will reflect well on you and others, and don’t even start typing a sentence you don’t want someone with enough permissions to find out from your AI.

For better, worse, or both: AI hears you and watches your screen. Protect your privacy before it is too late.