The biggest CyberSecurity issue with AI isn't the technology itself—it's how people use it without understanding the risks. This lesson will explain how AI can cause CyberSecurity issues - all while being controlled by humans.

Inadvertent data exposure

People routinely paste sensitive information into AI tools without thinking about where that data goes. Employees might input proprietary code or algorithms, customer data or personal information, confidential business strategies or financial data, passwords, data keys, system configurations, or internal documents and trade secrets. If they're using a public AI service, this information may be stored, used for training the LLM, or potentially exposed. Once data, text, documents or images are shared with an AI service, the person who shared the data is no long in control of how that data is used. The data is now considered lost - this term has special meanings when used with data especially in the context of PII - Personally Identifiable Information. The Course called Protecting your Data explains how the exposure or loss of control of PII is the cause of a Data Breach.

Lack of awareness about privacy settings

Most people don't realize that there is a difference between public and private AI instances. This has consequences around whether the conversations are being logged (for example in a Zoom or Teams meeting) or used for training the LLM. Knowing what the AI provider's data retention policies actually are, or how to enable enterprise or privacy-protected modes is a great starting point to help decide what is safe to include in a AI chat. When we assume AI conversations are private when they often aren't, we can fall into the trap of accidentally using a Public AI instance - a public AI chat means that the chat data is likely being used to enhance the LLM and therefore is possibly going to become part of someone else's answer to their chat question.

Using a Private AI instance (usually via a paid subscription) reduces the likelihood of data being used for training a public LLM and has more privacy controls available in the settings area. For example, to enable private and non-permanent chats in ChatGPT, under the settings menu, you can select the option Data Controls and disable Improve the model for everyone. This may prevent your chat from becoming part of the LLM.

Misplaced trust

People tend to become quite comfortable chatting with Generative AI - after all, it is built to have positive conversations with the user. It is common to treat AI like a trusted colleague rather than a third-party service. Once a person becomes comfortable with the chats and stops questioning whether they should be sharing certain information, it becomes more likely that they'll share private information. The conversational interface makes it feel safe and personal, which lowers natural caution.

The training gap

We know that people are using Generative AI tools without considering the implications of the use of those tools. LayerX Security recently published a report that explains how common it is for people to report that they're introducing AI tools into their business without any consideration of what data is being copied into the tools. We have added a link to this report in the Resources section for your reference. Some organisations are deploying AI tools faster than they're training employees on secure usage. When people don't understand what constitutes sensitive data in their business, or how to anonymise information before using AI to process that information, or when AI use violates industry compliance requirements, data breaches, CyberSecurity incidents or worse tend to follow.

The bottom line

AI is changing the way we work, often for the better. However, AI is currently skipping ahead of businesses. Technology like AI can be secured with proper privacy controls, business policies and user training. But human behavior—sharing sensitive data without thinking, trusting too readily, or lacking awareness—creates vulnerabilities that no technical solution can fully prevent. The solution isn't avoiding AI, but educating ourselves about responsible use, implementing clear policies, and using enterprise AI solutions with appropriate data protections for sensitive work.

Lesson 4 – AI and CyberSecurity