Responsible enterprise LLMs: Mitigating data privacy and LLM security risks (Part 2)

Following COVID-19, cybersecurity threats entered an era of massive growth and uncertainty. Organizations were transitioning en masse to digital solutions while netizens, who were trying to make sense of the global situation, were increasingly exposed to online misinformation. For adversaries, it created an ideal environment for exploitation—one we’re confronting again, in a new way, this time with artificial intelligence (AI) at the core.  

With large language model (LLM)-based tools like ChatGPT gaining traction, organizations are adopting more complex technology stacks and using software differently than they have before. LLMs, trained on vast datasets to provide human-like text responses to prompts, are revolutionizing business efficiency and strategy. However, they also expose organizations to additional security risks, many of which are still being understood by researchers. 

Organizations adopting responsible AI need to prioritize cybersecurity by staying up-to-date with evolving LLM security practices and technologies. 

Understanding LLM vulnerabilities and protecting your models 

As helpful as they are, the misuse of LLMs can also create an attack vector—a path that can be exploited to gain entry into a network or system. Adding any new tool to your tech stack can serve as an attack vector, but LLMs can create new security risks due to their complexity, necessitating updated security solutions. 

For instance, LLMs use vast amounts of data during training. While this enables models to provide helpful responses for a wide range of applications, it also makes them vulnerable to misuse. Adversaries can prompt LLMs to expose proprietary information or generate outputs that are influenced by sensitive data. 

Many organizations integrate third-party LLMs from providers like OpenAI and Google into their systems through an application programming interface (API), which can add entry points for adversaries and increase security risks. Protecting customizations can also be complex and resource-intensive if the provider lacks built-in security for these features. 

We’re still far from trusting LLMs to guide critical systems and infrastructure, but the potential is there. In these circumstances, model failure could have serious consequences. Even something as simple as an attacker manipulating outputs for a marketing email generator can significantly impact a company’s reputation and stakeholder trust. 

While enterprise LLM use cases are now common, it’s a relatively new technology. Developers and researchers are still navigating uncharted territory regarding LLM vulnerabilities and security requirements. Meanwhile, attackers are developing techniques to target LLMs, sometimes using LLMs as tools for attacks. 

Common LLM security vulnerabilities 

An authority on cybersecurity, the Open Worldwide Application Security Project (OWASP) publishes a list of the top 10 security challenges associated with LLMs. Some notable risks include: 

• Prompt injection is the process of manipulating an LLM into generating responses that are illegal, harmful, or that expose private information within the model’s training data. This is often accomplished using clever queries with reverse psychology or adversarial suffixes to get around the model’s safeguards. While prompting attacks raise concerns around LLMs and disinformation, researchers are uneasy about the technique’s potential for data theft and other high-risk attacks. 
• Insecure output handling means the model can’t sufficiently validate outputs for accuracy or compliance before passing them on to users or systems. 
• Training data poisoning may occur if an attacker gains access to a model’s training data before or during training. From there, the attacker can manipulate this data and compromise the model’s outputs. 
• Supply chain vulnerabilities include security risks at any point along an LLM’s development. For example, even a model that isn’t directly attacked could be compromised if it relied on training data from a third-party provider that was poisoned. 
• Over-reliance on LLMs—trusting their outputs without enough scrutiny—is dangerous. Lack of oversight on the quality of model outputs can lead to the spread of misinformation, biased content, security risks, and misinformed business decisions. 

Keep your LLM infrastructure secure: 4 essential LLM security practices 

Protecting your LLM is crucial for avoiding security breaches, which can cost organizations at least $300,000, according to Cisco’s 2024 Cybersecurity Readiness Index. Strategies like data anonymization, encryption, and comprehensive access control are essential security practices for any organization using LLMs. 

Data anonymization 

If your model is being trained with sensitive data, such as personally identifiable information (PII), anonymize this content before it is processed and used for training. Data anonymization uses techniques like data masking to obscure any confidential information. 

Encryption 

Encryption ensures that even if data is exposed, it will be unintelligible to the attacker. When using third-party vendors, such as a vector database provider for retrieval augmented generation (RAG), check that encryption is supported. Not all providers offer this feature. 

Validation 

Validation can be used to detect malicious or manipulative queries that could arise from techniques like jailbreaking. Before the LLM delivers an output, the system should validate the prompt and response for inaccurate, harmful, or sensitive content. Validation is still being explored in AI research with techniques like regular expression engines for language models (ReLM). 

Access control 

Apply comprehensive permission management settings that determine who can access different types of LLM outputs. For example, your finance team will likely require different access privileges than your development team, while users within a department may have different permissions depending on their role. 

Additional security protocols for LLM deployment 

Leading industry figures also advocate for an assortment of security best practices for LLMs, including: 

• Using standard network security strategies, such as firewalls, intrusion detection, red teaming, patches, data backups, and sandboxing. 
• Investing in continuous cybersecurity monitoring tools to detect threats. 
• Rate limiting database queries. This prevents attackers from attempting to overload the system in a denial of service (DoS) attack. 
• Regularly auditing your security infrastructure and performance. 
• Documenting your security practices and procedures in the event of an incident. 
• Educating employees and users on LLM security and fostering a security-first culture. 

Maintaining vigilance with evolving enterprise LLMs 

Adopting any emerging technology comes with security risks. While this remains true for LLMs, it doesn’t mean that you can’t—or shouldn’t—take advantage of the many benefits this AI innovation offers. The optimal strategy entails a keen awareness of the risks, seeking assurances from your technology partners that they are applying industry best practices for security, and investing in the right solutions to address gaps. LLM research and security techniques are constantly evolving, so it’s important to stay up-to-date with new findings and solutions.

Security practices keep your data safer and support more reliable LLM outputs, but what about the accuracy and integrity of AI outputs? If you haven't yet, read part 1 in our responsible enterprise LLM series, “Responsible enterprise LLMs: Addressing accuracy and LLM bias challenges.”