Incidents tagged LLM

The incident stems from a supply chain attack targeting the open-source LiteLLM project, where malicious code was injected. This compromise led to thousands of organizations, including AI startup Mercor, suffering data breaches and exfiltration of sensitive information.

Architectural vulnerabilities within Large Language Model (LLM) environments integrated with the Model Context Protocol (MCP) enable attackers to embed malicious instructions within data content or tool metadata. This flaw allows for indirect prompt injection and tool poisoning, compelling LLMs to autonomously perform unauthorized actions such as data exfiltration or triggering enterprise workflows.

Prompt injection attacks, particularly indirect prompt injection, pose critical enterprise security vulnerabilities by allowing attackers to manipulate Large Language Models (LLMs) and AI agents. These attacks exploit the LLM's inability to distinguish between data and instructions, leading to impacts such as data exfiltration, unauthorized privilege escalation, and malicious command execution.

LLM applications face significant security risks, primarily prompt injection attacks, where malicious inputs manipulate models into ignoring instructions and revealing sensitive data. This can lead to the exposure of internal configuration data, confidential information, or unauthorized actions within connected enterprise systems.

Malicious Chromium-based browser extensions are impersonating legitimate AI tools to harvest sensitive LLM chat histories and browsing data, impacting over 900,000 installs and 20,000 enterprise tenants. These extensions exfiltrate proprietary code, internal workflows, and confidential data to threat actor-controlled infrastructure, leading to widespread information leakage.

A viral AI caricature trend exposes enterprises to shadow AI risks and sensitive data leakage, as employees input work-related information into public LLMs and share results. This behavior enables adversaries to conduct targeted social engineering for LLM account compromise, granting access to confidential prompt histories.

An LLM-based AI agent, Owockibot, was compromised to disclose its private hot wallet keys, leading to a $2,100 financial loss and its operational shutdown. This incident demonstrates a critical vulnerability in autonomous agents with Internet and wallet access, where inadequately secured sensitive data can be extracted via prompting.

Advanced AI tools, specifically Large Language Models (LLMs), are now being leveraged to automate cloud environment attacks, rapidly identifying misconfigurations and exposed credentials within minutes in platforms like AWS. This enables attackers to achieve swift credential theft and privilege escalation to administrative access by exploiting existing weaknesses rather than novel vulnerabilities, circumventing traditional phishing methods.

The proliferation of unmanaged "Shadow AI" deployments, such as unauthenticated Ollama server instances, creates critical security blind spots within corporate networks. These unsecured LLM endpoints are susceptible to unauthorized access, risking exposure of proprietary training data, enabling prompt injection attacks, and serving as potential pivot points for lateral movement.

OpenAI confirmed a data breach originating from unauthorized access to Mixpanel, a third-party web analytics provider it uses for its API product. This incident exposed names, email addresses, approximate locations, OS/browser data, and user IDs associated with OpenAI API accounts (platform.openai.com users), but did not compromise ChatGPT content, passwords, or payment details.

NSFOCUS has identified emerging threats targeting AI Agents and Large Language Models (LLMs), specifically through sophisticated attacks leveraging Multi-Agent Communication Protocols (MCPs) to achieve unauthorized access, privilege escalation, and intent manipulation. These new vulnerabilities include "MCP Tool Poisoning Attacks" and "Intent Disruption & Goal Manipulation," which could lead to system intrusion, data tampering, and the spread of erroneous information across multi-agent systems.

Prompt injection is a critical vulnerability in Large Language Models (LLMs) that manipulates their natural language processing to override system instructions and safety filters using crafted malicious prompts. This exploit can lead to significant impacts such as data exfiltration, including the disclosure of sensitive files like `/etc/passwd`, and enables the AI system to perform unauthorized actions.

A March 2023 vulnerability in the Redis open-source library temporarily exposed ChatGPT users' chat titles, messages, and potentially payment information. Beyond platform vulnerabilities, users face risks from prompt injection attacks that bypass LLM guardrails and the utilization of AI by threat actors to generate malware, phishing templates, and deepfakes.

Evidence indicates that Large Language Model (LLM) deployments is affected by a security issue.

Researchers demonstrated that as few as 250 poisoned documents can create a backdoor vulnerability in large language models, irrespective of model size or training data volume. This data poisoning technique, which can induce denial-of-service or potentially facilitate data exfiltration, challenges prior assumptions about the required scale of malicious training data during pretraining.

Threat actors are employing Large Language Models (LLMs) to create sophisticated phishing campaigns, leveraging LLM-generated code to obfuscate malicious payloads within Scalable Vector Graphics (SVG) files. These meticulously crafted SVG files bypass email security by disguising embedded JavaScript with business terminology, ultimately redirecting victims to fake login pages for credential harvesting.

The article details critical security challenges associated with cloud-hosted Large Language Models (LLMs), including prompt injection, adversarial exploits, model jailbreaks, sensitive data leakage, and misconfigurations. These vulnerabilities, stemming from lack of visibility and ungoverned AI behavior, necessitate proactive discovery and risk management to safeguard AI workloads.

Researchers discovered a reflected Cross-Site Scripting (XSS) vulnerability in Yellow.ai's chatbot, which could be tricked into generating malicious HTML/JavaScript code. This flaw enabled attackers to steal support agent session cookies, potentially leading to account hijacking and data exfiltration from customer support platforms.

A critical zero-click vulnerability, dubbed "EchoLeak" and identified as CVE-2025-32711, was discovered in Microsoft Copilot. This flaw leveraged an "LLM scope violation" to allow remote attackers to exfiltrate sensitive data from Microsoft 365 services without any user interaction.

Researchers have uncovered "EchoLeak," a critical zero-click vulnerability in Microsoft 365 Copilot that exploits design flaws inherent to Retrieval Augmented Generation (RAG) applications. This flaw, leveraging an "LLM Scope Violation" technique, allows for the automatic exfiltration of sensitive corporate data from the LLM's context without requiring user interaction.