Incidents tagged LLM

OpenAI confirmed a data breach originating from unauthorized access to Mixpanel, a third-party web analytics provider it uses for its API product. This incident exposed names, email addresses, approximate locations, OS/browser data, and user IDs associated with OpenAI API accounts (platform.openai.com users), but did not compromise ChatGPT content, passwords, or payment details.

NSFOCUS has identified emerging threats targeting AI Agents and Large Language Models (LLMs), specifically through sophisticated attacks leveraging Multi-Agent Communication Protocols (MCPs) to achieve unauthorized access, privilege escalation, and intent manipulation. These new vulnerabilities include "MCP Tool Poisoning Attacks" and "Intent Disruption & Goal Manipulation," which could lead to system intrusion, data tampering, and the spread of erroneous information across multi-agent systems.

Prompt injection is a critical vulnerability in Large Language Models (LLMs) that manipulates their natural language processing to override system instructions and safety filters using crafted malicious prompts. This exploit can lead to significant impacts such as data exfiltration, including the disclosure of sensitive files like `/etc/passwd`, and enables the AI system to perform unauthorized actions.

A March 2023 vulnerability in the Redis open-source library temporarily exposed ChatGPT users' chat titles, messages, and potentially payment information. Beyond platform vulnerabilities, users face risks from prompt injection attacks that bypass LLM guardrails and the utilization of AI by threat actors to generate malware, phishing templates, and deepfakes.

Evidence indicates that Large Language Model (LLM) deployments is affected by a security issue.

Researchers demonstrated that as few as 250 poisoned documents can create a backdoor vulnerability in large language models, irrespective of model size or training data volume. This data poisoning technique, which can induce denial-of-service or potentially facilitate data exfiltration, challenges prior assumptions about the required scale of malicious training data during pretraining.

Threat actors are employing Large Language Models (LLMs) to create sophisticated phishing campaigns, leveraging LLM-generated code to obfuscate malicious payloads within Scalable Vector Graphics (SVG) files. These meticulously crafted SVG files bypass email security by disguising embedded JavaScript with business terminology, ultimately redirecting victims to fake login pages for credential harvesting.

The article details critical security challenges associated with cloud-hosted Large Language Models (LLMs), including prompt injection, adversarial exploits, model jailbreaks, sensitive data leakage, and misconfigurations. These vulnerabilities, stemming from lack of visibility and ungoverned AI behavior, necessitate proactive discovery and risk management to safeguard AI workloads.

Researchers discovered a reflected Cross-Site Scripting (XSS) vulnerability in Yellow.ai's chatbot, which could be tricked into generating malicious HTML/JavaScript code. This flaw enabled attackers to steal support agent session cookies, potentially leading to account hijacking and data exfiltration from customer support platforms.

A critical zero-click vulnerability, dubbed "EchoLeak" and identified as CVE-2025-32711, was discovered in Microsoft Copilot. This flaw leveraged an "LLM scope violation" to allow remote attackers to exfiltrate sensitive data from Microsoft 365 services without any user interaction.

Researchers have uncovered "EchoLeak," a critical zero-click vulnerability in Microsoft 365 Copilot that exploits design flaws inherent to Retrieval Augmented Generation (RAG) applications. This flaw, leveraging an "LLM Scope Violation" technique, allows for the automatic exfiltration of sensitive corporate data from the LLM's context without requiring user interaction.

The article highlights critical security gaps in Large Language Model (LLM) applications, detailing common vulnerabilities such as prompt injection, sensitive information disclosure, and supply chain compromises. These flaws, categorized by the OWASP Top 10 for LLM Applications, can lead to unintended LLM behavior, data exposure, and other serious consequences.

A critical "EchoLeak" zero-click vulnerability in Microsoft 365 Copilot allowed attackers to remotely exfiltrate sensitive internal data by sending emails containing hidden instructions. This flaw represents an LLM scope violation where the AI agent was tricked into accessing information beyond its intended permissions without user interaction.