Incidents tagged Large Language Models

Architectural vulnerabilities within Large Language Model (LLM) environments integrated with the Model Context Protocol (MCP) enable attackers to embed malicious instructions within data content or tool metadata. This flaw allows for indirect prompt injection and tool poisoning, compelling LLMs to autonomously perform unauthorized actions such as data exfiltration or triggering enterprise workflows.

Prompt injection attacks, particularly indirect prompt injection, pose critical enterprise security vulnerabilities by allowing attackers to manipulate Large Language Models (LLMs) and AI agents. These attacks exploit the LLM's inability to distinguish between data and instructions, leading to impacts such as data exfiltration, unauthorized privilege escalation, and malicious command execution.

An attack chain exploited exposed AWS credentials in public S3 buckets, leveraging Large Language Models (LLMs) to rapidly escalate privileges through a misconfigured Lambda function. This allowed threat actors to achieve full AWS administrative control and abuse GPU resources within an Amazon Bedrock environment in under eight minutes.

NSFOCUS has identified emerging threats targeting AI Agents and Large Language Models (LLMs), specifically through sophisticated attacks leveraging Multi-Agent Communication Protocols (MCPs) to achieve unauthorized access, privilege escalation, and intent manipulation. These new vulnerabilities include "MCP Tool Poisoning Attacks" and "Intent Disruption & Goal Manipulation," which could lead to system intrusion, data tampering, and the spread of erroneous information across multi-agent systems.

Prompt injection is a critical vulnerability in Large Language Models (LLMs) that manipulates their natural language processing to override system instructions and safety filters using crafted malicious prompts. This exploit can lead to significant impacts such as data exfiltration, including the disclosure of sensitive files like `/etc/passwd`, and enables the AI system to perform unauthorized actions.

Prompt injection attacks manipulate AI systems, particularly Large Language Models (LLMs), by overriding their intended instructions through malicious input, leading to sensitive data leakage, unauthorized actions, or corrupted outputs. A real-world example involved a phishing campaign leveraging hidden text-based prompt injection to bypass AI defenses, coupled with the exploitation of CVE-2022-30190 (Follina) for remote code execution.

Prompt injection is a prevalent AI-specific vulnerability where Large Language Models (LLMs) misinterpret external data as executable instructions, bypassing intended safeguards. This can lead to sensitive data exfiltration, the execution of rogue tasks, or malicious code execution, impacting various AI agents, coding assistants, and chatbots.

The UK's NCSC warns that Large Language Models (LLMs) possess an inherent architectural flaw, known as prompt injection, where they fail to distinguish between instructions and data within a single prompt. This fundamental vulnerability allows malicious actors to bypass security guardrails, hijack models, and potentially achieve remote code execution by embedding hidden instructions in seemingly benign inputs.

The article details critical security challenges associated with cloud-hosted Large Language Models (LLMs), including prompt injection, adversarial exploits, model jailbreaks, sensitive data leakage, and misconfigurations. These vulnerabilities, stemming from lack of visibility and ungoverned AI behavior, necessitate proactive discovery and risk management to safeguard AI workloads.