SquareX released critical research exposing a new class of attack targeting AI browsers. The AI Sidebar Spoofing attack leverages malicious browser extensions to impersonate trusted AI sidebar interfaces, which is used to trick users into executing dangerous commands that can lead to credential theft, device hijacking, and password exfiltration.
The research demonstrates how attackers can exploit users’ trust in AI browser sidebars – the primary interface through which users interact with AI browsers like Comet, as well as consumer browsers with AI features like Brave and Edge. By creating pixel-perfect replicas of legitimate AI sidebars, malicious extensions return AI-generated responses that include harmful instructions that unsuspecting users follow.
“AI has become an essential tool for millions of users to learn new skills and complete tasks. Unfortunately, this has created a dangerous dynamic where people blindly follow AI-generated instructions without the expertise to identify security risks,” explains Vivek Ramachandran, Founder and CEO of SquareX. “With no visual or workflow difference, the AI Sidebar Spoofing attack exploits the trust users place on these AI interfaces, tricking them into performing malicious tasks that they may not fully understand or are aware of.”
SquareX illustrates the AI Sidebar Spoofing attacks with three main case studies, but warns that we will likely see many variants of the attack develop. In one example, the user asks the AI sidebar how to withdraw cryptocurrency from their account. The fake AI Sidebar returns what looks like legitimate instructions but replaces the Binance login page URL with a phishing link. Thinking it was instructions generated by Comet, the user enters their credentials in the phishing site, which the attacker then uses to login to the victim’s account to access their cryptocurrency. In other examples, users were given false instructions to execute malicious commands that allowed attackers to exfiltrate passwords and hijack their device and execute ransomware attacks remotely.
The researchers also showed that other AI browsers and consumer browsers implementing AI sidebars like Edge, Firefox and Safari are equally vulnerable to the AI Sidebar Spoofing Attack. This means that even if organizations restrict the use of AI browsers, users are still subject to these attacks as it can be operated on any browser with an AI sidebar.
Surprisingly, these attacks require only basic browser extension permissions, commonly found in popular extensions like Grammarly and password managers, making them difficult to detect by simply looking at permission analysis. In fact, the AI Sidebar Spoofing extension can remain dormant, providing legitimate responses, until they see an opportunity to trick users into doing something malicious based on their prompt. Thus, it is absolutely critical that enterprises have both the ability to perform dynamic analysis on extension behavior at run time, as well as granular browser-native guardrails to warn and block users from following malicious instructions.
For more information, please refer to our technical blog.