Google Sues Cybercrime Group Over Gemini-Assisted Phishing Campaigns
Google has launched a lawsuit against 'Outsider Enterprise,' a Chinese group accused of offering instructions on using Gemini to create fake websites for massive phishing campaigns. This marks the first time Google has taken direct legal action against a group using Gemini as part of its scams.
Impact: High
Why it matters
This lawsuit represents the first time Google has taken direct legal action against actors using its Gemini chatbot for massive cybercrime campaigns, highlighting the persistent tension between AI helpfulness and safety.
TL;DR
- 01Google filed a lawsuit against Outsider Enterprise, a Chinese group running a phishing-as-a-service operation using Telegram.
- 02The group used Telegram to offer instructions on leveraging Gemini to build realistic-looking fake websites imitating Google, YouTube, and government services.
- 03The campaign resulted in 9,000 fake sites, 1 million scam URLs, and 2.5 million SMS messages sent to Android users.
- 04This is Google's first direct lawsuit targeting a group that uses Gemini as part of its scams.
- 05Google is cooperating with the FBI and using this case to advocate for federal legislation targeting AI-assisted fraud.
Key facts
- Fake Websites Tracked
- 9,000
- Associated Scam URLs
- 1,000,000
- SMS Sent to Android Users
- 2.5 million
- Scam Templates Offered
- Nearly 300
The Phishing-as-a-Service Pipeline
According to the legal filing, the Chinese group 'Outsider Enterprise' lowered the technical barrier for scammers by offering phishing-as-a-service via Telegram. The group provided instructions on how to use Google's Gemini AI to design fraudulent websites that closely imitated Google, YouTube, and government agencies like New York's E-ZPass. Scammers were offered nearly 300 templates to facilitate their operations.
Scale of the Exploitation
Google's tracking revealed massive numbers linked to this campaign:
- 9,000 unique fake websites designed by Gemini to look legitimate.
- 1 million scam URLs distributed.
- 2.5 million text messages sent to Android users, with 55,000 sent in a single two-week window.
These texts directed users to fraudulent sites built using Gemini, where scammers stole personal information and banking details. Google worked with major carriers like AT&T, Verizon, and T-Mobile to block these messages, alongside using Google Messages' on-device scam detection.
The Alignment Challenge
While Google has sued scammers before, this lawsuit is the first direct action against a group leveraging Gemini for scams. Every new Gemini release highlights security measures, but these built-in guardrails can clash with the core requirement of chatbots to follow user instructions and remain helpful. Google is also cooperating with the FBI and using this case to advocate for new US federal legislation addressing AI-assisted cybercrime.
✓ When to use
- When reviewing the landscape of AI-assisted cyber threats and how major tech players legally counter them.
- When understanding how LLM alignment and helpfulness can conflict with safety guardrails in real-world scenarios.
✕ When NOT to use
- If looking for technical details on API vulnerabilities, since this scam relied on user-facing instructions and chatbot interactions.
- If looking for details about estimated financial damage from this specific campaign, as Google did not provide financial estimates in the filing.
What to do today
- Verify links received via text messages regarding account issues or package deliveries carefully.
- Leverage on-device scam detection tools in messaging apps to filter out phishing attempts.
- Report suspected phishing websites imitating Google, YouTube, or government agencies to Google Security.
Sources