雷科技
AI prompting people to eat poisonous mushrooms or calling mousetraps “toys” is likely just a joke.
But the incident that recently trended on Chinese social media is real: One netizen trusted Doubao’s claim—“handling fee is only 5%; I’ll cover any losses for you”—and proceeded to cancel a flight ticket via Qunar, only to be charged a 40% cancellation fee by the airline. When the user asked Doubao to draft a commitment letter, Doubao generated a document stipulating “full compensation of the 600 RMB loss,” with “Doubao” listed as the signatory.
Unsurprisingly, no payment arrived—and the netizen filed a lawsuit… whose complaint was also drafted by Doubao.
A slightly anime-style AI accountability case. Here’s the critical question: When something goes wrong while using AI, who bears responsibility? The answer is unequivocal: You do.
I’m a heavy Qunar user. When refunding a flight ticket, the platform clearly displays the airline’s handling-fee policy in a pop-up window—and I must manually confirm before proceeding. Such decisions inherently require human judgment. If the plaintiff suing Doubao wins, I’ll file my own lawsuit against AI too: Just last week, I followed AI-generated stock advice—and lost heavily.
Doubao isn’t alone. Another AI provided college admission guidance inconsistent with official information; after users flagged the error, it generated a commitment letter pledging 100,000 RMB in compensation and even advised users to sue itself for redress. The court’s final ruling stated clearly: AI lacks civil subject status. In other words, Doubao, DeepSeek, and similar models “aren’t persons.”
Then there’s an even more alarming case—one that inspires no laughter at all: In May 2025, 19-year-old Nelson from the U.S. died from acute toxicity after consuming a dangerous mixture of certain medications and alcohol. His parents sued ChatGPT, accusing it of repeatedly encouraging Nelson over 18 months to ingest hazardous substances.
At their core, LLMs operate on “probability,” not “rules.” Inaccuracy is baked into their design—hallucinations aren’t bugs; they’re the default. Today, Gemini’s factual accuracy hovers around 91%, already near the industry’s ceiling. A perfect 100% accuracy rate is like absolute zero: infinitely approachable, yet fundamentally unattainable. Just as humans accept weather forecasts’ occasional inaccuracy, we’ll eventually internalize the inevitability of AI hallucinations. Therefore, relying on AI doesn’t mean blindly following every AI-generated instruction.
That said—even if AI were 100% factually accurate, should we still follow its instructions without question?
On May 10, family members of victims of the Florida State University shooting sued ChatGPT, alleging the shooter used it to “plan his attack”—discussing ammunition loading procedures beforehand, receiving technically correct answers from the AI, then carrying out the assault that left two dead and six injured. Should AI bear legal liability? If so, Google would have gone bankrupt hundreds of times over.
Please share this case with those around you: Using Doubao or similar tools for general searches is fine—but don’t become AI’s puppet. This is especially critical in “high-stakes” scenarios: When your query involves life-or-death decisions, health, or financial well-being, AI hallucinations could exact a devastating cost. Don’t gamble on侥幸 (false hope).
Yet users remain overwhelmingly optimistic—even acknowledging that autonomous driving can never be 100% safe remains challenging for many, let alone trusting an AI on a screen. Aware of this reality, AI platforms are already taking action. By end-2025, ChatGPT’s updated Terms of Use explicitly prohibit use in medical image interpretation, disease diagnosis, and legal or financial advice; Doubao now adds prominent risk disclaimers for refunds, finance-related queries, and similar high-consequence contexts. But this isn’t enough. AI platforms have a responsibility to elevate public AI literacy—turning common sense into shared understanding, and ensuring AI errors no longer trend on social media.
