Generative AI’s tendency to make up information (known in the industry as “hallucination”) can have negative consequences when the information is about a real person. In Europe, these hallucinations may also create some fallout for AI companies deploying the technology—starting with OpenAI.
Noyb, the privacy nonprofit founded by prominent Austrian activist-lawyer Max Schrems, said this morning that it had filed a complaint with the Austrian privacy regulator, alleging that ChatGPT’s hallucinations had broken the EU’s General Data Protection Regulation (GDPR) in multiple ways.
The GDPR gives Europeans the right to complain about companies holding incorrect data on them, and to force companies to correct it. According to Noyb’s complaint—filed on behalf of an unnamed “public figure,” though Schrems confirmed to me that it’s him—OpenAI claims it’s unable to stop ChatGPT from generating inaccurate information, or to correct it. As an example, the complaint says Schrems asked OpenAI to stop ChatGPT generating an incorrect date of birth for him, and was told this was impossible. It also says OpenAI didn’t disclose what data led ChatGPT to emit a false birth date, or where that data came from, again breaching the GDPR.
OpenAI’s reported claim sounds about right, given that: a) gen AI chatbots just predict the words that are most likely to fit in a response to a query, and have no concept of truth; b) the process through which training data influences those responses is super-opaque; and c) once a model is trained on data, it can’t just forget it. (OpenAI had not responded to a request for comment at the time of publication.) But even if it’s correctly describing the limitations of its tech, that doesn’t give the company a valid defense under the GDPR, Noyb lawyer Maartje de Graaf said in a statement.
“It’s clear that companies are currently unable to make chatbots like ChatGPT comply with EU law, when processing data about individuals,” she said. “If a system cannot produce accurate and transparent results, it cannot be used to generate data about individuals. The technology has to follow the legal requirements, not the other way around.”
This isn’t the first complaint of its kind; last year, Polish cybersecurity researcher Lukasz Olejnik also filed one about OpenAI with that country’s privacy regulator. “I’m happy that others also feel legitimate concerns about technical-legal circumstances,” Olejnik told me today, regarding Schrems’s complaint.
The Italian data-protection authority also warned earlier this year that OpenAI was breaching the GDPR, though it didn’t provide specifics. Outside the EU, the U.S. Federal Trade Commission is probing the capacity of ChatGPT’s hallucinations for “reputational harm,” though under current U.S. Law that’s a consumer-protection rather than privacy matter.
But Schrems is a formidable opponent—his lengthy crusade against Meta has repeatedly threatened (and still threatens) the ability of U.S. Tech firms to process the personal data of European users without breaking European law. As someone who began that crusade while a law student and who went on to build his career around it, Schrems has a strong track record of succeeding in the courts, and OpenAI should be very concerned that it’s fallen into his sights.
On the plus side for OpenAI, the wheels of GDPR justice are slow to turn. But in time, it does seem quite possible that the EU’s privacy regulators will force the Microsoft-sponsored AI darling to limit what its models say about people in Europe, and perhaps to address inaccuracies in its training data. If worst comes to worst, OpenAI might even have to pay a fine of up to 4% of its global revenues—and if that happens, I’m pretty sure Redmond will be thankful that OpenAI isn’t a Microsoft subsidiary.
More news below.
David Meyer
Want to send thoughts or suggestions to Data Sheet? Drop a line here.
NEWSWORTHY
Discord boots spies. Discord has blocked an operation through which bots were scraping users’ messages—over 3 billion in total—and selling them. 404 Media, which originally revealed the operation, reported Friday that Discord had shut down a bunch of accounts involved in the scheme and is considering legal action against the perpetrators.
Meta’s AI ad boo-boo. Meta has been offering advertisers a “set it and forget it” service called Advantage Plus, which has an AI-enabled, automated tool for running ad campaigns. But according to a report in The Verge, the system has a habit of going haywire, vastly overcharging for impressions and generating little revenue. Some businesses accused Meta of a lack of transparency and accountability.
Banning weak security. As of today, the U.K. Has banned the sale of smart devices with dumb security, the Guardian reports. The rules that came into force today will affect companies selling everything from smartphones and smart TVs to connected doorbells and refrigerators. They will no longer be allowed to ship their products with passwords like “admin” or “12345.” The aim is to reduce the likelihood of people’s devices becoming quietly dragooned into botnets that can then be used to launch cyberattacks.
ON OUR FEED
“At least let us make sure that the most profound and far-reaching decision, who lives and who dies, remains in the hands of humans and not of machines.”
—Austrian Foreign Minister Alexander Schallenberg pleads with delegates at a conference on AI-equipped weapons systems to agree on international rules around such technology. Schallenberg also referred to this as “the Oppenheimer Moment of our generation.”
IN CASE YOU MISSED IT
Why Tesla’s Elon Musk is lobbying Beijing to approve its self-driving car tech as part of his surprise trip to China, by Lionel Lim
Elon Musk says any company that isn’t spending $10 billion on AI this year like Tesla won’t be able to compete, by Christiaan Hetzner
Meta’s investors are worried about the billions it’s spending on AI—but its advertising empire makes it a positive, Deutsche Bank says, by Dylan Sloan
Apple renews talks to integrate OpenAI’s technology into the next iPhone operating system, by Bloomberg
YouTube was the real hero of Google’s Q1 earnings, and it should set off alarm bells at Netflix, by Alexandra Sternlicht
Daniel Ek has opted out of a salary since 2017—but the Spotify CEO already cashed in $178 million worth of shares this year, by Ryan Hogg
TikTok's effort to wall off U.S. User data only focused on the front door while leaving the back door wide open, former employees say, by Alexandra Sternlicht
BEFORE YOU GO
Defending Ukrainian bots. “Today, the management of the Telegram platform unreasonably blocked a number of official bots that opposed Russia’s military aggression against Ukraine, including the main Intelligence bot,” Ukraine’s intelligence agency complained in a statement last night. On Telegram—a messaging platform developed in Russia but these days headquartered in Dubai—bots are automated tools that let people submit information (among other things), and some of Ukraine’s intelligence bots give the country’s citizens an easy way to report the locations of Russian soldiers and military equipment inside their country. Soon after the agency complained, Telegram unblocked its bots.
This is the web version of Data Sheet, a daily newsletter on the business of tech. Sign up to get it delivered free to your inbox.
