Italy orders ChatGPT to be blocked due to data protection concerns

Two days after an open letter called for a moratorium on the development of more powerful generative AI models so regulators can catch up with ChatGPT, Italy’s data protection authority has just issued a timely reminder that some countries TO DO have laws that already apply to cutting-edge AI – ordering OpenAI to stop processing people’s data locally with immediate effect.

Italy’s DPA said it’s concerned the maker of ChatGPT may be in breach of the European Union’s General Data Protection Regulation (GDPR).

More specifically, the Guarantee said it issued the order to block ChatGPT due to concerns that OpenAI has illegally processed people’s data – and also the lack of any system to prevent minors from accessing the technology.

The San Francisco-based company has 20 days to respond to the order, with the threat of substantial penalties if it does not comply. (Reminder: fines for violating the EU data protection regime can reach 4% of annual turnover or €20 million, whichever is greater.)

It should be noted that since OpenAI does not have an established legal entity in the EU, any data protection authority is empowered to intervene, under the GDPR, if it finds risks for local users. (So ​​where Italy steps in, others can follow.)

Suite of GDPR issues

The GDPR applies whenever personal data of EU users is processed. And it’s clear that OpenAI’s big language model has parsed that kind of information — since it can, for example, produce biographies of named individuals in the region on demand (we know, we know that). have tried). Although OpenAI declined to provide details on the training data used for the technology’s latest iteration, GPT-4. But he revealed that previous models were trained on data pulled from the internet, including forums such as Reddit. So if you are reasonably online, chances are the bot knows your name.

Add to that, ChatGPT has been shown to produce completely false information about named individuals – apparently making up details that its training data is missing. Which potentially raises further concerns about the GDPR, since the regulation gives Europeans a range of rights over their data, including the right to rectify errors. And it’s unclear how/if people can ask OpenAI to correct misrepresentations about them generated by the bot, in a single example scenario.

THE GuaranteeThe statement also highlights a data breach suffered by the service earlier this month – when OpenAI admitted that a chat history feature had leaked users’ chats and said it could – be exposed the payment information of certain users.

Data breaches are another area regulated by the GDPR – with a focus on ensuring that entities processing personal data adequately protect the information. Pan-European legislation also contains obligations to notify competent supervisory authorities of material breaches within tight deadlines.

Above all of this is the big (more) question of what legal basis did OpenAI rely on to process the data of Europeans in the first place? Aka, the lawfulness of this processing.

The GDPR allows for a number of possibilities – from consent to public interest – but the extent of processing to form these large language patterns complicates the issue of legality, as the Guarantee notes (pointing to “mass collection and storage of personal data”), with data minimization being another big goal of the regulation – which also contains principles that require transparency and fairness. Yet at least the (now) for-profit company behind ChatGPT doesn’t appear to have informed the people whose data is reused to train its commercial AIs. Which could be quite a sticky problem for him.

If OpenAI illegally processed the Europeans’ data, DPAs across the bloc could order the data deleted – although that requires it to retrain models trained on illegally obtained data is an open question while an existing law is in place. grappling with cutting-edge technology.

On the other hand, Italy may have just banned all machine learning by, uh, accident… 😬

“[T]The Privacy Guarantor notes the lack of information for users and all interested parties whose data is collected by OpenAI but above all the absence of a legal basis justifying the massive collection and storage of personal data, with the aim of ” form “the algorithms underlying the operation of the platform,” the DPA wrote in its statement today. [which we’ve translated from Italian using AI].

“As evidenced by the checks carried out, the information provided by ChatGPT does not always correspond to the real data, thus determining inaccurate processing of personal data,” he added.

The authority added that it was concerned about the risk of minors’ data being processed by OpenAI, as the company does not actively prevent people under the age of 13 from signing up to use the chatbot, for example. by applying age verification technology.

Risks to children’s data is an area where the regulator has been very active – recently ordering a similar ban on the AI ​​virtual friendship chatbot, Replika, on child safety grounds. In recent years, it has also sued TikTok over underage use – forcing the company to purge more than half a million accounts it could not confirm did not belong to children.

So if OpenAI can’t definitively confirm the ages of the users it has registered in Italy, it could – at the very least – be forced to delete their accounts and start over with a more robust registration process.

OpenAI has been contacted for a response to the Guaranteeorder.