The EU’s General Data Protection Regulation (GDPR) has a strong emphasis on safeguarding children’s information and privacy - suggesting, for example, that services which are likely to have minors as users should think about incorporating child friendly design and be pro-active about conducting risk assessments to ensure they spot potential safety and other rights issues.
Replika was contacted for a response to the Garante’s order. “‘Replika’ is in breach of the EU data protection regulation: It does not comply with transparency requirements and it processes personal data unlawfully since performance of a contract cannot be invoked as a legal basis, even implicitly, given that children are incapable to enter into a valid contract under Italian law,” the Garante added, saying it has ordered its U.S.-based developer to cease processing data relating to Italian users - giving it 20 days to communicate measures taken to comply with the order.įailure to comply with the order risks a fine of up to €20 million, or 4% of total worldwide annual turnover, it further notes. Several reviews on the two main App Stores include comments by users flagging sexually inappropriate contents.”
“And the ‘replies’ served by the chatbot are often clearly in conflict with the enhanced safeguards children and vulnerable individuals are entitled to. During account creation, the platform merely requests a user’s name, email account and gender,” it observes.
#PERSONAL CHATBOT MAKER VERIFICATION#
“There is actually no age verification mechanism in place: no gating mechanism for children, no blocking of the app if a user declares that they are underage. And while under 18s are required to obtain authorization from a parent or guardian, the watchdog points out the app does not seek to verify the age of users, nor block minors who provide information about their age - hence its view that Replika is failing to protect children.
#PERSONAL CHATBOT MAKER ANDROID#
It also notes that while the app is listed as 17+, on Apple’s iOS and Google’s Android app stores, the developer’s terms of service only prohibit use by under 13s. The Garante’s order notes that several user reviews of the app report sexually inappropriate content being served up. But the Italian watchdog appears to the first regulator to take formal action over child safety. However concerns have been raised before now about the risks the technology might pose to children - ranging from worries over kids being exposed to inappropriate content to more general concerns, that they might get addicted to the interactions or just be encouraged into spending lots of money on customizing their avatars or to gain access to other paid content. Rather the startup claims it “ fine-tuned” GPT-3, using a network machine learning model trained on dialogue, to hone the generative technology for its particular use-case: Conversational (and it claims “empathic”) AI companions. Replika was an early API partner for OpenAI’s text-generating large language model technology, GPT-3 - although it’s service is not running on a carbon copy of GPT-3 (nor is it the same technology as OpenAI’s buzzy ChatGPT).
“Recent media reports along with tests the SA carried out on ‘Replika’ showed that the app carries factual risks to children - first and foremost, the fact that they are served replies which are absolutely inappropriate to their age,” it added. A provisional limitation on data processing was imposed by the Italian Garante on the U.S.-based company that has developed and operates the app the limitation will take effect immediately.” In a press release announcing its intervention, the watchdog said: “The AI-powered chatbot, which generates a ‘virtual friend’ using text and video interfaces, will not be able to process personal data of Italian users for the time being. The order to stop processing Italians’ data is effective immediately. It’s also accusing Luka Inc, the developer behind the Replika app, of failing to fulfil regional legal requirements to clearly convey how it’s using people’s data. The Garante said it’s concerned Replika’s chatbot technology poses risks to minors - and also that the company lacks a proper legal basis for processing children’s data under the EU’s data protection rules.Īdditionally, the regulator is worried about the risk the AI chatbots could pose to emotionally vulnerable people. San Francisco-based AI chatbot maker, Replika - which operates a freemium ‘virtual friendship’ service based on customizable digital avatars whose “personalized” responses are powered by artificial intelligence (and designed, per its pitch, to make human users feel better) - has been ordered by Italy’s privacy watchdog to stop processing local users’ data.
0 kommentar(er)
