“Instead of Alexa’s voice reading the book, it’s the voice of the child’s grandmother,” Rohit Prasad, senior vice president and chief scientist of artificial intelligence, explained enthusiastically on Wednesday. Alexa, during a keynote speech in Las Vegas. (Amazon founder Jeff Bezos owns The Washington Post.)
The demo was the first look at the new Alexa feature, which, although still in development, would allow the voice assistant to reproduce people’s voices from short audio clips. The goal, Prasad said, is to build greater trust with users by infusing artificial intelligence with the “human attributes of empathy and affect.”
The new feature could “make [loved ones’] memories last,” Prasad said. But while the prospect of hearing the voice of a deceased relative may touch the heart, it also raises myriad safety and ethical issues, experts have said.
“I don’t think our world is ready for user-friendly voice cloning technology,” Rachel Tobac, managing director of San Francisco-based SocialProof Security, told The Washington Post. Such technology, she added, could be used to manipulate audiences through fake audio or video clips.
“If a cybercriminal can easily and credibly reproduce another person’s voice with a small voice sample, he can use that voice sample to impersonate other people,” added Tobac, an expert in cybersecurity. “This bad actor can then trick others into thinking they are the person they are impersonating, which can lead to fraud, data loss, account takeover, etc.”
Then there’s the risk of blurring the lines between what’s human and what’s mechanical, said Tama Leaver, professor of internet studies at Curtin University in Australia.
“You won’t remember you’re speaking to the depths of Amazon…and its data collection services if it’s speaking with your grandmother or your grandfather’s voice or that of a lost loved one.”
“In some ways, it’s like an episode of ‘Black Mirror,'” Leaver said, referring to the sci-fi series envisioning a tech-themed future.
The Google engineer who thinks the company’s AI has come to life
The new Alexa feature also raises questions about consent, Leaver added — especially for people who never imagined their voice would be heard by a robotic personal assistant after they died.
“There is a real slippery slope to using data from deceased people in a way that is both frightening on the one hand, but deeply unethical on the other because they never considered that these traces are used in this way,” Leaver said.
Having recently lost his grandfather, Leaver said he understands the “temptation” of wanting to hear the voice of a loved one. But the possibility opens a floodgate of implications that society might not be ready to take on, he said – for example, who has the rights to the little snippets that people leave in the ethers of the World Wide Web?
“If my grandfather had sent me 100 messages, would I have the right to introduce that into the system? And if I do, who owns it? So does Amazon own this record? ” He asked. “Have I given up the rights to my grandfather’s voice?
Prasad did not address those details during Wednesday’s speech. He posited, however, that the ability to mimic voices was a product of “unquestionably living in the golden age of AI, where our dreams and science fiction become reality.”
This AI model attempts to recreate the mind of Ruth Bader Ginsburg
If Amazon’s demo becomes a real feature, Leaver said people might need to start thinking about how their voice and likeness might be used when they die.
“Am I to think in my will that I should say, ‘My voice and my pictorial social media story is the property of my children, and they can decide whether they want to resuscitate this by talking to me or not? ‘” Leaver wondered.
“That’s a weird thing to say now. But that’s probably a question we should have answered before Alexa starts talking like me tomorrow,” he added.