New York City Mayor Eric Adams has started creating audio deepfakes of himself in order to communicate with New Yorkers in several languages.
During a press conference on Monday to brief media on a recent “chatbot forum” at City Hall, Adams revealed that his staff has been robo-calling New Yorkers with an algorithmically generated facsimile of his voice speaking in multiple languages that the mayor does not speak.
The purpose of the Monday press conference was to announce the city’s impending use of “AI” voice chatbots that will speak to New Yorkers in their preferred language when they call 311 or other services, but the mayor casually mentioned that the technology has already been deployed for other purposes, and that New Yorkers are being misled about the range of languages he speaks.
“We’ve started doing robocalls with my voice in a variety of languages.” “Everyone stops me on the street and says, ‘I didn’t know you spoke Mandarin,'” Adams remarked during the news conference.
“The robocalls that we’re using, we’re using different languages to speak to the diversity of New Yorkers,” he went on to say.
In an email to Motherboard, the city stated that it has been delivering AI robocalls since March 2022, with 5,000 in Spanish, 250 in Yiddish, 160 in Mandarin, 89 in Cantonese, and 23 in Haitian Creole. It used the calls to inform New Yorkers about job opportunities as well as a free music series.
However, the robocalls do not include a disclaimer that they are generated artificially, giving New Yorkers the idea that Mayor Adams speaks Spanish, Mandarin, and Yiddish.
The Surveillance Technology Oversight Project (STOP) slammed the robocalls, calling them dishonest, in a statement issued following the press briefing on Monday. “This is deeply unethical, especially on the taxpayer’s dime,” stated executive director Albert Fox Cahn in a statement. “It’s deeply Orwellian to use AI to convince New Yorkers that he speaks languages he doesn’t.” Yes, we need announcements in every language spoken by New Yorkers, but the deep fakes are merely a disturbing vanity project.”
STOP also criticizes the city’s other applications of AI and related technology, such as facial recognition. “The Mayor continues to enable some of the worst AI abuses in the country, all while hiding behind the rhetoric of AI responsibility,” Cahn stated in an e-mail.
One of the major issues about the use of algorithmically created images, video, and sounds in political campaigns is that they can be misleading, and there have been proposals to require disclosures so that constituents understand what they’re seeing is artificially generated. A bill introduced in the New York state assembly would require disclosure of the use of such technology in political advertisements. However, there is no analogous legislation or regulation requiring disclosures in other forms of government contact, such as robocalls or 311 inquiries.
Eleven Labs’ Voice Lab was used to create the robocalls, according to the city. The popular solution, built by former Google and Palantir employees, allows users to clone their own voices by uploading an audio sample to its website.
Motherboard discovered in January that 4chan members were using a test version of the program to create voices of Joe Rogan, Ben Shapiro, and Emma Watson spouting racist and transphobic comments. After detecting “misuse cases,” the corporation stated that it will enhance measures. In February, a Motherboard reporter used the program to circumvent voice authentication and get access to his personal bank account. At the time, the corporation did not respond to a request for comment.
Mayor Adams has praised what he calls the “ethical” use of AI, and he has tried to position himself as someone who welcomes cutting-edge technology.
“We are not fleeing AI; we are going to properly govern how we use AI responsibly,” Adams stated at the start of the press conference. “You can use or abuse anything, and if we stay away from moving forward because we’re afraid someone is going to abuse it, you won’t get anything done.”
