Italian Comedies Are Being Dubbed In English Using AI

Some of the most famous Italian sketches, dubbed by AI in English, are on social media.

AI’s good or evil nature is being debated, but it is helping Italian comedy find an audience worldwide and dispelling the myth that comedy only works locally.

Tech journalist Andrea Indiano on Wired Italy explains that AI technology is venturing into the Italian comedy world, translating and dubbing comedy films into other languages. “An Italian project known on social media as @italiancomedydub offers some of the most famous sketches from Italian films with English dubbing, thanks to artificial intelligence. Hearing well-known voices such as those of Carlo Verdone, Checco Zalone or Aldo, Giovanni, and Giacomo while speaking in English intrigues historical spectators and could open up new scenarios for Italian comedy abroad,” writes Indiano. “The peculiarity is that the voices are those of the original performers. Therefore, Gigi Proietti’s voice recites from the cult Febbre da Cavallo, or Aldo Baglio uses his unmistakable cadence to deliver, in English, a sketch from Tre Uomini e Una Gamba.” The @italiancomedydub project is on TikTok, Instagram, and YouTube. “The authors’ identities behind the page are unknown, but they are undoubtedly artificial intelligence experts,” writes Idiano. What surprises the most, writes the author, is the “fact that those that could pass as ideal sketches for an Italian audience remain funny even in another language.”

This fact dispels the myth that comedy is a local affair. When voice, speech, and physical comedy, such as facial expressions and hand movements, are delivered together, it works everywhere.

Before AI, the dubbing process was costly and lengthy, with various steps: the lip-sinking attentive translation, the search for the right dubbing actor, and the taping. “Now, using deep learning algorithms, AI dubbing software can be trained to quickly learn the patterns of a particular voice sample and convey the specific tone and emotions of the original actor,” says Indiano. Moreover, modern AI technology can modify the video and perfectly sync the labial movements with the foreign language.

“Automated transcribing, text translation, and video dubbing with the help of AI are commonplace nowadays. These technologies are developing and improving year after year. United into one automatic mechanism, they may become an economically profitable, fast, and relatively accurate tool for video localization,” says Alexander Konovalov, the founder and CEO at Vidby, a Swiss startup offering AI-powered video translation and dubbing across 70 languages.

Of course, there are downsides: the growth of AI dubbing means that the relevant work sector will change, causing more than one concern among voice actors worldwide. There’s many ethical debates on the puppeteering of dead actors using AI and how AI can be used to steal and mimic voices of well-known figures into spreading misinformation or bigotry.

While AI technology continues to advance, concerns arise as voice artists face potential job loss due to this growing trend. AI companies are even hiring voice actors to train computer models, emphasizing the value of human creativity and emotion that machines cannot replicate. For the very same reason, actors stopped Hollywood two months ago.

Sources: Wired.it, Forbes