After ChatGPT and DALL-E, meet VALL-E – the text-to-speech AI that can mimic anyone’s voice

In the last year, artificial intelligence (AI) tools have emerged that can create images, artwork, or even videos with a text prompt.

There have also been major advances in AI writingwith OpenAI’s ChatGPT causing widespread excitement – and fear – about the future of writing.

Now, just a few days into 2023, another powerful use case for AI has come to the fore – a text-to-speech tool that can flawlessly mimic a person’s voice.

Developed by Microsoft, VALL-E can take a three-second recording of someone’s voice and replicate that voice, turning written words into speech, with realistic intonation and emotion, depending on the context of the text.

Trained on 60,000 hours of English speech recordings, he can deliver a speech in a “zero-fire situation”, meaning without any prior examples or training in a specific context or situation.

Introducing VALL-E in an article published by Cornell Universitythe developers explained that the recording data consisted of over 7,000 unique speakers.

The team says that their Text To Speech (TTS) system used hundreds of times more data than existing TTS systems, helping them overcome the zero triggering problem.

The tool is not currently available for public use – but it raises questions about security as it could be used to generate any text coming from anyone’s voice.

Microsoft betting big on AI

Its creators, however, provided a demofeaturing multiple three-second speaker prompts and a demonstration of text-to-speech in action, with the voice imitated correctly.

Along with the speaker prompt and VALL-E output, you can compare the results with the “ground truth” – the actual speaker reading the prompt text – and the “baseline” result of current TTS technology .

Microsoft has invested heavily in AI and is one of the backers of OpenAI, the company behind ChatGPT and DALL-E, a text-to-image or artwork tool.

The software giant invested $1 billion (€930 million) in OpenAI in 2019, and a report this week on semafor.com said it was considering investing another $10 billion (€9.3 billion) in the company.