Meta introduces multilingual speech translation model - SeamlessM4T

Meta has come up with a new all-in-one, multilingual multimodal AI translation and transcription model for up to 100 languages depending on the task. The model will be useful to communicate and understand information in language that people do not know.  

The model is called SeamlessM4T and it can perform speech-to-text, speech-to-speech, text-to-speech, and text-to-text translations. It supports speech recognition for almost 100 languages, speech-to-text translation for nearly 100 input and output languages, speech-to-speech translation, supporting nearly 100 input languages and 36 (including English) output languages and text-to-text translation for nearly 100 languages.

It can also support text-to-speech translation, supporting nearly 100 input languages and 35 (including English) output languages.

"We're also releasing the metadata of SeamlessAlign, the biggest open multimodal translation dataset to date, totalling 270,000 hours of mined speech and text alignments," Meta said in a blog post.

Meta, last year, launched a text-to-text machine translation model - No Language Left Behind (NLLB) which supports 200 languages, and has since been integrated into Wikipedia as one of the translation providers.

"We also shared a demo of our Universal Speech Translator, which was the first direct speech-to-speech translation system for Hokkien, a language without a widely used writing system," said the company.

Early this year, the company released Massively Multilingual Speech, which provides speech recognition, language identification and speech synthesis technology across more than 1,100 languages.

SeamlessM4T has been formed based on the findings from all of these projects to enable a multilingual and multimodal translation experience stemming from a single model, built across a wide range of spoken data sources with state-of-the-art results, Meta noted.