Microsoft today announced an update to its translation services that promises to dramatically improve translation between more language pairs thanks to new machine learning technologies. Based on the project’s Z-code, which uses an “additional recruitment of experts” approach, these new models now score 3-15% better than the company’s previous models in blind scoring. Z-Code is part of the broader Microsoft XYZ-Code initiative, which combines text, graphics, and sound models in multiple languages to create more powerful and useful AI systems.
The “combination of experts” is not a completely new technique, but it is especially useful in the context of translation. At its core, the system basically breaks down tasks into multiple subtasks and then assigns them to smaller, more specialized models called “experts”. The model then decides which expert to assign which task based on its predictions. In an oversimplified way, you can think of it as a model made up of many more specialized models.
“With Z-Code, we have made some truly amazing progress as we use both transfer learning and multi-task learning on monolingual and multilingual data to create a state of the art language model that we believe delivers quality and performance is the best combination. . and the efficiency we can offer our customers,” said Xuedong Huang, Microsoft CTO and CTO of Azure AI.
The result is a new system that can now directly translate between 10 languages, making multiple systems redundant, for example. Microsoft recently started using the Z-code model to improve other features of its AI system, including entity recognition, text digest, custom text classification, and key phrase extraction. However, this is the first time it has used this approach for a translation service.
Traditionally, translation models are very large, making them difficult to implement in a production environment. However, Microsoft Teams uses a “sparse” approach where only a few model settings are enabled for each task, rather than the entire system. “This makes them more cost-effective to use, as well as being cheaper and more efficient to heat your home in the winter only when you need it and in rooms you use regularly, on time, rather than running the stove at full blast.” explain the team in today’s announcement.