ELMo.”īERT achieved state-of-the-art results on a number of tests, and has been so successful that Google recently incorporated it into its search engine. In Google’s own blog post on the topic, the company says “BERT builds upon recent work in pre-training contextual representations - including. This model proved to be powerful and influential, and pushed a number of novel ideas about language generation into the AI mainstream.īERT itself officially stands for Bidirectional Encoder Representations from Transformers, and although Google refused multiple requests from The Verge to discuss the origins of the name, it’s widely assumed that the researchers, like those from Allen, had the Muppets in mind. “My oldest son was about three at the time and it was also my way of dedicating the paper to him.”ĮLMo might have been a one-off had it not been for BERT - a language model created by Google’s AI team in 2018. “I liked it because it is somewhat whimsical but memorable,” says Peters. “My oldest son was about three at the time and it was also my way of dedicating the paper to him.” But it’s a joke that happens to highlight a particular characteristic of AI research, demonstrating how labs pay homage to and build upon one another’s work. The naming convention is so well established that these systems are sometimes referred to as “Muppetware.” But who started the convention and why?Īs you might have guessed, the simple answer is: it’s an inside joke, with researchers naming AI models after Muppets because other researchers have named AI models after Muppets. Big tech players like Google, Facebook, and the Allen Institute for AI are all involved, and the craze has global reach, with Chinese search giant Baidu and Beijing’s Tsinghua University contributing models. To date, this new breed of language AIs includes an ELMo, a BERT, a Grover, a Big BIRD, a Rosita, a RoBERTa, at least two ERNIEs (three if you include ERNIE 2.0), and a KERMIT. ![]() These programs are game-changers, potentially supercharging computers’ ability to parse and produce language.īut something that’s gone largely unnoticed is a secondary trend - a shadow to the first - and that is: a surprising number of these tools are named after Muppets. One of the biggest trends in AI recently has been the creation of machine learning models that can generate the written word with unprecedented fluidity.
0 Comments
Leave a Reply. |
Details
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |