Meta unveiled earlier this month its new language processing model for artificial intelligence (AI) research. Called the Open Pretrained Transformer (OPT-175B), it will be entirely open source and can be used for non-commercial purposes. These models can be used to automate chatbots, translate texts, or even write product sheets.
Meta was inspired by the OpenAI model
The model shares the same capabilities as that created by OpenAI, a company co-founded by Elon Musk. Like GPT-3it has 175 billion parameters and is based on the machine learning.
If OPT-175B was designed to look so much like its big brother, it’s intentional. According to Joelle Pineau, co-CEO of Facebook AI Research, the model was designed so that the level of accuracy of the language tasks and its toxicity would match that of GPT-3.
The idea behind it? To make its code open source in order to allow researchers from all horizons to contribute to its development for free. Indeed, OpenAI had made the access to its API paying. Because of this, only the wealthiest laboratories were able to undertake research on its language processing model.
” We believe that the entire AI community – academic researchers, civil society, legislators, and industry – must be able to work together to develop clear guidelines around voluminous and responsible artificial intelligence and language models “, explained Meta in its communiqué.
Making technology more transparent
Making technology more and more transparent has always been Joelle Pineau’s hobbyhorse. ” This commitment to open science is why I am [chez Meta]. I would not be here under any other conditions “, said the researcher in an interview with MIT Technology Review.
Meta is not the company most inclined to share its secrets. It has only rarely revealed the workings of its algorithms and prefers instead to conceal its mistakes. However, Joelle Pineau managed to make OPT-175B open source and available for non-commercial use.
The group has also released the entire code of the model and a logbook to document the training process of the artificial intelligence. It already contains 3 months of research, from October 2021 to January 2022, and lists all bugs, crashes and reboots encountered.
Natural language processing models have become the next Holy Grail of artificial intelligence. However, they contain deep flaws, including the transmission of false information, the use of racist and sexist biases and toxic language. These problems are related to the texts used by the model to train and develop.
GPT-3 has been at the receiving end of this many times. In October 2020, the company Nabla was testing this model in the medical field when it advised a fake patient to end his life. Making the code of language processing models freely available in their entirety would therefore allow for more effective correction of these complications.
.
