BERT, a great step forward for the machine learning world


Artificial Intelligence has made enormous strides in natural language processing. Google and the Seattle lab Allen Institute for Artificial Intelligence have developed BERT (Bidirectional Encoder Representations from Transformer), an artificial intelligence system that can complete the missing parts of sentences (almost) as well as human beings.

Complete the sentence: “The man walked into a store and bought a _ of milk”. Simple, isn’t it? It is not for an AI system, yet BERT is able to do this and this opens the doors to an endless range of possibilities for the machine learning world. In fact, the programme can be used to improve technologies that are already used every day in our home like Alexa or Google Home, or automate business practices. 

By analysing millions of sentences written by humans, hundreds of books, including novels, science fiction novels and much more (including the entire catalogue offered by Wikipedia!), BERT can learn, understand and process the small idiosyncrasies of language and tackle small tasks, for example it can complete sentences. With these skills, the system can answer questions or gage, for example, the sentiment of a book review. It has even tackled the Allen Institute Common Sense test, which examines the ability of the computer to finish sentences.

Just like Google, many other companies are trying to develop new machine learning systems and many believe that this process will continue to accelerate the progress of different AI technologies, including natural language technologies.

Although BERT is a significant leap for researchers, machines are still far from an artificial version of human common sense; nonetheless, these are times of great progress for research in this field.