Название: Embeddings in Natural Language Processing: Theory and Advances in Vector Representations of Meaning
Автор: Mohammad Taher Pilehvar, Jose Camacho-Collados
Издательство: Morgan & Claypool
Год: 2021
Страниц: 175
Язык: английский
Формат: pdf (true)
Размер: 11.7 MB

Embeddings have undoubtedly been one of the most influential research areas in Natural Language Processing (NLP). Encoding information into a low-dimensional vector representation, which is easily integrable in modern machine learning models, has played a central role in the development of NLP. Embedding techniques initially focused on words, but the attention soon started to shift to other forms: from graph structures, such as knowledge bases, to other types of textual content, such as sentences and documents.

This book provides a high-level synthesis of the main embedding techniques in NLP, in the broad sense. The book starts by explaining conventional word vector space models and word embeddings (e.g., Word2Vec and GloVe) and then moves to other types of embeddings, such as word sense, sentence and document, and graph embeddings. The book also provides an overview of recent developments in contextualized representations (e.g., ELMo and BERT) and explains their potential in NLP.

Throughout the book, the reader can find both essential information for understanding a certain topic from scratch and a broad overview of the most successful techniques developed in the literature.

Artificial Intelligence (AI) has been one of the most important topics of discussion over the past years. The goal in AI is to design algorithms that transform computers into "intelligent" agents. By intelligence here we do not necessarily mean an extraordinary level of smartness; it often involves basic problems that humans solve frequently in their day-to-day lives. This can be as simple as recognizing faces in an image, driving a car, playing a board game, or reading (and understanding) an article in a newspaper. The intelligent behavior exhibited by humans when "reading" is one of the main goals for a subfield of AI called Natural Language Processing (NLP). Natural language 1 is one of the most complex tools used by humans for a wide range of reasons, for instance to communicate with others, to express thoughts, feelings, and ideas, to ask questions, or to give instructions. Therefore, it is crucial for computers to possess the ability to use the same tool in order to effectively interact with humans.

This book should be of interest to all AI researchers who work with natural language, especially those who are interested in semantics. Our goal is to introduce the topic of semantic representation to those who are new to the area and to provide those who are already familiar with the area with a broader perspective and a quick overview of recent developments and the state of the art in various branches. The book synthesizes the diverse literature on semantic representations and provides a high-level introduction to major semantic embedding models.