Natural Language Processing (NLP) offers a wide variety of approaches and applications to work with text, which allows us to process it in the form and format is beneficial to the end-user. In this blog post, we would like to talk about one of these applications: text summarization.
This technique is quite useful in case end users deal with large text volumes and have a need for fast and important extracts from large text chunks. There are two main methods for text summarization: Extractive and Abstractive text summarization.
The Extractive approach takes existing words and phrases from a text and uses them to create a shorter summary. Here is a simple example [sic]: “In the 21st-century application of smart systems will play a more important role in improving people lives”
Summarized text:
“In the 21st-century application of smart system will play an important role in people lives”
The Abstractive approach rephrases words and sentences and constructs summarized text in its own words: “In the 21st-century application of smart systems will play more important role in improving people lives”
Summarized text:
“People lives will be improving due to high-tech applications in the 21st century”
There are many ways to implement text summarization. We will briefly describe an extractive one as proposed in the Microsoft paper “Neural Document Summarization by Jointly Learning to Score and Select Sentences” (https://arxiv.org/abs/1807.02305). The diagram below shows a high-level overview of how this architecture works.
At first, this architecture reads each sentence word by word and produces sentence-level representations, which are encoded in a vector. Next, these sentence-level representations are being encoded into a document level representation. These representations, in turn, are then being used as inputs for joint sentence scorer and selector – a module that processes and select the desired sentences.
All in all text summarization is a very complex topic where new architectures are being proposed very regularly, using the latest breakthroughs in Machine Learning and NLP. Hopefully, the AI research community will come up soon with even more powerful architectures and models to take the accuracy of text summarization to the next level.
Ildar Abdrashitov
Business Intelligence Analyst
Missing Link Technologies