Neural Abstractive Text Summarization and Fake News Detection

Authors - Soheil Esmaeilzadeh, Gao Xian Peh, and Angela Xu

Download the Paper

Abstract - In this work, we study abstractive text summarization by exploring different models such as LSTM-encoder-decoder with attention, pointer-generator networks, coverage mechanisms, and transformers. Upon extensive and careful hyperparameter tuning we compare the proposed architectures against each other for the abstractive text summarization task. Finally, as an extension of our work, we apply our text summarization model as a feature extractor for a fake news detection task where the news articles prior to classification will be summarized and the results are compared against the classification using only the original news text.

Keywords: Abstractive text summarization, Pointer-generator, Coverage mechanism, Transformers, Fake news detection

Fig.1. Baseline sequence-to-sequence model’s architecture with attention, See, [See et al., 2017 Get To The Point: Summarization with Pointer-Generator Networks]

Fig.2. Figure 2: Pointer-generator model’s architecture, See, [See et al., 2017 Get To The Point: Summarization with Pointer-Generator Networks]

Fig.3. The Transformer - model architecture, (b). (left) Scaled Dot-Product Attention. (right) Multi-Head Attention consists of several attention layers running in parallel, See, [Miyagishima et al., 2014, Processing of S-cone signals in the inner plexiform layer of the mammalian retina]

Fig.4. Comparison of the generated summary using the summarization models v.s. the ground truth

Fig.5. Confusion matrix for test set of fake news detection task using three different input features - full body news text, news headline, and summary generated from the news text

Fig.6. Fake news classifier results - using full body news text, using news headline, and using summary generated from the news text