Detail View

Question answering has served as a primary task for understanding natural language by answering questions posed in natural language.
Over the past few years, question answering has been re-phrased as `machine comprehension', meaning reading comprehension ability of machines, and have started to outperform humans with the help of deep learning.
Combined with search engines, recent machine comprehension models are even leveraged to answer questions with a very large unstructured corpus, which is often called open-domain question answering.
As practical usage of machine comprehension models often includes understanding language of experts (e.g., biomedical, legal texts), machine comprehension models that answer domain-specific questions with large unstructured knowledge source could benefit various fields.

However, machine comprehension models are still far from understanding domain specific texts such as biomedical corpus as these texts have very different word distribution compared to the general domain corpora.
Also the performance of open-domain question answering is still very low considering the recent success of machine comprehension with relatively short paragraphs.
In this paper, we investigate how open-domain question answering could be improved with paragraph re-ranking, and also how machine comprehension models could be improved in domain specific texts, focusing on biomedical texts.
First, we introduce the current question answering system for a large unstructured corpus, and show how the open-domain question answering systems can be improved with paragraph re-ranking.
Then, we describe bidirecitional encoder representations from transformers for biomedical text mining (BioBERT) which understand biomedical texts with contextualized word representations.
BioBERT gives large improvement not only in biomedical named entity recognition, relation extraction, but also in question answering.
Finally, we show how machine comprehension models trained on general domain corpora could be also leveraged in domain specific texts without using any biomedical corpora.
We conclude by showing the possibility of building question answering systems that answer domain-specific questions with huge unstructured biomedical corpus.

Abstract
Contents i
List of Figures iii
List of Tables iv
1    Introduction 1
2    Ranking Paragraphs for Improving Answer Recall in Open-Domain Question Answering 4
 2.1 Background   4
 2.2 Open-Domain QA Pipeline   6
  2.2.1 Paragraph Ranker  6
  2.2.2 Answer Aggregation   8
 2.3 Experiments   9
  2.3.1 Datasets  9
  2.3.2 Implementation Details  9
  2.3.3 Results  9
  2.3.4 Analysis    10
 2.4 Discussion 11
3    BioBERT: a pre-trained biomedical language representation model for biomedicaltext mining 12
 3.1 Background  12
 3.2 Approach  14
 3.3 Methods   15
  3.3.1 BERT: Bidirectional Encoder Representations from Transformers  15
  3.3.2 Pre-training BioBERT   15
  3.3.3 Fine-tuning BioBERT    17
 3.4 Results 19
  3.4.1 Datasets 19
  3.4.2 Experimental Setups   20
  3.4.3 Experimental Results  21
 3.5 Discussion 25
4    Kernelized Sparse Phrase Representation Learning for Question Answering 27
 4.1 Background   27
 4.2 Analyzing Dense Phrase Encoding  29
  4.2.1 Dense Phrase Representation  29
  4.2.2 Analyses   30
 4.3 Sparse Phrase Encoding 31
 4.4 Experiments  35
  4.4.1 Results   37
  4.4.2 Qualitative Analyses   40
 4.5 Related Work   42
 4.6 Discussion 43
5    Conclusion 44
Bibliography 45