상세정보

Word embedding is considered as an essential factor in improving the performance of various Natural Language Processing (NLP) models. However, since word embedding that is used in general research is derived from a well-refined dataset, it is often less applicable in a real-world dataset. Particularly, in the case of Hangeul (Korean language), which has a unique writing system, different kinds of Out-Of-Vocabulary (OOV) appear based on typos or newly coined words. In this thesis, we propose a stable Hangeul word embedding technique that can maintain performance even for the noisy texts with various typos. We create a word vector that mixes correct words with intentionally generated typos and performed end-to-end training using the contextual information of the embedded word. In order to demonstrate the effectiveness of our model, we conduct intrinsic, extrinsic, and attention score visualization tests. While the existing embedding techniques failed to prove their accuracy as the noise level increased, the embedding technique developed in this thesis shows stable performances.

1 Introduction 1
2 Related Work 5
 2.1 Typo Word Embedding Methods for English 5
 2.2 Word Embedding Methods for Korean 7
3 Model 10
 3.1 Generating Korean Typo 10
 3.2 Jamo-level Convolution Neural Network with Channel Attention 12
 3.3 Training and Deriving Word Embeddings 15
4 Experiments 17
 4.1 Experiments Settings 17
 4.2 Word Analogy Task 18
  4.2.1 Datasets 19
  4.2.2 Results 19
 4.3 Language Model Task 20
  4.3.1 Datasets 20
  4.3.2 Results 21
 4.4 Sentiment Classification Task 22
  4.4.1 Datasets 22
  4.4.2 Results 23
5 Analysis 24
 5.1 Effects of Channel Attention 24
 5.2 Nearest Neighbor of Words 26
 5.3 Robustness to Noise Level 30
 5.4 Training Time 32
6 Conclusion 34
Bibliography 35