NLP

6-Spelling correction, edit distance and EM

Introduction to how to do spelling correction 1 Spelling correction ##1.1 Simple version Assumption: have a large dictionary of real word only correct non-word only consider corrections differ by a single character from the non-word Idea: generate a list of all words $y$ differ by 1 character from $x$ compute the probability of each $y$ and choose the highest one 1.2 Alignments and edit distance Edit distance: substitution deletion insertion Need to:

January 30, 2020 Read

5-Distributed representations

Brief introduction to word embedding

January 28, 2020 Read

4-Evaluation and smoothing

Introduction to smoothing word counts to get the probabilities

January 23, 2020 Read

3-N-gram

Brief introduction to N-gram language model

January 21, 2020 Read

2-Corpora

What is a corpora? 1. Corpora Def: a large or complete collection of writings For NLP, we need corpora with linguistic annotations Markup formats: XML, JSON, CoNLL-style e.g. Brown, WSJ, ECI, BNC, Redwoods, Gigaword, AMI, Google Books N-grams, Flickr 8K, English Visual Genome Why need corpora manual rules or database (rule-based, symbolic, knowledge-driven) learning: provide example input/output pairs for supervised learning 2. Sentiment Analysis Goal: predict the opinion expressed in a piece of text (e.

January 17, 2020 Read

1-Intro

Brief introduction to NLP

January 14, 2020 Read