Topic and assignment dates are tentative.
|Release Date||Due Date (all 11:59pm)|
Bold readings are the highest priority. J&Mxx, Gxx, and M&Sxx refer to recommended course readings.
|Feb 8||Introduction||NLP (circa 2001)|
|Mar 10||No class|
|Apr 26||No class|
As we schedule a topic, it will be moved to the schedule table above.
|Text classification||M&S 7.4,16.2-16.3, Collins: Naive Bayes (Sec 1-4), Collins: Log Linear (Sec 2), MaxEnt, Baselines, CNN Classification Naive Bayes prior derivation|
|Neural networks||Primer, Back-prop, Deep Averaging Networks, Gradient Checks (briefly), Gradient Checks (in details)|
|Computation graphs||NN Tips, Intro to Computation Graphs|
|Lexical semantics and embeddings||w2v explained, word2vec, word2vec phrases, Hill2016, Turney2010|
|Language modeling||J&M 4, M&S 6, Collins: LM, Smoothing, Char RNN|
|Sequence modeling||J&M 5.1-5.3, 6, M&S 3.1, 9, 10.1-10.3, Collins: HMM, Collins: MEMMs (Sec 3), Collins: CRF (sec 4), Collins: Forward-backward, SOTA Taggers, TnT Tagger, Stanford Tagger|
|Recurrent neural networks||G14, BPTT, RNN Tutorial, Effectiveness, Luong2015|
|Dependency parsing||J&M 12.7, Nivre2003, Chen2014|
|Convolutional neural networks||G13, Kim2014, Jacovi2018|
|Transformers||Annotated Transformer, Illustrated Transformer|
|Contextualized representations||BERT, The Illustrated BERT, ELMo, and co., Chen2019|
|IBM translation models||J&M 25.5, M&S 13.1-13.2, Collins: IBM Models, IBM Models, Collins: EM (Sec 5-6), HMM alignments, IBM Model 2 EM Notebook, BLEU Score, Neural MT Tutorial|
|Constituency parsing||J&M 12.1-12.6, 13.1-13.4, 14.1-14.4, M&S 11, 12.1, Collins: PCFGs, Eisner: Inside-outside, Collins: Inside-outside|
|Phrase-based machine translation||J&M 25.4, 25.8, M&S 13.3, Collins: PBT, Statistical PBT, Pharaoh decoder|
If time allows, we will discuss compositional semantics, summarization, and question answering.