CS5740 SP20

Time: MoWe 11:00am-12:15pm
Room: Bloomberg Center 131
Class listing: CS5740

Instructor: Yoav Artzi
Office hours: TBD
Location: Bloomberg 371

Teaching assistants: TBD
Graders: TBD
TA Office hours: TBD
Location: TBD

CMS | Forum

Schedule

Assignments and Exam

  Release Date Due Date
Assignment 1 TBD TBD
Assignment 2 TBD TBD
Assignment 3 TBD TBD
Assignment 4 TBD TBD
Assignment 5 TBD TBD
Final exam May 8 May 12

Lectures

Bold readings are the highest priority.

Date Topic Board Recommended Readings
Jan 22 Introduction   NLP (circa 2001)

Upcoming Topics

As we reach a topic, it will be moved to the schedule table above.

Topic Recommended Readings
Text classification M&S 7.4,16.2-16.3, Collins: Naive Bayes (Sec 1-4), Collins: Log Linear (Sec 2), MaxEnt, Baselines, CNN Classification Naive Bayes prior derivation
Neural networks Primer, Back-prop, Deep Averaging Networks, Gradient Checks (briefly), Gradient Checks (in details)
Computation graphs NN Tips, Intro to Computation Graphs
Lexical semantics and embeddings w2v explained, word2vec, word2vec phrases, Hill2016, Turney2010
Language modeling J&M 4, M&S 6, Collins: LM, Smoothing, Char RNN
Sequence modeling J&M 5.1-5.3, 6, M&S 3.1, 9, 10.1-10.3, Collins: HMM, Collins: MEMMs (Sec 3), Collins: CRF (sec 4), Collins: Forward-backward, SOTA Taggers, TnT Tagger, Stanford Tagger
Recurrent neural networks BPTT, RNN Tutorial, Effectiveness, Luong2015
Contextualized word representations Annotated Transformer, Illustrated Transformer, ELMo, BERT, The Illustrated BERT, ELMo, and co.
Dependency parsing J&M 12.7, Nivre2003, Chen2014
Constituency parsing J&M 12.1-12.6, 13.1-13.4, 14.1-14.4, M&S 11, 12.1, Collins: PCFGs, Eisner: Inside-outside, Collins: Inside-outside
Machine translation Neural MT Tutorial, BLEU Score
IBM translation models J&M 25.5, M&S 13.1-13.2, Collins: IBM Models, IBM Models, Collins: EM (Sec 5-6), HMM alignments, IBM Model 2 EM Notebook
Phrase-based machine translation J&M 25.4, 25.8, M&S 13.3, Collins: PBT, Statistical PBT, Pharaoh decoder

If time allows, we will discuss compositional semantics, summarization, and question answering.