CS 5740 SP21

Time: MoWe 1:35PM - 2:50PM
Instructor: Yoav Artzi
Teaching assistants: TBD
Graders: TBD

CMS | Canvas | Forum


Topic and assignment dates are tentative.

Assignments and Exam

  Release Date Due Date (all 11:59pm)
Assignment 1    
Assignment 2    
Assignment 3    
Assignment 4    
Assignment 5    
Take-home Exam    


Bold readings are the highest priority. J&Mxx, Gxx, and M&Sxx refer to recommended course readings.

Date Topic Board Recommended Readings
Feb 8 Introduction   NLP (circa 2001)
Feb 10      
Feb 15      
Feb 17      
Feb 22      
Feb 24      
Mar 1      
Mar 3      
Mar 8      
Mar 10 No class    
Mar 15      
Mar 17      
Mar 22      
Mar 24      
Mar 29      
Mar 31      
Apr 5      
Apr 7      
Apr 12      
Apr 14      
Apr 19      
Apr 21      
Apr 26 No class    
Apr 28      
May 3      
May 5      
May 10      
May 12      

Upcoming Topics

As we schedule a topic, it will be moved to the schedule table above.

Topic Recommended Readings
Text classification M&S 7.4,16.2-16.3, Collins: Naive Bayes (Sec 1-4), Collins: Log Linear (Sec 2), MaxEnt, Baselines, CNN Classification Naive Bayes prior derivation
Neural networks Primer, Back-prop, Deep Averaging Networks, Gradient Checks (briefly), Gradient Checks (in details)
Computation graphs NN Tips, Intro to Computation Graphs
Meta NLP  
Lexical semantics and embeddings w2v explained, word2vec, word2vec phrases, Hill2016, Turney2010
Language modeling J&M 4, M&S 6, Collins: LM, Smoothing, Char RNN
Sequence modeling J&M 5.1-5.3, 6, M&S 3.1, 9, 10.1-10.3, Collins: HMM, Collins: MEMMs (Sec 3), Collins: CRF (sec 4), Collins: Forward-backward, SOTA Taggers, TnT Tagger, Stanford Tagger
Recurrent neural networks G14, BPTT, RNN Tutorial, Effectiveness, Luong2015
Dependency parsing J&M 12.7, Nivre2003, Chen2014
Convolutional neural networks G13, Kim2014, Jacovi2018
Transformers Annotated Transformer, Illustrated Transformer
Contextualized representations BERT, The Illustrated BERT, ELMo, and co., Chen2019
IBM translation models J&M 25.5, M&S 13.1-13.2, Collins: IBM Models, IBM Models, Collins: EM (Sec 5-6), HMM alignments, IBM Model 2 EM Notebook, BLEU Score, Neural MT Tutorial
Constituency parsing J&M 12.1-12.6, 13.1-13.4, 14.1-14.4, M&S 11, 12.1, Collins: PCFGs, Eisner: Inside-outside, Collins: Inside-outside
Phrase-based machine translation J&M 25.4, 25.8, M&S 13.3, Collins: PBT, Statistical PBT, Pharaoh decoder

If time allows, we will discuss compositional semantics, summarization, and question answering.