This course focuses on models of dialogue in the context of Neural Networks for Natural Language Processing. The learning outcomes include understanding different paradigms of dialogue models, such as generation-based models and discourse-level VAE models. Students will learn about techniques to improve conversation coherence, diversity in responses, and incorporating personality into dialog agents. The course covers topics like dialog response retrieval, neural response retrieval, NLU with neural nets, dialog state tracking, and language generation from dialog state. The teaching method involves presenting various research papers and models in the field of dialogue systems. This course is intended for individuals interested in natural language processing, neural networks, and dialogue systems.
Overview
Syllabus
Types of Dialog
Two Paradigms
Generation-based Models (Ritter et al. 2011)
Neural Models for Dialog Response Generation
Dialog More Dependent on Global Coherence
One Solution: Use Standard Architecture w/ More Context
Discourse-level VAE Model (Zhao et al. 2017)
Diversity Promoting Objective for Conversation (Li et al. 2016) • Basic idea we want responses that are likely given the context, unlikely otherwise • Method: subtract weighted unconditioned log probability from conditioned probability (calculated only on first few words)
Using Multiple References with Human Evaluation Scores (Gallay et al. 2015)
Learning to Evaluate • Use context, true response, and actual response to learn a regressor that predicts goodness (Lowe et al. 2017)
Problem 3: Dialog Agents should have Personality
Personality Infused Dialog (Mairesse et al. 2007)
Dialog Response Retrieval
Retrieval-based Chat (Lee et al. 2009)
Neural Response Retrieval (Nio et al. 2014)
Smart Reply for Email Retrieval (Kannan et al. 2016)
NLU (for Slot Filling) w/ Neural Nets (Mesnil et al. 2015)
Dialog State Tracking
Language Generation from Dialog State w/ Neural Nets (Won et al. 2015)
Taught by
Graham Neubig
