Interested in joining our next batch? Applications open soon.

Interested in joining our next batch? Applications open soon.

Degree Level Course

Sequential Decision Making

At the end of the course, students will be able to understand the differences between the various sequential decision making problems based on the type of feedback involved, recognize practical ML problems as sequential decision making problems whenever they are, learn about optimal algorithms for several sequential decision making settings, and apply the algorithms studied in the course to various practical sequential decision making scenarios.

by Arun Rajkumar

Course ID: BSCCS4004

Course Credits: 4

Course Type: Elective

Pre-requisites: None

Course structure & Assessments

12 weeks of coursework, weekly online assignments, 2 in-person invigilated quizzes, 1 in-person invigilated end term exam. For details of standard course structure and assessments, visit Academics page.

WEEK 1 Introduction to Online Learning, Halving algorithm
WEEK 2 Online Machine Learning; Perceptron and Winnow
WEEK 3 Intro to Regret; Online learning with expert advice - Hedge algorithm
WEEK 4 Online linear optimization
WEEK 5 Online convex optimization; Online learning summary
WEEK 6 Introduction to Multi armed Bandits - EXP3
WEEK 7 Contextual MAB - EXP4
WEEK 8 Stochastic MAB, Epsilon Greedy, Explore then commit
WEEK 9 Stochastic MAB, UCB, Thompson Sampling
WEEK 10 Stochastic MAB - Linear Bandits - LinUCB algorithm; MAB summary
WEEK 11 Introduction to Reinforcement Learning - Markov Decision Process
WEEK 12 Q-learning
+ Show all weeks

About the Instructors

Arun Rajkumar
Assistant Professor, Department of Computer Sciences & Engineering, IIT Madras

I am currently an Assistant Professor at the Computer Science and Engineering department of IIT Madras. Prior to joining IIT Madras, I was a research scientist at the Xerox Research Center (now Conduent Labs), Bangalore for three years. I earned my Ph.D from the Indian Institute of Science where I worked on 'Ranking from Pairwise Comparisons'. My research interests are in the areas of Machine learning, statistical learning theory with applications to education and healthcare.


Other courses by the same instructor: BSCCS2004 - Machine Learning Foundations