Skip to content

Welcome

Welcome to Spring 2025 offering of Deep Reinforcement Learning course at Sharif University of Technology! We are excited to have you join us on this journey into the world of deep reinforcement learning.

Course Description

This course provides an in-depth introduction to the field of deep reinforcement learning. Initially, we will explore reinforcement learning conceptually and practically to help you grasp the fundamental concepts. This phase will take place before Nowrouz. After Nowrouz, we will delve deeper into the subject, focusing on advanced topics. The course will cover both classical reinforcement learning and deep reinforcement learning, including interesting topics such as multi-agent RL, offline methods, and meta RL. By the end of the course, you will have a solid understanding of how to apply deep reinforcement learning to solve complex problems in various domains.

Learning Objectives

  • Understand the fundamentals of reinforcement learning
  • Apply reinforcement learning to various domains
  • Use deep learning techniques to handle large state spaces in RL
  • Master the concepts and gain practical understanding of RL
  • Gain hands-on experience with important RL problems
  • Equip students with enough theoretical knowledge to understand research papers

Instructor

Guests

Schedule

Conceptual/Practical

Week #
Topic of the Week
Lecture 1
Lecture 2
Homework
Week 1 Introduction to RL
۲۱ بهمن
(February 9)
۲۳ بهمن
(February 11)
 HW 1
Week 2 Value-Based Methods
۲۸ بهمن
(February 16)
۳۰ بهمن
(February 18)
 HW 2
Week 3 Policy-Based Methods
۵ اسفند
(February 23)
۷ اسفند
(February 25)
 HW 3
Week 4 Advanced Methods
۱۲ اسفند
(March 2)
۱۴ اسفند
(March 4)
 HW 4
Week 5 Model-Based Methods
۱۹ اسفند
(March 9)
۲۱ اسفند
(March 11)
 HW 5
Week 6 Multi-Armed Bandits
۲۶ اسفند
(March 16)
۲۸ اسفند
(March 18)
 HW 6

In Depth/Theoritical

Week #
Topic of the Week
Lecture 1
Lecture 2
Homework
Week 7 Value-Based Theory
۱۷ فروردین
(April 6)
۱۹ فروردین
(April 8)
 HW 7
Week 8 Policy-Based Theory
۲۴ فروردین
(April 13)
۲۶ فروردین
(April 15)
 HW 8
Week 9 Advanced Theory
۳۱ فروردین
(April 20)
۲ اردیبهشت
(April 22)
 HW 9
Week 10 Exploration Methods
۷ اردیبهشت
(April 27)
۹ اردیبهشت
(April 29)
 HW 10
Week 11 Imitation & Inverse RL
۱۴ اردیبهشت
(May 4)
۱۶ اردیبهشت
(May 6)
 HW 11
Week 12 Offline Methods
۲۱ اردیبهشت
(May 11)
۲۳ اردیبهشت
(May 13)
 HW 12
Week 13 Multi-Agent Methods
۲۸ اردیبهشت
(May 18)
۳۰ اردیبهشت
(May 20)
 HW 13
Week 14 Hierarchical & Meta RL
۴ خرداد
(May 25)
۶ خرداد
(May 27)
 HW 14

Guest Lectures

Week #
Topic of the Week
Lecture 1
Lecture 2
Homework
Week 15 Guest Lectures
۱۱ خرداد
(June 1)
۱۳ خرداد
(June 3)
 -

Logistics & Policies

  • Lectures: Held on Sundays and Tuesdays from 1:30 PM to 3:00 PM in room 201 of the CE department.

  • Recitation Classes: Weekly sessions where TAs review the last two lectures and solve related problems. These sessions will be held in person on Wednesdays, except for week 15 when there will be no recitation class.

  • Homework: Will be released on Sunday. Due dates will be provided in the following table.

Homework Release Date Due Date Details
HW1-5 Sunday
of the week		 Sunday
of next week		 @ 11:59 PM
HW6
۲۶ اسفند
(March 16)
۱۷ فروردین
(April 6)
 @ 11:59 PM
HW7-9 Sunday
of the week
۲۱ اردیبهشت
(May 11)
 @ 11:59 PM
HW10-11 Sunday
of the week
۴ خرداد
(May 25)
 @ 11:59 PM
HW12-14 Sunday
of the week
۲۵ خرداد
(June 15)
 @ 11:59 PM

  • Homework Bonus: Some homeworks may have an optional bonus part that can earn you up to 0.75 bonus points.

  • Slack Days: You have a total of 14 slack days throughout the course with no penalty for submitting your homework late. For each homework, you can use up to 7 slack days. After 7 days, the solution will be released, and no further submissions will be accepted. Any additional delays beyond the slack days will result in a 0.5% reduction in the assignment grade for every hour of delay. We have a flat reduction policy from 3 AM to 11 AM (for your convenience to rest peacefully!). The 7 days for submitting your work for each homework is a hard deadline, and after that, you will receive a 0 grade because we will release the solution to the homework.

    For the 6th homework and the last homeworks (12th, 13th, and 14th), due to the midterm exam and final exam, the solutions will be released on 1404/01/20 [۲۰ فروردین] (April 9) and 1404/03/30 [۳۰ خرداد] (June 20), respectively. You have a 3-day hard deadline for the 6th homework and a 5-day hard deadline for the last homeworks.

  • Workshop Classes: Held for all weeks except weeks 7, 8, 9, and 15. These workshops will present practical implementations of the ideas covered in the lectures of the week. These sessions will be held online on Wednesdays.

  • Lecture Summaries and Quizzes: Summaries of the previous lecture will be released at 8:00 AM on the day of the next lecture. You must participate in a quiz before the start of the lecture at 1:30 PM. Participation in quizzes will earn you 0.75 bonus points.

  • Exams: Midterm questions will focus on conceptual understanding, while the final exam will be more theoretical.

  • Poster Session: There will be a poster session at the end of the course. Presenting at the poster session can earn you 1 point of course credit, with the ability to get an additional 0.25 bonus credit for extra work.

  • Feedback: Participation in all feedback sessions throughout the course will add up to 0.75 bonus points.

  • Prerequisite: Prerequisite classes will be held based on demand. A form will be released for each session, and we will decide to hold it based on your responses.

  • Journal Clubs: Journal clubs will be held weekly throughout the course. Their schedule and details will be announced. Participating in each of them can give you 0.1 bonus points, up to 0.5.

  • Course Calendar: Office hours, lecture schedules, recitations, workshops, deadlines, and all important events can be found on the course calendar.

  • Support: You can ask questions on Telegram Group or schedule office hours with a TA on the calender for additional guidance.

  • Optional Activities: There will be an optional visit to Taarlab in the middle of the course, and maybe a few more fun and inspiring activities that we will announce throughout the course! We are full of surprises this semester 🚀

Grading

The grading for the Deep Reinforcement Learning course is structured as follows:

Main Components

  • Homeworks: Gradual assessment through regular assignments
  • Midterm: Conceptual understanding tested mid-course
  • Final: Theoretical knowledge evaluated at the end of the course
  • Poster Session: Presentation at the end of the course
Component Points Date Details
Homeworks 7 - 14 HWs \(\times \approx\) 0.5 each
Midterm 5
۲۱ فروردین
(April 10)
 @ 9:00 AM
Final 7
۱ تیر
(June 22)
 @ 8:00 AM
Poster Session 1 End of course TBA

Bonus Components

Additional opportunities to earn bonus points:

Component Points
Quizzes 0.75
Feedback 0.75
Homeworks Bonus 0.75
Poster Session Bonus 0.25
Journal Clubs 0.5

Total possible points: 20 + 3 = 23

Head Assistants

Teaching Assistants

Acknowledgements

We would like to express our gratitude to the following individuals for their invaluable contributions to the Spring 2024 and 2023 offerings of this course. Their efforts have been instrumental in the development and success of this course.

Spring 2024
Alireza Ghahremani Alireza Sakhaei Rad Amirhossein Mohammadpour Azari Amirmohammad Izadi Arian Ahadinia Armin Behnamnia Armin Saghefian Behnia Soleimani Hossein Jafariniya Mahdi Ghaznavi Mohammadhassan Alikhani Ramtin Moslemi
Spring 2023
Ali Kaheh Amirhossein Mesbah Ayda AfsharMohammadian Bardia Mohammadi Hossein Hassani Hossein Khalili Kiana Asgari Mohammad Mozaffari Negin Hashemi Parsa Haghighi Roozbeh Razavi Sepehr Ghabadi Seyed Abolfazl Rahimi Seyed Mohammad Hosseini Seyed MohammadHossein Mousavi Soroush Vafaie Tabar

This offering and all of these changes are thanks to their effort in starting this course.