Fall 2020

Theory of Deep Learning

September 21 – December 12, 2020

Synopsis

Deep learning plays a central role in the recent revolution of artificial intelligence and data science. In a wide range of applications, such as computer vision, natural language processing, and robotics, deep learning achieves dramatic performance improvements over existing baselines and even human. Despite the empirical success of deep learning, its theoretical foundation remains less understood, which hinders the development of more principled methods with performance guarantees. In particular, such a lack of performance guarantees makes it challenging to incorporate deep learning into applications that involve decision making with critical consequences, such as healthcare and autonomous driving.

Towards theoretically understanding deep learning, many basic questions lack satisfying answers:

  1. The objective function for training a neural network is highly non-convex. From an optimization perspective, why does stochastic gradient descent often converge to a desired solution in practice?
  2. The number of parameters of a neural network generally far exceeds the number of training data points (also known as over-parametrization). From a statistical perspective, why can the learned neural network generalize to testing data points, even though classical ML theory suggests serious overfitting?
  3. From an information-theoretic perspective, how to characterize the form and/or the amount of information each hidden layer has about the input and output of a deep neural network?

Organizers

  • N. Srebro (ML, TTIC)
  • Z. Wang (OPT, NU)
  • D. Guo (IT, NU)

Events and Workshops

  • September 22, 2020: Kick-Off Social 
  • September 29, 2020: Mini-Workshop
  • October 30, 2020: Mini-Workshop
  • December 4, 2020: Mini-Workshop

Confirmed Speakers:

R. Srikan (UIUC)
Babak Hassibi (Caltech)
Andrea Montanari (Stanford)

Visit the events page events to see a full listing of workshops coming up.

Graduate Courses

The following graduate courses are coordinated to facilitate attendance by participants of the special quarter.

  • Computational and Statistical Learning Theory.
    Date and Time TBD, TTIC, Prof. Nathan Srebro.
  • Fundamentals of Deep Learning.
    Date and Time TBD, TTIC, Prof. David McAllester.
  • Information Theory and Statistics.
    Date and Time TBD, Northwestern, Prof. Zhaoran Wang.
  • Information Theory and Learning.
    Date and Time TBD, Northwestern, Prof. Dongning Guo.

Special Quarter Visitors

  • Name (Institution) 
  • Name (Institution) 
  • Name (Institution) 
  • Name (Institution) 

Calendar

April 2020

Mon Tue Wed Thu Fri Sat Sun
1
2
  • Canceled: IDEAL Quar…
3
4
5
6
7
8
9
10
11
12
13
  • Postponed: IDEAL Wor…
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30