Learning Theory#

Learning theory provides mathematical frameworks for understanding how and why machine learning algorithms work.

Topics#

• Perceptron
• Support Vector Machines
• Weighted Majority
• PAC Learning
• VC Dimension
• Ada Boost
• Random Graphs
• Fixed Point Iterations
• Empirical Risk Minimization
• Uniform Convergence
• Multi-armed Bandit
• Regret Minimization
• K-junta Problem

Resources#

Books#

• Understanding Machine Learning: From Theory to Algorithms by Shai Shalev-Shwartz, Shai Ben-David
• Foundations of Machine Learning by Mehryar Mohri, Afshin Rostamizadeh, Ameet Talwalkar
• Learning Theory from First Principles by Francis Bach

Online Resources#

• MIT: Machine Learning Theory
• Stanford: Learning Theory
• CMU: Machine Learning Theory

Practice Problems#

• Learning Theory Exercises
• Learning Theory Problems
• Learning Theory Challenges