Crash Course on Machine Learning

Simula hosted a crash course on Machine Learning in August 2016, with a series of lectures from international experts spanning from September 2016 to May 2017. The course was hosted in collaboration with MIT.

Course contents

Machine Learning (ML) is key to develop intelligent systems and analyse data in science and engineering. ML engines enable intelligent technologies such as Siri, Kinect, or Google’s self-driving car, just to mention a few examples. In addition, ML methods help deciphering information encrypted in our DNA and condense the flood of information gathered on the web, forming the basis of a new “Science of Data”. This course provides an introduction to the fundamental methods at the core of modern Machine Learning and Artificial Intelligence (AI).

The 3.5-day course will follow a modular structure allowing for participants with a busy schedule to attend the most relevant parts of the course.

The first part of the course (2 days) will focus on theoretical foundations and essential algorithms for data analysis. This part will cover foundations and recent advances in ML from a Statistical Learning and Regularization Theory point of view, which are key elements of high-dimensional learning. The presented methods allow for unified treatment of a huge class of diverse approaches, while also providing tools to design new ones.

The second part (1 day) will introduce ML as an Artificial Intelligence discipline, as it was developed chronologically. In this context, Learning is seen as the acquisition of explicitly represented domain knowledge, through the construction and modification of expressions in a formal language, such as Logical formulae or relations. This part will review basic knowledge, methods, and algorithms used in the symbolic approaches of ML by focusing on the most basic algorithms.

Classes on theoretical and algorithmic aspects will be complemented by practical lab sessions. We will illustrate the use of models throughout the course, implementing algorithms in PYTHON, and PROLOG. We will give a gentle introduction to the languages, and propose various exercises and tasks tailored to the participants’ programming skills and priorities.

The third part (0.5 day) will be devoted to discuss open problems from participants through presentations, poster presentations, and Q/A sessions. The goal of the session will be to uncover problems where ML techniques could be beneficial.

Finally, the fourth part, the so-called “Seminar and Mini-tutorial Series”, will span from Sep. 2016 until May 2017, where external experts will be invited to present their current research, perspectives of the field, and outline strategies for further development. Such seminars will be organized once a month, with half-day specialized presentations from world-class leading experts.


Prerequisites:

  • Basic probability
  • Basic calculus
  • Linear algebra

Blue form

Reading List

  1. Hastie, T., Tibshirani, R., and Friedman, J. (2013). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Series in Statistics. Springer New York. (pdf)
  2. Bousquet, O., S. Boucheron and G. Lugosi. Introduction to Statistical Learning Theory. Advanced Lectures on Machine Learning Lecture Notes in Artificial Intelligence 3176, 169-207. (Eds.) Bousquet, O., U. von Luxburg and G. Ratsch, Springer, Heidelberg, Germany (2004). (pdf)
  3. Evgeniou, T., Pontil, M., and Poggio, T. (2000). Regularization Networks and Support Vector Machines. Advances in Computational Mathematics. (pdf)
  4. F. Cucker and S. Smale. On The Mathematical Foundations of Learning. Bulletin of the American Mathematical Society, 2002. (pdf)
  5. Luger, G. (2005). Artificial Intelligence: Structures and Strategies for Complex Problem Solving. Fifth edition, Addison Wesley. (pdf)
  6. Mitchell, T. (1982). Generalization as search. Artificial Intelligence, 18(2): 203-226. (pdf)
  7. Quinlan, J. (1986). Induction of decision trees. Machine Learning, 1(1): 81-106. (pdf)
  8. Bessiere, C., Koriche, F., Lazaar, N., O’Sullivan, B. (2015) Constraint Acquisition. Artificial Intelligence, In Press. (pdf)
  9. Clocksin, W., Mellish, C. (2003) Programming In Prolog, 5th edition, Springer.(pdf)


Links and courses

  1. MIT 9.520, Statistical Learning Theory and Applications, Fall 2014.
  2. Stanford CS229, Machine Learning, Autumn 2014.
  3. Coursera, Machine Learning by Stanford.

Complementary courses at University of Oslo:

  1. INF4820 and INF 9825 Algorithms for Artificial Intelligence and Natural Language Processing: The courses provide foundational theory, with implementation in Common Lisp, concerning general techniques for searching, pattern matching, unification, knowledge representation, parsing and memoization, with particular weight given to algorithms and data structures for the analysis of natural languages.
  2. INF3490 and INF4490 Biologically inspired computing: The courses introduce self-adapting methods also called artificial intelligence or machine learning. Schemes for classification, search and optimization based on bio-inspired mechanisms are introduced. This includes evolutionary computation, artificial neural networks and more specialized approaches like e.g. swarm intelligence and artificial immune systems.