Crash Course on Machine Learning

About the Course:

Machine Learning (ML) is key to develop intelligent systems and analyse data in science and engineering. ML engines enable intelligent technologies such as Siri, Kinect, or Google’s self-driving car, just to mention a few examples. In addition, ML methods help deciphering information encrypted in our DNA and condense the flood of information gathered on the web, forming the basis of a new “Science of Data”. This course provides an introduction to the fundamental methods at the core of modern Machine Learning and Artificial Intelligence (AI).

The 3.5-day course will follow a modular structure allowing for participants with a busy schedule to attend the most relevant parts of the course.

The first part of the course (2 days) will focus on theoretical foundations and essential algorithms for data analysis. This part will cover foundations and recent advances in ML from a Statistical Learning and Regularization Theory point of view, which are key elements of high-dimensional learning. The presented methods allow for unified treatment of a huge class of diverse approaches, while also providing tools to design new ones.

The second part (1 day) will introduce ML as an Artificial Intelligence discipline, as it was developed chronologically. In this context, Learning is seen as the acquisition of explicitly represented domain knowledge, through the construction and modification of expressions in a formal language, such as Logical formulae or relations. This part will review basic knowledge, methods, and algorithms used in the symbolic approaches of ML by focusing on the most basic algorithms.

Classes on theoretical and algorithmic aspects will be complemented by practical lab sessions. We will illustrate the use of models throughout the course, implementing algorithms in PYTHON, and PROLOG. We will give a gentle introduction to the languages, and propose various exercises and tasks tailored to the participants’ programming skills and priorities.

The third part (0.5 day) will be devoted to discuss open problems from participants through presentations, poster presentations, and Q/A sessions. The goal of the session will be to uncover problems where ML techniques could be beneficial.

Finally, the fourth part, the so-called “Seminar and Mini-tutorial Series”, will span from Sep. 2016 until May 2017, where external experts will be invited to present their current research, perspectives of the field, and outline strategies for further development. Such seminars will be organized once a month, with half-day specialized presentations from world-class leading experts.


Prerequisites:

  • Basic probability
  • Basic calculus
  • Linear algebra

PDF iconCalculus Review


Schedule

Class

Day

Time

Instructor

Subject

D1.a

Mon 22

09:30-11:00

 

Valeriya Naumova

Introduction to Machine Learning

D1.b

11:30-13:00

Supervised Learning: Local Methods and Model Selection. Regularization I

D1.c

14:00-18:00

Lecture-Discussion: Advice for Applying Machine Learning

Laboratory: Local Methods for Classification. Regularization

 

18:00-20:00

 

Social event at Simula

D2.a

Tue 23

09:30-11:00

Valeriya Naumova

Supervised Learning: Regularization II

D2.b

11:30-13:00

Unsupervised learning: Dimensionality reduction, Variable Selection.

D2.c

 

14:00-18:00

Lecture-Discussion: Machine Learning Applications.

Laboratory: Regularization II. PCA and Sparsity

 

D3.a

Wed 24

09:30-11:00

Arnaud Gotlieb

Basics in Logic

Programming in Logic

Decision Tree Induction

D3.b

11:30-13:00

Version Space Search

Constraint Acquisition

D3.c

14:00-18:00

Laboratory session: Symbolic ML in PROLOG

D4.a

Thu 25

 

 

09:30-11:00

 

ALL

 

Open problems 1


Lab sessions:

If you are signed up for the laboratory session(s), please don't forget to bring your laptop with a working Python installation for Monday’s and Tuesday’s sessions, and SICStus Prolog 4.3 downloaded and installed for Wednesday’s sessions.

Please ensure that you have installed matplotlib and scipy packages in Python. SICStus Prolog 4.3 can be downloaded from this webpage.

Each registered participant should have received both download and installation temporary codes for SICStus Prolog 4.3. In case of problem with the installation, please refer first to the SICStus Prolog webpages. In case of problem with the installation codes, contact us.

You are free to use either Møterommet, Klasserommet, Storstua, or the Management meeting room, as well as any other common space at Simula for the lab sessions.


Confirmed guest speakers 

Jean-Luc Starck,

Director of Research and head of the CosmoStat Laboratory, France

(September 2016)

Expertise: application of statistical methods to cosmology

 

Massimo Fornasier,

Technical University Munich, Germany

(October 2016)

Expertise: sparse modeling, compressed sensing.

 

Holger Rauhut

RWTH Aachen University, Germany

(October 2016)

Expertise: applied Harmonic Analysis, compressive sensing, mathematical signal processing, random matrices, convex optimization, and approximation theory.

 

Lorenzo Rosasco,

MIT and University of Genova, Italy

(January 2017)

Expertise: learning and optimization; machine learning methods for cognitive and perceptual tasks in robotics; theory and algorithms for learning data representation.

 

Blue form

Reading List

  1. Hastie, T., Tibshirani, R., and Friedman, J. (2013). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Series in Statistics. Springer New York. (pdf)
  2. Bousquet, O., S. Boucheron and G. Lugosi. Introduction to Statistical Learning Theory. Advanced Lectures on Machine Learning Lecture Notes in Artificial Intelligence 3176, 169-207. (Eds.) Bousquet, O., U. von Luxburg and G. Ratsch, Springer, Heidelberg, Germany (2004). (pdf)
  3. Evgeniou, T., Pontil, M., and Poggio, T. (2000). Regularization Networks and Support Vector Machines. Advances in Computational Mathematics. (pdf)
  4. F. Cucker and S. Smale. On The Mathematical Foundations of Learning. Bulletin of the American Mathematical Society, 2002. (pdf)
  5. Luger, G. (2005). Artificial Intelligence: Structures and Strategies for Complex Problem Solving. Fifth edition, Addison Wesley. (pdf)
  6. Mitchell, T. (1982). Generalization as search. Artificial Intelligence, 18(2): 203-226. (pdf)
  7. Quinlan, J. (1986). Induction of decision trees. Machine Learning, 1(1): 81-106. (pdf)
  8. Bessiere, C., Koriche, F., Lazaar, N., O’Sullivan, B. (2015) Constraint Acquisition. Artificial Intelligence, In Press. (pdf)
  9. Clocksin, W., Mellish, C. (2003) Programming In Prolog, 5th edition, Springer.(pdf)


Links and courses

  1. MIT 9.520, Statistical Learning Theory and Applications, Fall 2014.
  2. Stanford CS229, Machine Learning, Autumn 2014.
  3. Coursera, Machine Learning by Stanford.

Complementary courses at University of Oslo:

  1. INF4820 and INF 9825 Algorithms for Artificial Intelligence and Natural Language Processing: The courses provide foundational theory, with implementation in Common Lisp, concerning general techniques for searching, pattern matching, unification, knowledge representation, parsing and memoization, with particular weight given to algorithms and data structures for the analysis of natural languages.
  2. INF3490 and INF4490 Biologically inspired computing: The courses introduce self-adapting methods also called artificial intelligence or machine learning. Schemes for classification, search and optimization based on bio-inspired mechanisms are introduced. This includes evolutionary computation, artificial neural networks and more specialized approaches like e.g. swarm intelligence and artificial immune systems.