Machine Learning, Course Information 2011

The course in brief

The course introduces various machine learning techniques, with a focus on natural computation methods in general and artificial neural networks in particular. Subtopics include artificial neural networks, reinforcement learning, evolutionary computing and swarm intelligence. See the formal course plan for details.

Course information on the Web

Course related information is maintained and updated under http://www.it.uu.se/edu/course/homepage/mil/vt11

Last year's course evaluation (when this was a 7.5hp credit course in period 4)

Teachers

Olle Gällmo (lectures, examinator)
Office 1256, tel: 018 - 471 10 09,
Email:

Pontus Ekberg (lab assistant)
Office 1237, tel: 018 - 471 73 41,
Email:

Joseph Scott (lab assistant)
Office 1345a, tel: 018 - 471 10 55,
Email:

Literature

  • Andries P. Engelbrecht, Computational Intelligence: An Introduction, 2nd edition, Wiley, 2007. ISBN 978-0-470-03561-0. Reading instructions will be provided for this book, and also for the 1st edition of the same book (which is also a good book, and probably cheaper, but it may be difficult to find in the shops).
  • Compendium

Lab assignments

The course includes five lab assignments:

  • Introductory lab: Introduction to machine learning (not mandatory)
  • Lab 1: Supervised learning and multilayer perceptrons
  • Lab 2: Reinforcement learning
  • Lab 3: Self organizing feature maps
  • Lab 4: Evolutionary computation and swarm intelligence

Lab assignments should be solved in groups of two students.

Assignments handed in before their respective deadlines (see table below) will be marked and returned as soon as possible. All assignments must be passed in order to pass the lab component of the course.

Note that one of the requirements for a top grade in the course as a whole is that the assignments (serious attempts) have been handed in before their respective deadlines. Assignments handed in after the last scheduled day (the second project seminar) will be accepted, but may not be marked until the end of the semester (Aug 31). Students who fail to hand in the reports before the end of the semester, fail the course.

Project assignment

In addition to the lab assignments, the course includes a larger project assignment which should also be done in groups of two students. The solution to this assignment is to be presented both as a written report, and orally at a seminar at the end of the course. Note that attendance at this seminar is mandatory for all students taking this course, not only the ones presenting that day.

Students may (and are encouraged to) define their own project assignment. Project proposals must be handed in and approved by the teacher before a specified deadline. Some examples of project assignments follow:

  • Apply machine learning to some interesting problem and analyze the result,
  • Implement and test (by simulation) a training algorithm,
  • Survey some sub-field of machine learning, i.e. search for scientific papers on a subject, summarize them, identify open and closed problems and define the 'state of the art',
  • Write an essay, in which different machine learning methods are compared to each other or to neighbouring fields, such as conventional AI, statistical methods, genetic algorithms, fuzzy logic, cellular automata, etc.

The written project report must be written in a 'scientific' manner, i.e. begin with a short abstract (of not more than 100 words) followed by an introductory section and end with a summary/conclusions and a list of references. In between, the work is described and discussed in one or several sections. Avoid making claims that you can't support with experiments, good reasoning or references to prevoius work. Allowed languages are Swedish and English.

A rough estimate of the amount of work required for the project is 2-3 fulltime weeks.

Supervision

Pontus Ekberg and Joseph Scott are the supervisors on lab assignments. They also mark and grade them. The project assignment is supervised, marked and graded by Olle, Pontus or Joseph, depending on subject.

Examination

Examination in this course is a sum of three parts; The lab assignments, the project assignment and a written exam, in total worth 10 credits.

The lab assignments are graded U (fail) or G (pass) and are worth 3 credits.

The project assignment is examined through the written project report and through the mandatory seminar. The project is graded U (fail), 3 (pass), 4 (very good) or 5 (excellent) and is worth 3 credits

The written exam is graded U, 3, 4 or 5 and is worth 4 credits.

The total grade on a complete course is usually the same as on the written exam, with one exception: A total grade 5 is only given to students who received grade 5 on the written exam, and at least grade 4 on the project, and who handed in all assignments on time.

Schedule

The course contains 17 lectures, all in English. The course material is in English. In addition to these lectures, there are five scheduled labs and two project seminars.

Preliminary contents:

(last checked against the official schedule Jan 05 2011)

(All lectures are in room 1211, and all labs in room 1312+1313)

Occasion Date Time Place Contents (preliminary) Chapter (*)
Lecture 1 Jan 17 13-14 P1211 Mandatory! Registration. Introduction to the course and the labs
Intro lab Jan 17 14-17 P1312D, P1313D Introduction to neural networks
Lecture 2 Jan 19 13-15 P1211 Introduction to artificial neural networks. Main areas. Some history. 1
Lecture 3 Jan 24 13-15 P1211 Applications. Pattern recognition and perceptrons. Nearest neighbour classifiers. Distance measures. More history. Perceptrons and the Perceptron Convergence Procedure. 2
Lecture 4 Jan 26 15-17 P1211 Deriving learning rules in general. The delta rule. Multilayer perceptrons. The credit assignment problem. Back propagation. 3
Lecture 5 Jan 31 13-15 P1211 Capacity of the MLP. Variants and extensions. RPROP. Automatic dimensioning. 7
Lab 1 Feb 3 8-12 P1312D, P1313D Supervised learning and multilayer perceptrons
Lecture 6 Feb 8 15-17 P1211 Practical issues: Pre-processing. Data representation. Exploiting prior knowledge. 7
Lecture 7 Feb 9 13-15 P1211 Reinforcement Learning 1: The basic problem and its history. Definitions. MENACE. 6
Lecture 8 Feb 14 13-15 P1211 Reinforcement Learning 2: Temporal Difference Learning. Q-Learning. 6
Lab 2 Feb 16 13-17 P1312D, P1313D Reinforcement learning
Deadline Feb 17 17.00 Lab 1 to be handed in
Lecture 9 Feb 21 13-15 P1211 Self organization and clustering. Competitive learning. Self Organizing Feature Maps. 4
Lecture 10 Feb 23 13-15 P1211 Radial Basis Functions 5
Lecture 11 Feb 28 13-15 P1211 Growing Neural Gas
Deadline Mar 2 17.00 Lab 2 to be handed in
Lab 3 Mar 3 13-17 P1312D, P1313D Self-organizing feature maps
Lecture 12 Mar 7 13-15 P1211 Evolutionary Computing, in general and genetic algorithms 8,9
Lecture 13 Mar 9 13-15 P1211 Evolutionary Computing continued. Genetic Programming 10
Lecture 14 Mar 11 10-12 P1211 Swarm Intelligence 1: General, Cellular Automata, Ant Colony Optimization 16, 17
Lecture 15 Mar 14 13-15 P1211 Swarm Intelligence 2: Particle Swarm Optimization 16
Deadline Mar 17 17.00 Lab 3 to be handed in
Lab 4 Mar 23 13-17 P1312D, P1313D Evolutionary computation and swarm intelligence
Lecture 16 Mar 28 13-15 P1211 Project information
Deadline Apr 6 17.00 Lab 4 to be handed in
Lecture 17 Apr 11 13-15 P1211 We will go through one of the written exams from last year. Also, time for questions (of general interest) on the lab course.
Deadline Apr 11 17.00 Project proposals to be handed in
Written exam Apr 15 8.00-13.00 Polacksbacken
Deadline May 27 17.00 Project reports to be handed in
Project seminars May 30 10-17 P1211 Mandatory! You talk, we sit, for once.
Project seminars May 31 10-17 P1211 Mandatory! You talk, we sit, for once.
Written reexam June 10 8.00-13.00 Polacksbacken For those who failed or missed the first exam
Written reexam Aug 24 14.00-19.00 Polacksbacken For those who failed or missed the first exams

(*) The Chapter column is a very rough guideline to which chapter in the 2nd edition of the book is the most relevant to each lecture. The lectures are not based on the book, however, since the book is fairly recent, so there is no one-to-one correspondence. See the reading instructions for more detailed instructions and comments to the book.