Machine Learning

Spring 2011: Machine Learning

General Information

Time: Wednesdays 12:00-3:00 PM	Place: CBIM 22
Instructor: Tina Eliassi-Rad	Office hours: Wednesdays 5:30-6:30 PM in CBIM 08
TA: Chetan Tonde	TA office hours: Tuesdays 1:00-3:00 PM in Hill 402
Course number: 16:198:536	Credits: 3

Overview

This graduate-level course introduces the theory, algorithms, and applications of machine learning. Topics covered include supervised learning, unsupervised learning, semi-supervised learning, learning theory, and reinforcement learning.

Prerequisites: Calculus and linear algebra. An introductory course on statistics and probability. Algorithms and programming (MATLAB).

Textbooks

(Required) Christopher Bishop, Pattern Recognition and Machine Learning. ISBN 0387310738.
(Recommended) Tom Mitchell, Machine Learning. ISBN 0070428077.
(Recommended) Trevor Hastie, Robert Tibshirani and Jerome Friedman, Elements of Statistical Learning. ISBN 0387952845.

Resources

Statistical Data Mining Tutorials (Andrew Moore, Google/CMU)

Theoretical CS Cheat Sheet (Princeton)

Grading

Homework assignments (3×10%)

In-class exam (30%)

Class project (40%)

Proposal report (10%) -- 2 pages plus 5-minute pitch

What is the problem?
Why is it interesting and important?
Why is it hard? Why have previous approaches failed?
What are the key components of your approach?
What data sets and metrics will be used to validate the approach?

Class presentation (15%) -- 20 minutes presentation
Final report (15%) -- 8 pages max

For guidance on writing the final report, see slide 70 of Eamonn Keogh's KDD'09 Tutorial on How to do good research, get it published in SIGKDD and get it cited!
Follow ACM formatting guidelines

Notes, Policies, and Guidelines

We will use the class sakai site for announcements, assignments, and your contributions.

Programming exercises will be in MATLAB. Rutgers holds a site license for MATLAB. You can download MATLAB to your computer from the university's software portal. MATLAB is also installed on the CS machines. Just type "matlab" at the prompt.

Homeworks must be done individually. Late homeworks are accepted up to 4 days after the deadline. A penalty of 20% be charged for each late day.

The class project can be done either individually or in groups of two.

Any regrading request must be submitted in writing and within one week of the returned material. The request must detail precisely and concisely the grading error.

Refresh your knowledge of the university's academic integrity policy and plagiarism . There is zero-tolerance for cheating!

Schedule / Syllabus (Subject to Change)

Date	Content	Readings	Notes
Jan 18	Introduction Decision Trees Naive Bayes Logistic Regression Generative vs. Discriminative Classifiers	Bishop 1.3, 1.4, 1.6, 14.4, Mitchell 1, 3, plus follow links in the previous cell. Optional: Discriminative vs. Generative Models, Guestrin's NB.
Jan 25	Linear Regression Regularization Bias-Variance Tradeoff Overfitting Cross-Validation	Bishop 3, 4, Optional: HTF 3, 4.
Feb 1	Learning Theory PAC Learning VC Dimension	Mitchell 7, HTF 7, COLT survey, Generalization Bounds, Schapire's Theoretical ML. Optional: Online Learning.	Guest Lecturer: Chris Mesterharm
Feb 8	Kernel Methods Support Vector Machines (SVM-1, SVM-2)	Bishop 6.1, 6.2, 6.3, 7.1 plus follow links in the previous cell. Optional: Bishop 6.4, HTF 6, 12.	HW#1 Out
Feb 15	Algorithms to Affect Influence Propagation on Large Graphs Heterogeneity Meets Rarity: Mining Multi-Faceted Diamond	Follow links in the previous cell.	Guest Speakers: Hanghang Tong & Jingrui He
Feb 22	Perceptron Neural Networks	Bishop 4.1.7, 5.1, 5.2, 5.3, 5.5. Optional: Mitchell 4, HTF 11.	HW #1 Due
Feb 29	Graphical Models	Bishop 8, Bayesian Networks. Optional: Mitchell 6, CRF, HMM, Ghahramani's HMM & BN, Bishop 13.1, 13.2, 11.
Mar 7	Ensemble Methods (Boosting, Random Forests) Active Learning In-class Project Pitches	Bishop 14.1, 14.2, 14.3 plus follow links in the previous cells. Optional: HTF: 10, 15, 16	HW #2 Out Project Pre-proposals Due
Mar 14	No Class -- Spring Break
Mar 21	K-means Expectation Maximization Mixture of Gaussians	Bishop 9.1, 9.2, 9.3, 9.4. Optional: HTF 14; x-means, k-means++
Mar 28	Dimensionality Reduction (PCA, ICA, CCA)	Bishop 12 plus follow links in the previous cell. Optional: M.E. Wall, et al.'s PCA	HW #2 Due
Apr 4	Anomaly Detection In-class Exam	Follow links in the previous cell.	HW #3 Out
Apr 11	Semi-Supervised Learning Transfer Learning Probabilistic Topic Models	Follow links in the previous cell. Optional: Co-training.
Apr 18	Reinforcement Learning Large-Scale Machine Learning	Follow links in the previous cell. Optional: Mitchell 13.	Guest Lecturer: Michael Littman HW #3 Due
Apr 25	In-class Project Presentations
May 2	No Class		Project Reports Due