### Syllabus

# CS 429/529

## Course description:

Introduction to principles and practice of systems that improve performance through experience. Topics include statistical learning framework, supervised and unsupervised learning, performance evaluation and empirical methodology; design tradeoffs.

### Textbook

Most lectures will be based on the following optional textbooks:

**Deep Learning**. Ian Goodfellow, Yoshua Begio, Aaron Courville. ISBN-10: 0262035618 | ISBN-13: 978-0262035613 (optional)**Machine Learning, Tom Mitchell**. ISBN-10: 0070428077 | ISBN-13: 978-0070428072 | Edition: 1 (optional)**Pattern Recognition and Machine Learning, Christopher Bishop**. ISBN-10: 0387310738 | ISBN-13: 978-0387310732 (optional)

## POLICIES

### Academic honesty:

Unless otherwise specified, you must write/code your own homework assignments. You cannot use the web to find answers to any assignment. If you do not have time to complete an assignment, it is better to submit your partial solutions than to get answers from someone else. Cheating students will be prosecuted according to University guidelines. Students should get acquainted with their rights and responsibilities as explained in the Student Code of Conduct

Any and all acts of plagiarism will result in an immediate dismissal from the course and an official report to the dean of students.

Instances of plagiarism include, but are not limited to: downloading code and snippets from the Internet without explicit permission from the instructor and/or without proper acknowledgment, citation, or license use; using code from a classmate or any other past or present student; quoting text directly or slightly paraphrasing from a source without proper reference; any other act of copying material and trying to make it look like it is yours.

Note that dismissal from the class means that the student will be dropped with an F from the course.

The best way of avoiding plagiarism is to start your assignments early. Whenever you feel like you cannot keep up with the course material, your instructor is happy to find a way to help you. Make an appointment or come to office hours, but DO NOT plagiarize; it is not worth it!

Change of grade to CR/NC after the semester deadline will be granted ONLY under special, documented extenuating circumstances.

### Class attendance:

Attendance to class is expected (read mandatory) and note taking encouraged. Important information (about exams, assignments, projects, policies) may be communicated only in the lectures. We may also cover additional material (not available in the book) during the lecture. If you miss a lecture, you should find what material was covered and if any announcement was made.

## ASSIGNMENTS

### Projects:

Projects and homework reinforce what you learned in class by materializing abstract concepts into practical problems. Every project usually consists of a programming assignment. Tentative schedule is as follows:

- Project 1, release date Sep 5,
**due date Sep 25**(decision trees) - Project 2, release date Sep 26,
**due date Oct 16**(logistic regression) - Project 3, release date Oct 24,
**due date Nov 13**(cnns) - Project 4, release date Nov 14,
**due date Dec 7**(kaggle competition or individual project)

You can discuss projects with other classmates but all the code have to be written by you. Any student suspected of plagiarizing code will be prosecuted according to the University guidelines.

- Project reports have to be uploaded to UNM Learn, code and solution have to be uploaded through Kaggle. Late projects will be accepted only within the following policy:
- Every student has 4 free days to be used at their own discretion across the multiple projects
- Once a student has used all of his/her free days, no other late project will be accepted

### Weekly quizzes

Readings in this syllabus are mandatory, you can expect short individual quizzes every week. Those quizzes will be graded and they account for 10% of your final grade

### Daily assignments

You can expect to have simple exercises every meeting. These daily assignments will be done in groups specified by the instructor and they will account for your participation grade (10% of your final grade)

## EXAM

Exams are our formal evaluation tool. In the exams you will be tested with respect to the learning goals of this course (see the schedule below for the list of learning goals). Exams will comprise a mix of practical exercises and concepts. I don't encourage you to learn concepts and definitions by heart, but to be able to explain them with your own words and to place these concepts into the broader context they belong to. There will be one **midterm exam around October 19** and one **final exam around Nov 16**

**The exam is open notes, but only personal, hand-written notes are accepted**. Restrictions in this matter include (but are not limited): you cannot download notes from Internet, you cannot use the electronic notes of the course, and you cannot photocopy notes from your classmates. In fact, the key point is that they must be **your own hand-written notes** because I expect you to reinforce what you learned in class by writing down key concepts.

## FEEDBACK

I value student's opinions regarding the course and I will take them in consideration to make this course as exciting and engaging as possible. Thus, through the semester I will ask students formal and informal feedback. Formal feedback includes short surveys on my teaching effectiveness, preferred teaching methods, and pace of the class. Informal feedback will be in the form of polls or in-class questions regarding learning preferences. You can also leave anonymous feedback in the form of a note in my departmental mail box, under my office door (FEC 325), or using this form. Remember that it is in the best interest of the class if you bring up to my attention if something is not working properly (e.g the pace of the class is too slow, the projects are boring, my teaching style is not effective) so that I can make the corrective steps.

## GRADING

- Participation 15 pts
- Quizzes 15 pts
- Projects 40 pts
- Exams 30 pts

Grades will be based on your earned points, following this grade scale. You need to get the specified number of points or more to obtain the grade from the same column. Scores will be rounded to the closest integer value.

A A- B+ B B- C+ C C- D+ D D- F 97 95 90 85 80 77 73 70 67 63 60 <60

- Incomplete can be assigned only for a documented medical reason

## ADA:

In accordance with University Policy 2310 and the Americans with Disabilities Act (ADA), academic accommodations may be made for any student who notifies the instructor of the need for an accommodation. If you have a disability, either permanent or temporary, contact Accessibility Resource Center at 277-3506 for additional information.

## SCHEDULE

### Introduction

- Readings: Mitchell, Chapter 1

### CART

- Readings: Mitchell, Chapter 3
- Quinlan's Induction of Decision Trees
- ID3 and Overfitting
- Prunning trees via chi-square

### Probability Review

- Readings: Bishop, Chapters 1 and 2
- Paola Sebastiani's tutorial

### Bayesian vs MLE Approach

- Readings: Bishop, Chapters 1 and 2
- Tutorial on MLE estimation
- ML, MAP, and Bayesian — The Holy Trinity of Parameter Estimation and Data Prediction
- MLE vs MAP

### Naive Bayes

- Readings: Tom Mitchell's book chapter http://www.cs.cmu.edu/~tom/mlbook/NBayesLogReg.pdf

### Linear and Logistic Regression

- Readings: Bishop, Chapter 3
- Tom Mitchell's http://www.cs.cmu.edu/~tom/mlbook/NBayesLogReg.pdf.
- Linear regression notes from Mark Girolami

### Bias/Variance Tradeoff, Boosting and Bagging

- Readings: Bishop, Chapter 14

### Bayesian networks: Representation & Inference

- Readings: Bishop, Chapter 8
- A Brief Introduction to Graphical Models and Bayesian Networks. By Kevin Murphy
- Tutoral and Slides by Andrew Moore
- Understanding your data with Bayesian networks (in Python) by Bartek Wilczynski
- Python Tutorials: Tutorial 1: Creating a Bayesian Network
- https://www.aaai.org/ojs/index.php/aimagazine/article/download/918/836?

### Hidden Markov Models

### EM algorithm and Gaussian Mixture models

- Readings: Bishop, Chapter 9
- Intuitive example of EM
- A Gentle Tutorial of the EM Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models

### Dimensionality reduction, PCA, SVD

- Readings: Bishop Chapter 12
- Vincent Spruyt's tutorial
- Eigenfaces tutorial

### Data representation LSI, ICA, Autoencoders

### Introduction to Kernel functions

- Readings: Bishop Chapter 6

### Support Vector Machines

- Readings: Bishop, Chapter 7
- SVM for classification and regression
- A User’s Guide to Support Vector Machines
- Support Vector Machines and Kernels for Computational Biology

### Instance Based Learning

- Readings: Mitchell, Chapter 8

### Unsupervised Learning and Clustering

- Readings: Bishop, Chapter 9

### Learning Theory

- Readings: Mitchell, Chapter 7

### Perceptrons & Neural Networks

- Readings: Mitchell, Chapter 4
- Readings: Bishop Chapter 5
- Super awesome set of videos Neural Networks Demystified (watch all 7 parts)
- An intoduction to Neural Networks, Ben Krose and Patrick van der Smagt

### Neural Architectures

- Restricted Boltzman Machines
- https://www.oreilly.com/learning/generative-adversarial-networks-for-beginners \ GANs for beginners, Jon BrunerAdit Deshpande
- GANs with tensorflow
- Tutorial on GAns by Ian Goodfellow

### Deep Learning

- Reducing the Dimensionality of Data with Neural Networks, Hinton and Salakhutdinov
- Andrew Ng on Deep Learning
- DL Demos
- DL advances (2016)