CS 506 Advanced Geometric and Probabilistic Methods Spring 2017

Instructor

• Jared Saia
• Email: "last name" at cs.unm.edu.
• Office: Travelstead Room B21, phone: 277-5446 The best way to reach me is generally via email. I usually check email once a day around noon.
• Office Hours: TBA or by appointment.
• Note: I will always be available in my office during office hours. At other times, if my door is open, feel free to come in. If the door is closed, I'm probably at work on a paper, grant or research problem. Please come by another time or make an appointment via email.

Class Info

Course Description

Text Book

Class E-mail List

• I have set up a google group to manage email for this class. Directions for joining this group are here. This link also contains an archive of emails.

Office Hours

Syllabus

Prereq: CS 362 or equivalent

Homework

• HW 1
• HW 2
• HW 3

Lecture Topics

• Convex Hull
• Jeff Erickson's Notes on Convex Hull
• Convex Crepes!
• High Dimensional Convex Hull; See also Quickhull
• Duality
• UFL lecture notes on Duality
• U. Maryland Notes, Pages 41-44 give a good connection between convex hulls, and upper/lower envelopes, Lecture 8 gives good connection between envelopes and linear programming. Lecture 16 gives good connections between convex hulls and Voronoi diagrams, and Delanauy triangulations
• Multiplicative Weights Update (MWU) Method
• "The Multiplicative Weights Update Method: a Meta Algorithm and Applications" by Arora, Hazan and Kale
• Arora Notes (Lecture 9 and 10 used in class)
• Gupta's notes We'll make use of Lectures 16 and 17 here for connections between MWU and Linear Programming
• MWU vs Gradient Descent (See Chapter 2.3)
• Linear Programming Application of MWU
• Linear Programming: Gupta's notes, Lecture 17
• Adaboost Application of MWU
• Adaboost Video Shows Adaboost learning a "swirly" dataset
• Our analysis of Adaboost followed the analysis in the Aurora survey paper
• Convex Optimization
• Online Convex Optimization via MWU (Section 3.9 of the Arora survey paper)
• Arora Lecture Notes on Gradient Descent and Stochastic Gradient Descent
• Andrew Ng video on Intuition of Gradient Descent
• Another interesting video on gradient descent
• Higher Dimensional Spaces and Dimension Reduction
• Arora Notes on Johnson-Lindenstrauss Projection
• You can embed an arbitrary metric into Euclidean space with O(log n) distortion (via Bourgain's theorem, see also here). Then, you can use Johnson-Lidenstrauss to project onto R^d where d = O(log n).
• Arora Notes on SVD (re low-rank approximation)
• Arora Notes on SVD (Part 2)
• Class Summary
• Class Summary Slides

Projects

• MIT Algorithm's Projects This is a general description of how to find a good CS theory project. The specific project ideas in this class are, of course, different from our own class - if you'd like specific ideas, please talk to me.

Useful Links for this Class

• Blum, Hopcroft and Kannon Foundations of Data Science
• David Mount's Computational Geometry Notes The University of Maryland notes
• David Mount's Homework Problems The University of Maryland notes
• Suresh Venkatasubramanian's Comp. Geometry Class
• Convex Optimization Notes by Vishnoi Formal Treatment of Convex Optimization Algorithms
• MWU as Gradient Descent (Section 3):
• Convex Optimization by Boyd and Vandenberghe Particularly of interest: Section 2.3 "Operations that Preserve Convexity" ; Chapter 4 and onward discuss optimization algorithms (albeit informally, without proofs of convergence time; NB that many problems discussed (i.e. quadratic programming)are NP-Hard). See Vishnoi's notes above for a more formal treatment.