Resources
CS591: Web Resources
Math and Miscellaneous Other Resources
- A terse but useful reference page on linear algebra, including a
great section on calculus identities (e.g., how to take the derivative
of a determinant or inverse of a matrix):
Matrix Reference Manual - Another nice set of notes on the calculus of matrices:
Some vector algebra and the generalized chain rule - A general math dictionary. It has entries on an amazing breadth
of subjects, though of varying depth on each. In general, it's not
enough to replace a set of good subject-specific reference books, but
it does give enough detail to get you oriented on a subject,
and it even occasionally contains a directly useful result:
Eric Weisstein's World of Mathematics (formerly, Eric's Math Treasure Trove). - A compendium of probability distributions. Lists more
distributions than many textbooks do, though gives only superficial
detail in each. I haven't examined the statistical software package
that the author is promoting, but the compendium itself has come in
useful a few times:
Compendium of probability distributions - An extended chapter on the properties of the multivariate
Gaussian. With respect to this class, probably the most useful
section is 11.5 on the derivation of the MLE for the multivariate
Gaussian:
The Multivariate Gaussian - A brief note on a graphical test for evaluating the normalcy of
data. (Part of a larger online statistics handbook.):
The Normal Probability Plot
Bayesian Nets (aka belief nets, graphical models,causal networks, etc.)
-
Pretty reasonable and accessible tutorial on Bayes Nets:
http://www.cs.berkeley.edu/~murphyk/Bayes/bayes.html -
The Charniak paper we mentioned in class,"Bayesian Networks without
Tears":
http://www.kddresearch.org/Resources/Papers/Intro/notears.pdf - A nice set of BN resources on Alex Dekhtyar's page at U of
Kentucky:
http://www.cs.engr.uky.edu/~dekhtyar/dblab/resources.html#bnets - An overview on inference in Bayes Nets, including the junction
tree algorithm. The original article is online via Elsevier Science,
but UNM doesn't have a subscription (at $900/journal, go figure.)
OTOH, you might be able to get a copy via the CiteSeer page:
Huang, C. and Darwiche, A., "Inference in Belief Networks: A Procedural Guide", International Journal of Approximate Reasoning, 15(3), 1996. pp. 225-263.
Support Vector Machines, Optimal Hyperplanes, and Kernel Methods
- The canonical introduction to SVMs. Pretty stiff going
(especially after the first few pages), but contains a lot of good
material:
http://www.kernel-machines.org/papers/Burges98.ps.gz - A general page on Kernel Methods, including notes on various
conferences, publications venues, etc. Has a good page of tutorial
papers (including the above):
http://www.kernel-machines.org/ - The webpage for a book on kernel methods. This is more advanced
material even than Burges, but it's good stuff by some leaders in the
field. Only some of the chapters are online, but you may be able to
get the full book or access it electronically through our library:
Learning With Kernels
Inductive Logic Programming
- Quinlan, R.
"Learning First-Order Definitions of Functions", JAIR vol 5, 139-161,
October 1996.
- Muggleton, S. and de Raedt, L., "Inductive Logic Programming: Theory and Methods." Journal of Logic Programming, 19(20), 1994.
- Flach, P.A., "The logic of learning: a brief introduction to Inductive Logic Programming". Proceedings of the CompulogNet Area Meeting on Computational Logic and Machine Learning, 1998.
Time-Series Analysis and Reinforcement Learning
- Angluin's original paper on learning DFAs:
Angluin, D., "Learning Regular Sets from Queries and Counterexamples", Information and Computation, Vol. 75, pp 87-106 (1987).
I don't believe that this paper is available online, but if you'd like to read it, you can come make a copy of mine. - The
canonical paper on Hidden Markov Models (HMMs) is:
Rabiner, L. R., "A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition", Proceedings of the IEEE, 77(2), pp 257-286 (February 1989).
Rabiner has also released a set of errata for this paper.
Thanks to a number of students in the F'2003 ML class for locating this web resource! - Another useful reference on HMMs is:
Rabiner, L. and Juang, B., Fundamentals of Speech Recognition. Prentice-Hall, 1993.
This text is, of course, focused on speech recognition and covers many more techniques than HMMs, but it does have an extended section on them. - A couple of useful references on reinforcement learning and Markov
decision processes
are:
Kaelbling, L. P., Littman, M. L., and Moore, A. W. "Reinforcement Learning: A Survey," Journal of Artificial Intelligence Research, Vol. 4, 1996.
Sutton, R. S. and Barto, A. Reinforcement Learning: An Introduction. MIT Press, 1998. (Conveniently, the entire text of the book is online.) - Finally, if we have time, we'll discuss partially observable MDPs
(a.k.a., POMDPs). Some of this topic is discussed in Sutton and
Barto's text, but an additional reference that focuses exclusively on
this subject is:
Kaelbling, L. P., Littman, M. L., and Cassandra, A. R. "Planning and Acting in Partially Observable Stochastic Domains," Artificial Intelligence, Vol. 101, (1998).
Journals, Conferences, Mailing Lists and related resources
Here are some of the top places that ML publications appear and that people discuss the subject.
- Leading paper journal. This page is mostly info for authors, but
has links to the publisher's web site which has electronic abstracts
and maybe accessible recent articles for UNM:
Machine Learning (MLJ) - Leading online journal. Publishing a lot of cutting-edge stuff
these days and prides itself on high standards and fast turnaround:
Journal of Machine Learning Research (JMLR) - The general meeting of ML researchers. There's no single overview
page, but this one lists a number of the recent meetings:
ICML 2002 - Another popular conference that is currently heavily involved with
some of the same statistical techniques we're using in class:
Conference on Uncertainty in Artificial Intelligence (UAI) - If you're into really big data sets, then Knowledge
Mining and Data Discovery (KDD) is for you:
Conference on Knowledge Discovery and Data Mining (KDD) - Though it's a much more general conference (originally concerned
primarily with neural functions and representations but now expanded
to address a wide variety of related topics), a number of ML
researchers currently publish at NIPS:
Conference on Neural Information Processing Systems (NIPS) - And, of course, the general premier meetings of AI researchers
have ML tracks:
American Association for Artificial Intelligence (AAAI)
The International Joint Conferences on Artificial Intelligence (IJCAI)
Other Stuff
This is the category for mailing lists, announcements, miscellaneous overview web pages, course pages, and other stuff that's relevant to ML but doesn't have a home elsewhere on this page.
- Andrew Moore (Carnegie Mellon) has some nice tutorials online:
http://www-2.cs.cmu.edu/~awm/tutorials - Leslie Kaelbling's research group at MIT has the "Statistical AI
Reading (STAIR) group":
http://www.ai.mit.edu/events/talks/stair/
