cs442/eece432: Introduction to Parallel Processing

Useful Links

MPI

This is a link to the MPI 1.1 standard. And this is a link to the MPI 2.0 standard.

The MPICH 2 and the Open MPI implementation for your laptop or desktop.

Mailing List

Computing Resources

The Center for High Performance Computing (HPC@UNM) where we will run our programs.

Request your account here (use project id 2006017).

A little information about Hammer, the machine we will be using, is available.

Books

This is the book we used last year. A table of contents and a review are available.

This is a good book to learn MPI from.

An early version of this book is available for download here.

I like Quinn's book.

Ian Foster's book on parallel programming is a little older, but still very popular. It is available online.

This one is online, available through the UNM library at books24x7. You may need to go here first and use your UNM NetID to get access.

Assignments

Assignment	Due Date	Feedback
Homework 1	February 7, 2008	Feedback hwk 1
Homework 2	February 21, 2008	Feedback hwk 2
Homework 3	March 4, 2008	Feedback hwk 3
Project 1 mandel.c main() for the project.	March 25, 2008	Feedback proj 1
Read paper for March 13 class Link to paper.	March 25, 2008
Homework 4	April 1, 2008	Feedback hwk 4
Project 2	April 15, 2008	Feedback proj 2
Homework 5	April 22, 2008	Feedback hwk 5
Homework 6	April 29, 2008
Project 3	Proposal: April 29, 2008, Project: May 8, 2008

Programs

An archive with all files is available here

Useful Utilities

A script to submit batch MPI jobs on hammer

A function to measure time with microsecond resolution

Selected Test Drivers

These are crude drivers that I use to test your homework. Link them with the appropriate function of yours.

Test ex_01 of hwk 1	Test ex_03 of hwk 1
Test ex_01 of hwk 2	Test ex_03 of hwk 2
Test ex_01 of hwk 3

Selected Solutions

Hwk2, ex_01: gsum() function using a ring

Hwk2, ex_01: gsum() function using recursive doubling

Hwk2, ex_01: gsum() function using a binary tree (thanks Lena)

Hwk2, ex_03: Synchronize two nodes

Hwk2, ex_04: Assumes left to right evaluation of tree

Hwk2, ex_04: This one uses a derived data type

Hwk2, ex_04: Uses MPI_2INT, but also assumes left to right evaluation

Hwk3, ex_01: A user-defined function to do selective sum

Hwk3, ex_02: Count ID numbers

Example Programs for Chapter 2

Example 2.1: Our first MPI program	Example 2.2: Causing an MPI abort
Example 2.3: Send a token around a ring	Example 2.4: Retrieve envelope information
Example 2.5: Ordering: one sender, one receiver	Example 2.6: Ordering: two senders, one receiver
Example 2.7: Ordering: multiple processes	Example 2.8: Ordering: matching tags
Example 2.9: Non-blocking transmission	MPI_Reduce example (not in book)

Example Programs for Chapter 3

Example 3.1: Using MPI_Bcast	Example 3.3: MPI_Reduce and MPI_MAXLOC
Example 3.4: User-defined reduce operation	Example 3.5: Distributed reduction
Example 3.7: Distributing a Lower-Triangular Matrix

Example Programs for Chapter 4

Example 4.1: Contiguous Data Type	Example 4.2: Flawed Vector Data Type
Example 4.3: Data Type Info	Example 4.4: Vector Data Type
Example 4.5: Indexed Data Type	Example 4.6: Structure Data Type
Example 4.7: Data Type Signature

Example Programs for Chapter 5

Example 5.1: Flawed Pthreads Program	Example 5.2: First Pthreads Program
Example 5.3 & 5.4: Matrix Multiply using Threads	Example 5.5: Load and Store
Example 5.6: Finding Maximum using Threads	Example 5.7: Thread Barrier

Example Programs for Chapter 8

Example 8.1: Lucky Numbers

Example 8.2: Global Sum Using Recursive Doubling

Test Patterns for the Game of Life

Test 0: All cells alive. Should die in one generation due to overcrowding	Test 1: Simple test file from assignment
Test 2: All cells dead. Should remain so.	Test 3: A blinker
Test 4: A static block	Test 5: A loaf, which is static
Test 6: A glider	Test 7: The Queen Bee Shuttle
Methuselah: Runs for 19258 gens	Infinite Growth: Keeps growing
Die658: Dies out completely after 658 gens	Rabbits: Runs for 17331 gens
Bi-gun: Fires bullets diagonally across grid	Blinker ship

Calendar

This is tentative and subject to change.

Lesson	Date	Topic	What is due
1	January 22, 2008	Class overview, introduction
2	January 24, 2008	MPI point-to-point operations
	January 29, 2008	Snow day, no class	Read Chapter 1 & 2
3	January 31, 2008	MPI p2p & collective operations
4	February 5, 2008	MPI collective operations
5	February 7, 2008	MPI advanced usage	Read Chapter 3, Hwk1
6	February 12, 2008	Finish MPI	Read Chapter 4
7	February 14, 2008	Discuss Hwk 1; Finnish MPI
8	February 19, 2008	Programming with threads	Read Chapter 5
9	February 21, 2008	Programming with threads	Hwk2
10	February 26, 2008	Parallel Computing Platforms
11	February 28, 2008	Discuss Homework 3
12	March 4, 2008	Topologies and cache coherence	Hwk3
13	March 6, 2008	Cache coherence
14	March 11, 2008	Mid-term exam
15	March 13, 2008	No class! Read and evaluate paper instead.	Assignement, paper
	March 18, 2008	Spring break
	March 20, 2008	Spring break
16	March 25, 2008	Routing, disucss hwk 3, parallel overhead	Project 1
17	March 27, 2008	Performance metrics
18	April 1, 2008	Discuss midterm	Hwk4
19	April 3, 2008	Amdahl, Karp-Flatt, Gustafson-Barsis Metrics
20	April 8, 2008	Discuss project 1 and homework 4
21	April 10, 2008	Scalability
22	April 15, 2008	Parallel program design (preliminaries)	Project 2
23	April 17, 2008	Parallel program design (decomposition)
24	April 22, 2008	Parallel program design (task interaction, mapping)	Hwk5
25	April 24, 2008	Parallel program design (task interaction, mapping)
26	April 29, 2008	Collective Algorithms, Proj 2 discussion	Hwk 6 and Proj 3 proposal
27	May 1, 2008	Collective Algorithms, Hwk 5 discussion
28	May 6, 2008	Hwk 6 discussion
29	May 8, 2008	Solving Linear Systems	Project 3
	May 13, 2008	Final exam: 8:00 -- 9:30

Meeting times and place

The class meets Tuesdays and Thursdays, 9:30 - 10:45, in Dane Smith Hall 232.

Class description

The short course description from the CS department web site:

Machine taxonomy and introduction to parallel programming. Performance issues, speed-up, and efficiency. Interconnection networks and embeddings. Parallel programming issues and models: control parallel, data parallel, and data flow. Programming assignments on massively parallel machines.

The class will be very hands-on with lots of small programming exercises and small projects. You should be a proficient programmer in a procedural language (preferably C or C++).

Office hours

I have an office in Farris Engineering Center 342 and will be there most Tuesdays and Thursdays. I'll have office hours there right after class, but talk to me, if another time works better for you.

Contacting me

The best way to reach me is via riesen@cs.unm.edu.

Textbook

We will be using a manuscript that I am working on.

Exams

There will be one in-class mid-term exam, tentatively scheduled for Tuesday, March 11. The final exam may be a take-home exam. I will decide that later in the semester.

Grading

Homework: 30%
Projects: 40%
Mid term: 15%
Final: 15%

Last updated May 8, 2008 by riesen@cs.unm.edu