Image Registration
Goal 1) Registration: because of subject and camera movement, different images will be shifted, rotated, and possibly scaled with respect to each other. We can think of this as a rigid-body transformation given by a per-image affine transform matrix, Ti: Yi=Ti Xi (where Yi is the "observed" image corresponding to the "true" underlying object Xi, under transform Ti.The goal of registration is to find Ti for each i so that it can be removed from Yi in order to recover the "true" image in the "master" coordinate frame. That is, if we can estimate Ti, then we can compute Xi=Ti-1 Yi, and all Xi should be represented in the same coordinate system.
It is possible to solve this problem under constraints forcing T to be a true affine transform matrix. However, in practice it works well to estimate T via simple least-squares, without additional constraints. That is, we seek the T that minimizes the squared error between T X and Y.
For the sake of this project, we can take one image, Y0 as defining the "master coordinate system" -- that is, T0=I. Then we want to align every other image in the population to that coordinate frame by solving for the necessary linear transform in a least-squares sense. After we apply the inverse transforms to each image, we get a set of images that are (close to) comparable.
There are two remaining issues, however:
[A] The data points in various images are not in the same index order in the data file, so it is not clear which points "correspond to" each other, for the purpose of the LSE solution. Indeed, there aren't even the same number of points in the various data images.
[B] Even if you knew what the coordinate transform was, the data points still wouldn't be precisely aligned in any pair of images, so you would still have a correspondance uncertainty to handle. This is not part of registration, but rather part of analysis, so we'll discuss it under variance estimation.
To solve problem [A], we usually use a subset of the image points that are "landmark" coordinates. An affine transform in Euclidean 3 space has six free parameters (three translation and three rotation) so, in principle, if you have at least two corresponding landmarks (i.e., six real values) in the two images, you should be able to solve for (an approximation to) the transform matrix. The trick is finding the landmarks.
If you are ambitious, you are welcome to work out a way to find landmarks automatically. However, our anthropology colleagues have a catlog of standard anatomical landmarks that they use, and they have been gracious enough to manually annotate the images with landmarks for us. These are available in auxilary data files. See the Data description file for more details on the landmarks.
Putting it together:
- For each image in a given data set: extract the corresponding landmarks.
- Use least-squares to estimate the transformation matrices between landmarks of image 1 and landmarks of each other image.
- Apply the inverse transform matrix to map every image in the set to image 1's coordinate frame.
Deliverables
For the registration section of the project, turn in:- Registration transform matrices for each image in the Registration/variance estimation data set, with respect to image 1 in that set, using the corresponding landmarks for estimation. (A total of seven transform matrices, assuming that image 1 has the identity transform matrix to itself.) Matrices should be saved in individual files, with identifying names, one row per line, tab-delimited columns, floating point values with at least 3 digits of precision printed.
- A single file containing a list of the determinants of each transform matrix, one per line (seven lines total); floating point values with at least 3 digits of precision printed.
- (New) An estimate of which landmarks are actually necessary for registration (alternatively: which are most valuable). The goal is to minimize manual labeling effort by identifying a (hopefully small) subset of landmarks that still enable high-quality registration. Provide your estimate of a minimal set of landmarks as well as supporting data and analysis that justifies your proposal.
