Working on Alignment: Walkthrough
To get a 3D scan of something, you take pictures all around it. Those pictures will naturally overlap. Our program helps you align these 3D pictures. Considering that properly aligning 3D snapshots is a pivot point between a good or bad 3D model, we are working this week on what the key points are in creating a good alignment the first time and what to look out for.
(We will not be focusing on color as we get these alignments done; rather, we will focus on the specific techniques for getting a colorless 3D model produced at the best match to real life.)
When aligning two scans, you need to choose distinguishing marks on the scans to match. For example, we could match the left ripple in the curtain from the first scan to the left ripple in the curtain on the second scan. The program merges those two points together. However, to make this work, you need: 1) distinguishable points, 2) enough points.
Two of our office scans were of flat, empty walls, so there were no distinguishable points to match. This makes alignment immensely difficult. The three scans we were able to merge effortlessly on the first try (see the picture below) were easy to do because there were multiple points we could easily distinguish to align the scans with (the fan blades, the curtains).
In the picture above, we had distinguishable points, like the curtains, the fan blades, and the door. However, if we marked only one of those distinguishing features, the two scans would not align very well. To align well, we mark multiple sets of points. So, in scan A, I could mark the left ripple in the curtain, the far right fan blade, and a point on the door. On scan B (which overlaps some of scan A), I would mark the same three points so the picture would align. Having enough points spread across the scan aligns the snapshots correctly.
Kinect distortion around edges
We’ve noticed that the Kinect seems to curve more near the edges of the scan. This is most likely a calibration issue with the Kinect (the factory calibration does not seem to be 100% accurate), but it could also be an issue with the infrared lighting. We will keep an eye on this to see if it’s impacting alignment, but with the surface reconstruction feature, this may end up not being a problem.
The edge distortion is a problem, though, if you align an image to it. In the image above, two scans of a wall and corner are aligned. The wall (light grey area) is aligned perfectly. However, the corner is not: you can see in the top red circle that there is a distortion–one of the scans veers off to the left, even though it should be aligned with the rest of the dark area. If you were to align a third scan to the circled distortion, it wouldn’t be aligned with the non-distorted scan. You would then align a fourth scan to the third, and a fifth to that one, and so forth, until you come full circle to align the seventh scan to the very first (remember, this is a 3D object, so we go 360° around something, requiring the end to match up to the beginning). If the third scan was aligned to a distortion, then all the following scans would be misaligned. The result: the final scan would be impossible to align to the first scan, and you couldn’t make a 3D model.
The solution: don’t align distorted scan edges to distorted scan edges. Instead, about 10% of the non-distorted area of the scans need to overlap. Align the non-distorted area.
Bad alignment indicator
In the picture above, you can see a display board has slightly different color in the middle (green circle) compared with the outside (blue circle), this is actually the two 3D snapshots that are aligned, it’s normal for the colors to weave in and out of each other like this.
However, here you can see an enlarged picture of the two 3D snapshots. Since they are snapshots of the same object, if they were aligned correctly, you would not be able to see the gap. Large gaps between layers indicates a bad alignment. Bad alignment distorts a 3D model.
End result and footnotes
The room as a whole maintained its shape, and all the data we got was merged as expected. The curtains look great; they retained their 3D pattern and lined up nicely.
Notes for quality alignment
- Groups of alignment points should be as far away from each other as possible; this increases accuracy.
- The edges of the scan have distortion and should not be used to align with.
- Scans need about 10% of good* scan area to overlap in order to get a good alignment. (*Bad would be edge of scan area)
- The majority of successful alignments are done on the first try. I found that the more I tried, the less likely the alignment would work.
- The scans need to have areas that are easy to match to: 3D features like curtain ripples, or color features like lines, or intersections of different colors. The lower quality of matching points, the less likely of aligning successfully.
The area being scanned needs to be in the center of the Kinect’s view. Scans need to capture different angles while at the same time provide enough overlap of points that are easy to match up. The points being matched up need to be spread out to lower the accuracy required for a good match. Align higher quality scans that are easier to match up first.
I’ll be applying these techniques to scanning a face tomorrow, and we will see if we can make it simple.