Target point cloud extraction from background

In this post, I will introduce how I extract target (human body) point cloud from background.
Here is a flow chart describes the process:

Figure 1 Flow chart for point cloud extraction.

In order to align cloud 1 and cloud 2 together, I perform two transformation on the cloud 1. The rough transformation used pre-calibration information which show the relative position between two camera. As the calibration was not perfect, the two cloud did not align perfectly. I apply ICP (iterative closest point) to cloud 1 so I got a perfect aligned cloud and corresponding transformation. In later process, I used this transformation from ICP directly rather than compute it at each iteration.

The combined cloud still contain target and background so far. As the point cloud is not aligned with camera coordinate, it is not intuitive to crop the cloud directly. I apply another transformation to the combined cloud manually so the horizontal plane of point cloud is aligned with x-z plane (can be other orthogonal plane) in camera coordinate. Then the point cloud is cropped at x, y and z coordinate in sequence. The result cloud after the cropping is our target point cloud.
Here are some results:

Figure 2 Point cloud of body model (rotated around)

Figure 3 Yang's model

Figure 4 Lin's model

Although left and right camera gave colour image in different brightness, we can see the alignment result was promising.

Future Improvements:
1. The transformation from object coordinate to camera coordinate was fine manually. It can be found automatically by finding a plane (table/ ground) of the scene then calculate the transformation from the plane to orthogonal plane in camera coordinate.
2. The cropping threshold was find manually as well. In future, I can detect the landmark (boundary) of bed, and therefore get the threshold.

The model shown above can be downloaded here: bodyModel.ply; Yang.ply; Lin.ply
You can view these models in MeshLab directly.