Experiments on interference problem of two Kinects

In this post, I will show the results of two experiments that can mitigate the cross-talk between two Kinects. One is time multiplexing method and another is to polarize infrared light by polarizing film.

- Time multiplexing method
As mentioned in another post, MS Kinect SDK provides only function (NuiSetForceInfraredEmitterOff) to turn the IR emitter on/off without knowing the speed. Nevertheless, it is worth experimenting the performance of this function.
The idea is turn off another Kinect's IR emitter while current Kinect is capturing depth frame. The codes is like this:

 // -- process 1st kinect
 kinect2->turnOffIR(true);
 kinect1->processDepthFrame();
 kinect2->turnOffIR(false);
 kinect1->processColorFrame();
 kinect1->processPointCloud(); 
 // -- process 2nd kinect
 kinect1->turnOffIR(true);
 kinect2->processDepthFrame();
 kinect1->turnOffIR(false);
 kinect2->processColorFrame();
 kinect2->processPointCloud();

The turnOffIR is a member function of depthSensor class that calls the NuiSetForceInfraredEmitterOff.
The result from above code is unstable because insufficient time for SDK to turn off the IR emitter before the depth frame is retrieved. Therefore, I add different delays after calling the function to turnOffIR and the result in shown below.

Figure 1 No delay.

Figure 2 100 ms delay.

Figure 3 150 ms delay.

Figure 4 200 ms delay.

 As the duration of delay is increasing, less holes caused by the interference. The best and most stable result can be achieved with about 200 ms delay per Kinect. For smaller delay, such as 50 ms, the point cloud become unstable.

Figure 5 50 ms delay.

The Figure 5 shows the point cloud captured in different time with same delay. The quality of the point cloud varying a lots. I guess that is because the amount of time required for turning off the IR emitter is different, so sometimes the IR emitter is turned off but sometimes it doesn't.
For the 200 ms delay, the frame rate of point cloud is decreased to roughly 2.5 fps (400 ms each frame).

- Polarization method
The idea is to distinguish the IR lights from different emitters by polarizing them in different direction. The Figure 6 shows the principle of the polarization of light. From the upper right diagram we can see letter 'A' easily  as two polarizing films are placed in same direction. The lower left diagram shows that the light is blocked completely when one film is rotated 90 degrees from another.

Figure 6 Polarization using two polarizing films. 

Finally, each Kinect is being attached two films with same direction of polarization, as shown in Figure 7. The other Kinect has two films with 90 degrees rotation. Hence, the IR camera can only see the IR light from its own emitter.

Figure 7 Two films with same direction of polarization are attached to IR emitter and camera. The blue stripes are just for illustration.

Here are some results of point cloud with polarization. Compared to the result without polarization, it is shows that the polarization method indeed improves the holes at the interference area while still maintaining real time frame rate. However, the point cloud are unstable, as shown in Figure 8 and 9, the appearance of the holes changing all the time. The situation happens less within the shared field of view and it is more likely to occur at far away scene and steep surface. I guess the reason is that some IR lights received by the IR camera in such situation are too small and ignored.

Figure 8 Top view of point cloud with polarization at different time.

Figure 9 Side view of point cloud with polarization at different time.

To sum up, for near field of interest the polarization method actually works relatively well. It is a non-invasive and simple implementation for the interference problem.
UPDATE: More details experiment described in another experiment (link) shows that the polarization method hardly improves the overall interference from two Kinects.