"Simultaneous three-dimensional geometry and color texture acquisition using a single-chip color camera," Opt. Eng., (2008)

[17] S Zhang and S-T Yau, "Simultaneous three-dimensional geometry and color texture acquisition using a single-chip color camera," Opt. Eng. 47(12), 123604, 2008; doi: 10.1117/1.3046715

A novel technique that uses a single color camera to capture high-resolution three-dimensional
3-D geometry and the perfectly aligned color texture simultaneously is discussed. A projector projects three phase-shifted black-and-white fringe patterns onto the object, and a color camera captures the fringe images reflected by the object. From these three fringe images, both 3-D shape and the color texture are obtained. Moreover, since only three fringe images are required, this proposed technique permits real-time 3-D shape measurement. Experiments are presented to demonstrate the success of this technique. 

"Absolute phase assisted three-dimensional data registration for a dual-camera structured light system," Appl. Opt., (2008)

[16] S Zhang and S-T Yau, "Absolute phase assisted three-dimensional data registration for a dual-camera structured light system,"  Appl. Opt., 47(17), 3134-3142, 2008 (Cover Feature); doi: 10.1364/AO.47.003134

For a three-dimensional shape measurement system with a single projector and multiple cameras, registering patches from different cameras is crucial. Registration usually involves a complicated and timeconsuming procedure. We propose a new method that can robustly match different patches via absolute phase without significantly increasing its cost. For y and z coordinates, the transformations from one camera to the other are approximated as third-order polynomial functions of the absolute phase. The x coordinates involve only translations and scalings. These functions are calibrated and only need to be determined once. Experiments demonstrated that the alignment error is within RMS 0:7 mm. © 2008 Optical Society of America

"Novel 3-D video for quantification of facial movement," Otolaryngol Head Neck Surg., (2008)

[15] RP Mehta, S Zhang, and TA Hadlock , "Novel 3-D video for quantification of facial movement,"  Otolaryngol Head Neck Surg., 138(4), 468-472, 2008; doi: 10.1016/j.otohns.2007.12.017

OBJECTIVES: To evaluate a novel 3-D geometry video acquisition system (3-D VAS) for quantifying facial movement. METHODS: Ten normal subjects and four patients with facial paralysis underwent recording of facial movement during three facial expressions. Facial feature points of interest were digitally marked on selected images. The maximum 3-D displacement of the brow and oral commissure and palpebral fissure width were determined. RESULTS: Test-retest reliability using the 3-D VAS in normal subjects was high (Spearman correlation coefficients 0.661-0.830, P
0.05). The predicted ratio of left to right facial measurements was 1.00 in normal subjects, and measured mean ratios were 1.05 (brow excursion), 1.01 (palpebral fissure width), and 1.03 (oral commissure excursion), respectively, matching the prediction. The 3-D VAS reliably quantified facial movement on both sides in patients with facial paralysis. CONCLUSIONS: The novel 3-D VAS can accurately and reliably quantify facial movement in healthy subjects. It is promising as a clinical tool to quantify facial movement in patients with facial paralysis.

"High resolution tracking of non-rigid 3D motion of densely sampled data using harmonic maps," Intl J. Comp. Vis., (2008)

[14] Y Wang, M Gupta, S Zhang, S Wang, X Gu, D Samaras, and P Huang, "High resolution tracking of non-rigid 3D motion of densely sampled data using harmonic maps," Intl J. Comp. Vis., 76(3), 283-300, 2008; doi: 10.1007/s11263-007-0063-y

We present a novel automatic method for high resolution, non-rigid dense 3D point tracking. High quality dense point clouds of non-rigid geometry moving at video speeds are acquired using a phase-shifting structured light ranging technique. To use such data for the temporal study of subtle motions such as those seen in facial expressions, an efficient non-rigid 3D motion tracking algorithm is needed to establish inter-frame correspondences. The novelty of this paper is the development of an algorithmic framework for 3D tracking that unifies tracking of intensity and geometric features, using harmonic maps with added feature correspondence constraints. While the previous uses of harmonic maps provided only global alignment, the proposed introduction of interior feature constraints allows to track non-rigid deformations accurately as well. The harmonic map between two topological disks is a diffeomorphism with minimal stretching energy and bounded angle distortion. The map is stable, insensitive to resolution changes and is robust to noise. Due to the strong implicit and explicit smoothness constraints imposed by the algorithm and the high-resolution data, the resulting registration/deformation field is smooth, continuous and gives dense one-to-one interframe correspondences. Our method is validated through a series of experiments demonstrating its accuracy and efficiency.

"Three-dimensional data merging using Holoimage," Opt. Eng., (2008)

[13] S Zhang and S-T Yau, "Three-dimensional data merging using Holoimage," Opt. Eng., 47(3), 033608, 2008 (Cover Feature); doi: 10.1117/1.2898902

Three-dimensional data merging is vital for full-field threedimensional
3D shape measurement. All 3D range data patches, acquired from either different sensors or the same sensor in different viewing angles, have to be merged into a single piece to facilitate future data analysis. A novel method for 3D data merging using Holoimage is proposed. Similar to the 3D shape measurement system using a phaseshifting method, Holoimage is a phase-shifting–based computer synthesized fringe image. The 3D information is retrieved from Holoimage using a phase-shifting method. If two patches of 3D data with overlapping areas are rendered by OpenGL, the overlapping areas are resolved by the graphics pipeline, that is, only the front geometry can be visualized. Therefore, the merging is performed if the front geometry information can be obtained. Holoimage is to obtain the front geometry by projecting the fringe patterns onto the rendered scene. We also demonstrated that each point of the geometry in the overlapping area can be obtained by averaging the corresponding point of the geometries reconstructed from Holoimage for each patch. Moreover, using Holoimage, the texture can also be obtained. Both simulation and experiments demonstrated the success of the proposed method. 

"Three-dimensional shape measurement using a structured light system with dual cameras," Opt. Eng., (2008)

[12] S Zhang and S-T Yau, "Three-dimensional shape measurement using a structured light system with dual cameras," Opt. Eng., 47(1), 013604, 2008; doi: 10.1117/1.2835686

A structured light system for three-dimensional shape measurement with single camera has the shortcoming of camera occlusion. To alleviate this problem, this paper introduces a structured light system with dual cameras for three-dimensional shape measurement. We discuss
1 system description,
2 system calibration,
3 three-dimensional data registration using the iterative closest-point
ICP algorithm, and
4 three-dimensional data merging using holoimage. The principle of the system is introduced, and experiments are presented to verify its performance.