1.

Introduction

High-speed three-dimensional (3-D) shape measurements based on a pattern projection technique are employed in mechanical, automobile, aircraft, medical, and heavy industries.¹ An important requirement for these applications is shorter measurement time. Among various techniques for 3-D shape measurement such as precision, accuracy, and full-field measurement, pattern projection is often preferred. If one could achieve faster phase shifting using smaller elements and devices, a more compact measurement system could be constructed that is both portable and easy to use. In addition to speed, precise shifting and an appropriate intensity distribution (preferably sinusoidal) are desired. Recently, a few papers have been published on 3-D shape measurements using a liquid crystal grating, a digital mirror device (DMD), a single micro-electro-mechanical system (MEMS) mirror, a linear LED array, and a polygonal mirror.^2–6 In particular, phase shifting using a digital projector with a built-in DMD has attained ultrafast 3-D shape measurements.^3,7–12 A DMD is at the heart of a digital light processing (DLP) unit. Digital projectors are readily available due to their popularity for industrial and consumer applications. In case of the use of the digital projectors, Pan et al.⁷ has suggested the importance of preferable sinusoidal intensity distribution of a fringe and has demonstrated the phase compensation algorithm.

As described by Zhang et al., conventional commercial DLP projectors are limited to a 120-Hz measurement rate. The speed of switching multiple 8-bit grayscale patterns is limited by the refresh rate in 3-D measurements.¹⁰ To overcome that drawback, techniques such as defocusing a square-shaped binary pattern has enabled 3-D shape measurements at a rate of 20 kHz.⁸ However, the measurement accuracy in the defocused area and the depth of the measured area are poor.

An alternative ultrafast phase-shifting technique using a single MEMS mirror (SMM) and a laser diode was proposed in 2007.^4,13–15 A SMM is scanned to accomplish the time-space transformation of the modulation signal. This spatiotemporal phase-shifting technique has merits, especially for applications that require high speeds. Use of a single mirror could enable compact equipment such as a commercial digital camera. Such a 3-D camera can cover a larger measurement area, determined by the focal depth of the projection optics. Eliminating the projection lenses will avoid aberrations caused by the optics. Therefore, this technique has potentiality that realizes the compact measurement system with superfast measurement speed. A compact system (P3D, Nikon Corp., Tokyo, Japan) shown in Fig. 1 has already been developed for practical applications in various industries.¹⁶ This measuring instrument has high accuracy of 0.3 mm covering the size of $230\times 230\mathrm{mm}$ and the dynamic range of depth is $\pm 50\mathrm{mm}$. It takes 5 s to display 3-D shape on the built-in LCD monitor after capturing images and processing the 3-D data.

Fig. 1

Compact system for practical applications in different fields of industry.

Another method accomplishes phase shifting using a linear LED array at a rate of 12 kHz. It can be applied to 3-D measurements at 4 kHz using a three-step algorithm.⁵ Some success was reported by Huang et al. They proposed a color-encoded fringe projection technique for three-step phase shifting, one in each color band, with a phase shift of $2\pi /3$ between neighboring channels.^17,18 Furthermore, Zhang et al.¹⁹ used a 3CCD camera to independently record fringe images in each RGB channel. However, they could not measure sinusoidal fringe patterns of different periods and phases, because their projector used a one-chip DMD.

To overcome this limitation and achieve high-speed 3-D measurements, a multiwavelength spatiotemporal phase-shifting technique is proposed in the present work.²⁰ This makes it possible to project three color fringe patterns with different periods using only one MEMS mirror and three laser diodes. This technique can control the initial phase of the periodic patterns. A novel projection technique is developed to control the periods and phases of the fringe patterns. In this article, we proposed high-speed 3-D shape measurement method using the multiwavelength spatiotemporal phase-shifting technique. The method is achieved to determine the phase distributions in each RGB channel. The strong point of this measurement is able to time-sequentially obtain the 3-D shape of sample without any color crosstalk. As a matter of course, this technique is also applied to determination of a phase distribution using RGB color channels. In addition to its application to 3-D shape measurements of objectives for a continuously varying surface, it can be used to measure samples having large steps or steep changes. It incorporates a gray code technique. In this experiment, we demonstrate to independently shift the phase of fringe patterns and measure 3-D shape of samples with large steps.

2.

Principle

2.1.

Multiwavelength Spatiotemporal Phase-Shifting Technique

Figure 2 sketches the phase-shifting technique using a red (R), a green (G), and a blue (B) lasers. To illuminate a two-dimensional area on a sample, an SMM, and a line generator lens (LG) are employed. The blue, red, and green laser beams are combined in dichroic mirrors DM1 and DM2. The LG converts the point beam into a fan, which is reflected by the SMM sympathetically vibrated at ${\omega }_c$. The swing $\pm {\theta }_0$ of the MEMS mirror can be adjusted from 0 to $\pm 20\mathrm{deg}$, so that the scanning angle is

Eq. (1)

$$\theta (t)={\theta }_0·\cos ({\omega }_{\mathrm{c}}·t).$$

Fig. 2

Scheme of the phase-shifting technique using RGB lasers and a single micro-electro-mechanical system (MEMS) mirror.

This scanning is used to transform from the time to the space domain.

The phase-shifting technique is illustrated in Fig. 3. Panel (a) plots the signals applied to the laser diodes. These signals $s_{\mathrm{MEMS}}$ are synchronized to the swing of the SMM and are output in binary form. During the positive swing of the mirror, the laser diodes are modulated at a frequency $f_0$, whereas during the return swing no signal is applied. This technique can thereby independently control the initial phase ${\mathrm{\Phi }}_0$ and the frequency of the signals. The output signals $s_{\mathrm{o}}(\lambda )$ are multiplied by the input signals $s_{\mathrm{MEMS}}$ and $s_{\mathrm{mod}}$, i.e., by both a sinusoidal and a square wave. Here, $\lambda$ denotes the wavelength of the laser diodes. These signals are expressed as

Eq. (2)

$$s_{\mathrm{MEMS}}=\{\begin{array}{cc}0& (\text{returning})\\ 1& (\text{going})\end{array},$$

Eq. (3)

$$s_{\mathrm{mod.}}=i^{\prime }+i^{\prime \prime }\cos (2\pi f_{0}t+{\mathrm{\Phi }}_0),$$

and

Eq. (4)

$$s_o(\lambda )=s_{\mathrm{MEMS}}·s_{\mathrm{mod.}},$$

where $i^{\prime }$ and $i^{\prime \prime }$ are the bias and amplitude of the signals, respectively. When the initial phases of the signals are different in each channel, colored fringe patterns result as shown in Fig. 3(b). The overall image then has the colorful pattern depicted in Fig. 3(c).

Fig. 3

Scheme of the multiwavelength and spatiotemporal phase-shifting technique, (a) applied signals to colored laser diodes, (b) colored fringe pattern, and (c) composed image by RGB fringe patterns.

2.2.

High Speed 3-D Shape Measurements

Figure 4 shows the experimental setup for 3-D shape measurements. Using a circuit controller such as a function generator to adjust the amplitude, bias, frequency, and initial phase, colorful periodic patterns can be projected. A 3CCD camera captures the fringe patterns and separates the red, green, and blue channels using an RGB prism inside the camera.

Fig. 4

Experimental setup for high-speed three-dimensional (3-D) shape measurement.

The intensity distribution is

Eq. (5)

$$\begin{gathered} I_i^{RGB}(x,y)=I_i^{\prime R}(x,y)+I_i^{\prime \prime R}(x,y)\cos ({\varphi }^R(x,y)+{\delta }_i^R) \\ +I_i^{\prime G}(x,y)+I_i^{\prime \prime G}(x,y)\cos ({\varphi }^G(x,y)+{\delta }_i^G) \\ +I_i^{\prime B}(x,y)+I_i^{\prime \prime B}(x,y)\cos ({\varphi }^B(x,y)+{\delta }_i^B). \end{gathered}$$

Here, $I_i^{\prime R}(x,y)$, $I_i^{\prime G}(x,y)$, and $I_i^{\prime B}(x,y)$ represent the bias components, $I_i^{\prime \prime R}(x,y)$, $I_i^{\prime \prime G}(x,y)$, and $I_i^{\prime \prime B}(x,y)$ the amplitude components of each channel, and ${\varphi }^R(x,y)$, ${\varphi }^G(x,y)$, and ${\varphi }^B(x,y)$ the phase distributions including sample height information, respectively. The intensity distributions captured by the three-chip CCD can also be written as

Eq. (6)

$$I_i^R(x,y)=I_i^{\prime R}(x,y)+I_i^{\prime \prime R}(x,y)\cos ({\varphi }^R(x,y)+{\delta }_i^R),$$

Eq. (7)

$$I_i^G(x,y)=I_i^{\prime G}(x,y)+I_i^{\prime \prime G}(x,y)\cos ({\varphi }^G(x,y)+{\delta }_i^G),$$

and

Eq. (8)

$$I_i^B(x,y)=I_i^{\prime B}(x,y)+I_i^{\prime \prime B}(x,y)\cos ({\varphi }^B(x,y)+{\delta }_i^B).$$

By controlling the initial phase, frequency, amplitude, and bias of the intensity distributions, it is possible to utilize the phase-shifting technique of Haung and Zhang.³ The technique uses a color fringe pattern with a $2\pi /3$ phase shift between neighboring-colored channels. A three-step phase shift gives values of ${{\delta }^{\lambda }}_i=2\pi /3\times i=0$, $2\pi /3$, and $4\pi /3$, where $i=0$, 1, and 2, respectively. When the amplitude and bias are ideally adjusted so that $I_i^{\prime R}(x,y)=I_i^{\prime G}(x,y)=I_i^{\prime B}(x,y)$ and $I_i^{\prime \prime R}(x,y)=I_i^{\prime \prime G}(x,y)=I_i^{\prime \prime B}(x,y)$ with a phase shift of $2\pi /3$ between neighboring color channels, the phase distribution can be obtained from the three-step algorithm as

Eq. (9)

$$\varphi (x,y)={\tan }^{-1}\lfloor \sqrt{3}(I_0^R-I_2^B)/(2I_1^G-I_0^R-I_2^B)\rfloor .$$

The present technique also provides phase distributions for the 3-D shape measurement in every frame. Figure 5 indicates the analysis procedure for high-speed measurements. The phase distributions can be written as

Eq. (10)

$${\varphi }_{\lambda }(x,y)={\tan }^{-1}\lfloor \sqrt{3}(I_0^{\lambda }-I_2^{\lambda })/(2I_1^{\lambda }-I_0^{\lambda }-I_2^{\lambda })\rfloor .$$

Fig. 5

Analyzing process for 3-D shape measurement.

By sliding the timing of the phase shift, the three phase distributions are sequentially obtained in three frames. The phase distribution can be measured in every frame. It is possible to obtain the phase distributions for all frames, including sample height information, when the fringe patterns have initial phases differing in steps of $\pi /3$. This phase shift technique is independent of wavelength because the intensity distributions are routed by internal RGB prisms in the 3CCD camera. That is, there is no color crosstalk because the system uses three laser diodes and a three-chip CCD. It is thus possible to develop various applications not only for high-speed measurements but also over a wide range for the gray code technique. In the latter case, height steps can be determined by analyzing the binary patterns after illumination with color patterns.

3.

Experimental Setup and Performance Check

Fast measurements of 3-D shapes having height steps are performed. The laser diode wavelengths and output powers are: red (R) at $\lambda =650\mathrm{nm}$ and 100 mW, green (G) at $\lambda =532\mathrm{nm}$ and 50 mW, and blue (B) at $\lambda =405\mathrm{nm}$ and 30 mW. The laser beams are mixed together and expanded by an LG. The expansion angle of the lens is 75 deg. The line beam is incident on the SMM (LSM-500CL, Venture Forum Mie, Mie, Japan) driven at 500 Hz. The sinusoidal and rectangular waves are applied as the signal voltages at different frequencies to the laser diodes using a circuit controller. The properties of the signals at each colored channel are individually controlled. The red channel has a 40-kHz sinusoidal wave applied to it, the blue channel a 10-kHz sinusoidal wave, and the green channel a rectangular wave at 2.5, 5, or 10 kHz. A 3CCD camera (IK-TF5C, Toshiba, Tokyo, Japan) synchronously captures the image with a periodic pattern at a frame rate of 60 fps.

A painting on plasterboard serves as the sample. The size of the painting is $130\times 100\times 7\mathrm{mm}$. Figure 6 shows an image measured using a digital camera. It does not capture the fringes because the period of the phase shift is 10 ms. The projected patterns are separately illuminated with two sinusoidal fringe sequences having different frequency and binary code patterns. Figure 7 presents the composite images captured by the 3CCD camera, and those separated into individual RGB channels. The blue and red channels consist of low-frequency and high-frequency fringe patterns, respectively. In contrast, the green channels are modulated using a gray code technique.

Fig. 6

Captured image using a digital camera with superfast phase shifting.

Fig. 7

Captured images using 3CCD camera and separated images at each wavelength, (a) composite images, (b) blue images, (c) green images, and (d) red images.

Figure 8 shows details of the gray code using the green channel. Panels (a) to (c) are binarized so that the bright area is “0” and the dark area is “1.” Panel (d) is an image encoded by the gray code listed in Table 1, whose scale ranges from $N=-2$ to $+1$. An additional gray code “3” is required to determine the height information in the area indicated by the values $-2$, $-1$, and $+1$. One can interpolate the height information using the low-frequency sinusoidal patterns. Figures 9(a) and 9(b) show the wrapped phase distributions at low and high frequencies, respectively. Both distributions are discontinuous in steps of $2\pi$, but the low-frequency phase can be deduced using the gray code as shown in Fig. 8(d). The 3-D shape can then be precisely recovered from the high-frequency-wrapped phase distribution, as demonstrated in Fig. 10. Using a calibration and a phase-coordinate conversion, the phase distribution can be obtained in terms of $(x,y,z)$ coordinates.¹⁵ In Fig. 10, the 3-D shape is presented at three different angles. The height step measurement is performed using gray-code-structured patterns. This experiment yields that the measurement has the accuracy of 0.5 mm within $130\times 100\mathrm{mm}$. In addition, the measurement time is less than $3/60\mathrm{s}=50\mathrm{ms}$. When we precisely control the sinusoidal shape of fringe pattern, it is possible to measure 3-D shape with high accuracy as same as the above-mentioned P3D. This technique has the potential not only to achieve high-speed 3-D and height-step measurements, but also to capture textured images from the RGB channels.

Fig. 8

Gray code sequence for $n=3$, (a) gray code “0,” (b) gray code “1,” (c) gray code “2,” and (d) encoding by gray codes.

Table 1

Gray code.

Fig. 9

Phase distribution at each wavelength, (a) 405 nm, (b) 650 nm.

Fig. 10

3-D shape at each angle, (a) view angle 1, (b) view angle 2, (c) view angle 3.

4.

Conclusions

We have developed a 3-D shape measurement system using a multiwavelength spatiotemporal phase-shifting technique and demonstrated its performance and applications. The key technology is a digital device using a SMM for pattern projection rather than a DLP or LC. The system employs laser diodes with a red, green, and blue channel. Two different sinusoidal waves are applied to the red and blue laser diodes, whereas a rectangular wave is input to the green laser. The modulated fan beams are converted into periodic patterns, specifically two sinusoidal patterns and a binary pattern following a time-space transformation using a SMM. A commercial digital projector with a one-chip DMD cannot simultaneously project different color periodic patterns. The 3CCD camera used here can individually capture the images because RGB prisms can completely separate the composite into three color channels. Specifically, the present projection method can sequentially determine the phase distributions of the RGB channels. Three-dimensional shape measurements with height steps were performed using the combination of two different frequency fringe and gray code patterns. The experimental results indicate that the technique can measure the 3-D shapes of slowly moving objects, which is difficult to attain using the traditional phase-shifting technique of commercial DLP projectors.