A new way of designing distortion-invariant filters for object detection is described. These filters are linear combinations of eigen-images of preprocessed training data. The joint preprocessing and eigen-image formulation leads to filters with good generalization, clutter rejection and object localization. Test results using these filters for synthetic aperture radar object detection are given. We show that with a systematic procedure to select filter parameters, a 100% detection rate with no false alarms can be obtained.

A new general method for linear and nonlinear feature extraction is presented. It is novel since it provides both representation and discrimination while most other methods are concerned with only one of these issues. We call this approach the maximum representation and discrimination feature (MRDF) method and show that the Bayes classifier and the Karhunen- Loeve transform are special cases of it. We refer to our nonlinear feature extraction technique as nonlinear eigen- feature extraction. It is new since it has a closed-form solution and produces nonlinear decision surfaces with higher rank than do iterative methods. Results on synthetic databases are shown and compared with results from standard Fukunaga- Koontz transform and Fisher discriminant function methods. The method is also applied to an automated product inspection problem (discrimination) and to the classification and pose estimation of two similar objects (representation and discrimination).

We report on the development of new mobile robots for Mars exploration missions. These 'lightweight survivable rover (LSR)' systems are of potential interest to both space and terrestrial applications, and are distinguished from more conventional designs by their use of new composite materials, collapsible running gear, integrated thermal-structural chassis, and other mechanical features enabling improved mobility and environmental robustness at reduced mass, volume, and power. Our first demonstrated such rover architecture, LSR-1, introduces running gear based on 2D composite struts and 3D machined composite joints, a novel collapsible hybrid composite-aluminum wheel design, a unit-body structural- thermal chassis with improved internal temperature isolation and stabilization, and a spot-pushbroom laser/CCD sensor enabling accurate, fast hazard detection and terrain mapping. LSR-1 is an approximately .7 $MIL 1.0 meter(Lambda) 2(W X L) footprint six-wheel (20 cm dia.) rocker-bogie geometry vehicle of approximately 30 cm ground clearance, weighing only 7 kilograms with an onboard .3 kilogram multi-spectral imager and spectroscopic photometer. By comparison, NASA/JPL's recently flown Mars Pathfinder rover Sojourner is an 11+ kilogram flight experiment (carrying a 1 kg APXS instrument) having approximately .45 X .6 meter(Lambda) 2(WXL) footprint and 15 cm ground clearance, and about half the warm electronics enclosure (WEE) volume with twice the diurnal temperature swing (-40 to +40 degrees Celsius) of LSR- 1 in nominal Mars environments. We are also developing a new, smaller 5 kilogram class LSR-type vehicle for Mars sample return -- the travel to, localization of, pick-up, and transport back to an Earth return ascent vehicle of a sample cache collected by earlier science missions. This sample retrieval rover R&D prototype has a completely collapsible mobility system enabling rover stowage to approximately 25% operational volume, as well an actively articulated axle, allowing changeable pose of the wheel strut geometry for improved transverse and manipulation characteristics.

An intelligent robot is a remarkably useful combination of a manipulator, sensors and controls. The use of these machines in factory automation can improve productivity, increase product quality and improve competitiveness. This paper presents a discussion of recent economic and technical trends. The robotics industry now has a billion-dollar market in the U.S. and is growing. Feasibility studies are presented which also show unaudited healthy rates of return for a variety of robotic applications. Technically, the machines are faster, cheaper, more repeatable, more reliable and safer. The knowledge base of inverse kinematic and dynamic solutions and intelligent controls is increasing. More attention is being given by industry to robots, vision and motion controls. New areas of usage are emerging for service robots, remote manipulators and automated guided vehicles. However, the road from inspiration to successful application is still long and difficult, often taking decades to achieve a new product. More cooperation between government, industry and universities is needed to speed the development of intelligent robots that will benefit both industry and society.

Inheritance or training in human vision? Training allows individual adaptation to environmental conditions, while genetics determine optimized constant i.e. long-term abilities. The biological engineering of the human eye needs both inheritance and training to realize its high performances. Particularly during postnatal training of an eye, adaptive freedom is necessary and available. To test the part of training it would become necessary to experimentally determine the in vivo refractive index differences between cellular nuclei and cytoplasm in retinal nuclear layers before and after birth to see if diffractive optical tuning of trichromatism in a retinal 3D-grating is synchronized with the differentiation of 3 photopigments in photopic vision or if the specialization in photochemistry depends on crystal- optical preprocessing.

This paper examines a novel approach for extracting motion information to allow the autonomous navigation of an intelligent mobile robot using computer vision in a moving camera, moving object environment. The approach begins by extracting low-level scene feature information using algorithms such as the SUSAN corner and edge detector. A routine is described for converting the information obtained from these stable features to initialization information for creating active contour models or 'snakes.' Multiple open and closed active contours are identified in an initialization frame from this primary feature extraction. These contours are allowed to converge more closely to the features to which they are attached. These contours are then allowed to converge to the features within each frame through image sequences, with criteria for the re-initialization of new contours when motion information in the sequence or a region becomes sparse. The information received from these contour models is then used to determine the motion information in the scene. Reasons for this approach are outlined and justified. This theoretical approach is then applied to the practical cases of a mobile robot navigating indoor scenes. Large sections of this approach have been implemented in the Khoros environment, with new routines written for this approach. Promising results are already available and this approach is being examined to allow the extraction of depth information in the scene for assisting navigation using a form of '3-D snakes.'

The discrete Fourier transform is of fundamental importance in the digital processing of signals. By using the Jacobian elliptic functions sn (u,m) and cn(u,m) as basis functions, in place of the trigonometric sine and cosine, one can obtain a generalized transform which includes the Fourier transform as a special case (viz., m equals 0). Since m, the squared modulus, can have any positive value less than 1, the new transform is extremely flexible. It is found that the associated inverse transform consists of basis functions whose appearance can be described as a set of dithered trigonometric functions. The dithering level increases in monotonic fashion with the parameter m. Sample applications of this non-linear form of signal processing are discussed.

This paper introduces a new method to solve the problem of matching correspondent feature points across two images containing a moving rigid object. Two successive time-varying images are used. Edge points are first extracted using a 3 by 3 Laplacian mask. Walsh transformation is then applied to the feature points in both images. The choice of Walsh transformation in contrast to other orthogonal transforms is a direct result of its computational simplicity and its interpretative meaning in terms of information contained in the spatial domain. Two premises are applied as matching rules. The first involves the speed of object and imaging system, while the other involves the selection of the best match from the set of candidate matches. Unlike other matching techniques, the computational complexity of the proposed technique does not grow up with to the number of detected feature points in either of the two images. This characteristic gives the technique a great flexibility. Experimental results are given and assessed in terms of both accuracy and computational complexity.

A method for object recognition and pose estimation for robotic bin picking is presented. The approach discussed is a variant on current approaches to eigenimage analysis. Compared to traditional approaches which use object geometry only (shape invariants), the implementation described uses the eigenspace determined by processing the eigenvalues and eigenvectors of the image set. The image set is obtained by varying pose while maintaining a constant level of illumination in space, and the eigenspace is computed for each object of interest. For an unknown input image, the recognition algorithm projects this image to each eigenspace and the object is recognized using space partitioning methods which determine the object and the position in space. Several experimental results have been obtained to demonstrate the robustness of this method when applied to the robotic bin picking task.

When interacting with intelligent agents, it is vital to understand something of their intentions. Agents' intentions provide context in spoken deluge, help to define their future plans (and thus actions), and reveal information about their beliefs. We propose a method by which agents' intentions are inferred by observing their actions. Explicit communication among agents is not permitted. The joint intentions framework specifies the behaviors and obligations of agents that share in a cooperative intention. Our work focuses on the creation of such joint intentions through observation and plan recognition. The plan recognition structure uses Bayes nets to reason from observations to actions, and hidden Markov models to reason from sequences of actions to intent.

Automated guided vehicles (AGVs) have many potential applications in manufacturing, medicine, space and defense. The purpose of this paper is to describe exploratory research on the design of a speed control for a modular autonomous mobile robot controller. The speed control of the traction motor is essential for safe operation of a mobile robot. The challenges of autonomous operation of a vehicle require safe, runaway and collision free operation. A mobile robot test-bed has been constructed using a golf cart base. The computer controlled speed control has been implemented and works with guidance provided by vision system and obstacle avoidance using ultrasonic sensors systems. A 486 computer through a 3- axis motion controller supervises the speed control. The traction motor is controlled via the computer by an EV-1 speed control. Testing of the system was done both in the lab and on an outside course with positive results. This design is a prototype and suggestions for improvements are also given. The autonomous speed controller is applicable for any computer controlled electric drive mobile vehicle.

Automated guided vehicles (AGVs) have many potential applications in manufacturing, medicine, space and defense. The purpose of this paper is to describe exploratory research on the design of an obstacle avoidance system using sonar sensors for a modular autonomous mobile robot controller. The advantages of a modular system are related to portability and the fact that any vehicle can become autonomous with minimal modifications. A mobile robot test-bed has been constructed using a golf cart base. The obstacle avoidance system is based on a micro-controller interfaced with multiple ultrasonic transducers. This micro-controller independently handles all timing and distance calculations and sends a distance measurement back to the computer via the serial line. This design yields a portable independent system. Testing of these systems has been done in the lab as well as on an outside test track with positive results that show that at five mph the vehicle can follow a line and at the same time avoid obstacles. This design, in its modularity, creates a portable autonomous obstacle avoidance controller applicable for any mobile vehicle with only minor adaptations.

In this paper, a method to refine an initial noisy optical flow map is presented. The method uses geometrical constraints induced by camera motion in order to improve an initial rough optical flow map obtained using the radiometric similarity measure. Under the assumption that the camera motion can be described by a prevalence of translational motion an appropriate cost function is derived, whose minimization is used to improve the initial displacement estimates and to obtain subpixel accuracy. Results are quantitatively compared with other known optical flow estimation techniques. Both synthetic and real image sequence are used in test.

Automated guided vehicles (AGVs) have many potential applications in manufacturing, medicine, space and defense. The purpose of this paper is to describe exploratory research on the design of the remote controlled emergency stop and vision systems for an autonomous mobile robot. The remote control provides human supervision and emergency stop capabilities for the autonomous vehicle. The vision guidance provides automatic operation. A mobile robot test-bed has been constructed using a golf cart base. The mobile robot (Bearcat) was built for the Association for Unmanned Vehicle Systems (AUVS) 1997 competition. The mobile robot has full speed control with guidance provided by a vision system and an obstacle avoidance system using ultrasonic sensors systems. Vision guidance is accomplished using two CCD cameras with zoom lenses. The vision data is processed by a high speed tracking device, communicating with the computer the X, Y coordinates of blobs along the lane markers. The system also has three emergency stop switches and a remote controlled emergency stop switch that can disable the traction motor and set the brake. Testing of these systems has been done in the lab as well as on an outside test track with positive results that show that at five mph the vehicle can follow a line and at the same time avoid obstacles.

Self-location is the capability of a mobile robot to determine its position in the environment referring to absolute landmarks. The possibility to use natural visual landmarks for self-location augments the autonomy and the flexibility of mobile vehicles. In this paper the use of junctions, detected in real images, as landmarks is proposed. The use of visual cues means that problems regarding variations of perspective and scale must be resolved. We propose to formulate the junction recognition as a graph matching problem and resolved using standard methods. Experimental results are shown on real contexts.

Automated guided vehicles (AGVs) have many potential applications in manufacturing, medicine, space and defense. The purpose of this paper is to describe exploratory research on the design of a modular autonomous mobile robot controller. The controller incorporates a fuzzy logic approach for steering and speed control, a neuro-fuzzy approach for ultrasound sensing (not discussed in this paper) and an overall expert system. The advantages of a modular system are related to portability and transportability, i.e. any vehicle can become autonomous with minimal modifications. A mobile robot test-bed has been constructed using a golf cart base. This cart has full speed control with guidance provided by a vision system and obstacle avoidance using ultrasonic sensors. The speed and steering fuzzy logic controller is supervised by a 486 computer through a multi-axis motion controller. The obstacle avoidance system is based on a micro-controller interfaced with six ultrasonic transducers. This micro- controller independently handles all timing and distance calculations and sends a steering angle correction back to the computer via the serial line. This design yields a portable independent system in which high speed computer communication is not necessary. Vision guidance is accomplished with a CCD camera with a zoom lens. The data is collected by a vision tracking device that transmits the X, Y coordinates of the lane marker to the control computer. Simulation and testing of these systems yielded promising results. This design, in its modularity, creates a portable autonomous fuzzy logic controller applicable to any mobile vehicle with only minor adaptations.

We review work conducted over the past several years and aimed at developing reinforcement learning architectures for solving difficult control problems and based on and inspired by associative control process (ACP) networks. We briefly review ACP networks able to reproduce many classical instrumental conditioning test results observed in animal research and to engage in real-time, closed-loop, goal-seeking interactions with their environment. Chronologically, our contributions include the ideally interfaced ACP network which is endowed with hierarchical, attention, and failure recognition interface mechanisms which greatly enhanced the capabilities of the original ACP network. When solving the cart-pole problem, it achieves 100 percent reliability and a reduction in training time similar to that of Baird and Klopf's modified ACP network and additionally an order of magnitude reduction in number of failures experienced for successful training. Next we introduced the command and control center/internal drive (Cid) architecture for artificial neural learning systems. It consists of a hierarchy of command and control centers governing motor selection networks. Internal drives, similar hunger, thirst, or reproduction in biological systems, are formed within the controller to facilitate learning. Efficiency, reliability, and adjustability of this architecture were demonstrated on the benchmark cart-pole control problem. A comparison with other artificial learning systems indicates that it learns over 100 times faster than Barto, et al's adaptive search element/adaptive critic element, experiencing less failures by more than an order of magnitude while capable of being fine-tuned by the user, on- line, for improved performance without additional training. Finally we present work in progress on a 'peaks and valleys' scheme which moves away from the one-dimensional learning mechanism currently found in Cid and shows promises in solving even more difficult learning control problems such as the truck backer-upper.

Automated guided vehicles (AGVs) have many potential applications in manufacturing, medicine, space and defense. The purpose of this paper is to describe the design of a steering mechanism for an autonomous mobile robot. The steering mechanism replaces a manually turned rack and pinion arrangement with a crank mechanism driven by a linear actuator that in turn is powered by a brushless dc motor. The system was modeled, analyzed, and redesigned to meet the requirements. A 486 computer through a 3-axis motion controller supervises the steering control. The steering motor is a brushless dc motor powered by three phase signals. It is run in current loop mode. The steering control system is supervised by a personal computer through a multi-axis motion controller. Testing of these systems has been done in the lab as well as on an outside test track with positive results.

Robotic exploration of the Martian surface will provide important scientific data on planetary climate, life history, and geologic resources. In particular, robotic arms will assist in the detailed visual inspection, instrumented analysis, extraction, and earth return of soil and rock samples. To this end, we are developing new robotic manipulation concepts for use on landers and rovers, wherein mass, volume, power and the ambient Mars environment are significant design constraints. Our earlier work led to MarsArmI, a 2.2 meter, 3-dof hybrid metal/composite, dc-motor actuated arm operating under coordinated joint-space control; NASA's Mars Surveyor '98 mission utilizes this design concept. More recently, we have conceived and implmented new, all- composite, very light robot arms: MarsArmII, a 4.0 kilogram, 2.3 meter arm for lander operations, and MicroArm-1 and MicroArm-2, two smaller 1.0+ kilogram, .7 meter rover arms for mobile sample acquisition and Mars sample return processing. Features of these arms include our creation of new 3D machined composites for critical load-bearing parts; actuation by high-torque density ultrasonic motors; and, visually-designated inverse kinematics positioning with contact force adaptation under a novel task-level, dexterous controls paradigm. Our demonstrated results include robotic trenching, sample grasp-manipulation-and-transfer, and fresh rock surface exposure-probing via the science operator's 'point-and-shoot' visual task designation in a stereo workspace. Sensor-referenced control capabilities include real-time adaptation to positioning error and environmental uncertainties (e.g., variable soil resistance and impediments), and the synthesis of power optimal trajectories for free space manipulation.

This paper describes the development of a robot fault diagnosis system (RFDS). Though designed ostensibly for the University of Cincinnati's autonomous, unmanned, mobile robot for a national competition, it has the flexibility to be adapted for industrial applications as well. Using a top-down approach the robot is sub-divided into different functional units, such as the vision guidance system, the ultrasonic obstacle avoidance system, the steering mechanism, the speed control system, the braking system and the power unit. The techniques of potential failure mode and effects analysis (PFMEA) are used to analyze faults, their visible symptoms, and probable causes and remedies. The relationships obtained therefrom are mapped in a database framework. This is then coded in a user-friendly interactive Visual Basic^TM program that guides the user to the likely cause(s) of failure through a question-answer format. A provision is made to ensure better accuracy of the system by incorporating historical data on failures as it becomes available. The RFDS thus provides a handy trouble-shooting tool that cuts down the time involved in diagnosing failures in the complex robot consisting of mechanical, electric, electronic and optical systems. This has been of great help in diagnosing failures and ensuring maximum performance from the robot during the contest in the face of pressure of the competition and the outdoor conditions.

The paper introduces a technique for constructing a look-up table that contains information about selected points in the state space of the nonlinear process to be controlled. The new states to which a specific state point transfers when the process is issued different control commands are registered. The controller designed to make use of the table depends on the heuristic assumption that the behavior of the process at a specific state space point is similar to its behavior at a nearby point provided that the process is issued a similar control command at each state. The table design aspects for reducing the storage requirements and minimizing the computational effort for producing control decisions are investigated. The applicability of the controller is tested for the tractor-trailer truck steering problem backing up at a constant speed. The truck backs up in an environment that contains obstacles to be avoided. Computer simulation of the truck and its controller has demonstrated workability.

This paper deals with the recovery of a scene from a pair of images, where each image is acquired from a different viewpoint. The central problem is the identification of corresponding points in all views. We use the feature-based approach to find corresponding points. Various types of features have been used previously, where Gabor features showed significant advantages in terms of accuracy and the complexity/accuracy trade-off. The accuracy is measured as the rate of correctly associated pixels. The matching process typically results in a certain number of ambiguous positions, where the best match found is not the desired match. The main contribution of this paper lies in the application of a genetic algorithm for feature selection. This method uses the previously illustrated fact that the amount of ambiguity in the matching process can be quantitatively measured via statistics on the back-matching distances. With this method, the quality of a matching result can be measured without reference disparity data (or ground truth). The fitness function required for the application of genetic feature optimization is defined using these back-matching statistics. The output of the genetic algorithm is an improved feature set, which contains fewer features as the initial set, but yields extremely improved accuracy. We show that the accuracy of the matching result can be much improved by our genetic optimization approach, and we describe the experiments illustrating the results.

Moments are important features used in pattern recognition, image analysis and computer vision. Geometric moments are calculated using window functions with great discontinuities at window boundary and the bases are not orthogonal. In order to better characterize noisy images, one should use the orthogonal moments with a smoothing window function. In the present paper, by use of the well-known Gaussian functions as smoothing widow function, we first introduce Gaussian-Hermite moments which are shown to be orthogonal smoothed moments. Then we present the recursive calculation of Gaussian-Hermite moments. The recursive calculation not only gives an efficient scheme for Gaussian-Hermite moment calculation, but also reveals the essential relation between orthogonal Gaussian- Hermite moments and Gaussian-filtered signal derivatives. From this relation, we see that the Gaussian-Hermite moments are good local features for noisy signal. Moreover, because Gaussian derivatives satisfy the conditions for mother wavelets, Gaussian-Hermite moments defined from Gaussian functions of different (sigma) correspond to wavelet development of the input signal. In fact, the n-order Gaussian-Hermite moment is a linear combination of different Gaussian-derivative wavelets of the input signal. So the use of Gaussian-Hermite moments gives also an efficient approach to representing the input signal in an orthogonal functional space from wavelet analysis. All the results above obtained for 1-D signals are generalized to 2-D image analysis.

The human visual system is very good at detecting geometric relationships such as colinearity, parallelism, connectivity, and repetitive patterns in a randomly distributed set of image elements. We believe that such capabilities are useful, if not essential, for the task of object detection and segmentation, shape description and object matching. We propose an algorithm based on some of these capabilities for organizing the fragmented low-level features into meaningfully mid-level description. This algorithm uses an estimated measurement of the curvature, the orientation and the gradient magnitude at each possible location of salient curves in the image. We then presented some geometric constraints between edge elements in term of a neighborhood relationships in order to organize the detected edges into groups. The grouping precess results from a procedure for computing the orientation and curvature by minimizing a natural functional that reduce ambiguity and noise effect. The minimization is accomplished using relaxation labeling technique. The procedure is applied to some images to evaluate the groupings processes and noise sensitivity.

An innovative model based object recognition system exploiting a priori knowledge called three dimensional model based approach (3D-MBA) is proposed in this paper. The idea behind 3D-MBA is to use virtual images to model the world and to utilize conventional vision to extract relevant clues in order to perform robust object recognition. By taking advantage of the different available sensors (range scanner, CCD camera) the system will combine top-down and bottom-up approaches by exploiting intensity, range and virtual images. Segmenting range images performs the feature extraction. These features are then combined to search in a geometrical constraint graph enabling object hypothesis generation, thus starting a prediction-verification process. We show that correlation techniques on range and intensity images can be used to validate the hypothesis. It is possible to refine those hypotheses to converge to the final recognition. Another technique to achieve 3-D mesh matching is ICP (iterative closest point). This iterative algorithm is used to find the best match between two sets of points. Each point of the first set is associated with the closest point in the second set. The best translation and rotation are evaluated in order to minimize point to point distance.

An automated product tag reading system based on CCD cameras and computer image processing has been developed by West Virginia University, and demonstrated at the Weirton Steel Corporation. The system was developed to read painted steel identification tags which are fastened to the ends of steel slabs. The prototype was mounted on a slab hauler and tested in a steel mill environment. It demonstrated the ability to survey a wide-angle image of a scene, locate the target tags in the image, pan and zoom on each one in turn, and read the contents of the tag -- both bar code and alphanumeric code. The system could communicate via radio modem to a stationary computer which could be interfaced to the plant's inventory management database. We are aware of no other system which can both point and read a bar code scanner. The pointing function is important, in that it allows operation in bright sunlight or at long distances: two conditions where humans encounter difficulties in aiming. Our system can read 50-mil bar codes at distances of 30 feet without the use of retro reflective targets. This paper provides an overview of the technical approach used, and the results of stem testing.

A machine vision system has been developed to separate half cut peaches with small splinters from clean ones. The system uses the different spectral profile of both together with an ad hoc illuminating system. The system is capable of processing 30 half peaches per second. The hardware and software solutions are described.

Face recognition, one of the most important abilities of intelligent vision, is discussed in this paper. A new idea of model-based matching is presented. The algorithms of face modeling, global matching and fine matching are given. Finally, a robust scheme for human face rapid recognition resulted.

Three-dimensional object recognition has always been one of the challenging fields in computer vision. In recent years, Ulman and Basri (1991) have proposed that this task can be done by using a database of 2-D views of the objects. The main problem in their proposed system is that the correspondent points should be known to interpolate the views. On the other hand, their system should have a supervisor to decide which class does the represented view belong to. In this paper, we propose a new momentum-Fourier descriptor that is invariant to scale, translation, and rotation. This descriptor provides the input feature vectors to our proposed system. By using the Dystal network, we show that the objects can be classified with over 95% precision. We have used this system to classify the objects like cube, cone, sphere, torus, and cylinder. Because of the nature of the Dystal network, this system reaches to its stable point by a single representation of the view to the system. This system can also classify the similar views to a single class (e.g., for the cube, the system generated 9 different classes for 50 different input views), which can be used to select an optimum database of training views. The system is also very flexible to the noise and deformed views.

The interpretation of the 'inverted' retina of primates as an 'optoretina' (a light cones transforming diffractive cellular 3D-phase grating) integrates the functional, structural, and oscillatory aspects of a cortical layer. It is therefore relevant to consider prenatal developments as a basis of the macro- and micro-geometry of the inner eye. This geometry becomes relevant for the postnatal trichromatic synchrony organization (TSO) as well as the adaptive levels of human vision. It is shown that the functional performances, the trichromatism in photopic vision, the monocular spatiotemporal 3D- and 4D-motion detection, as well as the Fourier optical image transformation with extraction of invariances all become possible. To transform light cones into reciprocal gratings especially the spectral phase conditions in the eikonal of the geometrical optical imaging before the retinal 3D-grating become relevant first, then in the von Laue resp. reciprocal von Laue equation for 3D-grating optics inside the grating and finally in the periodicity of Talbot-2/Fresnel-planes in the near-field behind the grating. It is becoming possible to technically realize -- at least in some specific aspects -- such a cortical optoretina sensor element with its typical hexagonal-concentric structure which leads to these visual functions.

This paper describes the shape recognition system that has been developed within the ESPRIT project 9052 ADAS on automatic disassembly of TV-sets using a robot cell. Depth data from a chirped laser radar are fused with color data from a video camera. The sensor data is pre-processed in several ways and the obtained representation is used to train a RAM neural network (memory based reasoning approach) to detect different components within TV-sets. The shape recognizing architecture has been implemented and tested in a demonstration setup.

Paper opacity is an important problem in the paper industry. In this paper we have studied methods to derive reflectance spectrum of an opaque pile of colored paper from the reflectance spectrum of the single sheet of the same paper type. Five opacity correction methods were tested. Four of them used algebraic equations for constructing corrected spectra and the fifth method was based on the multi-layer- perceptron (MLP) neural network. Two sample sets were used for testing opacity correction methods. The first set consisted of 9 colored newsprint samples and the second set was formed from 15 samples of colored photocopier paper. The measurements of both sample sets were done with Minolta CM-2002 spectrophotometer. The measured wavelength range was from 400 nm to 700 nm with 10 nm sampling interval. For both sample sets similar tests were made. The CIE L*a*b* color coordinates were calculated both for the spectra measured from opaque piles and for the opacity corrected spectra. For each sample the absolute color coordinate differences between the coordinates of the pile spectra and the corresponding corrected spectra were computed and the results of different methods were compared to each other. The method based on the transmittance of paper gave the best results with both test sets. The predefined error tolerance of 0.5 in the a* color coordinate was achieved. The MLP-network worked also fine but the results may have been affected by the small number of training samples. We have shown that paper opacity depends on wavelength and we have developed a method for deriving the color of an opaque pile of paper from the single sheet of paper of the same type.

Machine vision systems based on a color camera are increasingly used for color measurements under industrial conditions. An important problem in the measurement of small color differences is related to metamerism. But because color cameras and human vision have slightly different response functions, they produce different sets of 'tristimulus values' for the same object viewed under the same illumination conditions, and it can happen that metameric pairs for a human are discriminated very well by a color camera and vice versa. The purpose of the present research was to evaluate the performance of color camera for measuring small color differences. The tests were performed for the whole collection of the NCS Color Block viewed under three different illuminants: standard illuminant A, standard illuminant D₆₅, and illuminant F11, and for two options of color camera, 8-bit and 12-bit. The results show some limitations with lower bit camera for accurate colorimetric measurements and good performance with the 12-bit camera in discrimination of very similar colors.

This paper presents an evaluation of a stereo vision system using a unique wide-angle imaging device. Because of the extremely wide field of view, the use of such optics is potentially useful to stereo applications where large scenes or close-up features are of interest. A general problem and deterrent for the use of wide-angle optics has been the large amount of distortion which makes processing of the warped image features and modeling of the lens difficult when using conventional camera models. However, this research attempts to use a unique digital orientation and viewing apparatus, OMNIview, to eliminate this problem. The OMNIview motionless camera orientation system provides a method for video pan, tilt, zoom, rotation, and distortion correction without moving parts. The incentive for using such a device within a stereo vision system is the wide field of measurement which can be achieved without physical scanning of the camera. This research shows that use of OMNIview for acquisition and correction of a stereo pair of wide-angle images results in limitations to the accuracy of the system.

This work presents an image segmentation method for range data that uses multi-scale wavelet analysis in combination with statistical pattern recognition. We train a pattern- recognition system with scale-space data from the edge points of a training image. Once trained the system can determine the degree of edgeness of points in a new image. Before designing the segmentation system we set forth several goals. We desire that the system detect boundaries of small as well as large objects, be robust, and have few or no free parameters. Edges in an image respond to edge detectors at different scales; therefore combining edge detection information at multiple scales can create a more complete and robust edge detection. Scale-space refers to a family of derived signals where the fine-scale information is successively suppressed as scale increases. Edge points in images have a specific signature over scale space. We use a pattern recognition method to analyze these signatures as 1-D signals and therefore label edges in an image based on its multi-scale response to a wavelet transform. A fuzzy pattern classifier with one class determines the degree of membership in the edge class for each pixel in the image. Assigning this degree of membership to each pixel creates a fuzzy edge map. a watershed algorithm then creates a segmentation from this edge map. We use the wavelet transform to generate the scale space of a range image. We choose a spline wavelet used by Mallat. A simple, synthetic image with added noise and known edges provides a training set for the pattern recognition system. Known edge points from the image create a probability density function indicating membership in an edge class. The results from analyzing a complex real image are shown.

We advance new active computer vision algorithms that classify objects and estimate their pose from intensity images. Our algorithms automatically reposition the sensor if the class or pose of an object is ambiguous in a given image and incorporate data from multiple object views in determining the final object classification. A feature space trajectory (FST) in a global eigenfeature space is used to represent 3-D distorted views of an object. Assuming that an observed feature vector consists of Gaussian noise added to a point on the FST, we derive a probability density function (PDF) for the observation conditioned on the class and pose of the object. Bayesian estimation and hypothesis testing theory are then used to derive approximations to the maximum a posteriori probability pose estimate and the minimum probability of error classifier. New confidence measures for the class and pose estimates, derived using Bayes theory, determine when additional observations are required as well as where the sensor should be positioned to provide the most useful information.

This paper deals with strategies for reliably obtaining the edges and the surface texture of metallic objects. Since illumination is a critical aspect regarding robustness and image quality, it is considered here as an active component of the image acquisition system. The performance of the methods presented is demonstrated -- among other examples -- with images of needles for blood sugar tests. Such objects show an optimized form consisting of several planar grinded surfaces delimited by sharp edges. To allow a reliable assessment of the quality of each surface, and a measurement of their edges, methods for fusing data obtained with different illumination constellations were developed. The fusion strategy is based on the minimization of suitable energy functions. First, an illumination-based segmentation of the object is performed. To obtain the boundaries of each surface, directional light-field illumination is used. By formulating suitable criteria, nearly binary images are selected by variation of the illumination direction. Hereafter, the surface edges are obtained by fusing the contours of the areas obtained before. Following, an optimally illuminated image is acquired for each surface of the object by varying the illumination direction. For this purpose, a criterion describing the quality of the surface texture has to be maximized. Finally, the images of all textured surfaces of the object are fused to an improved result, in which the whole object is contained with high contrast. Although the methods presented were designed for inspection of needles, they also perform robustly in other computer vision tasks where metallic objects have to be inspected.

Hand-eye coordination is the coupling between vision and manipulation. Visual servoing is the term applied to hand-eye coordination in robots. In recent years, research has demonstrated that active vision -- active control of camera position and camera parameters -- facilitates a robot's interaction with the world. One aspect of active vision is centering an object in an image. This is known as gaze stabilization or fixation. This paper presents a new algorithm that applies target fixation to image-based visual servoing. This algorithm, called fixation point servoing (FPS), uses target fixation to eliminate the need for Jacobian computation Additionally, FPS reburies only the rotation relationship between the camera head and the gripper frames and does not require accurate tracking of the gripper. FPS was tested on a robotics system called ISAC and experimental results are shown. FPS was also compared to a classical Jacobian-based technique using simulations of both algorithms.

Earlier, the biologically plausible active vision, model for multiresolutional attentional representation and recognition (MARR) has been developed. The model is based on the scanpath theory of Noton and Stark and provides invariant recognition of gray-level images. In the present paper, the algorithm of automatic image viewing trajectory formation in the MARR model, the results of psychophysical experiments, and possible applications of the model are considered. Algorithm of automatic image viewing trajectory formation is based on imitation of the scanpath formed by operator. Several propositions about possible mechanisms for a consecutive selection of fixation points in human visual perception inspired by computer simulation results and known psychophysical data have been tested and confirmed in our psychophysical experiments. In particular, we have found that gaze switch may be directed (1) to a peripheral part of the vision field which contains an edge oriented orthogonally to the edge in the point of fixation, and (2) to a peripheral part of the vision field containing crossing edges. Our experimental results have been used to optimize automatic algorithm of image viewing in the MARR model. The modified model demonstrates an ability to recognize complex real world images invariantly with respect to scale, shift, rotation, illumination conditions, and, in part, to point of view and can be used to solve some robot vision tasks.

We present a number of applications of optical grating technology for use in machine vision and industrial inspection and measurement applications. The gratings are based on an array of prisms that are both very easy and very cheap to produce. Currently optical gratings are used in industry in temporal/spatial correlation systems using non-coherent white- light illumination. Such systems require either the displacement of the object or the displacement of the grating in order to achieve the time signals required for evaluation. New developments in grating manufacturing techniques and electronic signal processing now enable us to electronically simulate and generate the signals required eliminating the requirement for grating or object motion. We also present a range sensor based on a split pupil methodology upon which we are developing a multiple-channel range sensor. We also highlight some of the work we are presently pursuing in multiple-layer gratings. We are investigating the Talbot and Lau self-imaging effects with hopes of using the results to preprocess images. We have also calculated the effect of multiple Bragg planes recorded into holographic materials, which can be used to produce local image transformations. Also summarized is some of the work based on an inverted retina model of the eye which uses multi-layer gratings placed before the photoreceptors to general trichromatic separation of light and explain a number of physiological effects of vision.

We present an optical filter device doing local transformations. This is achieved by recording various sets of Bragg planes in a volume hologram with each set tilted by a different small angle. The incoming light is diffracted by such a bundle of planes resulting in a process which is similar to the refraction of light by a lens. The property of this optical transform is that the transform plane is located directly behind the filter plane. Spatially localized transformations of the input amplitude distribution are achieved. We calculate plane bundles and the field distribution behind the volume hologram using the coupled wave theory. The functionality of the principle is demonstrated for a simple example. This new filtering method results in a local transformation property which is performed in real time; such a filter can be placed directly on a detector array such as a CCD chip achieving a very robust and compact arrangement.

In this contribution, we propose an optoelectronic correlator system for real-time quality control by Moire analysis, that integrates a VanderLugt optical correlator and a fast digital signal processor associated to a vector co-processor. The phase-shifting Moire technique and optical correlation have been used for automatic 3D inspection of manufactured objects that are approximately positioned. Our algorithmic procedure for supervised real-time quality control by Moire analysis is a two stage process. First we preprocess phase Moire images by smoothing and subsampling to enhance the robustness of defect detection. Then, the detection of defects is performed directly from the phase pattern data by optical correlation, correspondence analysis and statistical supervised classification. Experimental results have shown that supervised detection of defects can be performed accurately in real-time.

We describe an integrated system developed for use onboard a moving work machine. The machine is targeted to such applications as e.g. automatic container handling at loading terminals. The main emphasis is on the various environment perception duties required by autonomous or semi-autonomous operation. These include obstacle detection, container position determination, localization needed for efficient navigation and measurement of docking and grasping locations of containers. Practical experience is reported on the use of several different types of technologies for the tasks. For close distance measurement, such as container row following, ultrasonic measurement was used, with associated control software. For obstacle and docking position detection, 3D active vision techniques were developed with structured lighting, utilizing also motion estimation techniques. Depth from defocus-based methods were developed for passive 3D vision. For localization, fusion of data from several sources was carried out. These included dead-reckoning data from odometry, an inertial unit, and several alternative external localization devices, i.e. real-time kinematic GPS, inductive and optical transponders. The system was integrated to run on a real-time operating system platform, using a high-level software specification tool that created the hierarchical control structure of the software.

This paper describes the design of a control system for a flexible automation system, AutoLab, for sample preparation and analysis in a chemistry laboratory without human assistance. The design has used the blackboard architecture for control which has to deal with problems of concurrency, deadlock and crash recovery while maintaining the real-time operation of AutoLab. Issues of the composition of blackboard area, coordination between communicating modules, robot operation are dealt with in the paper.

In object manipulation or material handling, the grasped object becomes unstable when external impacts exist. However, previous studies on grasping analysis seldom address such problems as the object is influenced by the external impacts. In this paper, we investigate the grasping stability and optimality issues under the influence of external disturbances. A rotation-displacement geometry model is used in analyzing the changes of equilibrium grasping forces under the disturbances. The changes of the grasping forces can be computed systematically through the invariant grasping configuration base constructed on this geometry model. Based on these results, we introduce the concept of perturbation closure, which plays a central role in our analysis. A method for finding the minimum finger grasping forces required for impact resist grasps is developed using the perturbation closure. A grasp so determined is guaranteed to be stable if the external impacts do not exceed the given threshold. A quantitative measurement is developed that can be used to evaluate the performance of different grasping configurations in resisting any kind of external impacts. Using this performance measurement, we can distinguish a good grasping configuration from a set of feasible grasping configurations of the given object. This actually gives a solution in finding the optimal grasping locations.

We discuss an algorithm to detect lines in low contrast images in the presence of noise. The image we consider are orientation imaging microscopy (OIM) images of metal surfaces. The objective is to locate lines (boundaries between grains) in the OIM images and use that information to determine where three grains intersect (triple junctions). We use a novel method for fusion edge enhancement and a new fast skeletonization procedure using a new and efficient hit and miss transform (HMT) that produces lines of one pixel width.

A merging algorithm is proposed for smooth and efficient merges of vehicles at intersections of an automated transportation system. First, a longitudinal control algorithm is designed to keep a safe headway between vehicles in a single lane. Secondly, a decision algorithm is designed to determine the sequence of vehicles entering a converging section. The sequencing algorithm is based on fuzzy rules considering relative vehicle speed, distance, and priority of the lane. The membership function of the fuzzy system is determined not by an intuitive method but by a learning method using a neural net, where a cost function considering energy consumption and ride comfortability is used for training of the neural net. Finally, feasibility of the algorithm is investigated and validated through a simulation. The vehicle merging algorithm can be used for a PRT (personal rapid transit) system as well as for IVHS (intelligent vehicle- highway system).

In the paper, we propose a systematic approach to object modeling by combining superquadric-fitting and segmentation into an interactive algorithm. It is assumed that the input data are a discrete description of the whole close-surface (CS) of the object, which can be acquired by range image registration and integration. Using the data as input, the method is a top-down, recursive procedure as follows: At first, it finds an initial approximation of the object by fitting a single superquadric to the whole CS data. The residual errors are examined to pick up data points locating in concave regions and far away from the fitted superquadric. A dividing plane is then extracted from the selected points to partition the original data set into two disjoint subsets, which are, respectively, approximated further by the same fitting-and-splitting process. This process is repeated until the whole data are decomposed into a number of primitive superquadrics each with a satisfactory accuracy. We present results of experiments using real range data for some complex objects.

Many computer vision, computer graphics and computer-aided design applications require mathematical models of existing objects to be generated from measured surface points. The geometric model of a complex surface can be created by joining numerous low-order bi-parametric surface patches, and adjusting the control parameters such that the constituent patches meet seamlessly at their common boundaries. In this paper a two-layer neural network, called the Bernstein Basis Function (BBF) network, is proposed for computing the control points of a defining polygon net that will generate a Bezier surface that 'best' approximates the data in a local segmented region. Complex surfaces are reconstructed by using several simultaneously updated networks, each corresponding to a separate surface patch. A smooth transition between the adjacent Bezier surface patches is achieved by imposing additional positional and tangential continuity constraints on the weights during the adaptation process. This method is illustrated by adaptively stitching together several patches to form a smooth surface.

This paper describes a method for expressing and manipulating position and orientation uncertainty in sensor-based robotics. The goal is to formulate a computational framework where the effects of accumulating uncertainties, originating from internal and external sensors, are shown as uncertainties in tool frame positions and orientations. The described method is based on using covariance matrices of position and orientation parameters. The used orientation parameters are xyz Euler angles. There are three different forms of spatial uncertainty and uncertainty manipulation involves transformations between these forms. These transformations are done by using linearization around nominal relations. Paper presents the basic formulas for these transformations and also three calculation examples.

Programming of frequently introduced new tasks becomes often a bottleneck for robotized surface treatment from the point of view of production performance. We have developed off-line programming tools relying on CAD product models, and, in more detail, on the surface and solid models of the products. Our system creates surface following paths over the parts or products automatically, or interactively guided by an operator. In automatic planning the paths are derived by decomposing the model into open surface segments over which the paths are created. In interactive planning a set of planar profiles are projected on the surfaces of the part model, composing the basis for the paths. We have implemented the planning system in WindowsNT^(TM) environment and confirmed the feasibility of the planning results for some demonstration products with tests in an IRB 1400 robot system.

During the summer of 1996, the topographical mapping system (TMS) for hazardous and radiological environments and its accompanying three-dimensional (3-D) visualization tool, the interactive computer-enhanced remote-viewing system (ICERVS), were delivered to Oak Ridge National Laboratory (ORNL). ORNL and Mechanical Technology, Inc., performed final acceptance testing of the TMS during the next eight months. The TMS was calibrated and characterized during this period. This paper covers the calibration, characterization, and acceptance testing of the TMS. Development of the TMS and the ICERVS was initiated by the U.S. Department of Energy (DOE) for the purpose of characterization and remediation of underground storage tanks (USTs) at DOE sites across the country. DOE required a 3-D, topographical mapping system suitable for use in hazardous and radiological environments. The intended application is the mapping of the interior of USTs as part of DOE's waste characterization and remediation efforts and to obtain baseline data on the content of the storage tank interiors as well as data on changes in the tank contents and levels brought about by waste remediation steps. Initially targeted for deployment at the Hanford Washington site, the TMS is designed to be a self-contained, compact, reconfigurable system that is capable of providing rapid, variable-resolution mapping information in poorly characterized workspaces with a minimum of operator intervention.