We propose a framework and impact of applying Machine Learning-based generated imagery to augment data variations for firearm detection in cargo x-ray images. Deep learning-based approaches for object detection have rapidly become the state-of-art and crucial technology for non-intrusive inspection (NII) based on x-ray radiography. The technology is widely employed to reduce or replace tedious labor-intensive inspection to verify cargo content and intercept potential threats at border crossings, ports, and other critical infrastructure facilities. However, the need for variations in the threat cargo content makes accumulating training data for such a system an increasing development cost. Even though threat image projection (TIP) is widely employed to simplify the process into artificially projecting the known threat, a considerable amount of threat object appearances is still needed. To further reduce the cost, we explore the use of GenerativeAdversarial-Network (GAN) to aid dataset creation. GAN is a successful deep learning technique for generating photo-real imagery in many domains. We propose a three-stage training framework dedicated to firearm detection. First, GAN is trained to generate variations of X-ray firearm appearance from binary masks for better image quality compared to the commonly used random noise. Second, the detection training dataset is created in combinations of generated images and actual firearms using TIP. Finally, the dataset is used to train RetinaNet for the detection. Our evaluations reveal that GAN can reduce the training cost in increase detection performance as using the combination of the real and generated firearms increase performance for unseen firearms detection.
We present a method for the automated detection of firearms in cargo x-ray images using RetinaNet. RetinaNet is a recently proposed powerful object detection framework that is shown to surpass the detection performance of state-of-art two-stage R-CNN family object detectors while matching the speed of one-stage object detection algorithms. We trained our models from scratch by generating training data with threat image projection (TIP) that alleviates the class imbalance problem inherent to the x-ray security inspection and eliminates the need for costly and tedious staged data collection. The method is tested on unseen weapons that are also injected into unseen cargo images using TIP. Variations in cargo content and background clutter is considered in training and testing datasets. We demonstrated RetinaNet-based firearm detection model matches the detection accuracies of traditional sliding-windows convolutional neural net firearm detectors while offering more precise object localization, and significantly faster detection speed.
Physics-based-theoretical models have been used to predict developmental patterning processes such as branching morphogenesis for over half a century. While such techniques are quite successful in understanding the patterning processes in organs such as the lung and the kidney, they are unable to accurately model the processes in other organs such as the submandibular salivary gland. One possible reason is the detachment of these models from data that describe the underlying biological process. This hypothesis coupled with the increasing availability of high quality data has made discrete, data-driven models attractive alternatives. These models are based on extracting features from data to describe the patterns and their time evolving multivariate statistics. These discrete models have low computational complexity and comparable or better accuracy than the continuous models. This paper presents a case study for coupling continuous-physics-based and discrete-empirical-models to address the prediction of cleft formation during the early stages of branching morphogenesis in mouse submandibular salivary glands (SMG). Given a time-lapse movie of a growing SMG, first we build a descriptive model that captures the underlying biological process and quantifies this ground truth. Tissue-scale (global) morphological features are used to characterize the biological ground truth. Second, we formulate a predictive model using the level-set method that simulates branching morphogenesis. This model successfully predicts the topological evolution, however, it is blind to the cellular organization, and cell-to-cell interactions occurring inside a gland; information that is available in the image data. Our primary objective via this study is to couple the continuous level set model with a discrete graph theory model that captures the cellular organization but ignores the forces that determine the evolution of the gland surface, i.e. formation of clefts and buds. We compared the prediction accuracy of our model to an on-lattice Monte-Carlo simulation model which has been used extensively for modeling morphogenesis and organogenesis. The results demonstrate that the coupled model yields comparable simulations of gland growth to that of the Monte-Carlo simulation model with a significantly lower computational complexity.
We present a method for the computer-aided histopathological grading of follicular lymphoma (FL) images based
on a multi-scale feature analysis. We analyze FL images using cell-graphs to characterize the structural organization
of the cells in tissues. Cell-graphs represent histopathological images with undirected and unweighted graphs
wherein the cytological components constitute the graph nodes and the approximate adjacencies of the components
are represented with edges. Using the features extracted from nuclei- and cytoplasm-based cell-graphs, a
classifier defines the grading of the follicular lymphoma images. The performance of this system is comparable
to that of our recently developed system that characterizes higher-level semantic description of tissues using
model-based intermediate representation (MBIR) and color-textural analysis. When tested with three different
classifiers, the combination of cell-graph based features with the MBIR and color-textural features followed by
a multi-scale feature selection is shown to achieve considerably higher classification accuracies than any set of
these feature sets can achieve separately.
Two-dimensional barcodes are widely used for encoding data in printed documents. In a number of applications,
the visual appearance of the barcode constitutes a fundamental restriction. In this paper, we propose high
capacity color image barcodes that encode data in an image while preserving its basic appearance. Our method
aims at high embedding rates and sacrifices image fidelity in favor of embedding robustness in regions where
these two goals conflict with each other. The method operates by utilizing cyan, magenta, and yellow printing
channels with elongated dots whose orientations are modulated in order to encode the data. At the receiver, by
using the complementary sensor channels to estimate the colorant channels, data is extracted in each individual
colorant channel. In order to recover from errors introduced in the channel, error correction coding is employed.
Our simulation and experimental results indicate that the proposed method can achieve high encoding rates
while preserving the appearance of the base image.
A framework for clustered-dot color halftone watermarking is considered, wherein watermark patterns are embedded in individual colorant halftones prior to printing and embedded watermarks are detected from scans of the printed images after obtaining estimates of the individual halftone separations. The principal challenge in this methodology arises in the watermark detection phase. Typical three-channel RGB scanner systems do not directly provide good estimates of the four CMYK colorant halftones that are commonly used in color printing systems. To address this challenge, we propose an estimation method that, when used with suitably selected halftone periodicities, jointly exploits the differences in the spatial periodicities and the color (spectra) of the halftone separations to obtain good estimates of the individual halftones from conventional RGB scans. We demonstrate the efficacy of this methodology experimentally using continuous phase modulation for the embedding of independent visual watermark patterns in the individual halftone separations. Watermarks detected from the estimates of halftone separations obtained using the proposed estimation method have a much higher contrast than those detected directly. We also evaluate the accuracy of the estimated halftones through simulations and demonstrate that the proposed estimation method offers high accuracy.
A framework for clustered-dot color halftone watermarking is proposed. Watermark patterns are embedded in
the color halftone on per-separation basis. For typical CMYK printing systems, common desktop RGB color
scanners are unable to provide the individual colorant halftone separations, which confounds per-separation
detection methods. Not only does the K colorant consistently appear in the scanner channels as it absorbs
uniformly across the spectrum, but cross-couplings between CMY separations are also observed in the scanner
color channels due to unwanted absorptions. We demonstrate that by exploiting spatial frequency and color
separability of clustered-dot color halftones, estimates of the individual colorant halftone separations can be
obtained from scanned RGB images. These estimates, though not perfect, allow per-separation detection to
operate efficiently. The efficacy of this methodology is demonstrated using continuous phase modulation for the
embedding of per-separation watermarks.
Halftoned separations of individual colorants, typically
cyan, magenta, yellow, and black, are overlaid on a print substrate
in typical color printing systems. Displacements between these
separations, commonly referred to as “interseparation misregistration”,
can cause objectionable color shifts in the prints. We study this
misregistration-induced color shift for periodic clustered-dot halftones
using a spatiospectral model for the printed output that combines
the Neugebauer model with a periodic lattice representation
for the individual halftones. Using Fourier analysis in the framework
of this model, we obtain an analytic characterization for the conditions
for misregistration invariance in terms of colorant spectra, periodicity
of the individual separation halftones, dot shapes, and misregistration
displacements. We further exploit the framework in a
hybrid analytical-numerical simulation that allows us to obtain quantitative
estimates of the color shifts due to misregistration, thereby
providing a characterization for these shifts as a function of the optical
dot gain, halftone periodicities, spot shapes, and interseparation
misregistration amounts. We present simulation results that
demonstrate the impact of each of these parameters on the color
shift and demonstrate qualitative agreement between our approximation
and experimental data.
The principal challenge in hardcopy data hiding is achieving robustness to the print-scan process. Conventional
robust hiding schemes are not well-suited because they do not adapt to the print-scan distortion channel, and hence are fundamentally limited in a detection theoretic sense. We consider data embedding in images printed with clustered dot halftones. The input to the print-scan channel in this scenario is a binary halftone image, and hence the distortions are also intimately tied to the nature of the halftoning algorithm employed. We propose a new framework for hardcopy data hiding based on halftone dot orientation modulation. We develop analytic halftone threshold functions that generate elliptically shaped halftone dots in any desired orientation. Our hiding strategy then embeds a binary symbol as a particular choice of the orientation. The orientation is identified at the decoder via statistically motivated moments following appropriate global and local synchronization to adress the geometric distortion introduced by the print scan channel. A probabilistic model of the print-scan process, which conditions received moments on input orientation, allows for Maximum Likelihood (ML) optimal decoding. Our method bears similarities to the paradigms of informed coding and QIM, but also makes departures from classical results in that constant and smooth image areas are better suited for embedding via our scheme as opposed to busy or "high entropy" regions. Data extraction is automatically done from a scanned hardcopy, and results indicate significantly higher embedding rate than existing methods, a majority of which rely on visual or manual detection.
We present a segmentation-based post-processing method to remove compression artifacts from JPEG compressed
document images. JPEG compressed images typically exhibit ringing and blocking artifacts, which can be
objectionable to the viewer above certain compression levels. The ringing is more dominant around textual
regions while the blocking is more visible in natural images. Despite extensive research, reducing these artifacts
in an effective manner still remains challenging. Document images are often segmented for various reasons. As a
result, the segmentation information in many instances is available without requiring additional computation. We
have developed a low computational cost method to reduce ringing and blocking artifacts for segmented document
images. The method assumes the textual parts and pictorial regions in the document have been separated from
each other by an automatic segmentation technique. It performs simple image processing techniques to clean
out ringing and blocking artifacts from these regions.
We present an analysis and model for evaluation of color shifts in halftone printing caused by inter-separation misregistration for periodic clustered dot halftones. Using a lattice framework, we present intuitive analysis that demonstrates conditions under which the average color is asymptotically invariant under inter-separation misregistration. Combining the framework with an analytic representation for the halftone dots, we develop a hybrid analytical-numerical model for quantitatively estimating color shifts as a function of inter-separation misregistration. The model is compared against experimental data for a xerographic printer.
Color-to-color misregistration refers to misregistration between color separations in a printed or display image. Such misregistration in printed halftoned images can result in several image defects, a primary one being shifts in average color. The present paper examines the variation in average color for two-color halftoned images as a function of color-to-color misregistration distance. Dot-on-dot/dot-off-dot and rotated dot screen configurations are examined via simulation and supported by print measurements. The color and color shifts were calculated using a spectral Neugebauer model for the underlying simulations. As expected, dot-on-dot/dot-off-dot color shifts were very high, while rotated dots screens exhibited very little color shift under the present idealized conditions. The simulations also demonstrate that optical dot gain significantly reduces the color shifts seen in practice.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.