This PDF file contains the front matter associated with SPIE Proceedings Volume 12106 including the Title Page, Copyright information, Table of Contents, and Committee Pages.

Combining planar optics such as metalenses or metacorrectors with conventional lenses can drastically improve the optical performance of imaging systems with additional benefits such as cost, size and weight improvements. However, incorporating metacorrectors into conventional lens design requires multiscale simulations to account for the different length scale interactions. Namely, full wave scattering and geometric optics analysis is needed for the metacorrector and hybrid lens design, respectively. Multiscale inverse optimization using Sandia National Laboratories’ MIRaGE along with different wave propagation techniques and commercial-off-the-shelf GO tools are considered to accurately predict hybrid design optical performance.

This paper describes the design, execution and analysis of a perception study measuring the capability of human observers to perform uncued detection of a ship in simulated video imagery. The results indicate the relative importance of the parameters varied: target size in pixels, contrast of the target with the background, time on screen, and operator experience with surveillance video. Based on these results, a set of logistic curves was derived and used to determine V50 and E values suitable for use with the U.S. Army Night Vision Electronic Systems Directorate NV-IPM package.

The time-limited search model was developed for military operations for evaluating human search performance as a function of time, originally using static imagery but later expanded to accommodate moving sensor situations. Previously, we introduced an application for using this moving sensor search model to optimize a forward-facing sensor look-down angle for a given forward vehicle speed. In this work, we build on the optimization model to accommodate sensors that may be pointed in any direction, using coordinate transforms. This allows us to determine probability of detection for a given target as a function of a more generalized camera pointing direction. While this methodology may be applied for any target of interest such as road potholes, tanks, or IEDs, here we determine probability of detection of a Burmese python against a grass background.

An accurate prediction of the number of pixels on a target is critical in modeling the performance of a cameras ability to perform a task. This requires an accurate knowledge of the angle subtended by a pixel of interest, which can be calculated from a specification sheet or lens prescription. When such information is not available, it can be retrieved through a measurement of a known sized target at a known distance. In this correspondence, we utilize canonical images (ideal simple functions) together with non-linear optimization to provide sub-pixel target localization. This allows for accurate and repeatable measurement of the angular sampling of a camera. Additionally, the use of well-defined shapes and accurate location determination can be used to determine the blur, rotation, motion, contrast, distortion, and other camera metrics.

Long-range target identification is well studied in the Visible (Vis) and near-infrared (NIR) bands, and more recently in the shortwave infrared (SWIR). The longer wavelength of SWIR (1-1.7μm) improves target detection for both long ranges and under challenging atmospheric conditions because it is less limited by scattering and absorption in the atmosphere. For these reasons, SWIR sensors are proliferating on military platforms. The extended shortwave infrared (eSWIR) band spanning from 2 to 2.5μm is not typically limited by diffraction, and, as a result, the band benefits target acquisition both at long ranges and for degraded visual environments. Theoretical and experimental data compare eSWIR to Vis, NIR, and SWIR for atmospheric transmission, reflectivity, illumination, and sensor resolution and sensitivity. The experimental setup includes two testbeds, each with four cameras. The first is a wide field of view (FOV) testbed matching FOV at 20 degrees for each camera. The second is a narrow FOV telescope testbed to match instantaneous FOV (IFOV) for consistent resolution across all four bands at long ranges. Both the theory and experiment demonstrate advantages of using eSWIR for long-range target identification under degraded visual environments.

One major struggle for modeling and simulation (M and S) over the past decades has been the development of individual models in isolation. Typically, models are developed for a single application area where they tend to become domain specific as the complexity of a single model grows. When a future application requires interaction of multiple M and S approaches that have developed independently, it is difficult, if not impossible, for the models to integrate into a common environment. Furthering this difficulty is that the models have likely developed disparate concepts of the world in which they operate. A prime example of this effect is the development of infrared (IR) and radio frequency (RF) models, which have different large scale phenomenology and have, therefore, developed as separate M and S domains. Attempting to combine the two modalities through integration of existing M and S tools specific to each application domain has historically proven nigh impossible. These factors led to the development of the Dynamic Model Integration and Simulation Engine (DMISE) which provides a flexible and extensible framework for integration of different models into a common simulation by defining the interfaces for the simulation components. For multi-spectral IR and RF simulations, the General High-Fidelity Omni-Spectral Toolbox (GHOST) has been built on the DMISE framework to allow for integration of models across the electromagnetic spectrum. This paper presents GHOST and the status of the current effort to provide a true multi-spectral, multi-sensor, and multi-actor M and S environment through simulation of scenarios with combined IR and RF sensors operating in a common environment.

The recent advancements in commercial drone performance and capability have seen their use in private industries proliferate. In terms of large area coverage, low-flying drones can accomplish the same tasks as larger unmanned aerial vehicles (UAVs) and small manned aircraft. Traditional methods of capturing this imagery, including single wide field of view (WFOV) cameras and gimbal-mounted systems, can be replaced by small camera arrays. Single WFOV lenses deliver poor resolution at the ground level. Similarly, the use of a narrow field of view (NFOV) lens would necessitate the use of a gimbal, a pivoted support used in camera stabilization – yielding a heavier, more expensive system that relies on additional moving parts. By utilizing multiple lightweight sensors, large area coverage while maintaining good ground sample resolution can be achieved as well as promise a more robust system. This paper will explore the creation and testing of one such system, describe a means by which more advanced systems can be developed, and introduce a metric so as to compare its performance against various modeled systems.

This paper examines the definitions of contrast as variously defined and used in target acquisition. A general definition of contrast is a difference normalized for significance. It is proposed that the difference be described as a signature metric and the normalization factor, which is usually in the same units as the signature metric, be characterized as a contrast reference. The various existing definitions are shown to be consistent with this definition. Common signature metrics are examined across a variety of target conditions. A new signature metric and contrast reference are proposed and examined. The development and use of contrast as a metric in image quality assessment is reviewed. Modern research methods involving image quality and local contrast are also examined and shown to be consistent with the general concept of contrast. We introduce local Michelson contrast, local Maximum Michelson contrast, natural scene statistics (NSS), and local Weber contrast. These local contrast measures produce a feature space that must be analyzed. We examine mean shift analysis as a means for analyzing those feature spaces. We apply mean shift analysis to imagery and discuss its benefits and shortcomings. We propose future paths forward for metric development in assessing target acquisition performance.

Target detection and identification are well-studied problems in the visible (VIS) and near infrared (NIR) bands, with recent work focusing on the short wave IR (SWIR) band. The extended SWIR (eSWIR) band (2 to 2.5 μm) offers an advantage over SWIR due to increased atmospheric transmission, while keeping greater angular resolution than the midwave and longwave IR. eSWIR should additionally improve object-sky contrast due to lower background sky path radiance than the SWIR. An analysis of drone signal-to-noise ratio (SNR) and contrast in the reflective bands is presented and compared to an NVIPM model of drone detection performance using equivalent reflectivities.

Modeling and simulation of the full electro-optical/infrared observation chain remains an incompletely solved problem, and approximations are made at many stages. Including support for current advances in sensor and imaging-system technology with greater spatial, spectral, and temporal resolution only increases the challenge. In this paper we will present results of a US Navy effort to develop an integrated tool that provides enhanced 3D physics-based EO/IR observation chain modeling support for complex dynamic scenes at hyperspectral radiometric fidelity levels, to support research and development in multiple areas of importance for EO and IR imaging systems. A new prototype software system integrates the US Navy TrueView EO/IR/hyperspectral scene simulation and signature modeling tool with the mature US Army Integrated Performance Model (IPM).

Clouds can increase the signal of the background and create non-uniformity behind an airborne target which results in low and varying contrast. Clear sky conditions provide a low noise, uniform background that gives a better chance of detection. In comparison, clouds in the immediate vicinity of a target can decrease the signal to noise ratio (SNR). Understanding key variables of this non-uniform structure can allow for better detection of small UAVs. The presented radiometric and spatial characteristics for both the midwave and longwave bands are the maximum and minimum blackbody equivalent temperature and the distributions of the cloud temperatures. The spatial metrics of measurements are a one-dimensional power spectrum to understand the random spatial structure of the clouds. These cloud properties are measured at night to avoid any solar contributions and obtain their emissive characteristics. An Empirical Model is created to predict cloud radiances in any atmosphere.

In operational environments, pilots of rotorcraft such as the Apache, Blackhawk, and other variants have been involved in catastrophic accidents, due to pilots’ inability to rely on visual indicators for landing. In 2019, the Army reported that over the past 10 years, there have been 87 rotorcraft accidents due Degraded Visual Environments (DVE)—resulting in 122 fatalities and over $1.18B in material losses [1]. This phenomenon poses a formidable hazard to advanced tilt rotor platforms. Dust clouds, rain, and other meteorological effects can obscure or degrade instrument readings and make it difficult for pilots to navigate safely—especially during takeoff and landing. In these types of degraded visibility environments, pilots must depend on instruments for situational awareness, making accurate sensing and reporting crucial to a real-time understanding of the environment. This research is intended to address gaps in the rotorcraft Hardware-InThe-Loop (HWIL) DVE simulation systems currently in use. Specifically, the research is intended to produce a physicsbased realistic representation of DVE conditions for a HWIL simulator to demonstrate the impact of DVE on sensor emulator performances. DVE testing in a simulated environment requires a representation of rotor induced aerosol concentration around the aircraft. Additionally, the simulation requires the ability to visualize the degradation of sensor performance by rotor-induced aerosols [2]. Having end-to-end control over the physical model, it is possible to extend the effects of DVE on sensors beyond just textures and statistical models to physics-based models.

This paper seeks to address whether active or passive tracking is preferable in terms of centroid-track error. Active tracking has the advantage of allowing for SWaP-limited source control to scale SNR. With coherent illumination, however, speckle noise gives rise to a fundamental limit in tracking precision. On the other hand, passive tracking relies on incoherent illumination with speckle-free return. The drawback in this case is that SNR itself is inherently limited, thus limiting precision with respect to tracking measurements. In our analysis, we first present the theory that drives limiting factors of both active and passive tracking schemes. From these limitations we then estimate Strehl ratio at various SNRs for direct comparison of active and passive performance. We consider objects of various shapes and sizes, study both well-resolved and unresolved objects, and anchor our findings to first-order simulation results that demonstrate significance in the design of tracking systems.

Virtual prototyping, using simulation and emulation of potential hardware prior to development, is quickly proving critical to the design and test procedures of novel imaging devices. This concept can also be applied to evaluate inverse problems, such as camera measurement and hardware in the loop testing. In this work, we introduce and discuss a simulation tool capable of capturing with high fidelity the nuances of scene and camera hardware. A variety of potential measurements of the camera system are simulated to replicate laboratory use, taking care to identify the influence of the scene generator on the accuracy of the measurement. This workflow provides valuable feedback for both hardware development as well as new measurement analysis development. Finally, we also describe uses of the tool for new measurements development, system studies and camera component development.

MWIR FPAs often contain a non-negligible number of dead pixels, and even clusters of pixels. In addition to the undesirable cosmetic effect that these flaws have on the output image, it can also be detrimental to downstream image exploitation efforts, including target detection and tracking algorithms. It is therefore necessary to mask such defects as early as possible, before it is exacerbated by the imaging pipeline processes, especially sharpening filters and contrast stretching, which are commonly used. This paper presents the results of an investigation into several dead-pixel replacement schemes of varying complexity, starting with simple replacement by the previous neighbor to interpolation using kernels of varying shapes and domain sizes. These are evaluated for accuracy, but also for their cluster-handling performance, latency impact, and suitability for real-time implementation on resource-constrained FPGA matrices. It is shown that asymmetric predictive kernels, optimized using neural nets or genetic algorithms, can offer significant improvements over naïve last-good-neighbor replacement, while affording improved cluster-handling capability, low FPGA resource requirements, and incurring minimal and even zero latency.

The continued technology push towards smaller pitch devices, the growing application of strained-layer-superlattice devices and the associated lateral carrier diffusion challenges with both trends make infrared (IR) detector resolution evaluation vital to the IR imaging community. Established methods for direct infrared detector modulation transfer function evaluation, namely laser speckle-based power spectral density methods, are reliant on Fresnel electric field propagation equations and are only applicable in regimes where small angle approximations are valid. This limitation prevents analysis of longer wavelength, smaller pixel pitch focal plane arrays (FPA). An alternative methodology is proposed, utilizing speckle autocorrelation functions to estimate the FPA impulse response. The major technique advantage is the input autocorrelation function is derived via Rayleigh-Sommerfeld propagation equations, making this method valid in a wider array of test geometries than conventional speckle-based methods. Therefore, this technique supports resolution estimation of smaller pixel pitch devices than previously possible with established techniques. This effort outlines an iterative maximum likelihood function-based approach proposed for impulse response estimation, demonstrates the proposed technique’s effectiveness via simulation and discusses the challenges associated with implementing the technique experimentally.

The authors propose a digital sensor figure of merit (FOM) to describe the performance of digital night vision sensors operating in low-light conditions with a single-number parameter in a manner directly comparative to the FOM of analog image intensifier tubes (IIT). Currently, this first iteration single-valued digital sensor FOM is the product of measured sensor limiting resolution and sensor signal-to-noise ratio using alternative but equivalent techniques to IIT FOM measurements.

Previous work has shown nonuniformly heated windows cause distortions which affect imaging a transmitted point source as the focused spot has an angular displacement. If this angular displacement is not compensated, the processing of the imaging system’s images of the transmitted point source could be misinterpreted. The in-situ estimation technique uses LWIR pyrometry to image the inside surface of the window, material properties, and fast computational techniques. Results from this in-situ estimation technique are presented and compared with experimentally measured results. The front surface temperatures, transmitted wavefront, and angular deviation of a transmitted point source are compared.

We describe a variable attenuator for use with conventional IR quantum cascade or carbon dioxide lasers to create a source with widely and rapidly controllable effective radiant temperature. This would have application to testing of imagers, which must observe scenes that change rapidly between ambient background and very hot objects. The mechanism is controllably frustrated surface plasmon resonance. The device comprises an IR transparent prism with one face coated by a semitransparent (optically-thin) semiconductor having suitable infrared plasma frequency, followed by a controllable gap to a conventional metal mirror. For the mid-wave infrared band (MWIR, 3-5 micron wavelength), we consider a sapphire hemicylindrical prism coated with the transparent conductor gallium-doped ZnO (GZO). For the long-wave infrared band (LWIR, 8-12 micron wavelength), we consider an undoped-Si prism with one heavily-doped surface. Due to the exponential decay of the surface-plasmon-polariton evanescent wave above the conducting film, the log of internal reflectance of the conducting film decreases linearly with increasing gap, typically by about 1 decade per micron, with a total variation of over 5 orders of magnitude. The effective radiance is determined by laser intensity, reflectance, and reflected-beam divergence. Comparison of the effective radiance values to the band radiance of a black body indicates effective radiant temperatures that can be varied from 300 to over 4000 K for a mirror diameter of 100 (MWIR) or 650 (LWIR) microns. At low effective radiant temperature the device can provide 0.1 K resolution.

Image intensified systems are a compact, low power device that converts visible through near-infrared illumination to visible imagery. These devices provide usable imagery in a variety of ambient illuminations, and they are a preferred means for night imaging. Even though the device consists of objective or relay optics and an image intensified tube, to perform critical measurements on the device performance one needs to dis-assemble the device to perform testing on only the image intensified tube. This is a non-trivial process that requires the hardware to be re-aligned and re-purged during re-assembly. Using proper sources, reference cameras, and image processing techniques, it is possible to fully characterize an image intensified device for its relevant measurable parameters (signal to noise ratio, tube gain, and limiting resolution) without disassembly. This paper outlines the classic component image intensified measurement methodology, assumptions on performance that support those measurement techniques, and the new methodology procedure. A comparison of measurement results using both methods will demonstrate the validity of this new measurement approach.

The characterization process of longwave infrared focal plane arrays (LWIR FPA) requires that some parameters be previously set, in order to evaluate the main figures of merit (FOM). Manufacturers of FPAs set these parameters differently, which makes it difficult to compare the performance of two FPAs from different manufacturers. In this sense, this work investigates the influence of two parameters - integration time and reference temperatures - on four FPA FOM: operability, uniformity, signal transfer function (SiTF), and noise equivalent temperature difference (NETD). The FOM were evaluated for different values of integration time and reference temperatures. As a result, it was found that, in the case of the FPA under test, the values of the FOM provided by the manufacturer's datasheet could be optimized. For reference temperatures and integration time, respectively, set as 20°C, 35°C, and 67 μs, the values obtained for the operability, NETD, and SiTF were 99.98%, 59 mK, and 3.0 mV/K, respectively. On the other hand, for reference temperatures and integration time set as 25°C, 60°C, and 240 μs, the values obtained for the operability, NETD, and SiTF were 99.98%, 30 mK, and 10.2 mV/K.

Numerous improvements have been made to the scene capabilities of ShipIR/NTCS since its early development (Vaitekunas and Lawrence, 1999). This paper will revisit some of the earlier technologies, how they remain largely unchanged except for two important upgrades relating to Open GL 3.0, namely off-screen rendering in hardware using Frame Buffer Objects and 32-bit floating-point colour. The net result is a two order of magnitude (100x) improvement in rendering precision of the infrared scene in ShipIR/NTCS (v4.2). A sample image analysis will investigate the sensitivity of the simulated seeker output to changes in frame buffer resolution (spatial and colour) and the simulation speed.

A key parameter associated with imaging device performance is read noise, which varies for each detector. Even when shot noise is dominant, the read component of noise may be useful as a means of determining which detectors are faulty so that replacement values can be used. Furthermore, the aggregate rate of faulty detectors is useful as a figure-of-merit for the imaging device. Finally, the RMS variability of noise is a commonly used metric of overall imaging device quality. Typically, when raw detector outputs are analyzed in the presence of a stable background, the component of noise that is attributable to individual detectors is stationary, additive and uncorrelated between consecutive frames. Given a finite sequence consisting of N frames, the RMS sampling error in the estimate of the RMS noise σ is σ/sqrt(2N), when N is sufficiently large; this forms the basis of determining the needed value of N. Another consideration is the level of shot noise, since the read noise and the shot noise combine in quadrature. A third factor is the fidelity to which the read noise must be determined. This paper investigates the values of N that are needed for determining the detector noise with sufficient accuracy under varying conditions. Examples using simulated sensor data will be included.

With the advent of cheap and available sensors, there is a need for intelligent sensor selection and placement for various purposes. While previous research was focused on the most efficient sensor networks, we present a new mathematical framework for efficient and resilient sensor network installation. Specifically, in this work we formulate and solve a sensor selection and placement problem when network resilience is also a factor in the optimization problem. Our approach is based on the binary linear programming problem. The generic formulation is probabilistic and applicable to any sensor types, line-of-site and non-line-of-site, and any sensor modality. It also incorporates several realistic constraints including finite sensor supply, cost, energy consumption, as well as specified redundancy in coverage areas that require resilience. While the exact solution is computationally prohibitive, we present a fast algorithm that produces a near-optimal solution that can be used in practice. We show how such formulation works on 2D examples, applied to infrared (IR) sensor networks designed to detect and track human presence and movements in a specified coverage area. Analysis of coverage and comparison of sensor placement with and without resilience considerations is also performed.

This study describes parametric modeling of diffuse reflectance for NIR/SWIR-absorbing dyes on substrates, which is based on diffuse reflectance theory, background subtraction and expansion of the reflectance function by linear combinations of basis functions representing segmented reflectance. The basis functions are constructed using normalized absorbance functions that are determined according to background subtraction and inverse analysis, which is of diffuse reflectance spectra for dyes on fabric. Inverse analysis is effected using the Kubelka-Munk model. Prototype simulations of diffuse reflectance spectra from dyed fabric are described, which demonstrate characteristics of the parametric model.