This PDF file contains the front matter associated with SPIE Proceedings Volume 12097, including the Title Page, Copyright information, Table of Contents, and Conference Committee listings.

With the outstanding accomplishments achieved in recent years, Reinforcement Learning (RL) has become an area where researchers have flocked to in order to find innovative ideas and solutions to challenge and conquer some of the most difficult tasks. While the bulk of the research has been focused on the learning algorithms such as SARSA, Q-Learning, and Genetic, not much attention has been paid to tools used to help these algorithms (e.g. the Experience Replay Buffer). This paper goes over what is believed to be the most accurate Taxonomy of the AI field and briefly covers the Q-Learning algorithm, as it is the base algorithm for this study. Most importantly, it proposes a new Experience Replay Buffer technique, the Round Robin Prioritized Experience Replay Buffer (RRPERB), which aims to help RL agents learn quicker and generalize better to rarely seen states by not completely depriving them of experiences which are ranked as less priority.

Most state-of-the-art Convolutional Neural Networks (CNNs) are bulky and cannot be deployed on resourceconstrained edge devices. In order to leverage the exceptional generalizability of CNNs on edge-devices, they need to be made efficient in terms of memory usage, model size, and power consumption, while maintaining acceptable performance. Neural architecture search (NAS) is a recent approach for developing efficient, edgedeployable CNNs. On the other hand, CNNs used for classification, albeit developed using NAS, often contain large fully-connected (FC) layers with thousands of parameters, contributing to the bulkiness of CNNs. Recent works have shown that FC layers can be compressed, with minimal loss in performance, if any, using tensor processing methods. In this work, for the first time in literature, we leverage tensor methods in the NAS framework to discover efficient CNNs. Specifically, we employ tensor contraction layers (TCLs) to compress fully connected layers in the NAS framework and control the trade-off between compressibility and classification performance by handcrafting the ranks of TCLs. Additionally, we modify the NAS procedure to incorporate automatic TCL rank search in an end-to-end fashion, without human intervention. Our numerical studies on a wide variety of datasets including CIFAR-10, CIFAR-100, and Imagenette (a subset of ImageNet) demonstrate the superior performance of the proposed method in the automatic discovery of CNNs, whose model sizes are manyfold smaller than other cutting-edge mobile CNNs, while maintaining similar classification performance.

As data collected through IoT systems worldwide increases and the deployment of IoT architectures is expanded across multiple domains, novel frameworks that focus on application-based criteria and constraints are needed. In recent years, big data processing has been addressed using cloud-based technology, although such implementations are not suitable for latency-sensitive applications. Edge and Fog computing paradigms have been proposed as a viable solution to this problem, expanding the computation and storage to data centers located at the network's edge and providing multiple advantages over sole cloud-based solutions. However, security and data integrity concerns arise in developing IoT architectures in such a framework, and blockchain-based access control and resource allocation are viable solutions in decentralized architectures. This paper proposes an architecture composed of a multilayered data system capable of redundant distributed storage and processing using encrypted data transmission and logging on distributed internal peer-to-peer networks.

Several classical statistical methods are commonly used for forecasting time-series data. However, due to a number of nonlinear characteristics, forecasting time-series data remains a challenge. Machine learning methods are better able to solve problems with high nonlinearity. RNNs (recurrent neural networks) are frequently used for time-series forecasting because their internal state, or memory, allows them to process a sequence of inputs. Specifically, LSTM (long short term memory), a type of RNN, is particularly useful, as it has both long-term and short-term components. Due to its feedback connections, ability to process a sequence of data of varying lengths, and ability to reset its own state, LSTMs are less sensitive to outliers and more forgiving to varying lags in time. Consequently, LSTMs are able to extract vital information and learn trends to forecast time-series data with high accuracy. We propose a novel neural network architecture using a combination of long short term memory and convolutional layers to predict time-series energy data with higher accuracy than comparable networks.

Digital twin has been envisioned as a key tool to enable data-driven real-time monitoring and prediction, automated modeling as well as zero-touch control and optimization in next-generation wireless networks. However, because of the mismatch between the dynamics in the source domain (i.e., the digital twin) and the target domain (i.e., the real network), policies generated in source domain by traditional machine learning algorithms may suffer from significant performance degradation when applied in the target domain, i.e., the so-called “source-to-target (S2T) gap” problem. In this work we investigate experimentally the S2T gap in digital twin-enabled wireless networks considering a new class of reinforcement learning algorithms referred to as robust deep reinforcement learning. We first design, based on a combination of double deep Q-learning and an R-contamination model, a robust learning framework to control the policy robustness through adversarial dynamics expected in the target domain. Then we test the robustness of the learning framework over UBSim, an event-driven universal simulator for broadband mobile wireless networks. The source domain is first constructed over UBSim by creating a virtual representation of an indoor testing environment at University at Buffalo, and then the target domain is constructed by modifying the source domain in terms of blockage distribution, user locations, among others. We compare the robust learning algorithm with traditional reinforcement learning algorithms in the presence of controlled model mismatch between the source and target domains. Through experiments we demonstrate that, with proper selection of parameter R, robust learning algorithms can reduce significantly the S2T gap, while they can be either too conservative or explorative otherwise. We observe that robust policy transfer is effective especially for target domains with time-varying blockage dynamics.

Deep neural networks (DNNs) designed for computer vision and natural language processing tasks cannot be directly applied to the radio frequency (RF) datasets. To address this challenge, we propose to convert the raw RF data to data types that are suitable for off-the-shelf DNNs by introducing a convolutional transform technique. In addition, we propose a simple 5-layer convolutional neural network architecture (CONV-5) that can operate with raw RF I/Q data without any transformation. Further, we put forward an RF dataset, referred to as RF1024, to facilitate the future RF research. RF1024 consists of 8 different RF modulation classes with each class having 1000/200 training/test samples. Each sample of the RF1024 dataset contains 1024 complex I/Q values. Lastly, the experiments are performed on the RadioML2016 and RF1024 datasets to demonstrate the improved classification performance.

Automatic RF modulation recognition is a primary signal intelligence (SIGINT) technique that serves as a physical layer authentication enabler and automated signal processing scheme for the beyond 5G and military networks. Most existing works rely on adopting deep neural network architectures to enable RF modulation recognition. The application of deep compression for the wireless domain, especially automatic RF modulation classification, is still in its infancy. Lightweight neural networks are key to sustain edge computation capability on resource-constrained platforms. In this letter, we provide an in-depth view of the state-of-the-art deep compression and acceleration techniques with an emphasis on edge deployment for beyond 5G networks. Finally, we present an extensive analysis of the representative acceleration approaches as a case study on automatic radar modulation classification and evaluate them in terms of the computational metrics.

Growing interest in utilizing wireless spectrum makes spectrum access congested, competitive, and often contested making wireless signals vulnerable to various attacks. This compels us to design a secure waveform that solves encryption and modulation as a joint problem. We propose a novel end-to-end symmetric key encryption algorithm where the transmitter encodes the confidential data bits using a shared secret key to generate a secure waveform and the legitimate receiver decrypts the waveform to retrieve the transmitted bits. The trusted pairs are trained adversarially to learn secure data communication by introducing an adversarial NN, that helps to separate the mutual information between secret bits and secured waveform. Cooperative learning takes place between the trusted pair to defeat the adversary and learn encryption and modulation jointly. Complex neural networks are used to build encryption/decryption networks to improve the secrecy-reliability trade-off compared to prior works. Extensive simulated data set is used to train the trusted pair to learn secure data transmission. Our results demonstrate that the trusted pair succeeds in achieving secure data transmission over wireless links while the adversary can not decode or recognize the received waveform.

We examine the problem of transmission control, i.e., when to transmit, in distributed wireless communications networks through the lens of multi-agent reinforcement learning. Most other works using reinforcement learning to control or schedule transmissions use some centralized control mechanism, whereas our approach is fully distributed. Each transmitter node is an independent reinforcement learning agent and does not have direct knowledge of the actions taken by other agents. We consider the case where only a subset of agents can successfully transmit at a time, so each agent must learn to act cooperatively with other agents. An agent may decide to transmit a certain number of steps into the future, but this decision is not communicated to the other agents, so it the task of the individual agents to attempt to transmit at appropriate times. We achieve this collaborative behavior through studying the effects of different actions spaces. We are agnostic to the physical layer, which makes our approach applicable to many types of networks. We submit that approaches similar to ours may be useful in other domains that use multi-agent reinforcement learning with independent agents.

In the rapidly advancing field of autonomous systems, real-time operation and monitoring in non-stationary environments frequently relies on analysis (filtering/prediction) of short sequences of sensed data that may be partly unreliable, missing, or faulty. Hankel-matrix representation and decomposition is a model-free approach that is becoming increasingly popular for the analysis of time-series data taking advantage of the progress in linear algebra methods in past years. In this work, we establish that novel L1-norm decompositions of Hankel matrices offer sturdy resistance against partially faulty sensed sequences and, therefore, creates a strong new framework for robust real-time monitoring of autonomous systems. The findings in this paper are illustrated and supported by extensive experimentation on artificial data.

In large-scale enterprise WiFi deployments, performance anomalies need to be detected autonomously without human intervention. The state of the art in detection and diagnosis of faults is a manual and slow process. Experts with domain-knowledge use alarms, traces, and Key Performance Indicator (KPI) data to understand the anomalies. In large-scale deployments, this is becoming an increasingly inefficient approach. The objective of this study is to determine anomalies in a WiFi network by exploiting big data, machine learning (ML) and deep learning techniques. Our work comprises of setting up WiFi access points and retrieving the KPI of client devices which are connected to access points. The metrics for client devices, access points and channel nodes are periodically logged on the cloud, and collected through an API. We develop a multivariate time series data which undergoes data preprocessing, such as data cleaning and data transformation prior to using ML algorithms. To detect anomalous instances, an unsupervised algorithm, Density Based Spatial Clustering of Applications with Noise (DBSCAN) is used. Clusters obtained through DBSCAN are analyzed under dimensionality reduction technique t-Distributed Stochastic Neighbor Embedding. While DBSCAN helps to obtain the outliers, the output of the dimensionality reduction technique enable us to coherently comprehend the dynamics of clusters. Our methodology involves a thorough analysis of the outliers from the dataset through legacy supervised and self-supervised ML models, as well as ensemble learning techniques.

Medical image analysis continues to evolve at an unprecedented rate with the integration of contemporary computer systems. Image registration is fundamental to the task of medical image analysis. Traditional methods of medical image registration are extremely time consuming and at times can be inaccurate. Novel techniques, including the amalgamation of machine learning, have proven to be fast, accurate and reliable. However, supervised learning models are difficult to train due to the lack of ground truth data. Therefore, researchers have endeavoured to explore variant avenues of machine learning, including the implementation of unsupervised learning. In this paper, we continue to explore the use of unsupervised learning for the task of image registration across medical imaging. We postulate that a greater focus on channel-wise data can largely improve model performance. To this end, we employ a sequence generation model, a squeeze excitation network, a convolutional neural network variation of long-short term memory and a spatial transformer network for a channel optimized image registration architecture. To test the proposed approach, we utilize a dataset of 2D brain scans and compare the results against a state-of-the-art baseline model.

Video data has occupied people’s daily professional and entertainment activities. It imposes a big pressure on the Internet bandwidth. Hence, it is important to develop effective video coding techniques to compress video data as much as possible and save the transmission bandwidth, while still providing visually pleasing decoded videos. In conventional video coding such as the high efficiency video coding (HEVC) and the versatile video coding (VVC), signal processing and information theory-based techniques are mainstream. In recent years, thanks to the advances in deep learning, a lot of deep learning-based approaches have emerged for image and video compression. In particular, the generative adversarial networks (GAN) have shown superior performance for image compression. The decoded images are usually sharper and present more details than pure convolutional neural network (CNN)-based image compression and are more consistent with human visual system (HVS). Nevertheless, most existing GAN-based methods are for still image compression, and truly little research investigates the potential of GAN for video compression. In this work, we propose a novel inter-frame video coding scheme that compresses both reference frames and target (residue) frames by GAN. Since residue signals contain less energy, the proposed method effectively reduces the bit rates. Meanwhile, since we adopt adversarial learning, the perceptual quality of decoded target frames is well-preserved. The effectiveness of our proposed algorithm is demonstrated by experimental studies on common test video sequences.

In this paper, we consider the problem of Poisson noise suppression in underwater laser line scan (LLS) imagery for image quality enhancement. We investigate two different denoising neural network architectures, one based on a convolutional autoencoder (CAE) and a commonly used convolutional neural network (CNN) based denoiser. Due to the relative abundance of camera images over underwater LLS imagery, we employ transfer learning with camera images as training data for the CAE and CNN architectures. Poisson noise is introduced in these training images at varying levels to mimic a noise-dominated LLS system. Images from an underwater scene under different turbidity conditions are used for testing. Using a composite loss function consisting of l1 and l2 norms, we demonstrate the noise suppression capabilities of both architectures in underwater LLS images via transfer learning. The results show that the CAE outperforms the CNN denoiser qualitatively and quantitatively in terms of the contrast ratio and contrast signal-to-noise ratio.

Modern displays are steadily increasing in resolution, though sensors can be prohibitively expensive to capture images and video at such high resolutions. Image super-resolution, or upsampling, has recently been applied to alleviate these shortcomings. There exist many deep learning image super-resolution models that reconstruct very high quality high-resolution images from a low-resolution base. However, most of these models use a tremendous amount of parameters, requiring a large amount of free memory and computational power to super resolve a single image. As a result, many modern super-resolution models are not entirely practical due to the computational or memory usage requirements. We propose a highly efficient, small super-resolution model utilizing the sub-pixel convolution block for single image super-resolution.

In recent years, the computational power of handheld devices has increased rapidly to the point of parity with computers of only a generation ago. The multiple tools integrated into these devices and the progressive expansion of cloud storage have created a need for novel compressing techniques for both storage and transmission. In this work, a novel L₁ principal component analysis (PCA) informed K-means approach is proposed. This new technique seeks to preserve the color definition of images through the application of K-means clustering algorithms. Assessment of the efficacy is carried out utilizing the structural similarity index (SSIM).

Compressed sensing theory (CS) is the most sensational topic of scientific research in the past century. CS relies on sparse representation and L1-minimization. This reliance leads to innate weaknesses in applicability. We hereby make insightful analysis on the CS weaknesses. This is the first appearance in the literature. Based on insight into CS weaknesses, we stride forward and exemplify a new mathematical theory and method for high efficiency sensing in contrast. It remedies the CS weaknesses with radically rectified mathematical rationale, which immensely improves technical performance in terms of both data quality and computation speed. High efficiency means high quality plus high speed. The pivotal innovation is simple yet powerful. Demo software and test data are downloadable at www.lucidsee.ca.

This article presents the experimental study of a biomass sensor to monitor the growth of macroalgae (seaweeds) in the Integrated Multi-Trophic Aquaculture (IMTA) at Harbor Branch Oceanographic Institute at Florida Atlantic University. Pseudorandom Encoded-light for Evaluating Biomass (PEEB) utilizes the measurements from a sequence of encoded light flashes to quantify the seaweed biomass. Such configuration ensures the sensor provides robust automated data acquisition under different ambient conditions and biomass densities. This data will be used to support a machine learning-based prediction biomass model, critical in any commercial-scale IMTA farm. A PEEB sensor based on an improved design has been developed based on an earlier feasibility study. The design of such a system and the initial tests at the macroalgal seaweed cultivation raceway in the HBOI IMTA system are discussed.

Aerial drones have great potential to monitor large areas quickly and efficiently. Aquaculture is an industry that requires continuous water quality data to successfully grow and harvest fish. The Hybrid Aerial Underwater Robotic System (HAUCS) is designed to collect water quality data of aquaculture ponds to reduce labor costs for farmers. The routing of drones to cover each fish pond on an aquaculture farm can be reduced to the Vehicle Routing Problem. A dataset is created to simulate the distribution of ponds on a farm and is used to assess the HAUCS Path Planning Algorithm (HPP). Its performance is compared with the Google Linear Optimization Package (GLOP) and a Graph Attention Model (GAM) for routing problems. GLOP is the most efficient solver for 50 to 200 ponds at the expense of long run times, while HPP outperforms the other methods in solution quality and run time for instances larger than 200 ponds.

Light field visualization technology has started emerging through commercially-available devices, and while its appearance on the consumer market is somewhat already on the horizon, the day light field displays integrate into our quotidian activities is still yet to come. However, just because light field displays are not part of our everyday lives yet, that does not mean that they do not already contribute to professional usage contexts. One such use case category is defense applications, where the daunting development and manufacturing expenses of these devices do not really intimidate the available budget. In fact, modern warfare is heavily investing into novel visualization technologies and cutting-edge innovations for both those who serve on the battlefield and those who devise strategic maneuvers and tactical decisions. The latter is essentially the process of making decisions based on a vast amount of data. The success of such process is fundamentally affected by the efficiency of the delivery of the available and the projected information. In this paper, we present our work on light field battlespace visualization and other relevant applications of light field for defense and warfare purposes. Beside conventional approaches, we propose multiple alternative solutions that best fit the investigated use cases. Our work prioritizes visualization quality and user interaction. Regarding system attributes, particular attention is paid to the field of view and the angular resolution of such devices. We separately address the utilization of commercial devices and the use-case-centric design of dedicated hardware.

High dynamic range imaging has become a technological trend in the past couple of decades, particularly through its integration into many applications. Numerous attempts were made to reconstruct HDR images from lowdynamic-range data. Such reconstruction techniques can be classified into single-camera and multi-camera approaches. Single-camera setups are less expensive, yet multi-camera setups are more efficient. At the time of this paper, there is already a great number of algorithms for single-camera HDR image reconstruction, but there are only a few for HDR video reconstruction. The latter takes into account the temporal coherence between consecutive video frames, leading to better results. For light field images, this remains a challenging open issue, as the HDR video reconstruction methods do not work as efficiently for light field images as HDR image reconstruction algorithms do. However, analogously to 2D videos, where consecutive frames have temporal coherence, many similarities can be found between the adjacent views of light field contents. In this paper, we investigate the theoretical possibilities of combining CNN architectures utilized for HDR images and videos, in order to enhance the outputs of HDR light field image reconstruction. The concept of our work is to exploit the similarities between light field images since they all visualize the same scene from different angular perspectives.

This paper investigates RF-based system for automatic American Sign Language (ASL) recognition. We consider radar for ASL by joint spatio-temporal preprocessing of radar returns using time frequency (TF) analysis and high-resolution receive beamforming. The additional degrees of freedom offered by joint temporal and spatial processing using a multiple antenna sensor can help to recognize ASL conversation between two or more individuals. This is performed by applying beamforming to collect spatial images in an attempt to resolve individuals communicating at the same time through hand and arm movements. The spatio-temporal images are fused and classified by a convolutional neural network (CNN) which is capable of discerning signs performed by different individuals even when the beamformer is unable to separate the respective signs completely. The focus group comprises individuals with varying expertise with sign language, and real time measurements at 77 GHz frequency are performed using Texas Instruments (TI) cascade radar.

For the past 4 decades the MIT-BIH dataset has become the industry standard for the analysis of a comparative metric of signal processing and machine learning techniques. This is because medical data is difficult to collect and use because it is not widely available and open-source. There exists a need to standardize the metric for comparative reasons. This paper proposes a set of datasets targeted at specific tasks currently under investigation in state-of-the-art works. The open sharing of these datasets in multiple formats will allow for the application of the benchmark data to multiple advanced classification algorithms. Published methods will be profiled using this new dataset building the foundation for its merit. A series of datasets are identified with applicable criteria as to their usage such as, TinyML for health monitoring and detection of heart disease.

Light field visualization technology has progressed significantly in the past two decades. With the emergence of commercially-available devices, both industry and academia have begun research on the potential use cases of future society, including medical visualization, 3D digital signage, telepresence, military applications and many more. During the evaluation and quality assessment of such usage contexts and display types, test participants are typically screened for visual acuity via the Snellen chart and color vision via the Ishihara plates. However, there is an unfortunate global trend that the eyesight of the new generations is getting notably worse, and other sight-related issues, such as color vision deficiency, are becoming more common as well. Therefore, while medical technologies do relentlessly combat the diseases and disorders of the human eye, long-term innovations of visualization must also account for such users. Yet at the time of this paper, those with imperfect vision are underrepresented in light field research. In this paper, we present the results of the series of subjective tests carried out on light field displays, exclusively with test participants that otherwise would not qualify to assess visualization quality in a regular study. The experiments aim at investigating the most relevant research questions of light field visualization quality, such as spatial resolution, angular resolution and viewing distance. Test participants with imperfect visual acuity are classified by the diopters of their corrective lenses, and correlations between diopters and subjective ratings are addressed. Similar analyses were performed for color-blind test participants as well.

Non-invasive classification of cultured cells is an important task for in vitro drug panel screening, phenotypic profiling for manufacturing therapeutic cell lines/proteins, and the evaluation of culture health and crosscontamination. Holographic microscopy and machine learning (ML) are attractive tools for these applications but must address two challenges: high performance in multi-class classification problems (< 2 classes), and identification of important image-derived features for a given classification problem. This paper aims to achieve high performance classification of four cell lines, two breast cancer (MDA-MB-231 and MCF-7) and two noncancer (human gingival fibroblast, HGF; and human gingival keratinocytes, GIE no3B11), with varied epithelial and mesenchymal morphologies; to determine the best machine learning model for this classification; and to identify features most strongly influencing model performance. We trained and evaluated three ML algorithms: Support Vector Machines (SVMs) with various kernels, Random Forest, and AdaBoost, using a previously defined set of 17 features derived from holographic microscopy images; selected components after Principal Component Analysis (PCA); and a subset of original features after feature selection. Grid searching was conducted to determine the optimal set of hyperparameters for each machine learning algorithm before training. The multi-class ML model created using the Random Forest algorithm was reliable and had an average F1 score of 0.89, 0.86, 0.84, and 0.84 for GIE, HGF, MCF7, and MDA cell lines respectively. Moreover, with the feature selection technique, the model performance for each cell line can be further improved. Top important features were geometric (area, perimeter, eccentricity), histogram-based (skew) and textural (contrast, correlation, homogeneity).