New articles on Electrical Engineering and Systems Science

[1] 2112.00729

Total-Body Low-Dose CT Image Denoising using Prior Knowledge Transfer Technique with Contrastive Regularization Mechanism

Reducing the radiation exposure for patients in Total-body CT scans has attracted extensive attention in the medical imaging community. Given the fact that low radiation dose may result in increased noise and artifacts, which greatly affected the clinical diagnosis. To obtain high-quality Total-body Low-dose CT (LDCT) images, previous deep-learning-based research work has introduced various network architectures. However, most of these methods only adopt Normal-dose CT (NDCT) images as ground truths to guide the training of the denoising network. Such simple restriction leads the model to less effectiveness and makes the reconstructed images suffer from over-smoothing effects. In this paper, we propose a novel intra-task knowledge transfer method that leverages the distilled knowledge from NDCT images to assist the training process on LDCT images. The derived architecture is referred to as the Teacher-Student Consistency Network (TSC-Net), which consists of the teacher network and the student network with identical architecture. Through the supervision between intermediate features, the student network is encouraged to imitate the teacher network and gain abundant texture details. Moreover, to further exploit the information contained in CT scans, a contrastive regularization mechanism (CRM) built upon contrastive learning is introduced.CRM performs to pull the restored CT images closer to the NDCT samples and push far away from the LDCT samples in the latent space. In addition, based on the attention and deformable convolution mechanism, we design a Dynamic Enhancement Module (DEM) to improve the network transformation capability.

[2] 2112.00730

Highly accelerated MR parametric mapping by undersampling the k-space and reducing the contrast number simultaneously with deep learning

Purpose: To propose a novel deep learning-based method called RG-Net (reconstruction and generation network) for highly accelerated MR parametric mapping by undersampling k-space and reducing the acquired contrast number simultaneously. Methods: The proposed framework consists of a reconstruction module and a generative module. The reconstruction module reconstructs MR images from the acquired few undersampled k-space data with the help of a data prior. The generative module then synthesizes the remaining multi-contrast images from the reconstructed images, where the exponential model is implicitly incorporated into the image generation through the supervision of fully sampled labels. The RG-Net was evaluated on the T1\r{ho} mapping data of knee and brain at different acceleration rates. Regional T1\r{ho} analysis for cartilage and the brain was performed to access the performance of RG-Net. Results: RG-Net yields a high-quality T1\r{ho} map at a high acceleration rate of 17. Compared with the competing methods that only undersample k-space, our framework achieves better performance in T1\r{ho} value analysis. Our method also improves quality of T1\r{ho} maps on patient with glioma. Conclusion: The proposed RG-Net that adopted a new strategy by undersampling k-space and reducing the contrast number simultaneously for fast MR parametric mapping, can achieve a high acceleration rate while maintaining good reconstruction quality. The generative module of our framework can also be used as an insert module in other fast MR parametric mapping methods. Keywords: Deep learning, convolutional neural network, fast MR parametric mapping

[3] 2112.00735

Reference-guided Pseudo-Label Generation for Medical Semantic Segmentation

Producing densely annotated data is a difficult and tedious task for medical imaging applications. To address this problem, we propose a novel approach to generate supervision for semi-supervised semantic segmentation. We argue that visually similar regions between labeled and unlabeled images likely contain the same semantics and therefore should share their label. Following this thought, we use a small number of labeled images as reference material and match pixels in an unlabeled image to the semantics of the best fitting pixel in a reference set. This way, we avoid pitfalls such as confirmation bias, common in purely prediction-based pseudo-labeling. Since our method does not require any architectural changes or accompanying networks, one can easily insert it into existing frameworks. We achieve the same performance as a standard fully supervised model on X-ray anatomy segmentation, albeit 95% fewer labeled images. Aside from an in-depth analysis of different aspects of our proposed method, we further demonstrate the effectiveness of our reference-guided learning paradigm by comparing our approach against existing methods for retinal fluid segmentation with competitive performance as we improve upon recent work by up to 15% mean IoU.

[4] 2112.00736

Ghost Imaging Based on Recurrent Neural Network

Benefit from the promising features of second-order correlation, ghost imaging (GI) has received extensive attentions in recent years. Simultaneously, GI is affected by the poor trade-off between sampling rate and imaging quality. The traditional image reconstruction method in GI is to accumulate the action result of each speckle and the corresponding bucket signal. We found that the image reconstruction process of GI is very similar to the Recurrent Neural Network (RNN), which is one of the deep learning algorithm. In this paper, we proposed a novel method that effectively implements GI on the RNN architecture, called GI-RNN. The state of each layer in RNN is determined by the output of the previous layer and the input of this layer, and the output of the network is the sum of all previous states. Therefore, we take the speckle of each illumination and the corresponding bucket signal as the input of each layer, and the output of the network is the sum of all previous speckle and bucket signal, which is the image of the target. The testing results show that the proposed method can achieve image reconstruction at a very low sampling rate (0.38$\%$). Moreover, we compare GI-RNN with traditional GI algorithm and compressed sensing algorithm. The results of different targets show that GI-RNN is 6.61 dB higher than compressed sensing algorithm and 12.58 dB higher than traditional GI algorithm on average. In our view, the proposed method makes an important step to applications of GI.

[5] 2112.00738

Aiding Medical Diagnosis Through the Application of Graph Neural Networks to Functional MRI Scans

Graph Neural Networks (GNNs) have been shown to be a powerful tool for generating predictions from biological data. Their application to neuroimaging data such as functional magnetic resonance imaging (fMRI) scans has been limited. However, applying GNNs to fMRI scans may substantially improve predictive accuracy and could be used to inform clinical diagnosis in the future. In this paper, we present a novel approach to representing resting-state fMRI data as a graph containing nodes and edges without omitting any of the voxels and thus reducing information loss. We compare multiple GNN architectures and show that they can successfully predict the disease and sex of a person. We hope to provide a basis for future work to exploit the power of GNNs when applied to brain imaging data.

[6] 2112.00794

DFTS2: Simulating Deep Feature Transmission Over Packet Loss Channels

In edge-cloud collaborative intelligence (CI), an unreliable transmission channel exists in the information path of the AI model performing the inference. It is important to be able to simulate the performance of the CI system across an imperfect channel in order to understand system behavior and develop appropriate error control strategies. In this paper we present a simulation framework called DFTS2, which enables researchers to define the components of the CI system in TensorFlow~2, select a packet-based channel model with various parameters, and simulate system behavior under various channel conditions and error/loss control strategies. Using DFTS2, we also present the most comprehensive study to date of the packet loss concealment methods for collaborative image classification models.

[7] 2112.00857

Phasor Modelling Approaches and Simulation Guidelines of Voltage-Source Converters in Grid-Integration Studies

This paper reviews Voltage-Source Converters (VSCs) EMT and Phasor models currently used to simulate converter-interfaced generation (CIG) and renewable energy resources integration to power systems. Several modelling guidelines and suitability analyses were provided based on a comprehensive comparative study among the models. Various studies were performed in a small system and a large system, modelled in Simulink. We address a gap related to the suitability of CIGs phasor models in studies where the boundary between electromagnetic and electromechanical transients overlap. An insightful analysis of the adequate simulation time step for each model and study is also provided, along with several simulation guidelines.

[8] 2112.00860

Robust Resource-Aware Self-triggered Model Predictive Control

The wide adoption of wireless devices in the Internet of Things requires controllers that are able to operate with limited resources, such as battery life. Operating these devices robustly in an uncertain environment, while managing available resources, increases the difficultly of controller design. This paper proposes a robust self-triggered model predictive control approach to optimize a control objective while managing resource consumption. In particular, a novel zero-order-hold aperiodic discrete-time feedback control law is developed to ensure robust constraint satisfaction for continuous-time linear systems.

[9] 2112.00862

Modelling and Simulation of Power Systems with Grid-Connected Converters in OpenModelica

This paper analyses the capabilities of the OpenModelica environment to perform electromagnetic transient (EMT) type simulations of power transmission systems with grid-connected converters. A power transmission system has been modelled and simulated in OpenModelica and Simulink to compare both tools in terms of accuracy, robustness, flexibility and computational performance. Power system transient studies such as faults and switching of capacitor banks have been performed. The results confirmed an excellent overall agreement between both software and demonstrated a remarkable potential for using OpenModelica for EMT-type modelling and simulation of future power electronic dominated grids.

[10] 2112.00867

Modelling Approaches of Power Systems Considering Grid-Connected Converters and Renewable Generation Dynamics

This paper presents a comparative analysis of several modelling approaches of key elements used in simulations of power systems with renewable energy sources. Different models of synchronous generators, transmission lines, converters, wind generators and photovoltaic (PV) power plants are compared to assess the most suitable models for grid-connection studies. It also analyses how the dynamics of PV power plants and the mechanical dynamics of wind generators affect the electrical variables on the grid side. The models were compared in terms of precision and computational time through simulations of load connection, short-circuits, disconnection of generators and lines in a benchmark system modelled in Simulink.

[11] 2112.00869

Definition of scenarios for modern power systems with a high renewable energy share

Recent environmental policies have led many academic, industrial and governmental stakeholders to design and plan scenarios with very high share of renewable energy sources (RES). New system elements such as High-voltage Direct Current (HVDC) transmission, Microgrids, Virtual Power Plants (VPP) and Dynamic Virtual Power Plants (DVPP) are being increasingly studied with respect to their contribution to integrate future RES plants in the main grid. Several future scenarios are being analysed for each system and region, to ensure that the future energy systems, composed mostly of RES, can remain stable, match the demand during the seasonal variations across the year and are economically feasible. In this article, different types of energy scenarios are considered to obtain a range of options in terms of size, renewable generation technologies, and electrical network configuration. The scenarios were studied in the context of the POSYTYF project and were quantified through an optimization-based algorithm, using specific locations in Europe, and real data related to the availability of different RES, as well as the demand. It has been shown that photovoltaic (PV) and wind generation can provide the renewable backbone but they lack the flexibility needed to achieve a very high share in the energy mix. Other technologies, such as solar thermal and pumped hydro, become important to cover the last range of integration, as they provide a high flexibility, which is crucial for high share.

[12] 2112.00883

Coded Estimation: Design of Backscatter Array Codes for 3D Orientation Estimation

We consider the problem of estimating the orientation of a 3D object with the assistance of configurable backscatter tags. We explore the idea of designing tag response codes to improve the accuracy of orientation estimation. To minimize the difference between the true and estimated orientation, we propose two code design criteria. We also derive a lower bound on the worst-case error using Le Cam's method and provide simulation results for multiple scenarios including line-of-sight only and multipath, comparing the theoretical bounds to those achieved by the designs.

[13] 2112.00913

CDLNet: Noise-Adaptive Convolutional Dictionary Learning Network for Blind Denoising and Demosaicing

Deep learning based methods hold state-of-the-art results in low-level image processing tasks, but remain difficult to interpret due to their black-box construction. Unrolled optimization networks present an interpretable alternative to constructing deep neural networks by deriving their architecture from classical iterative optimization methods without use of tricks from the standard deep learning tool-box. So far, such methods have demonstrated performance close to that of state-of-the-art models while using their interpretable construction to achieve a comparably low learned parameter count. In this work, we propose an unrolled convolutional dictionary learning network (CDLNet) and demonstrate its competitive denoising and joint denoising and demosaicing (JDD) performance both in low and high parameter count regimes. Specifically, we show that the proposed model outperforms state-of-the-art fully convolutional denoising and JDD models when scaled to a similar parameter count. In addition, we leverage the model's interpretable construction to propose a noise-adaptive parameterization of thresholds in the network that enables state-of-the-art blind denoising performance, and near perfect generalization on noise-levels unseen during training. Furthermore, we show that such performance extends to the JDD task and unsupervised learning.

[14] 2112.00926

Long-Term Recurrent Convolutional Network-based Inertia Estimation using Ambient Measurements

Conventional synchronous machines are gradually replaced by converter-based renewable resources. As a result, synchronous inertia, an important time-varying quantity, has substantially more impact on modern power systems stability. The increasing integration of renewable energy resources imports different dynamics into traditional power systems; therefore, the estimation of system inertia using mathematical model becomes more difficult. In this paper, we propose a novel learning-assisted inertia estimation model based on long-term recurrent convolutional network (LRCN) that uses system wide frequency and phase voltage measurements. The proposed approach uses a non-intrusive probing signal to perturb the system and collects ambient measurements with phasor measurement units (PMU) to train the proposed LRCN model. Case studies are conducted on the IEEE 24-bus system. Under a signal-to-noise ratio (SNR) of 60dB condition, the proposed LRCN based inertia estimation model achieves an accuracy of 97.56% with a mean squared error (MSE) of 0.0552. Furthermore, with a low SNR of 45dB, the proposed learning-assisted inertia estimation model is still able to achieve a high accuracy of 93.07%.

[15] 2112.01023

A higher order Minkowski loss for improved prediction ability of acoustic model in ASR

Conventional automatic speech recognition (ASR) system uses second-order minkowski loss during inference time which is suboptimal as it incorporates only first order statistics in posterior estimation [2]. In this paper we have proposed higher order minkowski loss (4th Order and 6th Order) during inference time, without any changes during training time. The main contribution of the paper is to show that higher order loss uses higher order statistics in posterior estimation, which improves the prediction ability of acoustic model in ASR system. We have shown mathematically that posterior probability obtained due to higher order loss is function of second order posterior and thus the method can be incorporated in standard ASR system in an easy manner. It is to be noted that all changes are proposed during test(inference) time, we do not make any change in any training pipeline. Multiple baseline systems namely, TDNN1, TDNN2, DNN and LSTM are developed to verify the improvement incurred due to higher order minkowski loss. All experiments are conducted on LibriSpeech dataset and performance metrics are word error rate (WER) on "dev-clean", "test-clean", "dev-other" and "test-other" datasets.

[16] 2112.01025

A Mixture of Expert Based Deep Neural Network for Improved ASR

This paper presents a novel deep learning architecture for acoustic model in the context of Automatic Speech Recognition (ASR), termed as MixNet. Besides the conventional layers, such as fully connected layers in DNN-HMM and memory cells in LSTM-HMM, the model uses two additional layers based on Mixture of Experts (MoE). The first MoE layer operating at the input is based on pre-defined broad phonetic classes and the second layer operating at the penultimate layer is based on automatically learned acoustic classes. In natural speech, overlap in distribution across different acoustic classes is inevitable, which leads to inter-class mis-classification. The ASR accuracy is expected to improve if the conventional architecture of acoustic model is modified to make them more suitable to account for such overlaps. MixNet is developed keeping this in mind. Analysis conducted by means of scatter diagram verifies that MoE indeed improves the separation between classes that translates to better ASR accuracy. Experiments are conducted on a large vocabulary ASR task which show that the proposed architecture provides 13.6% and 10.0% relative reduction in word error rates compared to the conventional models, namely, DNN and LSTM respectively, trained using sMBR criteria. In comparison to an existing method developed for phone-classification (by Eigen et al), our proposed method yields a significant improvement.

[17] 2112.01127

Wide-Sense Stationarity in Generalized Graph Signal Processing

We consider statistical graph signal processing (GSP) in a generalized framework where each vertex of a graph is associated with an element from a Hilbert space. This general model encompasses various signals such as the traditional scalar-valued graph signal, multichannel graph signal, and discrete- and continuous-time graph signals, allowing us to build a unified theory of graph random processes. We introduce the notion of joint wide-sense stationarity in this generalized GSP framework, which allows us to characterize a graph random process as a combination of uncorrelated oscillation modes across both the vertex and Hilbert space domains. We elucidate the relationship between the notions of wide-sense stationarity in different domains, and derive the Wiener filters for denoising and signal completion under this framework. Numerical experiments on both real and synthetic datasets demonstrate the utility of our generalized approach in achieving better estimation performance compared to traditional GSP or the time-vertex framework.

[18] 2112.01134

Data Stream Stabilization for Optical Coherence Tomography Volumetric Scanning

Optical Coherence Tomography (OCT) is an emerging medical imaging modality for luminal organ diagnosis. The non-constant rotation speed of optical components in the OCT catheter tip causes rotational distortion in OCT volumetric scanning. By improving the scanning process, this instability can be partially reduced. To further correct the rotational distortion in the OCT image, a volumetric data stabilization algorithm is proposed. The algorithm first estimates the Non-Uniform Rotational Distortion (NURD) for each B-scan by using a Convolutional Neural Network (CNN). A correlation map between two successive B-scans is computed and provided as input to the CNN. To solve the problem of accumulative error in iterative frame stream processing, we deploy an overall rotation estimation between reference orientation and actual OCT image orientation. We train the network with synthetic OCT videos by intentionally adding rotational distortion into real OCT images. As part of this article we discuss the proposed method in two different scanning modes: the first is a conventional pullback mode where the optical components move along the protection sheath, and the second is a self-designed scanning mode where the catheter is globally translated by using an external actuator. The efficiency and robustness of the proposed method are evaluated with synthetic scans as well as real scans under two scanning modes.

[19] 2112.01137

Deep Learning-Based Carotid Artery Vessel Wall Segmentation in Black-Blood MRI Using Anatomical Priors

Carotid artery vessel wall thickness measurement is an essential step in the monitoring of patients with atherosclerosis. This requires accurate segmentation of the vessel wall, i.e., the region between an artery's lumen and outer wall, in black-blood magnetic resonance (MR) images. Commonly used convolutional neural networks (CNNs) for semantic segmentation are suboptimal for this task as their use does not guarantee a contiguous ring-shaped segmentation. Instead, in this work, we cast vessel wall segmentation as a multi-task regression problem in a polar coordinate system. For each carotid artery in each axial image slice, we aim to simultaneously find two non-intersecting nested contours that together delineate the vessel wall. CNNs applied to this problem enable an inductive bias that guarantees ring-shaped vessel walls. Moreover, we identify a problem-specific training data augmentation technique that substantially affects segmentation performance. We apply our method to segmentation of the internal and external carotid artery wall, and achieve top-ranking quantitative results in a public challenge, i.e., a median Dice similarity coefficient of 0.813 for the vessel wall and median Hausdorff distances of 0.552 mm and 0.776 mm for lumen and outer wall, respectively. Moreover, we show how the method improves over a conventional semantic segmentation approach. These results show that it is feasible to automatically obtain anatomically plausible segmentations of the carotid vessel wall with high accuracy.

[20] 2112.01173

Directional Lifting Wavelet Transform for Image Edge Analysis

In this paper, we propose a new two-dimensional directional discrete wavelet transform that can decompose an image into 12 multiscale directional edge components. The proposed transform is designed in a fully discrete setting and thus is easy to implement in actual computations. The proposed transform is viewed as a category of redundant discrete wavelet transforms implemented by fast in-place computational algorithms by a lifting scheme that has been modified to incorporate redundancy. The redundancy is limited to $(N \times J+1)/4$, where $N=12$ is the directional selectivity and $J$ is a decomposition level of the multiscale transform. Numerical experiments in edge detection using various images demonstrate the advantages of the proposed method over some conventional standard methods. The proposed method outperforms several conventional edge detection methods in identifying both the location and orientation of edges, and well captures the directional and geometrical features of images.

[21] 2112.01183

Shallow geothermal energy potential for heating and cooling of buildings with regeneration under climate change scenarios

Shallow ground-source heat pumps (GSHPs) are a promising technology for contributing to the decarbonisation of the energy sector. In heating-dominated climates, the combined use of GSHPs for both heating and cooling increases their technical potential, defined as the maximum energy that can be exchanged with the ground, as the re-injection of excess heat from space cooling leads to a seasonal regeneration of the ground. This paper proposes a new approach to quantify the technical potential of GSHPs, accounting for effects of seasonal regeneration, and to estimate the useful energy to supply building energy demands at regional scale. The useful energy is obtained for direct heat exchange and for district heating and cooling (DHC) under several scenarios for climate change and market penetration levels of cooling systems. The case study in western Switzerland suggests that seasonal regeneration allows for annual maximum heat extraction densities above 300 kWh/m$^2$ at heat injection densities above 330 kWh/m$^2$. Results also show that GSHPs may cover up to 55% of heating demand while covering 57% of service-sector cooling demand for individual GSHPs in 2050, which increases to around 85% with DHC. The regional-scale results may serve to inform decision making on strategic areas for installing GSHPs.

[22] 2112.01188

Cost Functions over Feasible Power Transfer Regions of Virtual Power Plants

A virtual power plant (VPP) facilitates the integration of distributed energy resources (DERs) for the transmission-level operation. A challenge in operating a VPP is to characterize the cost function over its feasible power transfer region under DERs' uncertainties. To address this challenge, a characterization method is presented in this paper for the intraday operation of a VPP based on the concepts of nonanticipativity and robustness to DERs' uncertainties. The characterization stems from designing a second-order cone programming (SOCP) problem, based on which a feasible power transfer region across all time periods is constructed by exploring boundary points at each time period and establishing time coupling constraints. Furthermore, a cost function over the feasible power transfer region is formulated as a convex piecewise surface whose breakpoints are obtained by solving SOCP problems, together with a constant compensation cost from a linear programming problem. Finally, to alleviate the heavy computational burden brought by numerous DERs, an approximation method is presented by identifying the critical DERs whose uncertainties have dominant impacts. The effectiveness of the presented methods is verified by the numerical experiments in a 3-bus system and the IEEE 136-bus system.

[23] 2112.01204

Formal verification of a controller implementation in fixed-point arithmetic

For the implementations of controllers on digital processors, certain limitations, e.g. in the instruction set and register length, need to be taken into account, especially for safety-critical applications. This work aims to provide a computer-certified inductive definition for the control functions that are implemented on such processors accompanied with the fixed-point data type in a proof assistant. Using these inductive definitions we formally ensure correct realization of the controllers on a digital processor. Our results guarantee overflow-free computations of the implemented control algorithm. The method presented in this paper currently supports functions that are defined as polynomials within an arbitrary fixed-point structure. We demonstrate the verification process in the case study on an example with different scenarios of fixed-point type implementations.

[24] 2112.01253

Youla-REN: Learning Nonlinear Feedback Policies with Robust Stability Guarantees

This paper presents a parameterization of nonlinear controllers for uncertain systems building on a recently developed neural network architecture, called the recurrent equilibrium network (REN), and a nonlinear version of the Youla parameterization. The proposed framework has "built-in" guarantees of stability, i.e., all policies in the search space result in a contracting (globally exponentially stable) closed-loop system. Thus, it requires very mild assumptions on the choice of cost function and the stability property can be generalized to unseen data. Another useful feature of this approach is that policies are parameterized directly without any constraints, which simplifies learning by a broad range of policy-learning methods based on unconstrained optimization (e.g. stochastic gradient descent). We illustrate the proposed approach with a variety of simulation examples.

[25] 2112.01258

A framework for fitting quadratic-bilinear systems with applications to models of electrical circuits

In this contribution, we propose a data-driven procedure to fit quadratic-bilinear surrogate models from data. Although the dynamics characterizing the original model are strongly nonlinear, we rely on lifting techniques to embed the original model into a quadratic-bilinear format. Here, data represent generalized transfer function values. This method is an extension of methods that do bilinear, or quadratic inference, separately. It is based on first fitting a linear model with the classical Loewner framework, and then on inferring the best supplementing nonlinear operators, in a least-squares sense. The application scope of this method is given by electrical circuits with nonlinear components (such as diodes). We propose various test cases to illustrate the performance of the method.

[26] 2112.01320

Multi-task fusion for improving mammography screening data classification

Machine learning and deep learning methods have become essential for computer-assisted prediction in medicine, with a growing number of applications also in the field of mammography. Typically these algorithms are trained for a specific task, e.g., the classification of lesions or the prediction of a mammogram's pathology status. To obtain a comprehensive view of a patient, models which were all trained for the same task(s) are subsequently ensembled or combined. In this work, we propose a pipeline approach, where we first train a set of individual, task-specific models and subsequently investigate the fusion thereof, which is in contrast to the standard model ensembling strategy. We fuse model predictions and high-level features from deep learning models with hybrid patient models to build stronger predictors on patient level. To this end, we propose a multi-branch deep learning model which efficiently fuses features across different tasks and mammograms to obtain a comprehensive patient-level prediction. We train and evaluate our full pipeline on public mammography data, i.e., DDSM and its curated version CBIS-DDSM, and report an AUC score of 0.962 for predicting the presence of any lesion and 0.791 for predicting the presence of malignant lesions on patient level. Overall, our fusion approaches improve AUC scores significantly by up to 0.04 compared to standard model ensembling. Moreover, by providing not only global patient-level predictions but also task-specific model results that are related to radiological features, our pipeline aims to closely support the reading workflow of radiologists.

[27] 2112.01325

A New Look at AI-Driven NOMA-F-RANs: Features Extraction, Cooperative Caching, and Cache-Aided Computing

Non-orthogonal multiple access (NOMA) enabled fog radio access networks (NOMA-F-RANs) have been taken as a promising enabler to release network congestion, reduce delivery latency, and improve fog user equipments' (F-UEs') quality of services (QoS). Nevertheless, the effectiveness of NOMA-F-RANs highly relies on the charted feature information (preference distribution, positions, mobilities, etc.) of F-UEs as well as the effective caching, computing, and resource allocation strategies. In this article, we explore how artificial intelligence (AI) techniques are utilized to solve foregoing tremendous challenges. Specifically, we first elaborate on the NOMA-F-RANs architecture, shedding light on the key modules, namely, cooperative caching and cache-aided mobile edge computing (MEC). Then, the potentially applicable AI-driven techniques in solving the principal issues of NOMA-F-RANs are reviewed. Through case studies, we show the efficacy of AI-enabled methods in terms of F-UEs' latent feature extraction and cooperative caching. Finally, future trends of AI-driven NOMA-F-RANs, including open research issues and challenges, are identified.

[28] 2112.01326

A More Scalable Mixed-Integer Encoding for Metric Temporal Logic

The state-of-the-art in optimal control from timed temporal logic specifications, including Metric Temporal Logic (MTL) and Signal Temporal Logic (STL), is based on Mixed-Integer Convex Programming (MICP). The standard MICP approach is sound and complete, but struggles to scale to long and complex specifications. Drawing on recent advances in trajectory optimization for piecewise-affine systems, we propose a new MICP encoding for finite transition systems that significantly improves scalability to long and complex MTL specifications. Rather than seeking to reduce the number of variables in the MICP, we focus instead on designing an encoding with a tight convex relaxation. This leads to a larger optimization problem, but significantly improves branch-and-bound solver performance. In simulation experiments involving a mobile robot in a grid-world, the proposed encoding can reduce computation times by several orders of magnitude.

[29] 2112.01351

Equivalent Circuit Programming for Power Flow Analysis and Optimization

The utility of domain-specific knowledge for modeling, simulation, and optimization has been demonstrated for various research problem domains, including power systems. The concept of Equivalent Circuit Programming was previously developed and facilitated for robust, efficient, and scalable solution of network simulation and optimization problems. This paper extends the theoretical foundation of Equivalent Circuit Programming to enable the fusion of optimization theory and algorithms with the numerical methods that utilize the domain-specific knowledge of power flow models. The generality, scalability, and numerical robustness of the resulting framework are demonstrated on realistic AC power flow (ACPF) models of up to 70k buses with proper enforcement of industry-required operational and security constraints.

[30] 2112.01464

Time-Series Estimation from Randomly Time-Warped Observations

We consider the problem of estimating a signal from its warped observations. Such estimation is commonly performed by altering the observations through some inverse-warping, or solving a computationally demanding optimization formulation. While these may be unavoidable if observations are few, when large amounts of warped observations are available, the cost of running such algorithms can be prohibitive. We consider the scenario where we have many observations, and propose a computationally simple algorithm for estimating the function of interest. We demonstrate the utility of the algorithm on streaming biomedical signals.

[31] 2112.01496

Analysis of an adaptive lead weighted ResNet for multiclass classification of 12-lead ECGs

Background: Twelve lead ECGs are a core diagnostic tool for cardiovascular diseases. Here, we describe and analyse an ensemble deep neural network architecture to classify 24 cardiac abnormalities from 12-lead ECGs. Method: We proposed a squeeze and excite ResNet to automatically learn deep features from 12-lead ECGs, in order to identify 24 cardiac conditions. The deep features were augmented with age and gender features in the final fully connected layers. Output thresholds for each class were set using a constrained grid search. To determine why the model made incorrect predictions, two expert clinicians independently interpreted a random set of 100 misclassified ECGs concerning Left Axis Deviation. Results: Using the bespoke weighted accuracy metric, we achieved a 5-fold cross validation score of 0.684, and sensitivity and specificity of 0.758 and 0.969, respectively. We scored 0.520 on the full test data, and ranked 2nd out of 41 in the official challenge rankings. On a random set of misclassified ECGs, agreement between two clinicians and training labels was poor (clinician 1: kappa = -0.057, clinician 2: kappa = -0.159). In contrast, agreement between the clinicians was very high (kappa = 0.92). Discussion: The proposed prediction model performed well on the validation and hidden test data in comparison to models trained on the same data. We also discovered considerable inconsistency in training labels, which is likely to hinder development of more accurate models.

[32] 2112.00793

Using Deep Image Prior to Assist Variational Selective Segmentation Deep Learning Algorithms

Variational segmentation algorithms require a prior imposed in the form of a regularisation term to enforce smoothness of the solution. Recently, it was shown in the Deep Image Prior work that the explicit regularisation in a model can be removed and replaced by the implicit regularisation captured by the architecture of a neural network. The Deep Image Prior approach is competitive, but is only tailored to one specific image and does not allow us to predict future images. We propose to incorporate the ideas from Deep Image Prior into a more traditional learning algorithm to allow us to use the implicit regularisation offered by the Deep Image Prior, but still be able to predict future images.

[33] 2112.00910

IMRecoNet: Learn to Detect in Index Modulation Aided MIMO Systems with Complex Valued Neural Networks

Index modulation (IM) reduces the power consumption and hardware cost of the multiple-input multiple-output (MIMO) system by activating part of the antennas for data transmission. However, IM significantly increases the complexity of the receiver and needs accurate channel estimation to guarantee its performance. To tackle these challenges, in this paper, we design a deep learning (DL) based detector for the IM aided MIMO (IM-MIMO) systems. We first formulate the detection process as a sparse reconstruction problem by utilizing the inherent attributes of IM. Then, based on greedy strategy, we design a DL based detector, called IMRecoNet, to realize this sparse reconstruction process. Different from the general neural networks, we introduce complex value operations to adapt the complex signals in communication systems. To the best of our knowledge, this is the first attempt that introduce complex valued neural network to the design of detector for the IM-MIMO systems. Finally, to verify the adaptability and robustness of the proposed detector, simulations are carried out with consideration of inaccurate channel state information (CSI) and correlated MIMO channels. The simulation results demonstrate that the proposed detector outperforms existing algorithms in terms of antenna recognition accuracy and bit error rate under various scenarios.

[34] 2112.00931

Antenna Selection in Polarization Reconfigurable MIMO (PR-MIMO) Communication Systems

Adaptation of a wireless system to the polarization state of the propagation channel can improve reliability and throughput. This paper in particular considers polarization reconfigurable multiple input multiple output (PR-MIMO) systems, where both transmitter and receiver can change the (linear) polarization orientation at each element of their antenna arrays. We first introduce joint polarization pre-post coding to maximize bounds on the capacity and the maximum eigenvalue of the channel matrix. For this we first derive approximate closed form equations of optimal polarization vectors at one link end, and then use iterative joint polarization pre-post coding to pursue joint optimal polarization vectors at both link ends. Next we investigate the combination of PR-MIMO with hybrid antenna selection / maximum ratio transmission (PR-HS/MRT), which can achieve a remarkable improvement of channel capacity and symbol error rate (SER). Further, two novel schemes of element wise and global polarization reconfiguration are presented for PR-HS/MRT. Comprehensive simulation results indicate that the proposed schemes provide 3 to 5 dB SNR gain in PR-MIMO spatial multiplexing and approximately 3 dB SNR gain in PRHS/ MRT, with concomitant improvements of channel capacity and SER.

[35] 2112.00989

Embedding Decomposition for Artifacts Removal in EEG Signals

Electroencephalogram (EEG) recordings are often contaminated with artifacts. Various methods have been developed to eliminate or weaken the influence of artifacts. However, most of them rely on prior experience for analysis. Here, we propose an deep learning framework to separate neural signal and artifacts in the embedding space and reconstruct the denoised signal, which is called DeepSeparator. DeepSeparator employs an encoder to extract and amplify the features in the raw EEG, a module called decomposer to extract the trend, detect and suppress artifact and a decoder to reconstruct the denoised signal. Besides, DeepSeparator can extract the artifact, which largely increases the model interpretability. The proposed method is tested with a semi-synthetic EEG dataset and a real task-related EEG dataset, suggesting that DeepSeparator outperforms the conventional models in both EOG and EMG artifact removal. DeepSeparator can be extended to multi-channel EEG and data of any length. It may motivate future developments and application of deep learning-based EEG denoising. The code for DeepSeparator is available at

[36] 2112.01006

Distributed Control for a Robotic Swarm to Pass through a Curve Virtual Tube

Robotic swarm systems are now becoming increasingly attractive for many challenging applications. The main task for any robot is to reach the destination while keeping a safe separation from other robots and obstacles. In many scenarios, robots need to move within a narrow corridor, through a window or a doorframe. In order to guide all robots to move in a cluttered environment, a curve virtual tube with no obstacle inside is carefully designed in this paper. There is no obstacle inside the tube, namely the area inside the tube can be seen as a safety zone. Then, a distributed swarm controller is proposed with three elaborate control terms: a line approaching term, a robot avoidance term and a tube keeping term. Formal analysis and proofs are made to show that the curve virtual tube passing problem can be solved in a finite time. For the convenience in practical use, a modified controller with an approximate control performance is put forward. Finally, the effectiveness of the proposed method is validated by numerical simulations and real experiments. To show the advantages of the proposed method, the comparison between our method and the control barrier function method is also presented in terms of calculation speed.

[37] 2112.01034

Leveraging Human Selective Attention for Medical Image Analysis with Limited Training Data

The human gaze is a cost-efficient physiological data that reveals human underlying attentional patterns. The selective attention mechanism helps the cognition system focus on task-relevant visual clues by ignoring the presence of distractors. Thanks to this ability, human beings can efficiently learn from a very limited number of training samples. Inspired by this mechanism, we aim to leverage gaze for medical image analysis tasks with small training data. Our proposed framework includes a backbone encoder and a Selective Attention Network (SAN) that simulates the underlying attention. The SAN implicitly encodes information such as suspicious regions that is relevant to the medical diagnose tasks by estimating the actual human gaze. Then we design a novel Auxiliary Attention Block (AAB) to allow information from SAN to be utilized by the backbone encoder to focus on selective areas. Specifically, this block uses a modified version of a multi-head attention layer to simulate the human visual search procedure. Note that the SAN and AAB can be plugged into different backbones, and the framework can be used for multiple medical image analysis tasks when equipped with task-specific heads. Our method is demonstrated to achieve superior performance on both 3D tumor segmentation and 2D chest X-ray classification tasks. We also show that the estimated gaze probability map of the SAN is consistent with an actual gaze fixation map obtained by board-certified doctors.

[38] 2112.01288

How to quantify fields or textures? A guide to the scattering transform

Extracting information from stochastic fields or textures is a ubiquitous task in science, from exploratory data analysis to classification and parameter estimation. From physics to biology, it tends to be done either through a power spectrum analysis, which is often too limited, or the use of convolutional neural networks (CNNs), which require large training sets and lack interpretability. In this paper, we advocate for the use of the scattering transform (Mallat 2012), a powerful statistic which borrows mathematical ideas from CNNs but does not require any training, and is interpretable. We show that it provides a relatively compact set of summary statistics with visual interpretation and which carries most of the relevant information in a wide range of scientific applications. We present a non-technical introduction to this estimator and we argue that it can benefit data analysis, comparison to models and parameter inference in many fields of science. Interestingly, understanding the core operations of the scattering transform allows one to decipher many key aspects of the inner workings of CNNs.

[39] 2112.01363

Breaking the Convergence Barrier: Optimization via Fixed-Time Convergent Flows

Accelerated gradient methods are the cornerstones of large-scale, data-driven optimization problems that arise naturally in machine learning and other fields concerning data analysis. We introduce a gradient-based optimization framework for achieving acceleration, based on the recently introduced notion of fixed-time stability of dynamical systems. The method presents itself as a generalization of simple gradient-based methods suitably scaled to achieve convergence to the optimizer in a fixed-time, independent of the initialization. We achieve this by first leveraging a continuous-time framework for designing fixed-time stable dynamical systems, and later providing a consistent discretization strategy, such that the equivalent discrete-time algorithm tracks the optimizer in a practically fixed number of iterations. We also provide a theoretical analysis of the convergence behavior of the proposed gradient flows, and their robustness to additive disturbances for a range of functions obeying strong convexity, strict convexity, and possibly nonconvexity but satisfying the Polyak-{\L}ojasiewicz inequality. We also show that the regret bound on the convergence rate is constant by virtue of the fixed-time convergence. The hyperparameters have intuitive interpretations and can be tuned to fit the requirements on the desired convergence rates. We validate the accelerated convergence properties of the proposed schemes on a range of numerical examples against the state-of-the-art optimization algorithms. Our work provides insights on developing novel optimization algorithms via discretization of continuous-time flows.

[40] 2112.01413

Channel Estimation for STAR-RIS-aided Wireless Communication

In this letter, we study efficient uplink channel estimation design for a simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) assisted two-user communication systems. We first consider the time switching (TS) protocol for STAR-RIS and propose an efficient scheme to separately estimate the channels of the two users with optimized training (transmission/reflection) pattern. Next, we consider the energy splitting (ES) protocol for STAR-RIS under the practical coupled phase-shift model and devise a customized scheme to simultaneously estimate the channels of both users. Although the problem of minimizing the resultant channel estimation error for the ES protocol is difficult to solve, we propose an efficient algorithm to obtain a high-quality solution by jointly designing the pilot sequences, power-splitting ratio, and training patterns. Numerical results show the effectiveness of the proposed channel estimation designs and reveal that the STAR-RIS under the TS protocol achieves a smaller channel estimation error than the ES case.

[41] 2112.01484

Safe Reinforcement Learning for Grid Voltage Control

Under voltage load shedding has been considered as a standard approach to recover the voltage stability of the electric power grid under emergency conditions, yet this scheme usually trips a massive amount of load inefficiently. Reinforcement learning (RL) has been adopted as a promising approach to circumvent the issues; however, RL approach usually cannot guarantee the safety of the systems under control. In this paper, we discuss a couple of novel safe RL approaches, namely constrained optimization approach and Barrier function-based approach, that can safely recover voltage under emergency events. This method is general and can be applied to other safety-critical control problems. Numerical simulations on the 39-bus IEEE benchmark are performed to demonstrate the effectiveness of the proposed safe RL emergency control.