-
Combining X-Vectors and Bayesian Batch Active Learning: Two-Stage Active Learning Pipeline for Speech Recognition
Authors:
Ognjen Kundacina,
Vladimir Vincan,
Dragisa Miskovic
Abstract:
Emphasizing a data-centric AI approach, this paper introduces a novel two-stage active learning (AL) pipeline for automatic speech recognition (ASR), combining unsupervised and supervised AL methods. The first stage utilizes unsupervised AL by using x-vectors clustering for diverse sample selection from unlabeled speech data, thus establishing a robust initial dataset for the subsequent supervised…
▽ More
Emphasizing a data-centric AI approach, this paper introduces a novel two-stage active learning (AL) pipeline for automatic speech recognition (ASR), combining unsupervised and supervised AL methods. The first stage utilizes unsupervised AL by using x-vectors clustering for diverse sample selection from unlabeled speech data, thus establishing a robust initial dataset for the subsequent supervised AL. The second stage incorporates a supervised AL strategy, with a batch AL method specifically developed for ASR, aimed at selecting diverse and informative batches of samples. Here, sample diversity is also achieved using x-vectors clustering, while the most informative samples are identified using a Bayesian AL method tailored for ASR with an adaptation of Monte Carlo dropout to approximate Bayesian inference. This approach enables precise uncertainty estimation, thereby enhancing ASR model training with significantly reduced data requirements. Our method has shown superior performance compared to competing methods on homogeneous, heterogeneous, and OOD test sets, demonstrating that strategic sample selection and innovative Bayesian modeling can substantially optimize both labeling effort and data utilization in deep learning-based ASR applications.
△ Less
Submitted 3 May, 2024;
originally announced June 2024.
-
Application of Deep Learning Methods in Monitoring and Optimization of Electric Power Systems
Authors:
Ognjen Kundacina
Abstract:
This PhD thesis thoroughly examines the utilization of deep learning techniques as a means to advance the algorithms employed in the monitoring and optimization of electric power systems. The first major contribution of this thesis involves the application of graph neural networks to enhance power system state estimation. The second key aspect of this thesis focuses on utilizing reinforcement lear…
▽ More
This PhD thesis thoroughly examines the utilization of deep learning techniques as a means to advance the algorithms employed in the monitoring and optimization of electric power systems. The first major contribution of this thesis involves the application of graph neural networks to enhance power system state estimation. The second key aspect of this thesis focuses on utilizing reinforcement learning for dynamic distribution network reconfiguration. The effectiveness of the proposed methods is affirmed through extensive experimentation and simulations.
△ Less
Submitted 1 September, 2023;
originally announced September 2023.
-
Overview of Deep Learning Methods for Retinal Vessel Segmentation
Authors:
Gorana Gojić,
Ognjen Kundačina,
Dragiša Mišković,
Dinu Dragan
Abstract:
Methods for automated retinal vessel segmentation play an important role in the treatment and diagnosis of many eye and systemic diseases. With the fast development of deep learning methods, more and more retinal vessel segmentation methods are implemented as deep neural networks. In this paper, we provide a brief review of recent deep learning methods from highly influential journals and conferen…
▽ More
Methods for automated retinal vessel segmentation play an important role in the treatment and diagnosis of many eye and systemic diseases. With the fast development of deep learning methods, more and more retinal vessel segmentation methods are implemented as deep neural networks. In this paper, we provide a brief review of recent deep learning methods from highly influential journals and conferences. The review objectives are: (1) to assess the design characteristics of the latest methods, (2) to report and analyze quantitative values of performance evaluation metrics, and (3) to analyze the advantages and disadvantages of the recent solutions.
△ Less
Submitted 1 June, 2023;
originally announced June 2023.
-
Non-adversarial Robustness of Deep Learning Methods for Computer Vision
Authors:
Gorana Gojić,
Vladimir Vincan,
Ognjen Kundačina,
Dragiša Mišković,
Dinu Dragan
Abstract:
Non-adversarial robustness, also known as natural robustness, is a property of deep learning models that enables them to maintain performance even when faced with distribution shifts caused by natural variations in data. However, achieving this property is challenging because it is difficult to predict in advance the types of distribution shifts that may occur. To address this challenge, researche…
▽ More
Non-adversarial robustness, also known as natural robustness, is a property of deep learning models that enables them to maintain performance even when faced with distribution shifts caused by natural variations in data. However, achieving this property is challenging because it is difficult to predict in advance the types of distribution shifts that may occur. To address this challenge, researchers have proposed various approaches, some of which anticipate potential distribution shifts, while others utilize knowledge about the shifts that have already occurred to enhance model generalizability. In this paper, we present a brief overview of the most recent techniques for improving the robustness of computer vision methods, as well as a summary of commonly used robustness benchmark datasets for evaluating the model's performance under data distribution shifts. Finally, we examine the strengths and limitations of the approaches reviewed and identify general trends in deep learning robustness improvement for computer vision.
△ Less
Submitted 24 May, 2023;
originally announced May 2023.
-
Graph Neural Networks on Factor Graphs for Robust, Fast, and Scalable Linear State Estimation with PMUs
Authors:
Ognjen Kundacina,
Mirsad Cosovic,
Dragisa Miskovic,
Dejan Vukobratovic
Abstract:
As phasor measurement units (PMUs) become more widely used in transmission power systems, a fast state estimation (SE) algorithm that can take advantage of their high sample rates is needed. To accomplish this, we present a method that uses graph neural networks (GNNs) to learn complex bus voltage estimates from PMU voltage and current measurements. We propose an original implementation of GNNs ov…
▽ More
As phasor measurement units (PMUs) become more widely used in transmission power systems, a fast state estimation (SE) algorithm that can take advantage of their high sample rates is needed. To accomplish this, we present a method that uses graph neural networks (GNNs) to learn complex bus voltage estimates from PMU voltage and current measurements. We propose an original implementation of GNNs over the power system's factor graph to simplify the integration of various types and quantities of measurements on power system buses and branches. Furthermore, we augment the factor graph to improve the robustness of GNN predictions. This model is highly efficient and scalable, as its computational complexity is linear with respect to the number of nodes in the power system. Training and test examples were generated by randomly sampling sets of power system measurements and annotated with the exact solutions of linear SE with PMUs. The numerical results demonstrate that the GNN model provides an accurate approximation of the SE solutions. Furthermore, errors caused by PMU malfunctions or communication failures that would normally make the SE problem unobservable have a local effect and do not deteriorate the results in the rest of the power system.
△ Less
Submitted 28 April, 2023;
originally announced April 2023.
-
Supporting Future Electrical Utilities: Using Deep Learning Methods in EMS and DMS Algorithms
Authors:
Ognjen Kundacina,
Gorana Gojic,
Mile Mitrovic,
Dragisa Miskovic,
Dejan Vukobratovic
Abstract:
Electrical power systems are increasing in size, complexity, as well as dynamics due to the growing integration of renewable energy resources, which have sporadic power generation. This necessitates the development of near real-time power system algorithms, demanding lower computational complexity regarding the power system size. Considering the growing trend in the collection of historical measur…
▽ More
Electrical power systems are increasing in size, complexity, as well as dynamics due to the growing integration of renewable energy resources, which have sporadic power generation. This necessitates the development of near real-time power system algorithms, demanding lower computational complexity regarding the power system size. Considering the growing trend in the collection of historical measurement data and recent advances in the rapidly develo** deep learning field, the main goal of this paper is to provide a review of recent deep learning-based power system monitoring and optimization algorithms. Electrical utilities can benefit from this review by re-implementing or enhancing the algorithms traditionally used in energy management systems (EMS) and distribution management systems (DMS).
△ Less
Submitted 1 March, 2023;
originally announced March 2023.
-
Scalability and Sample Efficiency Analysis of Graph Neural Networks for Power System State Estimation
Authors:
Ognjen Kundacina,
Gorana Gojic,
Mirsad Cosovic,
Dragisa Miskovic,
Dejan Vukobratovic
Abstract:
Data-driven state estimation (SE) is becoming increasingly important in modern power systems, as it allows for more efficient analysis of system behaviour using real-time measurement data. This paper thoroughly evaluates a phasor measurement unit-only state estimator based on graph neural networks (GNNs) applied over factor graphs. To assess the sample efficiency of the GNN model, we perform multi…
▽ More
Data-driven state estimation (SE) is becoming increasingly important in modern power systems, as it allows for more efficient analysis of system behaviour using real-time measurement data. This paper thoroughly evaluates a phasor measurement unit-only state estimator based on graph neural networks (GNNs) applied over factor graphs. To assess the sample efficiency of the GNN model, we perform multiple training experiments on various training set sizes. Additionally, to evaluate the scalability of the GNN model, we conduct experiments on power systems of various sizes. Our results show that the GNN-based state estimator exhibits high accuracy and efficient use of data. Additionally, it demonstrated scalability in terms of both memory usage and inference time, making it a promising solution for data-driven SE in modern power systems.
△ Less
Submitted 2 March, 2023; v1 submitted 28 February, 2023;
originally announced March 2023.
-
GP CC-OPF: Gaussian Process based optimization tool for Chance-Constrained Optimal Power Flow
Authors:
Mile Mitrovic,
Ognjen Kundacina,
Aleksandr Lukashevich,
Petr Vorobev,
Vladimir Terzija,
Yury Maximov,
Deepjyoti Deka
Abstract:
The Gaussian Process (GP) based Chance-Constrained Optimal Power Flow (CC-OPF) is an open-source Python code developed for solving economic dispatch (ED) problem in modern power grids. In recent years, integrating a significant amount of renewables into a power grid causes high fluctuations and thus brings a lot of uncertainty to power grid operations. This fact makes the conventional model-based…
▽ More
The Gaussian Process (GP) based Chance-Constrained Optimal Power Flow (CC-OPF) is an open-source Python code developed for solving economic dispatch (ED) problem in modern power grids. In recent years, integrating a significant amount of renewables into a power grid causes high fluctuations and thus brings a lot of uncertainty to power grid operations. This fact makes the conventional model-based CC-OPF problem non-convex and computationally complex to solve. The developed tool presents a novel data-driven approach based on the GP regression model for solving the CC-OPF problem with a trade-off between complexity and accuracy. The proposed approach and developed software can help system operators to effectively perform ED optimization in the presence of large uncertainties in the power grid.
△ Less
Submitted 16 February, 2023;
originally announced February 2023.
-
Distributed Nonlinear State Estimation in Electric Power Systems using Graph Neural Networks
Authors:
Ognjen Kundacina,
Mirsad Cosovic,
Dragisa Miskovic,
Dejan Vukobratovic
Abstract:
Nonlinear state estimation (SE), with the goal of estimating complex bus voltages based on all types of measurements available in the power system, is usually solved using the iterative Gauss-Newton method. The nonlinear SE presents some difficulties when considering inputs from both phasor measurement units and supervisory control and data acquisition system. These include numerical instabilities…
▽ More
Nonlinear state estimation (SE), with the goal of estimating complex bus voltages based on all types of measurements available in the power system, is usually solved using the iterative Gauss-Newton method. The nonlinear SE presents some difficulties when considering inputs from both phasor measurement units and supervisory control and data acquisition system. These include numerical instabilities, convergence time depending on the starting point of the iterative method, and the quadratic computational complexity of a single iteration regarding the number of state variables. This paper introduces an original graph neural network based SE implementation over the augmented factor graph of the nonlinear power system SE, capable of incorporating measurements on both branches and buses, as well as both phasor and legacy measurements. The proposed regression model has linear computational complexity during the inference time once trained, with a possibility of distributed implementation. Since the method is noniterative and non-matrix-based, it is resilient to the problems that the Gauss-Newton solver is prone to. Aside from prediction accuracy on the test set, the proposed model demonstrates robustness when simulating cyber attacks and unobservable scenarios due to communication irregularities. In those cases, prediction errors are sustained locally, with no effect on the rest of the power system's results.
△ Less
Submitted 8 September, 2022; v1 submitted 23 July, 2022;
originally announced July 2022.
-
Near Real-Time Distributed State Estimation via AI/ML-Empowered 5G Networks
Authors:
Ognjen Kundacina,
Miodrag Forcan,
Mirsad Cosovic,
Darijo Raca,
Merim Dzaferagic,
Dragisa Miskovic,
Mirjana Maksimovic,
Dejan Vukobratovic
Abstract:
Fifth-Generation (5G) networks have a potential to accelerate power system transition to a flexible, softwarized, data-driven, and intelligent grid. With their evolving support for Machine Learning (ML)/Artificial Intelligence (AI) functions, 5G networks are expected to enable novel data-centric Smart Grid (SG) services. In this paper, we explore how data-driven SG services could be integrated wit…
▽ More
Fifth-Generation (5G) networks have a potential to accelerate power system transition to a flexible, softwarized, data-driven, and intelligent grid. With their evolving support for Machine Learning (ML)/Artificial Intelligence (AI) functions, 5G networks are expected to enable novel data-centric Smart Grid (SG) services. In this paper, we explore how data-driven SG services could be integrated with ML/AI-enabled 5G networks in a symbiotic relationship. We focus on the State Estimation (SE) function as a key element of the energy management system and focus on two main questions. Firstly, in a tutorial fashion, we present an overview on how distributed SE can be integrated with the elements of the 5G core network and radio access network architecture. Secondly, we present and compare two powerful distributed SE methods based on: i) graphical models and belief propagation, and ii) graph neural networks. We discuss their performance and capability to support a near real-time distributed SE via 5G network, taking into account communication delays.
△ Less
Submitted 22 July, 2022;
originally announced July 2022.
-
Robust and Fast Data-Driven Power System State Estimator Using Graph Neural Networks
Authors:
Ognjen Kundacina,
Mirsad Cosovic,
Dejan Vukobratovic
Abstract:
The power system state estimation (SE) algorithm estimates the complex bus voltages based on the available set of measurements. Because phasor measurement units (PMUs) are becoming more widely employed in transmission power systems, a fast SE solver capable of exploiting PMUs' high sample rates is required. To accomplish this, we present a method for training a model based on graph neural networks…
▽ More
The power system state estimation (SE) algorithm estimates the complex bus voltages based on the available set of measurements. Because phasor measurement units (PMUs) are becoming more widely employed in transmission power systems, a fast SE solver capable of exploiting PMUs' high sample rates is required. To accomplish this, we present a method for training a model based on graph neural networks (GNNs) to learn estimates from PMU voltage and current measurements, which, once it is trained, has a linear computational complexity with respect to the number of nodes in the power system. We propose an original GNN implementation over the power system's factor graph to simplify the incorporation of various types and numbers of measurements both on power system buses and branches. Furthermore, we augment the factor graph to improve the robustness of GNN predictions. Training and test examples were generated by randomly sampling sets of power system measurements and annotated with the exact solutions of linear SE with PMUs. The numerical results demonstrate that the GNN model provides an accurate approximation of the SE solutions. Additionally, errors caused by PMU malfunctions or the communication failures that make the SE problem unobservable have a local effect and do not deteriorate the results in the rest of the power system.
△ Less
Submitted 6 June, 2022;
originally announced June 2022.
-
State Estimation in Electric Power Systems Leveraging Graph Neural Networks
Authors:
Ognjen Kundacina,
Mirsad Cosovic,
Dejan Vukobratovic
Abstract:
The goal of the state estimation (SE) algorithm is to estimate complex bus voltages as state variables based on the available set of measurements in the power system. Because phasor measurement units (PMUs) are increasingly being used in transmission power systems, there is a need for a fast SE solver that can take advantage of high sampling rates of PMUs. This paper proposes training a graph neur…
▽ More
The goal of the state estimation (SE) algorithm is to estimate complex bus voltages as state variables based on the available set of measurements in the power system. Because phasor measurement units (PMUs) are increasingly being used in transmission power systems, there is a need for a fast SE solver that can take advantage of high sampling rates of PMUs. This paper proposes training a graph neural network (GNN) to learn the estimates given the PMU voltage and current measurements as inputs, with the intent of obtaining fast and accurate predictions during the evaluation phase. GNN is trained using synthetic datasets, created by randomly sampling sets of measurements in the power system and labelling them with a solution obtained using a linear SE with PMUs solver. The presented results display the accuracy of GNN predictions in various test scenarios and tackle the sensitivity of the predictions to the missing input data.
△ Less
Submitted 12 April, 2022; v1 submitted 11 January, 2022;
originally announced January 2022.
