-
Deep Learning for In-Orbit Cloud Segmentation and Classification in Hyperspectral Satellite Data
Authors:
Daniel Kovac,
Jan Mucha,
Jon Alvarez Justo,
Jiri Mekyska,
Zoltan Galaz,
Krystof Novotny,
Radoslav Pitonak,
Jan Knezik,
Jonas Herec,
Tor Arne Johansen
Abstract:
This article explores the latest Convolutional Neural Networks (CNNs) for cloud detection aboard hyperspectral satellites. The performance of the latest 1D CNN (1D-Justo-LiuNet) and two recent 2D CNNs (nnU-net and 2D-Justo-UNet-Simple) for cloud segmentation and classification is assessed. Evaluation criteria include precision and computational efficiency for in-orbit deployment. Experiments utili…
▽ More
This article explores the latest Convolutional Neural Networks (CNNs) for cloud detection aboard hyperspectral satellites. The performance of the latest 1D CNN (1D-Justo-LiuNet) and two recent 2D CNNs (nnU-net and 2D-Justo-UNet-Simple) for cloud segmentation and classification is assessed. Evaluation criteria include precision and computational efficiency for in-orbit deployment. Experiments utilize NASA's EO-1 Hyperion data, with varying spectral channel numbers after Principal Component Analysis. Results indicate that 1D-Justo-LiuNet achieves the highest accuracy, outperforming 2D CNNs, while maintaining compactness with larger spectral channel sets, albeit with increased inference times. However, the performance of 1D CNN degrades with significant channel reduction. In this context, the 2D-Justo-UNet-Simple offers the best balance for in-orbit deployment, considering precision, memory, and time costs. While nnU-net is suitable for on-ground processing, deployment of lightweight 1D-Justo-LiuNet is recommended for high-precision applications. Alternatively, lightweight 2D-Justo-UNet-Simple is recommended for balanced costs between timing and precision in orbit.
△ Less
Submitted 13 March, 2024;
originally announced March 2024.
-
A Comparative Study of Rapidly-exploring Random Tree Algorithms Applied to Ship Trajectory Planning and Behavior Generation
Authors:
Trym Tengesdal,
Tom Arne Pedersen,
Tor Arne Johansen
Abstract:
Rapidly Exploring Random Tree (RRT) algorithms, notably used for nonholonomic vehicle navigation in complex environments, are often not thoroughly evaluated for their specific challenges. This paper presents a first such comparison study of the variants Potential-Quick RRT* (PQ-RRT*), Informed RRT* (IRRT*), RRT*, and RRT, in maritime single-query nonholonomic motion planning. Additionally, the pra…
▽ More
Rapidly Exploring Random Tree (RRT) algorithms, notably used for nonholonomic vehicle navigation in complex environments, are often not thoroughly evaluated for their specific challenges. This paper presents a first such comparison study of the variants Potential-Quick RRT* (PQ-RRT*), Informed RRT* (IRRT*), RRT*, and RRT, in maritime single-query nonholonomic motion planning. Additionally, the practicalities of using these algorithms in maritime environments are discussed and outlined. We also contend that these algorithms are beneficial not only for trajectory planning in Collision Avoidance Systems (CAS) but also for CAS verification when used as vessel behavior generators.
Optimal RRT variants tend to produce more distance-optimal paths but require more computational time due to complex tree wiring and nearest neighbor searches. Our findings, supported by Welch`s t-test at a significance level of Alpha = 0.05, indicate that PQ-RRT* slightly outperform IRRT* and RRT* in achieving shorter trajectory length but at the expense of higher tuning complexity and longer run-times. Based on the results, we argue that these RRT algorithms are better suited for smaller-scale problems or environments with low obstacle congestion ratio. This is attributed to the curse of dimensionality, and trade-off with available memory and computational resources.
△ Less
Submitted 17 April, 2024; v1 submitted 2 March, 2024;
originally announced March 2024.
-
A Comparative Study of Compressive Sensing Algorithms for Hyperspectral Imaging Reconstruction
Authors:
Jon Alvarez Justo,
Daniela Lupu,
Milica Orlandic,
Ion Necoara,
Tor Arne Johansen
Abstract:
Hyperspectral Imaging comprises excessive data consequently leading to significant challenges for data processing, storage and transmission. Compressive Sensing has been used in the field of Hyperspectral Imaging as a technique to compress the large amount of data. This work addresses the recovery of hyperspectral images 2.5x compressed. A comparative study in terms of the accuracy and the perform…
▽ More
Hyperspectral Imaging comprises excessive data consequently leading to significant challenges for data processing, storage and transmission. Compressive Sensing has been used in the field of Hyperspectral Imaging as a technique to compress the large amount of data. This work addresses the recovery of hyperspectral images 2.5x compressed. A comparative study in terms of the accuracy and the performance of the convex FISTA/ADMM in addition to the greedy gOMP/BIHT/CoSaMP recovery algorithms is presented. The results indicate that the algorithms recover successfully the compressed data, yet the gOMP algorithm achieves superior accuracy and faster recovery in comparison to the other algorithms at the expense of high dependence on unknown sparsity level of the data to recover.
△ Less
Submitted 26 January, 2024;
originally announced January 2024.
-
Sea-Land-Cloud Segmentation in Satellite Hyperspectral Imagery by Deep Learning
Authors:
Jon Alvarez Justo,
Joseph L. Garrett,
Mariana-Iuliana Georgescu,
Jesus Gonzalez-Llorente,
Radu Tudor Ionescu,
Tor Arne Johansen
Abstract:
Satellites are increasingly adopting on-board AI for enhanced autonomy through in-orbit inference. In this context, the use of deep learning (DL) techniques for segmentation in hyperspectral (HS) satellite imagery offers advantages for remote sensing applications, and therefore, we train 16 different models, whose codes are made available through our study, which we consider to be relevant for on-…
▽ More
Satellites are increasingly adopting on-board AI for enhanced autonomy through in-orbit inference. In this context, the use of deep learning (DL) techniques for segmentation in hyperspectral (HS) satellite imagery offers advantages for remote sensing applications, and therefore, we train 16 different models, whose codes are made available through our study, which we consider to be relevant for on-board multi-class segmentation of HS imagery, focusing on classifying oceanic (sea), terrestrial (land), and cloud formations. We employ the HYPSO-1 mission as an illustrative case for sea-land-cloud segmentation, and to demonstrate the utility of the segments, we introduce a novel sea-land-cloud ranking application scenario. We consider how to prioritize HS image downlink based on sea, land, and cloud coverage levels from the segmented images. We comparatively evaluate the models for future in-orbit deployment, considering performance, parameter count, and inference time. The models include both shallow and deep models, and after we propose four new DL models, we demonstrate that segmenting single spectral signatures (1D) outperforms 3D data processing comprising both spectral (1D) and spatial (2D) contexts. We conclude that our lightweight DL model, called 1D-Justo-LiuNet, consistently surpasses state-of-the-art models for sea-land-cloud segmentation, such as U-Net and its variations, in terms of performance (0.93 accuracy) and parameter count (4,563). However, the 1D models present longer inference time (15s) in the tested processing architecture, which seems to be a suboptimal architecture for this purpose. Finally, after demonstrating that in-orbit segmentation should occur post L1b radiance calibration rather than on raw data, we also show that reducing spectral channels down to 3 lowers models' parameter counts and inference time, at the cost of weaker segmentation performance.
△ Less
Submitted 28 December, 2023; v1 submitted 24 October, 2023;
originally announced October 2023.
-
An Open Hyperspectral Dataset with Sea-Land-Cloud Ground-Truth from the HYPSO-1 Satellite
Authors:
Jon A. Justo,
Joseph Garrett,
Dennis D. Langer,
Marie B. Henriksen,
Radu T. Ionescu,
Tor A. Johansen
Abstract:
Hyperspectral Imaging, employed in satellites for space remote sensing, like HYPSO-1, faces constraints due to few labeled data sets, affecting the training of AI models demanding these ground-truth annotations. In this work, we introduce The HYPSO-1 Sea-Land-Cloud-Labeled Dataset, an open dataset with 200 diverse hyperspectral images from the HYPSO-1 mission, available in both raw and calibrated…
▽ More
Hyperspectral Imaging, employed in satellites for space remote sensing, like HYPSO-1, faces constraints due to few labeled data sets, affecting the training of AI models demanding these ground-truth annotations. In this work, we introduce The HYPSO-1 Sea-Land-Cloud-Labeled Dataset, an open dataset with 200 diverse hyperspectral images from the HYPSO-1 mission, available in both raw and calibrated forms for scientific research in Earth observation. Moreover, 38 of these images from different countries include ground-truth labels at pixel-level totaling about 25 million spectral signatures labeled for sea/land/cloud categories. To demonstrate the potential of the dataset and its labeled subset, we have additionally optimized a deep learning model (1D Fully Convolutional Network), achieving superior performance to the current state of the art. The complete dataset, ground-truth labels, deep learning model, and software code are openly accessible for download at the website https://ntnu-smallsat-lab.github.io/hypso1_sea_land_clouds_dataset/ .
△ Less
Submitted 3 September, 2023; v1 submitted 25 August, 2023;
originally announced August 2023.
-
Data-Efficient Deep Reinforcement Learning for Attitude Control of Fixed-Wing UAVs: Field Experiments
Authors:
Eivind Bøhn,
Erlend M. Coates,
Dirk Reinhardt,
Tor Arne Johansen
Abstract:
Attitude control of fixed-wing unmanned aerial vehicles (UAVs) is a difficult control problem in part due to uncertain nonlinear dynamics, actuator constraints, and coupled longitudinal and lateral motions. Current state-of-the-art autopilots are based on linear control and are thus limited in their effectiveness and performance. Deep reinforcement learning (DRL) is a machine learning method to au…
▽ More
Attitude control of fixed-wing unmanned aerial vehicles (UAVs) is a difficult control problem in part due to uncertain nonlinear dynamics, actuator constraints, and coupled longitudinal and lateral motions. Current state-of-the-art autopilots are based on linear control and are thus limited in their effectiveness and performance. Deep reinforcement learning (DRL) is a machine learning method to automatically discover optimal control laws through interaction with the controlled system, which can handle complex nonlinear dynamics. We show in this paper that DRL can successfully learn to perform attitude control of a fixed-wing UAV operating directly on the original nonlinear dynamics, requiring as little as three minutes of flight data. We initially train our model in a simulation environment and then deploy the learned controller on the UAV in flight tests, demonstrating comparable performance to the state-of-the-art ArduPlane proportional-integral-derivative (PID) attitude controller with no further online learning required. Learning with significant actuation delay and diversified simulated dynamics were found to be crucial for successful transfer to control of the real UAV. In addition to a qualitative comparison with the ArduPlane autopilot, we present a quantitative assessment based on linear analysis to better understand the learning controller's behavior.
△ Less
Submitted 19 April, 2023; v1 submitted 7 November, 2021;
originally announced November 2021.
-
Optimization of the Model Predictive Control Meta-Parameters Through Reinforcement Learning
Authors:
Eivind Bøhn,
Sebastien Gros,
Signe Moe,
Tor Arne Johansen
Abstract:
Model predictive control (MPC) is increasingly being considered for control of fast systems and embedded applications. However, the MPC has some significant challenges for such systems. Its high computational complexity results in high power consumption from the control algorithm, which could account for a significant share of the energy resources in battery-powered embedded systems. The MPC param…
▽ More
Model predictive control (MPC) is increasingly being considered for control of fast systems and embedded applications. However, the MPC has some significant challenges for such systems. Its high computational complexity results in high power consumption from the control algorithm, which could account for a significant share of the energy resources in battery-powered embedded systems. The MPC parameters must be tuned, which is largely a trial-and-error process that affects the control performance, the robustness and the computational complexity of the controller to a high degree. In this paper, we propose a novel framework in which any parameter of the control algorithm can be jointly tuned using reinforcement learning(RL), with the goal of simultaneously optimizing the control performance and the power usage of the control algorithm. We propose the novel idea of optimizing the meta-parameters of MPCwith RL, i.e. parameters affecting the structure of the MPCproblem as opposed to the solution to a given problem. Our control algorithm is based on an event-triggered MPC where we learn when the MPC should be re-computed, and a dual mode MPC and linear state feedback control law applied in between MPC computations. We formulate a novel mixture-distribution policy and show that with joint optimization we achieve improvements that do not present themselves when optimizing the same parameters in isolation. We demonstrate our framework on the inverted pendulum control task, reducing the total computation time of the control system by 36% while also improving the control performance by 18.4% over the best-performing MPC baseline.
△ Less
Submitted 7 November, 2021;
originally announced November 2021.
-
Reinforcement Learning of the Prediction Horizon in Model Predictive Control
Authors:
Eivind Bøhn,
Sebastien Gros,
Signe Moe,
Tor Arne Johansen
Abstract:
Model predictive control (MPC) is a powerful trajectory optimization control technique capable of controlling complex nonlinear systems while respecting system constraints and ensuring safe operation. The MPC's capabilities come at the cost of a high online computational complexity, the requirement of an accurate model of the system dynamics, and the necessity of tuning its parameters to the speci…
▽ More
Model predictive control (MPC) is a powerful trajectory optimization control technique capable of controlling complex nonlinear systems while respecting system constraints and ensuring safe operation. The MPC's capabilities come at the cost of a high online computational complexity, the requirement of an accurate model of the system dynamics, and the necessity of tuning its parameters to the specific control application. The main tunable parameter affecting the computational complexity is the prediction horizon length, controlling how far into the future the MPC predicts the system response and thus evaluates the optimality of its computed trajectory. A longer horizon generally increases the control performance, but requires an increasingly powerful computing platform, excluding certain control applications.The performance sensitivity to the prediction horizon length varies over the state space, and this motivated the adaptive horizon model predictive control (AHMPC), which adapts the prediction horizon according to some criteria. In this paper we propose to learn the optimal prediction horizon as a function of the state using reinforcement learning (RL). We show how the RL learning problem can be formulated and test our method on two control tasks, showing clear improvements over the fixed horizon MPC scheme, while requiring only minutes of learning.
△ Less
Submitted 22 February, 2021;
originally announced February 2021.
-
Optimization of the Model Predictive Control Update Interval Using Reinforcement Learning
Authors:
Eivind Bøhn,
Sebastien Gros,
Signe Moe,
Tor Arne Johansen
Abstract:
In control applications there is often a compromise that needs to be made with regards to the complexity and performance of the controller and the computational resources that are available. For instance, the typical hardware platform in embedded control applications is a microcontroller with limited memory and processing power, and for battery powered applications the control system can account f…
▽ More
In control applications there is often a compromise that needs to be made with regards to the complexity and performance of the controller and the computational resources that are available. For instance, the typical hardware platform in embedded control applications is a microcontroller with limited memory and processing power, and for battery powered applications the control system can account for a significant portion of the energy consumption. We propose a controller architecture in which the computational cost is explicitly optimized along with the control objective. This is achieved by a three-part architecture where a high-level, computationally expensive controller generates plans, which a computationally simpler controller executes by compensating for prediction errors, while a recomputation policy decides when the plan should be recomputed. In this paper, we employ model predictive control (MPC) as the high-level plan-generating controller, a linear state feedback controller as the simpler compensating controller, and reinforcement learning (RL) to learn the recomputation policy. Simulation results for two examples showcase the architecture's ability to improve upon the MPC approach and find reasonable compromises weighing the performance on the control objective and the computational resources expended.
△ Less
Submitted 26 November, 2020;
originally announced November 2020.
-
Accelerating Reinforcement Learning with Suboptimal Guidance
Authors:
Eivind Bøhn,
Signe Moe,
Tor Arne Johansen
Abstract:
Reinforcement Learning in domains with sparse rewards is a difficult problem, and a large part of the training process is often spent searching the state space in a more or less random fashion for any learning signals. For control problems, we often have some controller readily available which might be suboptimal but nevertheless solves the problem to some degree. This controller can be used to gu…
▽ More
Reinforcement Learning in domains with sparse rewards is a difficult problem, and a large part of the training process is often spent searching the state space in a more or less random fashion for any learning signals. For control problems, we often have some controller readily available which might be suboptimal but nevertheless solves the problem to some degree. This controller can be used to guide the initial exploration phase of the learning controller towards reward yielding states, reducing the time before refinement of a viable policy can be initiated. In our work, the agent is guided through an auxiliary behaviour cloning loss which is made conditional on a Q-filter, i.e. it is only applied in situations where the critic deems the guiding controller to be better than the agent. The Q-filter provides a natural way to adjust the guidance throughout the training process, allowing the agent to exceed the guiding controller in a manner that is adaptive to the task at hand and the proficiency of the guiding controller. The contribution of this paper lies in identifying shortcomings in previously proposed implementations of the Q-filter concept, and in suggesting some ways these issues can be mitigated. These modifications are tested on the OpenAI Gym Fetch environments, showing clear improvements in adaptivity and yielding increased performance in all robotic environments tested.
△ Less
Submitted 21 November, 2019;
originally announced November 2019.
-
Deep Reinforcement Learning Attitude Control of Fixed-Wing UAVs Using Proximal Policy Optimization
Authors:
Eivind Bøhn,
Erlend M. Coates,
Signe Moe,
Tor Arne Johansen
Abstract:
Contemporary autopilot systems for unmanned aerial vehicles (UAVs) are far more limited in their flight envelope as compared to experienced human pilots, thereby restricting the conditions UAVs can operate in and the types of missions they can accomplish autonomously. This paper proposes a deep reinforcement learning (DRL) controller to handle the nonlinear attitude control problem, enabling exten…
▽ More
Contemporary autopilot systems for unmanned aerial vehicles (UAVs) are far more limited in their flight envelope as compared to experienced human pilots, thereby restricting the conditions UAVs can operate in and the types of missions they can accomplish autonomously. This paper proposes a deep reinforcement learning (DRL) controller to handle the nonlinear attitude control problem, enabling extended flight envelopes for fixed-wing UAVs. A proof-of-concept controller using the proximal policy optimization (PPO) algorithm is developed, and is shown to be capable of stabilizing a fixed-wing UAV from a large set of initial conditions to reference roll, pitch and airspeed values. The training process is outlined and key factors for its progression rate are considered, with the most important factor found to be limiting the number of variables in the observation vector, and including values for several previous time steps for these variables. The trained reinforcement learning (RL) controller is compared to a proportional-integral-derivative (PID) controller, and is found to converge in more cases than the PID controller, with comparable performance. Furthermore, the RL controller is shown to generalize well to unseen disturbances in the form of wind and turbulence, even in severe disturbance conditions.
△ Less
Submitted 13 November, 2019;
originally announced November 2019.
-
Cooperative decentralized circumnavigation with application to algal bloom tracking
Authors:
Joana Fonseca,
Jieqiang Wei,
Karl H. Johansson,
Tor Arne Johansen
Abstract:
Harmful algal blooms occur frequently and deteriorate water quality. A reliable method is proposed in this paper to track algal blooms using a set of autonomous surface robots. A satellite image indicates the existence and initial location of the algal bloom for the deployment of the robot system. The algal bloom area is approximated by a circle with time varying location and size. This circle is…
▽ More
Harmful algal blooms occur frequently and deteriorate water quality. A reliable method is proposed in this paper to track algal blooms using a set of autonomous surface robots. A satellite image indicates the existence and initial location of the algal bloom for the deployment of the robot system. The algal bloom area is approximated by a circle with time varying location and size. This circle is estimated and circumnavigated by the robots which are able to locally sense its boundary. A multi-agent control algorithm is proposed for the continuous monitoring of the dynamic evolution of the algal bloom. Such algorithm comprises of a decentralized least squares estimation of the target and a controller for circumnavigation. We prove the convergence of the robots to the circle and in equally spaced positions around it. Simulation results with data provided by the SINMOD ocean model are used to illustrate the theoretical results.
△ Less
Submitted 14 March, 2019;
originally announced March 2019.
-
Towards autonomous ocean observing systems using Miniature Underwater Gliders with UAV deployment and recovery capabilities
Authors:
Erik Sollesnes,
Ole Martin Brokstad,
Rolf Klæboe,
Bendik Vågen,
Alfredo Carella,
Alex Alcocer,
Artur Piotr Zolich,
Tor Arne Johansen
Abstract:
This paper presents preliminary results towards the development of an autonomous ocean observing system using Miniature Underwater Gliders (MUGs) that can operate with the support of Unmanned Aerial Vehicles (UAVs) and Unmanned Surface Vessels (USVs) for deployment, recovery, battery charging, and communication relay. The system reduces human intervention to the minimum, revolutionizing the afford…
▽ More
This paper presents preliminary results towards the development of an autonomous ocean observing system using Miniature Underwater Gliders (MUGs) that can operate with the support of Unmanned Aerial Vehicles (UAVs) and Unmanned Surface Vessels (USVs) for deployment, recovery, battery charging, and communication relay. The system reduces human intervention to the minimum, revolutionizing the affordability of a broad range of surveillance and data collection operations. The MUGs are equipped with a small Variable Buoyancy System (VBS) composed of a gas filled piston and a linear actuator powered by brushless DC motor and a rechargable lithium ion battery in an oil filled flexible enclosure. By using a fully pressure tolerant electronic design the aim is to reduce the total complexity, weight, and cost of the overall system. A first prototype of the VBS was built and demonstrated in a small aquarium. The electronic components were tested in a pressure testing facility to a minimum of 20bar. Preliminary results are promising and future work will focus on system and weight optimization, UAV deployment/recovery strategies, as well as sea trials to an operating depth of 200m.
△ Less
Submitted 8 February, 2019;
originally announced February 2019.
