-
Harfang3D Dog-Fight Sandbox: A Reinforcement Learning Research Platform for the Customized Control Tasks of Fighter Aircrafts
Authors:
Muhammed Murat Özbek,
Süleyman Yıldırım,
Muhammet Aksoy,
Eric Kernin,
Emre Koyuncu
Abstract:
The advent of deep learning (DL) gave rise to significant breakthroughs in Reinforcement Learning (RL) research. Deep Reinforcement Learning (DRL) algorithms have reached super-human level skills when applied to vision-based control problems as such in Atari 2600 games where environment states were extracted from pixel information. Unfortunately, these environments are far from being applicable to…
▽ More
The advent of deep learning (DL) gave rise to significant breakthroughs in Reinforcement Learning (RL) research. Deep Reinforcement Learning (DRL) algorithms have reached super-human level skills when applied to vision-based control problems as such in Atari 2600 games where environment states were extracted from pixel information. Unfortunately, these environments are far from being applicable to highly dynamic and complex real-world tasks as in autonomous control of a fighter aircraft since these environments only involve 2D representation of a visual world. Here, we present a semi-realistic flight simulation environment Harfang3D Dog-Fight Sandbox for fighter aircrafts. It is aimed to be a flexible toolbox for the investigation of main challenges in aviation studies using Reinforcement Learning. The program provides easy access to flight dynamics model, environment states, and aerodynamics of the plane enabling user to customize any specific task in order to build intelligent decision making (control) systems via RL. The software also allows deployment of bot aircrafts and development of multi-agent tasks. This way, multiple groups of aircrafts can be configured to be competitive or cooperative agents to perform complicated tasks including Dog Fight. During the experiments, we carried out training for two different scenarios: navigating to a designated location and within visual range (WVR) combat, shortly Dog Fight. Using Deep Reinforcement Learning techniques for both scenarios, we were able to train competent agents that exhibit human-like behaviours. Based on this results, it is confirmed that Harfang3D Dog-Fight Sandbox can be utilized as a 3D realistic RL research platform.
△ Less
Submitted 13 October, 2022;
originally announced October 2022.
-
Reinforcement Learning based Air Combat Maneuver Generation
Authors:
Muhammed Murat Ozbek,
Emre Koyuncu
Abstract:
The advent of artificial intelligence technology paved the way of many researches to be made within air combat sector. Academicians and many other researchers did a research on a prominent research direction called autonomous maneuver decision of UAV. Elaborative researches produced some outcomes, but decisions that include Reinforcement Learning(RL) came out to be more efficient. There have been…
▽ More
The advent of artificial intelligence technology paved the way of many researches to be made within air combat sector. Academicians and many other researchers did a research on a prominent research direction called autonomous maneuver decision of UAV. Elaborative researches produced some outcomes, but decisions that include Reinforcement Learning(RL) came out to be more efficient. There have been many researches and experiments done to make an agent reach its target in an optimal way, most prominent are Genetic Algorithm(GA) , A star, RRT and other various optimization techniques have been used. But Reinforcement Learning is the well known one for its success. In DARPHA Alpha Dogfight Trials, reinforcement learning prevailed against a real veteran F16 human pilot who was trained by Boeing. This successor model was developed by Heron Systems. After this accomplishment, reinforcement learning bring tremendous attention on itself. In this research we aimed our UAV which has a dubin vehicle dynamic property to move to the target in two dimensional space in an optimal path using Twin Delayed Deep Deterministic Policy Gradients (TD3) and used in experience replay Hindsight Experience Replay(HER).We did tests on two different environments and used simulations.
△ Less
Submitted 14 January, 2022;
originally announced January 2022.
-
Large-scale 3D map** of the intergalactic medium using the Lyman Alpha Forest
Authors:
Melih Ozbek,
Rupert A. C. Croft,
Nishikanta Khandai
Abstract:
Maps of the large-scale structure of the Universe at redshifts 2-4 can be made with the Lyman-alpha forest which are complementary to low redshift galaxy surveys. We apply the Wiener interpolation method of Caucci et al. to construct three-dimensional maps from sets of Lyman-alpha forest spectra taken from cosmological hydrodynamic simulations. We mimic some current and future quasar redshift surv…
▽ More
Maps of the large-scale structure of the Universe at redshifts 2-4 can be made with the Lyman-alpha forest which are complementary to low redshift galaxy surveys. We apply the Wiener interpolation method of Caucci et al. to construct three-dimensional maps from sets of Lyman-alpha forest spectra taken from cosmological hydrodynamic simulations. We mimic some current and future quasar redshift surveys (BOSS, eBOSS and MS-DESI) by choosing similar sightline densities. We use these appropriate subsets of the Lyman-alpha absorption sightlines to reconstruct the full three dimensional Lyman-alpha flux field and perform comparisons between the true and the reconstructed fields. We study global statistical properties of the intergalactic medium (IGM) maps with auto-correlation and cross-correlation analysis, slice plots, local peaks and point by point scatter. We find that both the density field and the statistical proper- ties of the IGM are recovered well enough that the resulting IGM maps can be meaningfully considered to represent large-scale maps of the Universe in agreement with Caucci et al., on larger scales and for sparser sightlines than had been tested previously. Quantitatively, for sightline parameters comparable to current and near future surveys the correlation coefficient between true and reconstructed fields is r > 0.9 on scales > 30 h^-1 Mpc. The properties of the maps are relatively insensitive to the precise form of the covariance matrix used. The final BOSS quasar Lyman-alpha forest sample will allow maps to be made with a resolution of ~ 30 h^-1 Mpc over a volume of ~ 15 h^-3 Gpc^3 between redshifts 1.9 and 2.3.
△ Less
Submitted 4 February, 2016;
originally announced February 2016.
-
Nonparametric 3D map of the IGM using the Lyman-alpha forest
Authors:
Jessi Cisewski,
Rupert A. C. Croft,
Peter E. Freeman,
Christopher R. Genovese,
Nishikanta Khandai,
Melih Ozbek,
Larry Wasserman
Abstract:
Visualizing the high-redshift Universe is difficult due to the dearth of available data; however, the Lyman-alpha forest provides a means to map the intergalactic medium at redshifts not accessible to large galaxy surveys. Large-scale structure surveys, such as the Baryon Oscillation Spectroscopic Survey (BOSS), have collected quasar (QSO) spectra that enable the reconstruction of HI density fluct…
▽ More
Visualizing the high-redshift Universe is difficult due to the dearth of available data; however, the Lyman-alpha forest provides a means to map the intergalactic medium at redshifts not accessible to large galaxy surveys. Large-scale structure surveys, such as the Baryon Oscillation Spectroscopic Survey (BOSS), have collected quasar (QSO) spectra that enable the reconstruction of HI density fluctuations. The data fall on a collection of lines defined by the lines-of-sight (LOS) of the QSO, and a major issue with producing a 3D reconstruction is determining how to model the regions between the LOS. We present a method that produces a 3D map of this relatively uncharted portion of the Universe by employing local polynomial smoothing, a nonparametric methodology. The performance of the method is analyzed on simulated data that mimics the varying number of LOS expected in real data, and then is applied to a sample region selected from BOSS. Evaluation of the reconstruction is assessed by considering various features of the predicted 3D maps including visual comparison of slices, PDFs, counts of local minima and maxima, and standardized correlation functions. This 3D reconstruction allows for an initial investigation of the topology of this portion of the Universe using persistent homology.
△ Less
Submitted 8 January, 2014;
originally announced January 2014.
-
Observational Requirements for Lyman-alpha Forest Tomographic Map** of Large-Scale Structure at z ~ 2
Authors:
Khee-Gan Lee,
Joseph F. Hennawi,
Martin White,
Rupert Croft,
Melih Ozbek
Abstract:
The z > 2 Lyman-alpha (Lya) forest traces the underlying dark-matter distribution on large scales and, given sufficient sightlines, can be used to create 3D maps of large-scale structure. We examine the observational requirements to construct such maps and estimate the signal-to-noise as a function of exposure time and sightline density. Sightline densities at z = 2.25 are n_los = [360, 1200,3300]…
▽ More
The z > 2 Lyman-alpha (Lya) forest traces the underlying dark-matter distribution on large scales and, given sufficient sightlines, can be used to create 3D maps of large-scale structure. We examine the observational requirements to construct such maps and estimate the signal-to-noise as a function of exposure time and sightline density. Sightline densities at z = 2.25 are n_los = [360, 1200,3300] deg^{-2} at limiting magnitudes of g =[24.0, 24.5,25.0], resulting in transverse sightline separations of d_perp = [3.6, 1.9, 1.2] h^{-1} Mpc, which roughly sets the reconstruction scale. We simulate these reconstructions using mock spectra with realistic noise properties, and find that spectra with S/N = 4 per angstrom can be used to generate maps that clearly trace the underlying dark-matter at overdensities of rho/<rho> ~ 1. For the VLT/VIMOS spectrograph, exposure times t_exp = [4, 6, 10] hrs are sufficient for maps with spatial resolution epsilon_3d = [5.0, 3.2, 2.3] h^{-1} Mpc. Assuming ~ 250 h^{-1} Mpc is probed along the line-of-sight, 1 deg^2 of survey area would cover a comoving volume of ~ 10^6 h^{-3} Mpc^3 at <z>=2.3, enabling efficient map** of large volumes with 8-10m telescopes. These maps could be used to study galaxy environments, detect proto-clusters, and study the topology of large-scale structure at high-z.
△ Less
Submitted 15 May, 2014; v1 submitted 5 September, 2013;
originally announced September 2013.
