-
Reciprocal Visibility
Authors:
Rakesh John Amala Arokia Nathan,
Sigrid Strand,
Dmitriy Shutin,
Oliver Bimber
Abstract:
We propose a guidance strategy to optimize real-time synthetic aperture sampling for occlusion removal with drones by pre-scanned point-cloud data. Depth information can be used to compute visibility of points on the ground for individual drone positions in the air. Inspired by Helmholtz reciprocity, we introduce reciprocal visibility to determine the dual situation - the visibility of potential s…
▽ More
We propose a guidance strategy to optimize real-time synthetic aperture sampling for occlusion removal with drones by pre-scanned point-cloud data. Depth information can be used to compute visibility of points on the ground for individual drone positions in the air. Inspired by Helmholtz reciprocity, we introduce reciprocal visibility to determine the dual situation - the visibility of potential sampling position in the air from given points of interest on the ground. The resulting visibility map encodes which point on the ground is visible by which magnitude from any position in the air. Based on such a map, we demonstrate a first greedy sampling optimization.
△ Less
Submitted 10 February, 2024;
originally announced February 2024.
-
Stereoscopic Depth Perception Through Foliage
Authors:
Robert Kerschner,
Rakesh John Amala Arokia Nathan,
Rafal Mantiuk,
Oliver Bimber
Abstract:
Both humans and computational methods struggle to discriminate the depths of objects hidden beneath foliage. However, such discrimination becomes feasible when we combine computational optical synthetic aperture sensing with the human ability to fuse stereoscopic images. For object identification tasks, as required in search and rescue, wildlife observation, surveillance, and early wildfire detect…
▽ More
Both humans and computational methods struggle to discriminate the depths of objects hidden beneath foliage. However, such discrimination becomes feasible when we combine computational optical synthetic aperture sensing with the human ability to fuse stereoscopic images. For object identification tasks, as required in search and rescue, wildlife observation, surveillance, and early wildfire detection, depth assists in differentiating true from false findings, such as people, animals, or vehicles vs. sun-heated patches at the ground level or in the tree crowns, or ground fires vs. tree trunks. We used video captured by a drone above dense woodland to test users' ability to discriminate depth. We found that this is impossible when viewing monoscopic video and relying on motion parallax. The same was true with stereoscopic video because of the occlusions caused by foliage. However, when synthetic aperture sensing was used to reduce occlusions and disparity-scaled stereoscopic video was presented, whereas computational (stereoscopic matching) methods were unsuccessful, human observers successfully discriminated depth. This shows the potential of systems which exploit the synergy between computational methods and human vision to perform tasks that neither can perform alone.
△ Less
Submitted 24 October, 2023;
originally announced October 2023.
-
Synthetic Aperture Anomaly Imaging
Authors:
Rakesh John Amala Arokia Nathan,
Oliver Bimber
Abstract:
Previous research has shown that in the presence of foliage occlusion, anomaly detection performs significantly better in integral images resulting from synthetic aperture imaging compared to applying it to conventional aerial images. In this article, we hypothesize and demonstrate that integrating detected anomalies is even more effective than detecting anomalies in integrals. This results in enh…
▽ More
Previous research has shown that in the presence of foliage occlusion, anomaly detection performs significantly better in integral images resulting from synthetic aperture imaging compared to applying it to conventional aerial images. In this article, we hypothesize and demonstrate that integrating detected anomalies is even more effective than detecting anomalies in integrals. This results in enhanced occlusion removal, outlier suppression, and higher chances of visually as well as computationally detecting targets that are otherwise occluded. Our hypothesis was validated through both: simulations and field experiments. We also present a real-time application that makes our findings practically available for blue-light organizations and others using commercial drone platforms. It is designed to address use-cases that suffer from strong occlusion caused by vegetation, such as search and rescue, wildlife observation, early wildfire detection, and sur-veillance.
△ Less
Submitted 26 April, 2023;
originally announced April 2023.
-
Synthetic Aperture Sensing for Occlusion Removal with Drone Swarms
Authors:
Rakesh John Amala Arokia Nathan,
Indrajit Kurmi,
Oliver Bimber
Abstract:
We demonstrate how efficient autonomous drone swarms can be in detecting and tracking occluded targets in densely forested areas, such as lost people during search and rescue missions. Exploration and optimization of local viewing conditions, such as occlusion density and target view obliqueness, provide much faster and much more reliable results than previous, blind sampling strategies that are b…
▽ More
We demonstrate how efficient autonomous drone swarms can be in detecting and tracking occluded targets in densely forested areas, such as lost people during search and rescue missions. Exploration and optimization of local viewing conditions, such as occlusion density and target view obliqueness, provide much faster and much more reliable results than previous, blind sampling strategies that are based on pre-defined waypoints. An adapted real-time particle swarm optimization and a new objective function are presented that are able to deal with dynamic and highly random through-foliage conditions. Synthetic aperture sensing is our fundamental sampling principle, and drone swarms are employed to approximate the optical signals of extremely wide and adaptable airborne lenses.
△ Less
Submitted 30 December, 2022;
originally announced December 2022.
-
Inverse Airborne Optical Sectioning
Authors:
Rakesh John Amala Arokia Nathan,
Indrajit Kurmi,
Oliver Bimber
Abstract:
We present Inverse Airborne Optical Sectioning (IAOS) an optical analogy to Inverse Synthetic Aperture Radar (ISAR). Moving targets, such as walking people, that are heavily occluded by vegetation can be made visible and tracked with a stationary optical sensor (e.g., a hovering camera drone above forest). We introduce the principles of IAOS (i.e., inverse synthetic aperture imaging), explain how…
▽ More
We present Inverse Airborne Optical Sectioning (IAOS) an optical analogy to Inverse Synthetic Aperture Radar (ISAR). Moving targets, such as walking people, that are heavily occluded by vegetation can be made visible and tracked with a stationary optical sensor (e.g., a hovering camera drone above forest). We introduce the principles of IAOS (i.e., inverse synthetic aperture imaging), explain how the signal of occluders can be further suppressed by filtering the Radon transform of the image integral, and present how targets motion parameters can be estimated manually and automatically. Finally, we show that while tracking occluded targets in conventional aerial images is infeasible, it becomes efficiently possible in integral images that result from IAOS.
△ Less
Submitted 27 July, 2022;
originally announced July 2022.
-
Vildehaye: A Family of Versatile, Widely-Applicable, and Field-Proven Lightweight Wildlife Tracking and Sensing Tags
Authors:
Sivan Toledo,
Shai Mendel,
Anat Levi,
Yoni Vortman,
Wiebke Ullmann,
Lena-Rosa Scherer,
Jan Pufelski,
Frank van Maarseveen,
Bas Denissen,
Allert Bijleveld,
Yotam Orchan,
Yoav Bartan,
Sivan Margalit,
Idan Talmon,
Ran Nathan
Abstract:
We describe the design and implementation of Vildehaye, a family of versatile, widely-applicable, and field-proven tags for wildlife sensing and radio tracking. The family includes 6 distinct hardware designs for tags, 3 add-on boards, a programming adapter, and base stations; modular firmware for tags and base stations (both standalone low-power embedded base stations and base stations tethered t…
▽ More
We describe the design and implementation of Vildehaye, a family of versatile, widely-applicable, and field-proven tags for wildlife sensing and radio tracking. The family includes 6 distinct hardware designs for tags, 3 add-on boards, a programming adapter, and base stations; modular firmware for tags and base stations (both standalone low-power embedded base stations and base stations tethered to a computer running Linux or Windows); and desktop software for programming and configuring tags, monitoring tags, and downloading and processing sensor data. The tags are versatile: they support multiple packet formats, data rates, and frequency bands; they can be configured for minimum mass (down to less than 1g), making them applicable to a wide range of flying and terrestrial animals, or for inclusion of important sensors and large memories; they can transmit packets compatible with time-of-arrival transmitter-localization systems, tag identification and state packets, and they can reliably upload sensor data through their radio link. The system has been designed, upgraded, and maintained as an academic research project, but it has been extensively used by 5 different groups of ecologists in 4 countries over a period of 5 years. More than 7100 tags have been produced and most of these have been deployed. Production used 41 manufacturing runs. The tags have been used in studies that so far resulted in 9 scientific publications in ecology (including in Science). The paper describes innovative design aspects of Vildehaye, field-use experiences, and lessons from the design, implementation, and maintenance of the system. Both the hardware and software of the system are open.
△ Less
Submitted 4 May, 2022;
originally announced June 2022.
-
On the Role of Field of View for Occlusion Removal with Airborne Optical Sectioning
Authors:
Francis Seits,
Indrajit Kurmi,
Rakesh John Amala Arokia Nathan,
Rudolf Ortner,
Oliver Bimber
Abstract:
Occlusion caused by vegetation is an essential problem for remote sensing applications in areas, such as search and rescue, wildfire detection, wildlife observation, surveillance, border control, and others. Airborne Optical Sectioning (AOS) is an optical, wavelength-independent synthetic aperture imaging technique that supports computational occlusion removal in real-time. It can be applied with…
▽ More
Occlusion caused by vegetation is an essential problem for remote sensing applications in areas, such as search and rescue, wildfire detection, wildlife observation, surveillance, border control, and others. Airborne Optical Sectioning (AOS) is an optical, wavelength-independent synthetic aperture imaging technique that supports computational occlusion removal in real-time. It can be applied with manned or unmanned aircrafts, such as drones. In this article, we demonstrate a relationship between forest density and field of view (FOV) of applied imaging systems. This finding was made with the help of a simulated procedural forest model which offers the consideration of more realistic occlusion properties than our previous statistical model. While AOS has been explored with automatic and autonomous research prototypes in the past, we present a free AOS integration for DJI systems. It enables bluelight organizations and others to use and explore AOS with compatible, manually operated, off-the-shelf drones. The (digitally cropped) default FOV for this implementation was chosen based on our new finding.
△ Less
Submitted 28 April, 2022;
originally announced April 2022.
-
Through-Foliage Tracking with Airborne Optical Sectioning
Authors:
Rakesh John Amala Arokia Nathan,
Indrajit Kurmi,
David C. Schedl,
Oliver Bimber
Abstract:
Detecting and tracking moving targets through foliage is difficult, and for many cases even impossible in regular aerial images and videos. We present an initial light-weight and drone-operated 1D camera array that supports parallel synthetic aperture aerial imaging. Our main finding is that color anomaly detection benefits significantly from image integration when compared to conventional raw ima…
▽ More
Detecting and tracking moving targets through foliage is difficult, and for many cases even impossible in regular aerial images and videos. We present an initial light-weight and drone-operated 1D camera array that supports parallel synthetic aperture aerial imaging. Our main finding is that color anomaly detection benefits significantly from image integration when compared to conventional raw images or video frames (on average 97% vs. 42% in precision in our field experiments). We demonstrate, that these two contributions can lead to the detection and tracking of moving people through densely occluding forest.
△ Less
Submitted 30 November, 2021; v1 submitted 12 November, 2021;
originally announced November 2021.
-
Profile-Driven Automated Mixed Precision
Authors:
Ralph Nathan,
Helia Naeimi,
Daniel J. Sorin,
Xiaobai Sun
Abstract:
We present a scheme to automatically set the precision of floating point variables in an application. We design a framework that profiles applications to measure undesirable numerical behavior at the floating point operation level. We use this framework to perform mixed precision analysis to heuristically set the precision of all variables in an application based on their numerical profiles. We ex…
▽ More
We present a scheme to automatically set the precision of floating point variables in an application. We design a framework that profiles applications to measure undesirable numerical behavior at the floating point operation level. We use this framework to perform mixed precision analysis to heuristically set the precision of all variables in an application based on their numerical profiles. We experimentally evaluate the mixed precision analysis to show that it can generate a range of results with different accuracy and performance characteristics.
△ Less
Submitted 1 June, 2016;
originally announced June 2016.
-
Topic Modeling of Behavioral Modes Using Sensor Data
Authors:
Yehezkel S. Resheff,
Shay Rotics,
Ran Nathan,
Daphna Weinshall
Abstract:
The field of Movement Ecology, like so many other fields, is experiencing a period of rapid growth in availability of data. As the volume rises, traditional methods are giving way to machine learning and data science, which are playing an increasingly large part it turning this data into science-driving insights. One rich and interesting source is the bio-logger. These small electronic wearable de…
▽ More
The field of Movement Ecology, like so many other fields, is experiencing a period of rapid growth in availability of data. As the volume rises, traditional methods are giving way to machine learning and data science, which are playing an increasingly large part it turning this data into science-driving insights. One rich and interesting source is the bio-logger. These small electronic wearable devices are attached to animals free to roam in their natural habitats, and report back readings from multiple sensors, including GPS and accelerometer bursts. A common use of accelerometer data is for supervised learning of behavioral modes. However, we need unsupervised analysis tools as well, in order to overcome the inherent difficulties of obtaining a labeled dataset, which in some cases is either infeasible or does not successfully encompass the full repertoire of behavioral modes of interest. Here we present a matrix factorization based topic-model method for accelerometer bursts, derived using a linear mixture property of patch features. Our method is validated via comparison to a labeled dataset, and is further compared to standard clustering algorithms.
△ Less
Submitted 16 November, 2015;
originally announced November 2015.
-
Recycled Error Bits: Energy-Efficient Architectural Support for Higher Precision Floating Point
Authors:
Ralph Nathan,
Bryan Anthonio,
Shih-Lien Lu,
Helia Naeimi,
Daniel J. Sorin,
Xiaobai Sun
Abstract:
In this work, we provide energy-efficient architectural support for floating point accuracy. Our goal is to provide accuracy that is far greater than that provided by the processor's hardware floating point unit (FPU). Specifically, for each floating point addition performed, we "recycle" that operation's error: the difference between the finite-precision result produced by the hardware and the re…
▽ More
In this work, we provide energy-efficient architectural support for floating point accuracy. Our goal is to provide accuracy that is far greater than that provided by the processor's hardware floating point unit (FPU). Specifically, for each floating point addition performed, we "recycle" that operation's error: the difference between the finite-precision result produced by the hardware and the result that would have been produced by an infinite-precision FPU. We make this error architecturally visible such that it can be used, if desired, by software. Experimental results on physical hardware show that software that exploits architecturally recycled error bits can achieve accuracy comparable to a 2B-bit FPU with performance and energy that are comparable to a B-bit FPU.
△ Less
Submitted 27 September, 2013;
originally announced September 2013.
