-
Wild Berry image dataset collected in Finnish forests and peatlands using drones
Authors:
Luigi Riz,
Sergio Povoli,
Andrea Caraffa,
Davide Boscaini,
Mohamed Lamine Mekhalfi,
Paul Chippendale,
Marjut Turtiainen,
Birgitta Partanen,
Laura Smith Ballester,
Francisco Blanes Noguera,
Alessio Franchi,
Elisa Castelli,
Giacomo Piccinini,
Luca Marchesotti,
Micael Santos Couceiro,
Fabio Poiesi
Abstract:
Berry picking has long-standing traditions in Finland, yet it is challenging and can potentially be dangerous. The integration of drones equipped with advanced imaging techniques represents a transformative leap forward, optimising harvests and promising sustainable practices. We propose WildBe, the first image dataset of wild berries captured in peatlands and under the canopy of Finnish forests u…
▽ More
Berry picking has long-standing traditions in Finland, yet it is challenging and can potentially be dangerous. The integration of drones equipped with advanced imaging techniques represents a transformative leap forward, optimising harvests and promising sustainable practices. We propose WildBe, the first image dataset of wild berries captured in peatlands and under the canopy of Finnish forests using drones. Unlike previous and related datasets, WildBe includes new varieties of berries, such as bilberries, cloudberries, lingonberries, and crowberries, captured under severe light variations and in cluttered environments. WildBe features 3,516 images, including a total of 18,468 annotated bounding boxes. We carry out a comprehensive analysis of WildBe using six popular object detectors, assessing their effectiveness in berry detection across different forest regions and camera types. We will release WildBe publicly.
△ Less
Submitted 15 May, 2024; v1 submitted 13 May, 2024;
originally announced May 2024.
-
FreeZe: Training-free zero-shot 6D pose estimation with geometric and vision foundation models
Authors:
Andrea Caraffa,
Davide Boscaini,
Amir Hamza,
Fabio Poiesi
Abstract:
Estimating the 6D pose of objects unseen during training is highly desirable yet challenging. Zero-shot object 6D pose estimation methods address this challenge by leveraging additional task-specific supervision provided by large-scale, photo-realistic synthetic datasets. However, their performance heavily depends on the quality and diversity of rendered data and they require extensive training. I…
▽ More
Estimating the 6D pose of objects unseen during training is highly desirable yet challenging. Zero-shot object 6D pose estimation methods address this challenge by leveraging additional task-specific supervision provided by large-scale, photo-realistic synthetic datasets. However, their performance heavily depends on the quality and diversity of rendered data and they require extensive training. In this work, we show how to tackle the same task but without training on specific data. We propose FreeZe, a novel solution that harnesses the capabilities of pre-trained geometric and vision foundation models. FreeZe leverages 3D geometric descriptors learned from unrelated 3D point clouds and 2D visual features learned from web-scale 2D images to generate discriminative 3D point-level descriptors. We then estimate the 6D pose of unseen objects by 3D registration based on RANSAC. We also introduce a novel algorithm to solve ambiguous cases due to geometrically symmetric objects that is based on visual features. We comprehensively evaluate FreeZe across the seven core datasets of the BOP Benchmark, which include over a hundred 3D objects and 20,000 images captured in various scenarios. FreeZe consistently outperforms all state-of-the-art approaches, including competitors extensively trained on synthetic 6D pose estimation data. Code will be publicly available at https://andreacaraffa.github.io/freeze.
△ Less
Submitted 3 April, 2024; v1 submitted 1 December, 2023;
originally announced December 2023.
-
The MONET dataset: Multimodal drone thermal dataset recorded in rural scenarios
Authors:
Luigi Riz,
Andrea Caraffa,
Matteo Bortolon,
Mohamed Lamine Mekhalfi,
Davide Boscaini,
André Moura,
José Antunes,
André Dias,
Hugo Silva,
Andreas Leonidou,
Christos Constantinides,
Christos Keleshis,
Dante Abate,
Fabio Poiesi
Abstract:
We present MONET, a new multimodal dataset captured using a thermal camera mounted on a drone that flew over rural areas, and recorded human and vehicle activities. We captured MONET to study the problem of object localisation and behaviour understanding of targets undergoing large-scale variations and being recorded from different and moving viewpoints. Target activities occur in two different la…
▽ More
We present MONET, a new multimodal dataset captured using a thermal camera mounted on a drone that flew over rural areas, and recorded human and vehicle activities. We captured MONET to study the problem of object localisation and behaviour understanding of targets undergoing large-scale variations and being recorded from different and moving viewpoints. Target activities occur in two different land sites, each with unique scene structures and cluttered backgrounds. MONET consists of approximately 53K images featuring 162K manually annotated bounding boxes. Each image is timestamp-aligned with drone metadata that includes information about attitudes, speed, altitude, and GPS coordinates. MONET is different from previous thermal drone datasets because it features multimodal data, including rural scenes captured with thermal cameras containing both person and vehicle targets, along with trajectory information and metadata. We assessed the difficulty of the dataset in terms of transfer learning between the two sites and evaluated nine object detection algorithms to identify the open challenges associated with this type of data. Project page: https://github.com/fabiopoiesi/monet_dataset.
△ Less
Submitted 19 July, 2023; v1 submitted 11 April, 2023;
originally announced April 2023.
