-
Planted: a dataset for planted forest identification from multi-satellite time series
Authors:
Luis Miguel Pazos-Outón,
Cristina Nader Vasconcelos,
Anton Raichuk,
Anurag Arnab,
Dan Morris,
Maxim Neumann
Abstract:
Protecting and restoring forest ecosystems is critical for biodiversity conservation and carbon sequestration. Forest monitoring on a global scale is essential for prioritizing and assessing conservation efforts. Satellite-based remote sensing is the only viable solution for providing global coverage, but to date, large-scale forest monitoring is limited to single modalities and single time points…
▽ More
Protecting and restoring forest ecosystems is critical for biodiversity conservation and carbon sequestration. Forest monitoring on a global scale is essential for prioritizing and assessing conservation efforts. Satellite-based remote sensing is the only viable solution for providing global coverage, but to date, large-scale forest monitoring is limited to single modalities and single time points. In this paper, we present a dataset consisting of data from five public satellites for recognizing forest plantations and planted tree species across the globe. Each satellite modality consists of a multi-year time series. The dataset, named \PlantD, includes over 2M examples of 64 tree label classes (46 genera and 40 species), distributed among 41 countries. This dataset is released to foster research in forest monitoring using multimodal, multi-scale, multi-temporal data sources. Additionally, we present initial baseline results and evaluate modality fusion and data augmentation approaches for this dataset.
△ Less
Submitted 24 May, 2024;
originally announced June 2024.
-
Greedy Growing Enables High-Resolution Pixel-Based Diffusion Models
Authors:
Cristina N. Vasconcelos,
Abdullah Rashwan,
Austin Waters,
Trevor Walker,
Keyang Xu,
Jimmy Yan,
Rui Qian,
Shixin Luo,
Zarana Parekh,
Andrew Bunner,
Hongliang Fei,
Roopal Garg,
Mandy Guo,
Ivana Kajic,
Yeqing Li,
Henna Nandwani,
Jordi Pont-Tuset,
Yasumasa Onoe,
Sarah Rosston,
Su Wang,
Wenlei Zhou,
Kevin Swersky,
David J. Fleet,
Jason M. Baldridge,
Oliver Wang
Abstract:
We address the long-standing problem of how to learn effective pixel-based image diffusion models at scale, introducing a remarkably simple greedy growing method for stable training of large-scale, high-resolution models. without the needs for cascaded super-resolution components. The key insight stems from careful pre-training of core components, namely, those responsible for text-to-image alignm…
▽ More
We address the long-standing problem of how to learn effective pixel-based image diffusion models at scale, introducing a remarkably simple greedy growing method for stable training of large-scale, high-resolution models. without the needs for cascaded super-resolution components. The key insight stems from careful pre-training of core components, namely, those responsible for text-to-image alignment {\it vs.} high-resolution rendering. We first demonstrate the benefits of scaling a {\it Shallow UNet}, with no down(up)-sampling enc(dec)oder. Scaling its deep core layers is shown to improve alignment, object structure, and composition. Building on this core model, we propose a greedy algorithm that grows the architecture into high-resolution end-to-end models, while preserving the integrity of the pre-trained representation, stabilizing training, and reducing the need for large high-resolution datasets. This enables a single stage model capable of generating high-resolution images without the need of a super-resolution cascade. Our key results rely on public datasets and show that we are able to train non-cascaded models up to 8B parameters with no further regularization schemes. Vermeer, our full pipeline model trained with internal datasets to produce 1024x1024 images, without cascades, is preferred by 44.0% vs. 21.4% human evaluators over SDXL.
△ Less
Submitted 26 May, 2024;
originally announced May 2024.
-
ConformalLayers: A non-linear sequential neural network with associative layers
Authors:
Eduardo Vera Sousa,
Leandro A. F. Fernandes,
Cristina Nader Vasconcelos
Abstract:
Convolutional Neural Networks (CNNs) have been widely applied. But as the CNNs grow, the number of arithmetic operations and memory footprint also increase. Furthermore, typical non-linear activation functions do not allow associativity of the operations encoded by consecutive layers, preventing the simplification of intermediate steps by combining them. We present a new activation function that a…
▽ More
Convolutional Neural Networks (CNNs) have been widely applied. But as the CNNs grow, the number of arithmetic operations and memory footprint also increase. Furthermore, typical non-linear activation functions do not allow associativity of the operations encoded by consecutive layers, preventing the simplification of intermediate steps by combining them. We present a new activation function that allows associativity between sequential layers of CNNs. Even though our activation function is non-linear, it can be represented by a sequence of linear operations in the conformal model for Euclidean geometry. In this domain, operations like, but not limited to, convolution, average pooling, and dropout remain linear. We take advantage of associativity to combine all the "conformal layers" and make the cost of inference constant regardless of the depth of the network.
△ Less
Submitted 9 November, 2021; v1 submitted 22 October, 2021;
originally announced October 2021.
-
Spatially and color consistent environment lighting estimation using deep neural networks for mixed reality
Authors:
Bruno Augusto Dorta Marques,
Esteban Walter Gonzalez Clua,
Anselmo Antunes Montenegro,
Cristina Nader Vasconcelos
Abstract:
The representation of consistent mixed reality (XR) environments requires adequate real and virtual illumination composition in real-time. Estimating the lighting of a real scenario is still a challenge. Due to the ill-posed nature of the problem, classical inverse-rendering techniques tackle the problem for simple lighting setups. However, those assumptions do not satisfy the current state-of-art…
▽ More
The representation of consistent mixed reality (XR) environments requires adequate real and virtual illumination composition in real-time. Estimating the lighting of a real scenario is still a challenge. Due to the ill-posed nature of the problem, classical inverse-rendering techniques tackle the problem for simple lighting setups. However, those assumptions do not satisfy the current state-of-art in computer graphics and XR applications. While many recent works solve the problem using machine learning techniques to estimate the environment light and scene's materials, most of them are limited to geometry or previous knowledge. This paper presents a CNN-based model to estimate complex lighting for mixed reality environments with no previous information about the scene. We model the environment illumination using a set of spherical harmonics (SH) environment lighting, capable of efficiently represent area lighting. We propose a new CNN architecture that inputs an RGB image and recognizes, in real-time, the environment lighting. Unlike previous CNN-based lighting estimation methods, we propose using a highly optimized deep neural network architecture, with a reduced number of parameters, that can learn high complex lighting scenarios from real-world high-dynamic-range (HDR) environment images. We show in the experiments that the CNN architecture can predict the environment lighting with an average mean squared error (MSE) of \num{7.85e-04} when comparing SH lighting coefficients. We validate our model in a variety of mixed reality scenarios. Furthermore, we present qualitative results comparing relights of real-world scenes.
△ Less
Submitted 17 August, 2021;
originally announced August 2021.
-
Convolutional Neural Network Committees for Melanoma Classification with Classical And Expert Knowledge Based Image Transforms Data Augmentation
Authors:
Cristina Nader Vasconcelos,
Bárbara Nader Vasconcelos
Abstract:
Skin cancer is a major public health problem, as is the most common type of cancer and represents more than half of cancer diagnoses worldwide. Early detection influences the outcome of the disease and motivates our work. We investigate the composition of CNN committees and data augmentation for the the ISBI 2017 Melanoma Classification Challenge (named Skin Lesion Analysis towards Melanoma Detect…
▽ More
Skin cancer is a major public health problem, as is the most common type of cancer and represents more than half of cancer diagnoses worldwide. Early detection influences the outcome of the disease and motivates our work. We investigate the composition of CNN committees and data augmentation for the the ISBI 2017 Melanoma Classification Challenge (named Skin Lesion Analysis towards Melanoma Detection) facing the peculiarities of dealing with such a small, unbalanced, biological database. For that, we explore committees of Convolutional Neural Networks trained over the ISBI challenge training dataset artificially augmented by both classical image processing transforms and image war** guided by specialist knowledge about the lesion axis and improve the final classifier invariance to common melanoma variations.
△ Less
Submitted 15 March, 2017; v1 submitted 22 February, 2017;
originally announced February 2017.
-
Minimizing cyber sickness in head mounted display systems: design guidelines and applications
Authors:
Thiago M. Porcino,
Esteban W. Clua,
Cristina N. Vasconcelos,
Daniela Trevisan,
Luis Valente
Abstract:
We are experiencing an upcoming trend of using head mounted display systems in games and serious games, which is likely to become an established practice in the near future. While these systems provide highly immersive experiences, many users have been reporting discomfort symptoms, such as nausea, sickness, and headaches, among others. When using VR for health applications, this is more critical,…
▽ More
We are experiencing an upcoming trend of using head mounted display systems in games and serious games, which is likely to become an established practice in the near future. While these systems provide highly immersive experiences, many users have been reporting discomfort symptoms, such as nausea, sickness, and headaches, among others. When using VR for health applications, this is more critical, since the discomfort may interfere a lot in treatments. In this work we discuss possible causes of these issues, and present possible solutions as design guidelines that may mitigate them. In this context, we go deeper within a dynamic focus solution to reduce discomfort in immersive virtual environments, when using first-person navigation. This solution applies an heuristic model of visual attention that works in real time. This work also discusses a case study (as a first-person spatial shooter demo) that applies this solution and the proposed design guidelines.
△ Less
Submitted 18 November, 2016;
originally announced November 2016.
