-
VR-NeRF: High-Fidelity Virtualized Walkable Spaces
Authors:
Linning Xu,
Vasu Agrawal,
William Laney,
Tony Garcia,
Aayush Bansal,
Changil Kim,
Samuel Rota Bulò,
Lorenzo Porzi,
Peter Kontschieder,
Aljaž Božič,
Dahua Lin,
Michael Zollhöfer,
Christian Richardt
Abstract:
We present an end-to-end system for the high-fidelity capture, model reconstruction, and real-time rendering of walkable spaces in virtual reality using neural radiance fields. To this end, we designed and built a custom multi-camera rig to densely capture walkable spaces in high fidelity and with multi-view high dynamic range images in unprecedented quality and density. We extend instant neural g…
▽ More
We present an end-to-end system for the high-fidelity capture, model reconstruction, and real-time rendering of walkable spaces in virtual reality using neural radiance fields. To this end, we designed and built a custom multi-camera rig to densely capture walkable spaces in high fidelity and with multi-view high dynamic range images in unprecedented quality and density. We extend instant neural graphics primitives with a novel perceptual color space for learning accurate HDR appearance, and an efficient mip-map** mechanism for level-of-detail rendering with anti-aliasing, while carefully optimizing the trade-off between quality and speed. Our multi-GPU renderer enables high-fidelity volume rendering of our neural radiance field model at the full VR resolution of dual 2K$\times$2K at 36 Hz on our custom demo machine. We demonstrate the quality of our results on our challenging high-fidelity datasets, and compare our method and datasets to existing baselines. We release our dataset on our project website.
△ Less
Submitted 4 November, 2023;
originally announced November 2023.
-
Artificial Intelligence Model for Tumoral Clinical Decision Support Systems
Authors:
Guillermo Iglesias,
Edgar Talavera,
Jesús Troya Garcìa,
Alberto Díaz-Álvarez,
Miguel Gracía-Remesal
Abstract:
Comparative diagnostic in brain tumor evaluation makes possible to use the available information of a medical center to compare similar cases when a new patient is evaluated. By leveraging Artificial Intelligence models, the proposed system is able of retrieving the most similar cases of brain tumors for a given query. The primary objective is to enhance the diagnostic process by generating more a…
▽ More
Comparative diagnostic in brain tumor evaluation makes possible to use the available information of a medical center to compare similar cases when a new patient is evaluated. By leveraging Artificial Intelligence models, the proposed system is able of retrieving the most similar cases of brain tumors for a given query. The primary objective is to enhance the diagnostic process by generating more accurate representations of medical images, with a particular focus on patient-specific normal features and pathologies. The proposed model uses Artificial Intelligence to detect patient features to recommend the most similar cases from a database. The system not only suggests similar cases but also balances the representation of healthy and abnormal features in its design. This not only encourages the generalization of its use but also aids clinicians in their decision-making processes. We conducted a comparative analysis of our approach in relation to similar studies. The proposed architecture obtains a Dice coefficient of 0.474 in both tumoral and healthy regions of the patients, which outperforms previous literature. Our proposed model excels at extracting and combining anatomical and pathological features from brain \glspl{mr}, achieving state-of-the-art results while relying on less expensive label information. This substantially reduces the overall cost of the training process. This paper provides substantial grounds for further exploration of the broader applicability and optimization of the proposed architecture to enhance clinical decision-making. The novel approach presented in this work marks a significant advancement in the field of medical diagnosis, particularly in the context of Artificial Intelligence-assisted image retrieval, and promises to reduce costs and improve the quality of patient care using Artificial Intelligence as a support tool instead of a black box system.
△ Less
Submitted 24 May, 2024; v1 submitted 9 January, 2023;
originally announced January 2023.
-
Assessing the Syntactic Capabilities of Transformer-based Multilingual Language Models
Authors:
Laura Pérez-Mayos,
Alba Táboas García,
Simon Mille,
Leo Wanner
Abstract:
Multilingual Transformer-based language models, usually pretrained on more than 100 languages, have been shown to achieve outstanding results in a wide range of cross-lingual transfer tasks. However, it remains unknown whether the optimization for different languages conditions the capacity of the models to generalize over syntactic structures, and how languages with syntactic phenomena of differe…
▽ More
Multilingual Transformer-based language models, usually pretrained on more than 100 languages, have been shown to achieve outstanding results in a wide range of cross-lingual transfer tasks. However, it remains unknown whether the optimization for different languages conditions the capacity of the models to generalize over syntactic structures, and how languages with syntactic phenomena of different complexity are affected. In this work, we explore the syntactic generalization capabilities of the monolingual and multilingual versions of BERT and RoBERTa. More specifically, we evaluate the syntactic generalization potential of the models on English and Spanish tests, comparing the syntactic abilities of monolingual and multilingual models on the same language (English), and of multilingual models on two different languages (English and Spanish). For English, we use the available SyntaxGym test suite; for Spanish, we introduce SyntaxGymES, a novel ensemble of targeted syntactic tests in Spanish, designed to evaluate the syntactic generalization capabilities of language models through the SyntaxGym online platform.
△ Less
Submitted 10 May, 2021;
originally announced May 2021.
-
Estimating city-level travel patterns using street imagery: a case study of using Google Street View in Britain
Authors:
Rahul Goel,
Leandro M. T. Garcia,
Anna Goodman,
Rob Johnson,
Rachel Aldred,
Manoradhan Murugesan,
Soren Brage,
Kavi Bhalla,
James Woodcock
Abstract:
Street imagery is a promising big data source providing current and historical images in more than 100 countries. Previous studies used this data to audit built environment features. Here we explore a novel application, using Google Street View (GSV) to predict travel patterns at the city level. We sampled 34 cities in Great Britain. In each city, we accessed GSV images from 1000 random locations…
▽ More
Street imagery is a promising big data source providing current and historical images in more than 100 countries. Previous studies used this data to audit built environment features. Here we explore a novel application, using Google Street View (GSV) to predict travel patterns at the city level. We sampled 34 cities in Great Britain. In each city, we accessed GSV images from 1000 random locations from years overlap** with the 2011 Census and the 2011-2013 Active People Survey (APS). We manually annotated images into seven categories of road users. We developed regression models with the counts of images of road users as predictors. Outcomes included Census-reported commute shares of four modes (walking plus public transport, cycling, motorcycle, and car), and APS-reported past-month participation in walking and cycling. In bivariate analyses, we found high correlations between GSV counts of cyclists (GSV-cyclists) and cycle commute mode share (r=0.92) and past-month cycling (r=0.90). Likewise, GSV-pedestrians was moderately correlated with past-month walking for transport (r=0.46), GSV-motorcycles was moderately correlated with commute share of motorcycles (r=0.44), and GSV-buses was highly correlated with commute share of walking plus public transport (r=0.81). GSV-car was not correlated with car commute mode share (r=-0.12). However, in multivariable regression models, all mode shares were predicted well. Cross-validation analyses showed good prediction performance for all the outcomes except past-month walking. Street imagery is a promising new big data source to predict urban mobility patterns. Further testing across multiple settings is warranted both for cross-sectional and longitudinal assessments.
△ Less
Submitted 8 February, 2018;
originally announced February 2018.
