-
DALL-M: Context-Aware Clinical Data Augmentation with LLMs
Authors:
Chihcheng Hsieh,
Catarina Moreira,
Isabel Blanco Nobre,
Sandra Costa Sousa,
Chun Ouyang,
Margot Brereton,
Joaquim Jorge,
Jacinto C. Nascimento
Abstract:
X-ray images are vital in medical diagnostics, but their effectiveness is limited without clinical context. Radiologists often find chest X-rays insufficient for diagnosing underlying diseases, necessitating comprehensive clinical features and data integration. We present a novel technique to enhance the clinical context through augmentation techniques with clinical tabular data, thereby improving…
▽ More
X-ray images are vital in medical diagnostics, but their effectiveness is limited without clinical context. Radiologists often find chest X-rays insufficient for diagnosing underlying diseases, necessitating comprehensive clinical features and data integration. We present a novel technique to enhance the clinical context through augmentation techniques with clinical tabular data, thereby improving its applicability and reliability in AI medical diagnostics. To address this, we introduce a pioneering approach to clinical data augmentation that employs large language models (LLMs) to generate patient contextual synthetic data. This methodology is crucial for training more robust deep learning models in healthcare. It preserves the integrity of real patient data while enriching the dataset with contextually relevant synthetic features, significantly enhancing model performance. DALL-M uses a three-phase feature generation process: (i) clinical context storage, (ii) expert query generation, and (iii) context-aware feature augmentation. DALL-M generates new, clinically relevant features by synthesizing chest X-ray images and reports. Applied to 799 cases using nine features from the MIMIC-IV dataset, it created an augmented set of 91 features. This is the first work to generate contextual values for existing and new features based on patients' X-ray reports, gender, and age and to produce new contextual knowledge during data augmentation. Empirical validation with machine learning models, including Decision Trees, Random Forests, XGBoost, and TabNET, showed significant performance improvements. Incorporating augmented features increased the F1 score by 16.5% and Precision and Recall by approximately 25%. DALL-M addresses a critical gap in clinical data augmentation, offering a robust framework for generating contextually enriched datasets.
△ Less
Submitted 11 July, 2024;
originally announced July 2024.
-
Impact of Social Relationships on Peer Assessment in E-Learning
Authors:
Francisco Sousa,
Tomás Alves,
Sandra Gama,
Joaquim Jorge,
Daniel Gonçalves
Abstract:
Peer assessment has been widely studied as a replacement for traditional evaluation, not only by reducing the professors' workload but mainly by benefiting students' engagement and learning. Although several works successfully validate its accuracy and fairness, more research must be done on how students' pre-existing social relationships affect the grades they give their peers in an e-learning co…
▽ More
Peer assessment has been widely studied as a replacement for traditional evaluation, not only by reducing the professors' workload but mainly by benefiting students' engagement and learning. Although several works successfully validate its accuracy and fairness, more research must be done on how students' pre-existing social relationships affect the grades they give their peers in an e-learning course. We developed a Moodle plugin to provide the platform with peer assessment capabilities in forums and used it on an MSc course. The plugin curated the reviewer set for a post based on the author's relationships and included rubrics to counter the possible interpersonal effects of peer assessment. Results confirm that peer assessment is reliable and accurate for works with at least three peer assessments, although students' grades are slightly higher. The impact of social relationships is noticeable when students who do not like another peer grade their work consistently lower than students who have a positive connection. However, this has little influence on the final aggregate peer grade. Our findings show that peer assessment can replace traditional evaluation in an e-learning environment where students are familiar with each other.
△ Less
Submitted 1 July, 2024;
originally announced July 2024.
-
SPARC: Shared Perspective with Avatar Distortion for Remote Collaboration in VR
Authors:
João Simões,
Anderson Maciel,
Catarina Moreira,
Joaquim Jorge
Abstract:
Telepresence VR systems allow for face-to-face communication, promoting the feeling of presence and understanding of nonverbal cues. However, when discussing virtual 3D objects, limitations to presence and communication cause deictic gestures to lose meaning due to disparities in orientation. Current approaches use shared perspective, and avatar overlap to restore these references, which cause occ…
▽ More
Telepresence VR systems allow for face-to-face communication, promoting the feeling of presence and understanding of nonverbal cues. However, when discussing virtual 3D objects, limitations to presence and communication cause deictic gestures to lose meaning due to disparities in orientation. Current approaches use shared perspective, and avatar overlap to restore these references, which cause occlusions and discomfort that worsen when multiple users participate. We introduce a new approach to shared perspective in multi-user collaboration where the avatars are not co-located. Each person sees the others' avatars at their positions around the workspace while having a first-person view of the workspace. Whenever a user manipulates an object, others will see his/her arms stretching to reach that object in their perspective. SPARC combines a shared orientation and supports nonverbal communication, minimizing occlusions. We conducted a user study (n=18) to understand how the novel approach impacts task performance and workspace awareness. We found evidence that SPARC is more efficient and less mentally demanding than life-like settings.
△ Less
Submitted 7 June, 2024;
originally announced June 2024.
-
SelfReDepth: Self-Supervised Real-Time Depth Restoration for Consumer-Grade Sensors
Authors:
Alexandre Duarte,
Francisco Fernandes,
João M. Pereira,
Catarina Moreira,
Jacinto C. Nascimento,
Joaquim Jorge
Abstract:
Depth maps produced by consumer-grade sensors suffer from inaccurate measurements and missing data from either system or scene-specific sources. Data-driven denoising algorithms can mitigate such problems. However, they require vast amounts of ground truth depth data. Recent research has tackled this limitation using self-supervised learning techniques, but it requires multiple RGB-D sensors. More…
▽ More
Depth maps produced by consumer-grade sensors suffer from inaccurate measurements and missing data from either system or scene-specific sources. Data-driven denoising algorithms can mitigate such problems. However, they require vast amounts of ground truth depth data. Recent research has tackled this limitation using self-supervised learning techniques, but it requires multiple RGB-D sensors. Moreover, most existing approaches focus on denoising single isolated depth maps or specific subjects of interest, highlighting a need for methods to effectively denoise depth maps in real-time dynamic environments. This paper extends state-of-the-art approaches for depth-denoising commodity depth devices, proposing SelfReDepth, a self-supervised deep learning technique for depth restoration, via denoising and hole-filling by inpainting full-depth maps captured with RGB-D sensors. The algorithm targets depth data in video streams, utilizing multiple sequential depth frames coupled with color data to achieve high-quality depth videos with temporal coherence. Finally, SelfReDepth is designed to be compatible with various RGB-D sensors and usable in real-time scenarios as a pre-processing step before applying other depth-dependent algorithms. Our results demonstrate our approach's real-time performance on real-world datasets. They show that it outperforms state-of-the-art denoising and restoration performance at over 30fps on Commercial Depth Cameras, with potential benefits for augmented and mixed-reality applications.
△ Less
Submitted 5 June, 2024;
originally announced June 2024.
-
Pseudo-Haptics Survey: Human-Computer Interaction in Extended Reality & Teleoperation
Authors:
Rui Xavier,
José Luís Silva,
Rodrigo Ventura,
Joaquim Jorge
Abstract:
Pseudo-haptic techniques are becoming increasingly popular in human-computer interaction. They replicate haptic sensations by leveraging primarily visual feedback rather than mechanical actuators. These techniques bridge the gap between the real and virtual worlds by exploring the brain's ability to integrate visual and haptic information. One of the many advantages of pseudo-haptic techniques is…
▽ More
Pseudo-haptic techniques are becoming increasingly popular in human-computer interaction. They replicate haptic sensations by leveraging primarily visual feedback rather than mechanical actuators. These techniques bridge the gap between the real and virtual worlds by exploring the brain's ability to integrate visual and haptic information. One of the many advantages of pseudo-haptic techniques is that they are cost-effective, portable, and flexible. They eliminate the need for direct attachment of haptic devices to the body, which can be heavy and large and require a lot of power and maintenance. Recent research has focused on applying these techniques to extended reality and mid-air interactions. To better understand the potential of pseudo-haptic techniques, the authors developed a novel taxonomy encompassing tactile feedback, kinesthetic feedback, and combined categories in multimodal approaches, ground not covered by previous surveys. This survey highlights multimodal strategies and potential avenues for future studies, particularly regarding integrating these techniques into extended reality and collaborative virtual environments.
△ Less
Submitted 3 June, 2024;
originally announced June 2024.
-
Eery Space: Facilitating Virtual Meetings Through Remote Proxemics
Authors:
Maurício Sousa,
Daniel Mendes,
Alfredo Ferreira,
João Madeiras Pereira,
Joaquim Jorge
Abstract:
Virtual meetings have become increasingly common with modern video-conference and collaborative software. While they allow obvious savings in time and resources, current technologies add unproductive layers of protocol to the flow of communication between participants, rendering the interactions far from seamless. In this work we introduce Remote Proxemics, an extension of proxemics aimed at bring…
▽ More
Virtual meetings have become increasingly common with modern video-conference and collaborative software. While they allow obvious savings in time and resources, current technologies add unproductive layers of protocol to the flow of communication between participants, rendering the interactions far from seamless. In this work we introduce Remote Proxemics, an extension of proxemics aimed at bringing the syntax of co-located proximal interactions to virtual meetings. We propose Eery Space, a shared virtual locus that results from merging multiple remote areas, where meeting participants' are located side-by-side as if they shared the same physical location. Eery Space promotes collaborative content creation and seamless mediation of communication channels based on virtual proximity. Results from user evaluation suggest that our approach is effective at enhancing mutual awareness between participants and sufficient to initiate proximal exchanges regardless of their geolocation, while promoting smooth interactions between local and remote people alike. These results happen even in the absence of visual avatars and other social devices such as eye contact, which are largely the focus of previous approaches.
△ Less
Submitted 1 June, 2024;
originally announced June 2024.
-
4Doodle: Two-handed Gestures for Immersive Sketching of Architectural Models
Authors:
Fernando Fonseca,
Maurício Sousa,
Daniel Mendes,
Alfredo Ferreira,
Joaquim Jorge
Abstract:
Three-dimensional immersive sketching for content creation and modeling has been studied for some time. However, research in this domain mainly focused on CAVE-like scenarios. These setups can be expensive and offer a narrow interaction space. Building more affordable setups using head-mounted displays is possible, allowing greater immersion and a larger space for user physical movements. This pap…
▽ More
Three-dimensional immersive sketching for content creation and modeling has been studied for some time. However, research in this domain mainly focused on CAVE-like scenarios. These setups can be expensive and offer a narrow interaction space. Building more affordable setups using head-mounted displays is possible, allowing greater immersion and a larger space for user physical movements. This paper presents a fully immersive environment using bi-manual gestures to sketch and create content freely in the virtual world. This approach can be applied to many scenarios, allowing people to express their ideas or review existing designs. To cope with known motor difficulties and inaccuracy of freehand 3D sketching, we explore proxy geometry and a laser-like metaphor to draw content directly from models and create content surfaces. Our current prototype offers 24 cubic meters for movement, limited by the room size. It features infinite virtual drawing space through pan and scale techniques and is larger than the typical 6-sided cave at a fraction of the cost. In a preliminary study conducted with architects and engineers, our system showed a clear promise as a tool for sketching and 3D content creation in virtual reality with a great emphasis on bi-manual gestures.
△ Less
Submitted 27 June, 2024; v1 submitted 29 May, 2024;
originally announced May 2024.
-
In-vivo imaging of the human thalamus: a comprehensive evaluation of structural magnetic resonance imaging approaches for thalamic nuclei differentiation at 7T
Authors:
Cristina Sainz Martinez,
José P. Marques,
Gabriele Bonanno,
Tom Hilbert,
Constantin Tuleasca,
Meritxell Bach Cuadra,
João Jorge
Abstract:
The thalamus is a subcortical structure of central importance to brain function, which is organized in smaller nuclei with specialized roles. Despite significant functional and clinical relevance, locating and distinguishing the different thalamic nuclei in vivo, non-invasively, has proved challenging with conventional imaging techniques, such as T$_{1}$ and T$_{2}$-weighted magnetic resonance ima…
▽ More
The thalamus is a subcortical structure of central importance to brain function, which is organized in smaller nuclei with specialized roles. Despite significant functional and clinical relevance, locating and distinguishing the different thalamic nuclei in vivo, non-invasively, has proved challenging with conventional imaging techniques, such as T$_{1}$ and T$_{2}$-weighted magnetic resonance imaging (MRI). This key limitation has prompted extensive research efforts, and several new candidate MRI sequences for thalamic imaging have been proposed, especially at 7T. However, studies to date have mainly been centered on individual techniques, and often focused on subsets of specific nuclei. It is now critical to evaluate which options are best for which nuclei, and which are globally the most informative. This work addresses these questions through a comprehensive evaluation of thalamic structural imaging techniques in humans at 7T, including several variants of T$_{1}$, T$_{2}$, T$_{2}$* and magnetic susceptibility-based contrasts. All images were obtained from the same participants, to allow direct comparisons without anatomical variability confounds. The different contrasts were qualitatively and quantitatively analyzed with dedicated approaches, referenced to well-established thalamic atlases. Overall, the analyses showed that quantitative susceptibility map** (QSM) and T$_{1}$-weighted MP2RAGE tuned to maximize gray-to-white matter contrast are currently the most valuable options. The two contrasts display unique, complementary features and, together, enable the distinction of the majority of known nuclei. Likewise, their combined information could provide a powerful input for automatic segmentation approaches. To our knowledge, this study represents the most comprehensive assessment of structural MRI contrasts for thalamic imaging to date.
△ Less
Submitted 9 April, 2024;
originally announced April 2024.
-
SleepVST: Sleep Staging from Near-Infrared Video Signals using Pre-Trained Transformers
Authors:
Jonathan F. Carter,
João Jorge,
Oliver Gibson,
Lionel Tarassenko
Abstract:
Advances in camera-based physiological monitoring have enabled the robust, non-contact measurement of respiration and the cardiac pulse, which are known to be indicative of the sleep stage. This has led to research into camera-based sleep monitoring as a promising alternative to "gold-standard" polysomnography, which is cumbersome, expensive to administer, and hence unsuitable for longer-term clin…
▽ More
Advances in camera-based physiological monitoring have enabled the robust, non-contact measurement of respiration and the cardiac pulse, which are known to be indicative of the sleep stage. This has led to research into camera-based sleep monitoring as a promising alternative to "gold-standard" polysomnography, which is cumbersome, expensive to administer, and hence unsuitable for longer-term clinical studies. In this paper, we introduce SleepVST, a transformer model which enables state-of-the-art performance in camera-based sleep stage classification (sleep staging). After pre-training on contact sensor data, SleepVST outperforms existing methods for cardio-respiratory sleep staging on the SHHS and MESA datasets, achieving total Cohen's kappa scores of 0.75 and 0.77 respectively. We then show that SleepVST can be successfully transferred to cardio-respiratory waveforms extracted from video, enabling fully contact-free sleep staging. Using a video dataset of 50 nights, we achieve a total accuracy of 78.8\% and a Cohen's $κ$ of 0.71 in four-class video-based sleep staging, setting a new state-of-the-art in the domain.
△ Less
Submitted 4 April, 2024;
originally announced April 2024.
-
Building An Efficient Grid On GPU
Authors:
Vasco Costa,
João M. Pereira,
Joaquim Jorge
Abstract:
Grid space partitioning is a technique to speed up queries to graphics databases. We present a parallel grid construction algorithm which can efficiently construct a structured grid on GPU hardware. Our approach is substantially faster than existing uniform grid construction algorithms, especially on non-homogeneous scenes. Indeed, it can populate a grid in real-time (at rates over 25 Hz), for arc…
▽ More
Grid space partitioning is a technique to speed up queries to graphics databases. We present a parallel grid construction algorithm which can efficiently construct a structured grid on GPU hardware. Our approach is substantially faster than existing uniform grid construction algorithms, especially on non-homogeneous scenes. Indeed, it can populate a grid in real-time (at rates over 25 Hz), for architectural scenes with 10 million triangles.
△ Less
Submitted 15 March, 2024;
originally announced March 2024.
-
Polygon Detection from a Set of Lines
Authors:
Alfredo Ferreira Jr.,
Manuel J. Fonseca,
Joaquim A. Jorge
Abstract:
Detecting polygons defined by a set of line segments in a plane is an important step in analyzing vector drawings. This paper presents an approach combining several algorithms to detect basic polygons from arbitrary line segments. The resulting algorithm runs in polynomial time and space, with complexities of $O\bigl((N + M)^4\bigr)$ and $O\bigl((N + M)^2\bigr)$, where $N$ is the number of line se…
▽ More
Detecting polygons defined by a set of line segments in a plane is an important step in analyzing vector drawings. This paper presents an approach combining several algorithms to detect basic polygons from arbitrary line segments. The resulting algorithm runs in polynomial time and space, with complexities of $O\bigl((N + M)^4\bigr)$ and $O\bigl((N + M)^2\bigr)$, where $N$ is the number of line segments and $M$ is the number of intersections between line segments. Our choice of algorithms was made to strike a good compromise between efficiency and ease of implementation. The result is a simple and efficient solution to detect polygons from lines.
△ Less
Submitted 26 December, 2023;
originally announced December 2023.
-
Deep Learning-Enabled Sleep Staging From Vital Signs and Activity Measured Using a Near-Infrared Video Camera
Authors:
Jonathan Carter,
João Jorge,
Bindia Venugopal,
Oliver Gibson,
Lionel Tarassenko
Abstract:
Conventional sleep monitoring is time-consuming, expensive and uncomfortable, requiring a large number of contact sensors to be attached to the patient. Video data is commonly recorded as part of a sleep laboratory assessment. If accurate sleep staging could be achieved solely from video, this would overcome many of the problems of traditional methods. In this work we use heart rate, breathing rat…
▽ More
Conventional sleep monitoring is time-consuming, expensive and uncomfortable, requiring a large number of contact sensors to be attached to the patient. Video data is commonly recorded as part of a sleep laboratory assessment. If accurate sleep staging could be achieved solely from video, this would overcome many of the problems of traditional methods. In this work we use heart rate, breathing rate and activity measures, all derived from a near-infrared video camera, to perform sleep stage classification. We use a deep transfer learning approach to overcome data scarcity, by using an existing contact-sensor dataset to learn effective representations from the heart and breathing rate time series. Using a dataset of 50 healthy volunteers, we achieve an accuracy of 73.4\% and a Cohen's kappa of 0.61 in four-class sleep stage classification, establishing a new state-of-the-art for video-based sleep staging.
△ Less
Submitted 6 June, 2023;
originally announced June 2023.
-
MDF-Net for abnormality detection by fusing X-rays with clinical data
Authors:
Chihcheng Hsieh,
Isabel Blanco Nobre,
Sandra Costa Sousa,
Chun Ouyang,
Margot Brereton,
Jacinto C. Nascimento,
Joaquim Jorge,
Catarina Moreira
Abstract:
This study investigates the effects of including patients' clinical information on the performance of deep learning (DL) classifiers for disease location in chest X-ray images. Although current classifiers achieve high performance using chest X-ray images alone, our interviews with radiologists indicate that clinical data is highly informative and essential for interpreting images and making prope…
▽ More
This study investigates the effects of including patients' clinical information on the performance of deep learning (DL) classifiers for disease location in chest X-ray images. Although current classifiers achieve high performance using chest X-ray images alone, our interviews with radiologists indicate that clinical data is highly informative and essential for interpreting images and making proper diagnoses.
In this work, we propose a novel architecture consisting of two fusion methods that enable the model to simultaneously process patients' clinical data (structured data) and chest X-rays (image data). Since these data modalities are in different dimensional spaces, we propose a spatial arrangement strategy, spatialization, to facilitate the multimodal learning process in a Mask R-CNN model. We performed an extensive experimental evaluation using MIMIC-Eye, a dataset comprising modalities: MIMIC-CXR (chest X-ray images), MIMIC IV-ED (patients' clinical data), and REFLACX (annotations of disease locations in chest X-rays).
Results show that incorporating patients' clinical data in a DL model together with the proposed fusion methods improves the disease localization in chest X-rays by 12\% in terms of Average Precision compared to a standard Mask R-CNN using only chest X-rays. Further ablation studies also emphasize the importance of multimodal DL architectures and the incorporation of patients' clinical data in disease localization. The architecture proposed in this work is publicly available to promote the scientific reproducibility of our study (https://github.com/ChihchengHsieh/multimodal-abnormalities-detection)
△ Less
Submitted 27 December, 2023; v1 submitted 26 February, 2023;
originally announced February 2023.
-
MAGIC: Manipulating Avatars and Gestures to Improve Remote Collaboration
Authors:
Catarina G. Fidalgo,
Maurício Sousa,
Daniel Mendes,
Rafael Kuffner dos Anjos,
Daniel Medeiros,
Karan Singh,
Joaquim Jorge
Abstract:
Remote collaborative work has become pervasive in many settings, from engineering to medical professions. Users are immersed in virtual environments and communicate through life-sized avatars that enable face-to-face collaboration. Within this context, users often collaboratively view and interact with virtual 3D models, for example, to assist in designing new devices such as customized prosthetic…
▽ More
Remote collaborative work has become pervasive in many settings, from engineering to medical professions. Users are immersed in virtual environments and communicate through life-sized avatars that enable face-to-face collaboration. Within this context, users often collaboratively view and interact with virtual 3D models, for example, to assist in designing new devices such as customized prosthetics, vehicles, or buildings. However, discussing shared 3D content face-to-face has various challenges, such as ambiguities, occlusions, and different viewpoints that all decrease mutual awareness, leading to decreased task performance and increased errors. To address this challenge, we introduce MAGIC, a novel approach for understanding pointing gestures in a face-to-face shared 3D space, improving mutual understanding and awareness. Our approach distorts the remote userś gestures to correctly reflect them in the local userś reference space when face-to-face. We introduce a novel metric called pointing agreement to measure what two users perceive in common when using pointing gestures in a shared 3D space. Results from a user study suggest that MAGIC significantly improves pointing agreement in face-to-face collaboration settings, improving co-presence and awareness of interactions performed in the shared space. We believe that MAGIC improves remote collaboration by enabling simpler communication mechanisms and better mutual awareness.
△ Less
Submitted 15 February, 2023;
originally announced February 2023.
-
SURVIVRS: Surround Video-Based Virtual Reality for Surgery Guidance
Authors:
Amani Taweel,
Joaquim Jorge,
Anderson Maciel,
João Ricardo Nickenig Vissoci,
Regis Kopper
Abstract:
There is a strong demand for virtual reality (VR) to bring quality healthcare to underserved populations. This paper addresses this need with the design and prototype of SURVIVRS: Surround Video-Based Virtual Reality for Surgery Guidance. SURVIVRS allows a remote specialist to guide a local surgery team through a virtual reality (VR) telepresence interface. SURVIVRS is motivated by a need for medi…
▽ More
There is a strong demand for virtual reality (VR) to bring quality healthcare to underserved populations. This paper addresses this need with the design and prototype of SURVIVRS: Surround Video-Based Virtual Reality for Surgery Guidance. SURVIVRS allows a remote specialist to guide a local surgery team through a virtual reality (VR) telepresence interface. SURVIVRS is motivated by a need for medical expertise in remote and hard-to-reach areas, such as low-to-middle-income countries (LMICs). The remote surgeon interface allows the live observation of a procedure and combines 3D user interface annotation and communication tools on streams of the surgical site and the patient vitals monitor. SURVIVRS also supports debriefing and educational experiences by offering the ability for users to watch recorded surgeries from the point of view of the remote expert. The main contributions of this work are: the feasibility demonstration of the SURVIVRS system through a rigorous 3D user interface design process; the implementation of a prototype application that realizes the proposed design; and a usability evaluation of SURVIVRS showing that the tool was highly favored by users from the general population. The paper discusses the next steps in this line of research aimed at more equitable and diverse access to healthcare.
△ Less
Submitted 8 February, 2023;
originally announced February 2023.
-
Exploring Affordances for AR in Laparoscopy
Authors:
Matheus Negrão,
Joaquim Jorge,
João Vissoci,
Regis Kopper,
Anderson Maciel
Abstract:
This paper explores the possibilities of designing AR interfaces to be used during laparoscopy surgery. It suggests that the laparoscopic video be displayed on AR headsets and that surgeons can consult preoperative image data on that display. Interaction with these elements is necessary, and no patterns exist to design them. Thus the paper proposes a head-gaze and clicker approach that is effectiv…
▽ More
This paper explores the possibilities of designing AR interfaces to be used during laparoscopy surgery. It suggests that the laparoscopic video be displayed on AR headsets and that surgeons can consult preoperative image data on that display. Interaction with these elements is necessary, and no patterns exist to design them. Thus the paper proposes a head-gaze and clicker approach that is effective and minimalist. Finally, a prototype is presented, and an evaluation protocol is briefly discussed.
△ Less
Submitted 8 February, 2023;
originally announced February 2023.
-
Development of an Immersive Virtual Colonoscopy Viewer for Colon Growths Diagnosis
Authors:
João Serras,
Anderson Maciel,
Soraia Paulo,
Andrew Duchowski,
Regis Kopper,
Catarina Moreira,
Joaquim Jorge
Abstract:
Desktop-based virtual colonoscopy has been proven to be an asset in the identification of colon anomalies. The process is accurate, although time-consuming. The use of immersive interfaces for virtual colonoscopy is incipient and not yet understood. In this work, we present a new design exploring elements of the VR paradigm to make the immersive analysis more efficient while still effective. We al…
▽ More
Desktop-based virtual colonoscopy has been proven to be an asset in the identification of colon anomalies. The process is accurate, although time-consuming. The use of immersive interfaces for virtual colonoscopy is incipient and not yet understood. In this work, we present a new design exploring elements of the VR paradigm to make the immersive analysis more efficient while still effective. We also plan the conduction of experiments with experts to assess the multi-factor influences of coverage, duration, and diagnostic accuracy.
△ Less
Submitted 4 May, 2023; v1 submitted 6 February, 2023;
originally announced February 2023.
-
Integrating Eye-Gaze Data into CXR DL Approaches: A Preliminary study
Authors:
André Luís,
Chihcheng Hsieh,
Isabel Blanco Nobre,
Sandra Costa Sousa,
Anderson Maciel,
Catarina Moreira,
Joaquim Jorge
Abstract:
This paper proposes a novel multimodal DL architecture incorporating medical images and eye-tracking data for abnormality detection in chest x-rays. Our results show that applying eye gaze data directly into DL architectures does not show superior predictive performance in abnormality detection chest X-rays. These results support other works in the literature and suggest that human-generated data,…
▽ More
This paper proposes a novel multimodal DL architecture incorporating medical images and eye-tracking data for abnormality detection in chest x-rays. Our results show that applying eye gaze data directly into DL architectures does not show superior predictive performance in abnormality detection chest X-rays. These results support other works in the literature and suggest that human-generated data, such as eye gaze, needs a more thorough investigation before being applied to DL architectures.
△ Less
Submitted 6 February, 2023;
originally announced February 2023.
-
Virtual Reality for medical education and training of Diabetic Foot
Authors:
Gabriel Riva,
Wellington Dores,
Artur Damasio,
Daniel Guimarães Cacione,
Joaquim Jorge,
Ezequiel Zorzal
Abstract:
Diabetic Foot is one of the most common complications of Diabetes Mellitus and the leading non-traumatic cause of lower-limb amputations worldwide. These complications are preventable with early diagnosis and timely care. However, even in the context of expanding primary health care, this problem continues to increase, which suggests a gap in the training of primary health care professionals regar…
▽ More
Diabetic Foot is one of the most common complications of Diabetes Mellitus and the leading non-traumatic cause of lower-limb amputations worldwide. These complications are preventable with early diagnosis and timely care. However, even in the context of expanding primary health care, this problem continues to increase, which suggests a gap in the training of primary health care professionals regarding the diagnosis and treatment of Diabetic Foot. This project proposes the development of a Virtual Reality simulator for training students and professionals in primary health care, aiming to collaborate in filling this gap. The application features gamification elements to increase user engagement. The context of medical care in primary care will be simulated with various clinical cases, including several virtual patients with different stages related to the Diabetic Foot, seeking to provide credible and distinct experiences to potential users. In addition, we aim to verify usability, effectiveness, and any side effects (cybersickness). Finally, we plan to conduct field studies with qualified students and professionals to identify the main benefits and obstacles to applying the technology.
△ Less
Submitted 6 February, 2023;
originally announced February 2023.
-
BigBrain-MR: a new digital phantom with anatomically-realistic magnetic resonance properties at 100-μm resolution for magnetic resonance methods development
Authors:
Cristina Sainz Martinez,
Meritxell Bach Cuadra,
João Jorge
Abstract:
The benefits, opportunities and growing availability of ultra-high field magnetic resonance imaging (MRI) for humans have prompted an expansion in research and development efforts towards increasingly more advanced high-resolution imaging techniques. To maximize their effectiveness, these efforts need to be supported by powerful computational simulation platforms that can adequately reproduce the…
▽ More
The benefits, opportunities and growing availability of ultra-high field magnetic resonance imaging (MRI) for humans have prompted an expansion in research and development efforts towards increasingly more advanced high-resolution imaging techniques. To maximize their effectiveness, these efforts need to be supported by powerful computational simulation platforms that can adequately reproduce the biophysical characteristics of MRI, with high spatial resolution. In this work, we have sought to address this need by develo** a novel digital phantom with realistic anatomical detail up to 100-um resolution, including multiple MRI properties that affect image generation. This phantom, termed BigBrain-MR, was generated from the publicly available BigBrain histological dataset and lower-resolution in-vivo 7T-MRI data, using a newly-developed image processing framework that allows map** the general properties of the latter into the fine anatomical scale of the former. Overall, the map** framework was found to be effective and robust, yielding a diverse range of realistic "in-vivo-like" MRI contrasts and maps at 100-um resolution. BigBrain-MR was then tested in three different imaging applications (motion effects and interpolation, super-resolution imaging, and parallel imaging reconstruction) to investigate its properties, value and validity as a simulation platform. The results consistently showed that BigBrain-MR can closely approximate the behavior of real in-vivo data, more realistically and with more extensive features than a more classic option such as the Shepp-Logan phantom. This novel phantom is therefore deemed a favorable choice to support methodological development in brain MRI, and has been made freely available to the community.
△ Less
Submitted 15 December, 2022;
originally announced December 2022.
-
Benchmarking Instance-Centric Counterfactual Algorithms for XAI: From White Box to Black Box
Authors:
Catarina Moreira,
Yu-Liang Chou,
Chihcheng Hsieh,
Chun Ouyang,
Joaquim Jorge,
João Madeiras Pereira
Abstract:
This study investigates the impact of machine learning models on the generation of counterfactual explanations by conducting a benchmark evaluation over three different types of models: a decision tree (fully transparent, interpretable, white-box model), a random forest (semi-interpretable, grey-box model), and a neural network (fully opaque, black-box model). We tested the counterfactual generati…
▽ More
This study investigates the impact of machine learning models on the generation of counterfactual explanations by conducting a benchmark evaluation over three different types of models: a decision tree (fully transparent, interpretable, white-box model), a random forest (semi-interpretable, grey-box model), and a neural network (fully opaque, black-box model). We tested the counterfactual generation process using four algorithms (DiCE, WatcherCF, prototype, and GrowingSpheresCF) in the literature in 25 different datasets. Our findings indicate that: (1) Different machine learning models have little impact on the generation of counterfactual explanations; (2) Counterfactual algorithms based uniquely on proximity loss functions are not actionable and will not provide meaningful explanations; (3) One cannot have meaningful evaluation results without guaranteeing plausibility in the counterfactual generation. Algorithms that do not consider plausibility in their internal mechanisms will lead to biased and unreliable conclusions if evaluated with the current state-of-the-art metrics; (4) A counterfactual inspection analysis is strongly recommended to ensure a robust examination of counterfactual explanations and the potential identification of biases.
△ Less
Submitted 11 June, 2024; v1 submitted 4 March, 2022;
originally announced March 2022.
-
Anatomy Studio II: A Cross-Reality Application for Teaching Anatomy
Authors:
Joaquim Jorge,
Pedro Belchior,
Abel Gomes,
Maurício Sousa,
João Pereira,
Jean-François Uhl
Abstract:
Virtual Reality has become an important educational tool, due to the pandemic and increasing globalization of education. This paper presents a framework for teaching Virtual Anatomy at the university level. Virtual classes have become a staple of today's curricula because of the isolation and quarantine requirements and the increased international collaboration. Our work builds on the Visible Huma…
▽ More
Virtual Reality has become an important educational tool, due to the pandemic and increasing globalization of education. This paper presents a framework for teaching Virtual Anatomy at the university level. Virtual classes have become a staple of today's curricula because of the isolation and quarantine requirements and the increased international collaboration. Our work builds on the Visible Human Projects for Virtual Dissection material and provides a medium for groups of students to do collaborative anatomical dissections in real-time using sketching and 3D visualizations and audio coupled with interactive 2D tablets for precise drawing. We describe the system architecture, compare requirements with those of previous development, and discuss the preliminary results. Discussions with Anatomists show that this is an effective tool. We introduce avenues for further research and discuss collaboration challenges posed by this context.
△ Less
Submitted 4 March, 2022;
originally announced March 2022.
-
Design requirements to improve laparoscopy via XR
Authors:
Ezequiel R. Zorzal,
Maurício Sousa,
Pedro Belchior,
João Madeiras Pereira,
Nuno Figueiredo,
Joaquim Jorge
Abstract:
Laparoscopic surgery has the advantage of avoiding large open incisions and thereby decreasing blood loss, pain, and discomfort to patients. However, on the other side, it is hampered by restricted workspace, ambiguous communication, and surgeon fatigue caused by non-ergonomic head positioning. We aimed to identify critical problems and suggest design requirements and solutions. We used user and t…
▽ More
Laparoscopic surgery has the advantage of avoiding large open incisions and thereby decreasing blood loss, pain, and discomfort to patients. However, on the other side, it is hampered by restricted workspace, ambiguous communication, and surgeon fatigue caused by non-ergonomic head positioning. We aimed to identify critical problems and suggest design requirements and solutions. We used user and task analysis methods to learn about practices performed in an operating room by observing surgeons in their working environment to understand how they performed tasks and achieved their intended goals. Drawing on observations and analysis from recorded laparoscopic surgeries, we have identified several constraints and design requirements to propose potential solutions to address the issues. Surgeons operate in a dimly lit environment, surrounded by monitors, and communicate through verbal commands and pointing gestures. Therefore, performing user and task analysis allowed us to better understand the existing problems in laparoscopy while identifying several communication constraints and design requirements, which a solution has to follow to address those problems. Our contributions include identifying design requirements for laparoscopy surgery through a user and task analysis. These requirements propose design solutions towards improved surgeons' comfort and make the surgical procedure less laborious.
△ Less
Submitted 3 March, 2022;
originally announced March 2022.
-
Improving X-ray Diagnostics through Eye-Tracking and XR
Authors:
Catarina Moreira,
Isabel Blanco Nobre,
Sandra Costa Sousa,
João Madeiras Pereira,
Joaquim Jorge
Abstract:
There is a growing need to assist radiologists in performing X-ray readings and diagnoses fast, comfortably, and effectively. As radiologists strive to maximize productivity, it is essential to consider the impact of reading rooms in interpreting complex examinations and ensure that higher volume and reporting speeds do not compromise patient outcomes. Virtual Reality (VR) is a disruptive technolo…
▽ More
There is a growing need to assist radiologists in performing X-ray readings and diagnoses fast, comfortably, and effectively. As radiologists strive to maximize productivity, it is essential to consider the impact of reading rooms in interpreting complex examinations and ensure that higher volume and reporting speeds do not compromise patient outcomes. Virtual Reality (VR) is a disruptive technology for clinical practice in assessing X-ray images. We argue that conjugating eye-tracking with VR devices and Machine Learning may overcome obstacles posed by inadequate ergonomic postures and poor room conditions that often cause erroneous diagnostics when professionals examine digital images.
△ Less
Submitted 3 March, 2022;
originally announced March 2022.
-
Controlling camera movement in VR colonography
Authors:
Soraia F Paulo,
Daniel Medeiros,
Daniel Lopes,
Joaquim Jorge
Abstract:
Immersive Colonography allows medical professionals to navigate inside the intricate tubular geometries of subject-specific 3D colon images using Virtual Reality displays. Typically, camera travel is performed via Fly-Through or Fly-Over techniques that enable semi-automatic traveling through a constrained, well-defined path at user-controlled speeds. However, Fly-Through is known to limit the vis…
▽ More
Immersive Colonography allows medical professionals to navigate inside the intricate tubular geometries of subject-specific 3D colon images using Virtual Reality displays. Typically, camera travel is performed via Fly-Through or Fly-Over techniques that enable semi-automatic traveling through a constrained, well-defined path at user-controlled speeds. However, Fly-Through is known to limit the visibility of lesions located behind or inside haustral folds. At the same time, Fly-Over requires splitting the entire colon visualization into two specific halves. In this paper, we study the effect of immersive Fly-Through and Fly-Over techniques on lesion detection and introduce a camera travel technique that maintains a fixed camera orientation throughout the entire medial axis path. While these techniques have been studied in non-VR desktop environments, their performance is not well understood in VR setups. We performed a comparative study to ascertain which camera travel technique is more appropriate for constrained path navigation in Immersive Colonography and validated our conclusions with two radiologists. To this end, we asked 18 participants to navigate inside a 3D colon to find specific marks. Our results suggest that the Fly-Over technique may lead to enhanced lesion detection at the cost of higher task completion times. Nevertheless, the Fly-Through method may offer a more balanced trade-off between speed and effectiveness, whereas the fixed camera orientation technique provided seemingly inferior performance results. Our study further provides design guidelines and informs future work.
△ Less
Submitted 8 January, 2022;
originally announced January 2022.
-
On the assumptions underlying KS-like contradictions
Authors:
J. Acacio de Barros,
Juan Pablo Jorge,
Federico Holik
Abstract:
The Kochen-Specker theorem is one of the fundamental no-go theorems in quantum theory. It has far-reaching consequences for all attempts trying to give an interpretation of the quantum formalism. In this work, we examine the hypotheses that, at the ontological level, lead to the KochenSpecker contradiction. We emphasize the role of the assumptions about identity and distinguishability of quantum o…
▽ More
The Kochen-Specker theorem is one of the fundamental no-go theorems in quantum theory. It has far-reaching consequences for all attempts trying to give an interpretation of the quantum formalism. In this work, we examine the hypotheses that, at the ontological level, lead to the KochenSpecker contradiction. We emphasize the role of the assumptions about identity and distinguishability of quantum objects in the argument.
△ Less
Submitted 11 March, 2021;
originally announced March 2021.
-
Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applications
Authors:
Yu-Liang Chou,
Catarina Moreira,
Peter Bruza,
Chun Ouyang,
Joaquim Jorge
Abstract:
There has been a growing interest in model-agnostic methods that can make deep learning models more transparent and explainable to a user. Some researchers recently argued that for a machine to achieve a certain degree of human-level explainability, this machine needs to provide human causally understandable explanations, also known as causability. A specific class of algorithms that have the pote…
▽ More
There has been a growing interest in model-agnostic methods that can make deep learning models more transparent and explainable to a user. Some researchers recently argued that for a machine to achieve a certain degree of human-level explainability, this machine needs to provide human causally understandable explanations, also known as causability. A specific class of algorithms that have the potential to provide causability are counterfactuals. This paper presents an in-depth systematic review of the diverse existing body of literature on counterfactuals and causability for explainable artificial intelligence. We performed an LDA topic modelling analysis under a PRISMA framework to find the most relevant literature articles. This analysis resulted in a novel taxonomy that considers the grounding theories of the surveyed algorithms, together with their underlying properties and applications in real-world data. This research suggests that current model-agnostic counterfactual algorithms for explainable AI are not grounded on a causal theoretical formalism and, consequently, cannot promote causability to a human decision-maker. Our findings suggest that the explanations derived from major algorithms in the literature provide spurious correlations rather than cause/effects relationships, leading to sub-optimal, erroneous or even biased explanations. This paper also advances the literature with new directions and challenges on promoting causability in model-agnostic approaches for explainable artificial intelligence.
△ Less
Submitted 8 June, 2021; v1 submitted 6 March, 2021;
originally announced March 2021.
-
Towards augmented reality for corporate training
Authors:
Bruno R. Martins,
Joaquim A. Jorge,
Ezequiel R. Zorzal
Abstract:
Corporate training relates to employees acquiring essential skills to operate equipment or effectively performing required tasks both competently and safely. Unlike formal education, training can be incorporated into the task workflow and performed during working hours. Increasingly, organizations adopt different technologies to develop both individual skills and improve their organization. Studie…
▽ More
Corporate training relates to employees acquiring essential skills to operate equipment or effectively performing required tasks both competently and safely. Unlike formal education, training can be incorporated into the task workflow and performed during working hours. Increasingly, organizations adopt different technologies to develop both individual skills and improve their organization. Studies indicate that Augmented Reality (AR) is quickly becoming an effective technology for training programs. This systematic literature review (SLR) aims to screen works published on AR for corporate training. We describe AR training applications, discuss current challenges, literature gaps, opportunities, and tendencies of corporate AR solutions. We structured a protocol to define keywords, the semantics of research, and databases used as sources of this SLR. From a primary analysis, we considered 1952 articles in the review for qualitative synthesis. We selected 60 among the selected articles for this study. The survey shows a large number of 41.7% of applications focused on automotive and medical training. Additionally, 20% of selected publications use a camera-display with a tablet device, while 40% refer to head-mounted-displays, and many surveyed approaches (45%) adopt marker-based tracking. Results indicate that publications on AR for corporate training increased significantly in recent years. AR has been used in many areas, exhibiting high quality and provides viable approaches to On-The-Job training. Finally, we discuss future research issues related to increasing relevance regarding AR for corporate training.
△ Less
Submitted 18 February, 2021;
originally announced February 2021.
-
Indistinguishability right from the start in standard quantum mechanics
Authors:
F. Holik,
J. P. Jorge,
C. Massri
Abstract:
We discuss a reconstruction of standard quantum mechanics assuming indistinguishability right from the start, by appealing to quasi-set theory. After recalling the fundamental aspects of the construction and introducing some improvements in the original formulation, we extract some conclusions for the interpretation of quantum theory.
We discuss a reconstruction of standard quantum mechanics assuming indistinguishability right from the start, by appealing to quasi-set theory. After recalling the fundamental aspects of the construction and introducing some improvements in the original formulation, we extract some conclusions for the interpretation of quantum theory.
△ Less
Submitted 21 November, 2020;
originally announced November 2020.
-
Camera Travel for Immersive Colonography
Authors:
Soraia F. Paulo,
Daniel Medeiros,
Pedro Borges,
Joaquim Jorge,
Daniel Simões Lopes
Abstract:
Immersive Colonography allows medical professionals to navigate inside the intricate tubular geometries of subject-specific 3D colon images using Virtual Reality displays. Typically, camera travel is performed via Fly-Through or Fly-Over techniques that enable semi-automatic traveling through a constrained, well-defined path at user controlled speeds. However, Fly-Through is known to limit the vis…
▽ More
Immersive Colonography allows medical professionals to navigate inside the intricate tubular geometries of subject-specific 3D colon images using Virtual Reality displays. Typically, camera travel is performed via Fly-Through or Fly-Over techniques that enable semi-automatic traveling through a constrained, well-defined path at user controlled speeds. However, Fly-Through is known to limit the visibility of lesions located behind or inside haustral folds, while Fly-Over requires splitting the entire colon visualization into two specific halves. In this paper, we study the effect of immersive Fly-Through and Fly-Over techniques on lesion detection, and introduce a camera travel technique that maintains a fixed camera orientation throughout the entire medial axis path. While these techniques have been studied in non-VR desktop environments, their performance is yet not well understood in VR setups. We performed a comparative study to ascertain which camera travel technique is more appropriate for constrained path navigation in Immersive Colonography. To this end, we asked 18 participants to navigate inside a 3D colon to find specific marks. Our results suggest that the Fly-Over technique may lead to enhanced lesion detection at the cost of higher task completion times, while the Fly-Through method may offer a more balanced trade-off between both speed and effectiveness, whereas the fixed camera orientation technique provided seemingly inferior performance results. Our study further provides design guidelines and informs future work.
△ Less
Submitted 15 October, 2020;
originally announced October 2020.
-
Technical Design Report for the PANDA Endcap Disc DIRC
Authors:
Panda Collaboration,
F. Davi,
W. Erni,
B. Krusche,
M. Steinacher,
N. Walford,
H. Liu,
Z. Liu,
B. Liu,
X. Shen,
C. Wang,
J. Zhao,
M. Albrecht,
T. Erlen,
F. Feldbauer,
M. Fink,
V. Freudenreich,
M. Fritsch,
F. H. Heinsius,
T. Held,
T. Holtmann,
I. Keshk,
H. Koch,
B. Kopf,
M. Kuhlmann
, et al. (441 additional authors not shown)
Abstract:
PANDA (anti-Proton ANnihiliation at DArmstadt) is planned to be one of the four main experiments at the future international accelerator complex FAIR (Facility for Antiproton and Ion Research) in Darmstadt, Germany. It is going to address fundamental questions of hadron physics and quantum chromodynamics using cooled antiproton beams with a high intensity and and momenta between 1.5 and 15 GeV/c.…
▽ More
PANDA (anti-Proton ANnihiliation at DArmstadt) is planned to be one of the four main experiments at the future international accelerator complex FAIR (Facility for Antiproton and Ion Research) in Darmstadt, Germany. It is going to address fundamental questions of hadron physics and quantum chromodynamics using cooled antiproton beams with a high intensity and and momenta between 1.5 and 15 GeV/c. PANDA is designed to reach a maximum luminosity of 2x10^32 cm^2 s. Most of the physics programs require an excellent particle identification (PID). The PID of hadronic states at the forward endcap of the target spectrometer will be done by a fast and compact Cherenkov detector that uses the detection of internally reflected Cherenkov light (DIRC) principle. It is designed to cover the polar angle range from 5° to 22° and to provide a separation power for the separation of charged pions and kaons up to 3 standard deviations (s.d.) for particle momenta up to 4 GeV/c in order to cover the important particle phase space. This document describes the technical design and the expected performance of the novel PANDA Disc DIRC detector that has not been used in any other high energy physics experiment (HEP) before. The performance has been studied with Monte-Carlo simulations and various beam tests at DESY and CERN. The final design meets all PANDA requirements and guarantees suffcient safety margins.
△ Less
Submitted 29 December, 2019;
originally announced December 2019.
-
Safe Walking In VR using Augmented Virtuality
Authors:
Maurício Sousa,
Daniel Mendes,
Joaquim Jorge
Abstract:
New technologies allow ordinary people to access Virtual Reality at affordable prices in their homes. One of the most important tasks when interacting with immersive Virtual Reality is to navigate the virtual environments (VEs). Arguably, the best methods to accomplish this use of direct control interfaces. Among those, natural walking (NW) makes for enjoyable user experience. However, common tech…
▽ More
New technologies allow ordinary people to access Virtual Reality at affordable prices in their homes. One of the most important tasks when interacting with immersive Virtual Reality is to navigate the virtual environments (VEs). Arguably, the best methods to accomplish this use of direct control interfaces. Among those, natural walking (NW) makes for enjoyable user experience. However, common techniques to support direct control interfaces in VEs feature constraints that make it difficult to use those methods in cramped home environments. Indeed, NW requires unobstructed and open space. To approach this problem, we propose a new virtual locomotion technique, Combined Walking in Place (CWIP). CWIP allows people to take advantage of the available physical space and empowers them to use NW to navigate in the virtual world. For longer distances, we adopt Walking in Place (WIP) to enable them to move in the virtual world beyond the confines of a cramped real room. However, roaming in immersive alternate reality, while moving in the confines of a cluttered environment can lead people to stumble and fall. To approach these problems, we developed Augmented Virtual Reality (AVR), to inform users about real-world hazards, such as chairs, drawers, walls via proxies and signs placed in the virtual world. We propose thus CWIP-AVR as a way to safely explore VR in the cramped confines of your own home. To our knowledge, this is the first approach to combined different locomotion modalities in a safe manner. We evaluated it in a user study with 20 participants to validate their ability to navigate a virtual world while walking in a confined and cluttered real space. Our results show that CWIP-AVR allows people to navigate VR safely, switching between locomotion modes flexibly while maintaining a good immersion.
△ Less
Submitted 29 November, 2019;
originally announced November 2019.
-
Europarl-ST: A Multilingual Corpus For Speech Translation Of Parliamentary Debates
Authors:
Javier Iranzo-Sánchez,
Joan Albert Silvestre-Cerdà,
Javier Jorge,
Nahuel Roselló,
Adrià Giménez,
Albert Sanchis,
Jorge Civera,
Alfons Juan
Abstract:
Current research into spoken language translation (SLT),or speech-to-text translation, is often hampered by the lack of specific data resources for this task, as currently available SLT datasets are restricted to a limited set of language pairs. In this paper we present Europarl-ST, a novel multilingual SLT corpus containing paired audio-text samples for SLT from and into 6 European languages, for…
▽ More
Current research into spoken language translation (SLT),or speech-to-text translation, is often hampered by the lack of specific data resources for this task, as currently available SLT datasets are restricted to a limited set of language pairs. In this paper we present Europarl-ST, a novel multilingual SLT corpus containing paired audio-text samples for SLT from and into 6 European languages, for a total of 30 different translation directions. This corpus has been compiled using the debates held in the European Parliament in the period between 2008 and 2012. This paper describes the corpus creation process and presents a series of automatic speech recognition, machine translation and spoken language translation experiments that highlight the potential of this new resource. The corpus is released under a Creative Commons license and is freely accessible and downloadable.
△ Less
Submitted 12 February, 2020; v1 submitted 8 November, 2019;
originally announced November 2019.
-
Negative Space: Workspace Awareness in 3D Face-to-Face Remote Collaboration
Authors:
Maurício Sousa,
Daniel Mendes,
Rafael Kuffner dos Anjos,
Daniel Simões Lopes,
Joaquim Jorge
Abstract:
Face-to-face telepresence promotes the sense of "being there" and can improve collaboration by allowing immediate understanding of remote people's nonverbal cues. Several approaches successfully explored interactions with 2D content using a see-through whiteboard metaphor. However, with 3D content, there is a decrease in awareness due to ambiguities originated by participants' opposing points-of-v…
▽ More
Face-to-face telepresence promotes the sense of "being there" and can improve collaboration by allowing immediate understanding of remote people's nonverbal cues. Several approaches successfully explored interactions with 2D content using a see-through whiteboard metaphor. However, with 3D content, there is a decrease in awareness due to ambiguities originated by participants' opposing points-of-view. In this paper, we investigate how people and content should be presented for discussing 3D renderings within face-to-face collaborative sessions. To this end, we performed a user evaluation to compare four different conditions, in which we varied reflections of both workspace and remote people representation. Results suggest potentially more benefits to remote collaboration from workspace consistency rather than people's representation fidelity. We contribute a novel design space, the Negative Space, for remote face-to-face collaboration focusing on 3D content.
△ Less
Submitted 8 October, 2019;
originally announced October 2019.
-
Non-deterministic semantics for quantum states
Authors:
Juan Pablo Jorge,
Federico Holik
Abstract:
In this work we discuss the failure of the principle of truth functionality in the quantum formalism. By exploiting this failure, we import the formalism of N-matrix theory and non-deterministic semantics to the foundations of quantum mechanics. This is done by describing quantum states as particular valuations associated to infinite non-deterministic truth tables. This allows us to introduce a na…
▽ More
In this work we discuss the failure of the principle of truth functionality in the quantum formalism. By exploiting this failure, we import the formalism of N-matrix theory and non-deterministic semantics to the foundations of quantum mechanics. This is done by describing quantum states as particular valuations associated to infinite non-deterministic truth tables. This allows us to introduce a natural interpretation of quantum states in terms of a non-deterministic semantics. We also provide a similar construction for arbitrary probabilistic theories based in orthomodular lattices, allowing to study post-quantum models using logical techniques.
△ Less
Submitted 28 January, 2020; v1 submitted 8 June, 2019;
originally announced June 2019.
-
Understanding Individual Differences: Towards Effective Mobile Interface Design and Adaptation for the Blind
Authors:
Tiago Guerreiro,
Hugo Nicolau,
João Oliveira,
Joaquim Jorge,
Daniel Gonçalves
Abstract:
No two people are alike. We usually ignore this diversity as we have the capability to adapt and, without noticing, become experts in interfaces that were probably misadjusted to begin with. This adaptation is not always at the user's reach. One neglected group is the blind. Spatial ability, memory, and tactile sensitivity are some characteristics that diverge between users. Regardless, all are pr…
▽ More
No two people are alike. We usually ignore this diversity as we have the capability to adapt and, without noticing, become experts in interfaces that were probably misadjusted to begin with. This adaptation is not always at the user's reach. One neglected group is the blind. Spatial ability, memory, and tactile sensitivity are some characteristics that diverge between users. Regardless, all are presented with the same methods ignoring their capabilities and needs. Interaction with mobile devices is highly visually demanding which widens the gap between blind people. Our research goal is to identify the individual attributes that influence mobile interaction, considering the blind, and match them with mobile interaction modalities in a comprehensive and extensible design space. We aim to provide knowledge both for device design, device prescription and interface adaptation.
△ Less
Submitted 5 February, 2014;
originally announced February 2014.
