Showing 1–11 of 11 results for author: Keshavarzi, M

Mutual Scene Synthesis for Mixed Reality Telepresence

Authors: Mohammad Keshavarzi, Michael Zollhoefer, Allen Y. Yang, Patrick Peluse, Luisa Caldas

Abstract: Remote telepresence via next-generation mixed reality platforms can provide higher levels of immersion for computer-mediated communications, allowing participants to engage in a wide spectrum of activities, previously not possible in 2D screen-based communication methods. However, as mixed reality experiences are limited to the local physical surrounding of each user, finding a common virtual grou… ▽ More Remote telepresence via next-generation mixed reality platforms can provide higher levels of immersion for computer-mediated communications, allowing participants to engage in a wide spectrum of activities, previously not possible in 2D screen-based communication methods. However, as mixed reality experiences are limited to the local physical surrounding of each user, finding a common virtual ground where users can freely move and interact with each other is challenging. In this paper, we propose a novel mutual scene synthesis method that takes the participants' spaces as input, and generates a virtual synthetic scene that corresponds to the functional features of all participants' local spaces. Our method combines a mutual function optimization module with a deep-learning conditional scene augmentation process to generate a scene mutually and physically accessible to all participants of a mixed reality telepresence scenario. The synthesized scene can hold mutual walkable, sittable and workable functions, all corresponding to physical objects in the users' real environments. We perform experiments using the MatterPort3D dataset and conduct comparative user studies to evaluate the effectiveness of our system. Our results show that our proposed approach can be a promising research direction for facilitating contextualized telepresence systems for next-generation spatial computing platforms. △ Less

Submitted 31 March, 2022; originally announced April 2022.

Comments: 11 pages

Contextual Scene Augmentation and Synthesis via GSACNet

Authors: Mohammad Keshavarzi, Flaviano Christian Reyes, Ritika Shrivastava, Oladapo Afolabi, Luisa Caldas, Allen Y. Yang

Abstract: Indoor scene augmentation has become an emerging topic in the field of computer vision and graphics with applications in augmented and virtual reality. However, current state-of-the-art systems using deep neural networks require large datasets for training. In this paper we introduce GSACNet, a contextual scene augmentation system that can be trained with limited scene priors. GSACNet utilizes a n… ▽ More Indoor scene augmentation has become an emerging topic in the field of computer vision and graphics with applications in augmented and virtual reality. However, current state-of-the-art systems using deep neural networks require large datasets for training. In this paper we introduce GSACNet, a contextual scene augmentation system that can be trained with limited scene priors. GSACNet utilizes a novel parametric data augmentation method combined with a Graph Attention and Siamese network architecture followed by an Autoencoder network to facilitate training with small datasets. We show the effectiveness of our proposed system by conducting ablation and comparative studies with alternative systems on the Matterport3D dataset. Our results indicate that our scene augmentation outperforms prior art in scene synthesis with limited scene priors available. △ Less

Submitted 29 March, 2021; originally announced March 2021.

Comments: arXiv admin note: text overlap with arXiv:2009.12395 by other authors

GenFloor: Interactive Generative Space Layout System via Encoded Tree Graphs

Authors: Mohammad Keshavarzi, Mohammad Rahmani-Asl

Abstract: Automated floorplanning or space layout planning has been a long-standing NP-hard problem in the field of computer-aided design, with applications in integrated circuits, architecture, urbanism, and operational research. In this paper, we introduce GenFloor, an interactive design system that takes geometrical, topological, and performance goals and constraints as input and provides optimized spati… ▽ More Automated floorplanning or space layout planning has been a long-standing NP-hard problem in the field of computer-aided design, with applications in integrated circuits, architecture, urbanism, and operational research. In this paper, we introduce GenFloor, an interactive design system that takes geometrical, topological, and performance goals and constraints as input and provides optimized spatial design solutions as output. As part of our work, we propose three novel permutation methods for existing space layout graph representations, namely O-Tree and B*-Tree representations. We implement our proposed floorplanning methods as a package for Dynamo, a visual programming tool, with a custom GUI and additional evaluation functionalities to facilitate designers in their generative design workflow. Furthermore, we illustrate the performance of GenFloor in two sets of case-study experiments for residential floorplanning tasks by (a) measuring the ability of the proposed system to find a known optimal solution, and (b) observing how the system can generate diverse floorplans while addressing given a constant residential design problem. Our results indicate convergence to the global optimum is achieved while offering a diverse set of solutions of a residential floorplan corresponding to the Pareto-optimums of the solution landscape. △ Less

Submitted 20 February, 2021; originally announced February 2021.

GenScan: A Generative Method for Populating Parametric 3D Scan Datasets

Authors: Mohammad Keshavarzi, Oladapo Afolabi, Luisa Caldas, Allen Y. Yang, Avideh Zakhor

Abstract: The availability of rich 3D datasets corresponding to the geometrical complexity of the built environments is considered an ongoing challenge for 3D deep learning methodologies. To address this challenge, we introduce GenScan, a generative system that populates synthetic 3D scan datasets in a parametric fashion. The system takes an existing captured 3D scan as an input and outputs alternative vari… ▽ More The availability of rich 3D datasets corresponding to the geometrical complexity of the built environments is considered an ongoing challenge for 3D deep learning methodologies. To address this challenge, we introduce GenScan, a generative system that populates synthetic 3D scan datasets in a parametric fashion. The system takes an existing captured 3D scan as an input and outputs alternative variations of the building layout including walls, doors, and furniture with corresponding textures. GenScan is a fully automated system that can also be manually controlled by a user through an assigned user interface. Our proposed system utilizes a combination of a hybrid deep neural network and a parametrizer module to extract and transform elements of a given 3D scan. GenScan takes advantage of style transfer techniques to generate new textures for the generated scenes. We believe our system would facilitate data augmentation to expand the currently limited 3D geometry datasets commonly used in 3D computer vision, generative design, and general 3D deep learning tasks. △ Less

Submitted 7 December, 2020; originally announced December 2020.

SceneGen: Generative Contextual Scene Augmentation using Scene Graph Priors

Authors: Mohammad Keshavarzi, Aakash Parikh, Xiyu Zhai, Melody Mao, Luisa Caldas, Allen Y. Yang

Abstract: Spatial computing experiences are constrained by the real-world surroundings of the user. In such experiences, augmenting virtual objects to existing scenes require a contextual approach, where geometrical conflicts are avoided, and functional and plausible relationships to other objects are maintained in the target environment. Yet, due to the complexity and diversity of user environments, automa… ▽ More Spatial computing experiences are constrained by the real-world surroundings of the user. In such experiences, augmenting virtual objects to existing scenes require a contextual approach, where geometrical conflicts are avoided, and functional and plausible relationships to other objects are maintained in the target environment. Yet, due to the complexity and diversity of user environments, automatically calculating ideal positions of virtual content that is adaptive to the context of the scene is considered a challenging task. Motivated by this problem, in this paper we introduce SceneGen, a generative contextual augmentation framework that predicts virtual object positions and orientations within existing scenes. SceneGen takes a semantically segmented scene as input, and outputs positional and orientational probability maps for placing virtual content. We formulate a novel spatial Scene Graph representation, which encapsulates explicit topological properties between objects, object groups, and the room. We believe providing explicit and intuitive features plays an important role in informative content creation and user interaction of spatial computing settings, a quality that is not captured in implicit models. We use kernel density estimation (KDE) to build a multivariate conditional knowledge model trained using prior spatial Scene Graphs extracted from real-world 3D scanned data. To further capture orientational properties, we develop a fast pose annotation tool to extend current real-world datasets with orientational labels. Finally, to demonstrate our system in action, we develop an Augmented Reality application, in which objects can be contextually augmented in real-time. △ Less

Submitted 30 September, 2020; v1 submitted 25 September, 2020; originally announced September 2020.

Comments: 19 pages, 19 figures

SketchOpt: Sketch-based Parametric Model Retrieval for Generative Design

Authors: Mohammad Keshavarzi, Clayton Hutson, Chin-Yi Cheng, Mehdi Nourbakhsh, Michael Bergin, Mohammad Rahmani Asl

Abstract: Develo** fully parametric building models for performance-based generative design tasks often requires proficiency in many advanced 3D modeling and visual programming, limiting its use for many building designers. Moreover, iterations of such models can be time-consuming tasks and sometimes limiting, as major changes in the layout design may result in remodeling the entire parametric definition.… ▽ More Develo** fully parametric building models for performance-based generative design tasks often requires proficiency in many advanced 3D modeling and visual programming, limiting its use for many building designers. Moreover, iterations of such models can be time-consuming tasks and sometimes limiting, as major changes in the layout design may result in remodeling the entire parametric definition. To address these challenges, we introduce a novel automated generative design system, which takes a basic floor plan sketch as an input and provides a parametric model prepared for multi-objective building optimization as output. Furthermore, the user-designer can assign various design variables for its desired building elements by using simple annotations in the drawing. The system would recognize the corresponding element and define variable constraints to prepare for a multi-objective optimization problem. △ Less

Submitted 1 September, 2020; originally announced September 2020.

V-Dream: Immersive Exploration of Generative Design Solution Space

Authors: Mohammad Keshavarzi, Ardavan Bidgoli, Hans Kellner

Abstract: Generative Design workflows have introduced alternative paradigms in the domain of computational design, allowing designers to generate large pools of valid solutions by defining a set of goals and constraints. However, analyzing and narrowing down the generated solution space, which usually consists of various high-dimensional properties, has been a major challenge in current generative workflows… ▽ More Generative Design workflows have introduced alternative paradigms in the domain of computational design, allowing designers to generate large pools of valid solutions by defining a set of goals and constraints. However, analyzing and narrowing down the generated solution space, which usually consists of various high-dimensional properties, has been a major challenge in current generative workflows. By taking advantage of the interactive unbounded spatial exploration, and the visual immersion offered in virtual reality platforms, we propose V-Dream, a virtual reality generative analysis framework for exploring large-scale solution spaces. V-Dream proposes a hybrid search workflow in which a spatial stochastic search approach is combined with a recommender system allowing users to pick desired candidates and eliminate the undesired ones iteratively. In each cycle, V-Dream reorganizes the remaining options in clusters based on the defined features. Moreover, our framework allows users to inspect design solutions and evaluate their performance metrics in various hierarchical levels, assisting them in narrowing down the solution space through iterative cycles of search/select/re-clustering of the solutions in an immersive fashion. Finally, we present a prototype of our proposed framework, illustrating how users can navigate and narrow down desired solutions from a pool of over 16000 monitor stands generated by Autodesk's Dreamcatcher software. △ Less

Submitted 19 June, 2020; originally announced June 2020.

Comments: Accepted to HCI International 2020

Optimization and Manipulation of Contextual Mutual Spaces for Multi-User Virtual and Augmented Reality Interaction

Authors: Mohammad Keshavarzi, Allen Y. Yang, Woo** Ko, Luisa Caldas

Abstract: Spatial computing experiences are physically constrained by the geometry and semantics of the local user environment. This limitation is elevated in remote multi-user interaction scenarios, where finding a common virtual ground physically accessible for all participants becomes challenging. Locating a common accessible virtual ground is difficult for the users themselves, particularly if they are… ▽ More Spatial computing experiences are physically constrained by the geometry and semantics of the local user environment. This limitation is elevated in remote multi-user interaction scenarios, where finding a common virtual ground physically accessible for all participants becomes challenging. Locating a common accessible virtual ground is difficult for the users themselves, particularly if they are not aware of the spatial properties of other participants. In this paper, we introduce a framework to generate an optimal mutual virtual space for a multi-user interaction setting where remote users' room spaces can have different layout and sizes. The framework further recommends movement of surrounding furniture objects that expand the size of the mutual space with minimal physical effort. Finally, we demonstrate the performance of our solution on real-world datasets and also a real HoloLens application. Results show the proposed algorithm can effectively discover optimal shareable space for multi-user virtual interaction and hence facilitate remote spatial computing communication in various collaborative workflows. △ Less

Submitted 9 February, 2020; v1 submitted 14 October, 2019; originally announced October 2019.

Comments: Accepted at 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)

RadVR: A 6DOF Virtual Reality Daylighting Analysis Tool

Authors: Mohammad Keshavarzi, Luisa Caldas, Luis Santos

Abstract: This work introduces RadVR, a virtual reality tool for daylighting analysis that simultaneously combines qualitative assessments through immersive real-time renderings with quantitative physically correct daylighting simulations in a 6DOF virtual environment. By taking a 3D building model with material properties as input, RadVR allows users to (1) perform physically-based daylighting simulations… ▽ More This work introduces RadVR, a virtual reality tool for daylighting analysis that simultaneously combines qualitative assessments through immersive real-time renderings with quantitative physically correct daylighting simulations in a 6DOF virtual environment. By taking a 3D building model with material properties as input, RadVR allows users to (1) perform physically-based daylighting simulations via Radiance, (2) study sunlight in different hours-of-the-year, (3) interact with a 9-point-in-time matrix for the most representative times of the year, and (4) visualize, compare, and analyze daylighting simulation results. With an end-to-end workflow, RadVR integrates with 3D modeling software that is commonly used by building designers. Additionally, by conducting user experiments we compare the proposed system with DIVA for Rhino, a Radiance-based tool that uses conventional 2D-displays. The results show that RadVR can provide promising assistance in spatial understanding tasks, navigation, and sun position analysis in virtual reality. △ Less

Submitted 6 February, 2021; v1 submitted 2 July, 2019; originally announced July 2019.

Comments: Accepted to Automation in Construction

Affordance Analysis of Virtual and Augmented Reality Mediated Communication

Authors: Mohammad Keshavarzi, Michael Wu, Michael N. Chin, Robert N. Chin, Allen Y. Yang

Abstract: Virtual and augmented reality communication platforms are seen as promising modalities for next-generation remote face-to-face interactions. Our study attempts to explore non-verbal communication features in relation to their conversation context for virtual and augmented reality mediated communication settings. We perform a series of user experiments, triggering nine conversation tasks in 4 setti… ▽ More Virtual and augmented reality communication platforms are seen as promising modalities for next-generation remote face-to-face interactions. Our study attempts to explore non-verbal communication features in relation to their conversation context for virtual and augmented reality mediated communication settings. We perform a series of user experiments, triggering nine conversation tasks in 4 settings, each containing corresponding non-verbal communication features. Our results indicate that conversation types which involve less emotional engagement are more likely to be acceptable in virtual reality and augmented reality settings with low-fidelity avatar representation, compared to scenarios that involve high emotional engagement or intellectually difficult discussions. We further systematically analyze and rank the impact of low-fidelity representation of micro-expressions, body scale, head pose, and hand gesture in affecting the user experience in one-on-one conversations, and validate that preserving micro-expression cues plays the most effective role in improving bi-directional conversations in future virtual and augmented reality settings. △ Less

Submitted 9 April, 2019; originally announced April 2019.

Brain-Computer Interface in Virtual Reality

Authors: Reza Abbasi-Asl, Mohammad Keshavarzi, Dorian Yao Chan

Abstract: We study the performance of brain computer interface (BCI) system in a virtual reality (VR) environment and compare it to 2D regular displays. First, we design a headset that consists of three components: a wearable electroencephalography (EEG) device, a VR headset and an interface. Recordings of brain and behavior from human subjects, performing a wide variety of tasks using our device are collec… ▽ More We study the performance of brain computer interface (BCI) system in a virtual reality (VR) environment and compare it to 2D regular displays. First, we design a headset that consists of three components: a wearable electroencephalography (EEG) device, a VR headset and an interface. Recordings of brain and behavior from human subjects, performing a wide variety of tasks using our device are collected. The tasks consist of object rotation or scaling in VR using either mental commands or facial expression (smile and eyebrow movement). Subjects are asked to repeat similar tasks on regular 2D monitor screens. The performance in 3-D virtual reality environment is considerably higher compared to the to the 2D screen. Particularly, the median number of success rate across trials for VR setting is double of that for the 2D setting (8 successful command in VR setting compared to 4 successful command in 2D screen in 1 minute trials). Our results suggest that the design of future BCI systems can remarkably benefit from the VR setting. △ Less

Submitted 13 November, 2018; originally announced November 2018.