-
UniSG^GA: A 3D scenegraph powered by Geometric Algebra unifying geometry, behavior and GNNs towards generative AI
Authors:
Manos Kamarianakis,
Antonis Protopsaltis,
Dimitris Angelis,
Paul Zikas,
Mike Kentros,
George Papagiannakis
Abstract:
This work presents the introduction of UniSG^GA, a novel integrated scenegraph structure, that to incorporates behavior and geometry data on a 3D scene. It is specifically designed to seamlessly integrate Graph Neural Networks (GNNs) and address the challenges associated with transforming a 3D scenegraph (3D-SG) during generative tasks. To effectively capture and preserve the topological relations…
▽ More
This work presents the introduction of UniSG^GA, a novel integrated scenegraph structure, that to incorporates behavior and geometry data on a 3D scene. It is specifically designed to seamlessly integrate Graph Neural Networks (GNNs) and address the challenges associated with transforming a 3D scenegraph (3D-SG) during generative tasks. To effectively capture and preserve the topological relationships between objects in a simplified way, within the graph representation, we propose UniSG^GA, that seamlessly integrates Geometric Algebra (GA) forms. This novel approach enhances the overall performance and capability of GNNs in handling generative and predictive tasks, opening up new possibilities and aiming to lay the foundation for further exploration and development of graph-based generative AI models that can effectively incorporate behavior data for enhanced scene generation and synthesis.
△ Less
Submitted 18 June, 2023;
originally announced June 2023.
-
Project Elements: A computational entity-component-system in a scene-graph pythonic framework, for a neural, geometric computer graphics curriculum
Authors:
George Papagiannakis,
Manos Kamarianakis,
Antonis Protopsaltis,
Dimitris Angelis,
Paul Zikas
Abstract:
We present the Elements project, a lightweight, open-source, computational science and computer graphics (CG) framework, tailored for educational needs, that offers, for the first time, the advantages of an Entity-Component-System (ECS) along with the rapid prototy** convenience of a Scenegraph-based pythonic framework. This novelty allows advances in the teaching of CG: from heterogeneous direc…
▽ More
We present the Elements project, a lightweight, open-source, computational science and computer graphics (CG) framework, tailored for educational needs, that offers, for the first time, the advantages of an Entity-Component-System (ECS) along with the rapid prototy** convenience of a Scenegraph-based pythonic framework. This novelty allows advances in the teaching of CG: from heterogeneous directed acyclic graphs and depth-first traversals, to animation, skinning, geometric algebra and shader-based components rendered via unique systems all the way to their representation as graph neural networks for 3D scientific visualization. Taking advantage of the unique ECS in a a Scenegraph underlying system, this project aims to bridge CG curricula and modern game engines (MGEs), that are based on the same approach but often present these notions in a black-box approach. It is designed to actively utilize software design patterns, under an extensible open-source approach. Although Elements provides a modern (i.e., shader-based as opposed to fixed-function OpenGL), simple to program approach with Jupyter notebooks and unit-tests, its CG pipeline is not black-box, exposing for teaching for the first time unique challenging scientific, visual and neural computing concepts.
△ Less
Submitted 27 June, 2023; v1 submitted 15 February, 2023;
originally announced February 2023.
-
MAGES 4.0: Accelerating the world's transition to VR training and democratizing the authoring of the medical metaverse
Authors:
Paul Zikas,
Antonis Protopsaltis,
Nick Lydatakis,
Mike Kentros,
Stratos Geronikolakis,
Steve Kateros,
Manos Kamarianakis,
Giannis Evangelou,
Achilleas Filippidis,
Eleni Grigoriou,
Dimitris Angelis,
Michail Tamiolakis,
Michael Dodis,
George Kokiadis,
John Petropoulos,
Maria Pateraki,
George Papagiannakis
Abstract:
In this work, we propose MAGES 4.0, a novel Software Development Kit (SDK) to accelerate the creation of collaborative medical training applications in VR/AR. Our solution is essentially a low-code metaverse authoring platform for developers to rapidly prototype high-fidelity and high-complexity medical simulations. MAGES breaks the authoring boundaries across extended reality, since networked par…
▽ More
In this work, we propose MAGES 4.0, a novel Software Development Kit (SDK) to accelerate the creation of collaborative medical training applications in VR/AR. Our solution is essentially a low-code metaverse authoring platform for developers to rapidly prototype high-fidelity and high-complexity medical simulations. MAGES breaks the authoring boundaries across extended reality, since networked participants can also collaborate using different virtual/augmented reality as well as mobile and desktop devices, in the same metaverse world. With MAGES we propose an upgrade to the outdated 150-year-old master-apprentice medical training model. Our platform incorporates, in a nutsell, the following novelties: a) 5G edge-cloud remote rendering and physics dissection layer, b) realistic real-time simulation of organic tissues as soft-bodies under 10ms, c) a highly realistic cutting and tearing algorithm, d) neural network assessment for user profiling and, e) a VR recorder to record and replay or debrief the training simulation from any perspective.
△ Less
Submitted 19 January, 2023; v1 submitted 19 September, 2022;
originally announced September 2022.
-
"Deep Cut": An all-in-one Geometric Algorithm for Unconstrained Cut, Tear and Drill of Soft-bodies in Mobile VR
Authors:
Manos Kamarianakis,
Nick Lydatakis,
Antonis Protopsaltis,
John Petropoulos,
Michail Tamiolakis,
Paul Zikas,
George Papagiannakis
Abstract:
In this work, we present an integrated geometric framework: "deep- cut" that enables for the first time a user to geometrically and algorithmically cut, tear and drill the surface of a skinned model without prior constraints, layered on top of a custom soft body mesh deformation algorithm. Both layered algorithms in this frame- work yield real-time results and are amenable for mobile Virtual Reali…
▽ More
In this work, we present an integrated geometric framework: "deep- cut" that enables for the first time a user to geometrically and algorithmically cut, tear and drill the surface of a skinned model without prior constraints, layered on top of a custom soft body mesh deformation algorithm. Both layered algorithms in this frame- work yield real-time results and are amenable for mobile Virtual Reality, in order to be utilized in a variety of interactive application scenarios. Our framework dramatically improves real-time user experience and task performance in VR, without pre-calculated or artificially designed cuts, tears, drills or surface deformations via predefined rigged animations, which is the current state-of-the-art in mobile VR. Thus our framework improves user experience on one hand, on the other hand saves both time and costs from expensive, manual, labour-intensive design pre-calculation stages.
△ Less
Submitted 11 August, 2021;
originally announced August 2021.
-
MAGES 3.0: Tying the knot of medical VR
Authors:
George Papagiannakis,
Paul Zikas,
Nick Lydatakis,
Steve Kateros,
Mike Kentros,
Efstratios Geronikolakis,
Manos Kamarianakis,
Ioanna Kartsonaki,
Giannis Evangelou
Abstract:
In this work, we present MAGES 3.0, a novel Virtual Reality (VR)-based authoring SDK platform for accelerated surgical training and assessment. The MAGES Software Development Kit (SDK) allows code-free prototy** of any VR psychomotor simulation of medical operations by medical professionals, who urgently need a tool to solve the issue of outdated medical training. Our platform encapsulates the f…
▽ More
In this work, we present MAGES 3.0, a novel Virtual Reality (VR)-based authoring SDK platform for accelerated surgical training and assessment. The MAGES Software Development Kit (SDK) allows code-free prototy** of any VR psychomotor simulation of medical operations by medical professionals, who urgently need a tool to solve the issue of outdated medical training. Our platform encapsulates the following novel algorithmic techniques: a) collaborative networking layer with Geometric Algebra (GA) interpolation engine b) supervised machine learning analytics module for real-time recommendations and user profiling c) GA deformable cutting and tearing algorithm d) on-the-go configurable soft body simulation for deformable surfaces.
△ Less
Submitted 5 May, 2020; v1 submitted 3 May, 2020;
originally announced May 2020.
-
From Readership to Usership and Education, Entertainment, Consumption to Valuation: Embodiment and Aesthetic Experience in Literature-based MR Presence
Authors:
Stéphanie Bertrand,
Martha Vassiliadi,
Paul Zikas,
Efstratios Geronikolakis,
George Papagiannakis
Abstract:
This chapter will extend its preliminary scope by examining how literary transportation further amplifies presence and affects user response vis-á-vis virtual heritage by focusing on embodiment and aesthetic experience. To do so, it will draw on recent findings emerging from the fields of applied psychology, neuroaesthetics and cognitive literary studies; and consider a case study advancing the us…
▽ More
This chapter will extend its preliminary scope by examining how literary transportation further amplifies presence and affects user response vis-á-vis virtual heritage by focusing on embodiment and aesthetic experience. To do so, it will draw on recent findings emerging from the fields of applied psychology, neuroaesthetics and cognitive literary studies; and consider a case study advancing the use of literary travel narratives in the design of DCH applications for Antiquities - in this case the well-known ancient Greek monument of Acropolis. Subsequently, the chapter will discuss how Literary-based MR Presence shifts public reception from an education-entertainment-touristic consumption paradigm to a response predicated on valuation. It will show that this type of public engagement is more closely aligned both with MR applications' default mode of usership, and with newly emerging conceptions of a user-centered museum (e.g., the Museum 3.0), thus providing a Virtual Museum model expressly suited to cultural heritage.
△ Less
Submitted 21 October, 2019;
originally announced October 2019.
-
A True AR Authoring Tool for Interactive Virtual Museums
Authors:
Efstratios Geronikolakis,
Paul Zikas,
Steve Kateros,
Nick Lydatakis,
Stelios Georgiou,
Mike Kentros,
George Papagiannakis
Abstract:
In this work, a new and innovative way of spatial computing that appeared recently in the bibliography called True Augmented Reality (AR), is employed in cultural heritage preservation. This innovation could be adapted by the Virtual Museums of the future to enhance the quality of experience. It emphasises, the fact that a visitor will not be able to tell, at a first glance, if the artefact that h…
▽ More
In this work, a new and innovative way of spatial computing that appeared recently in the bibliography called True Augmented Reality (AR), is employed in cultural heritage preservation. This innovation could be adapted by the Virtual Museums of the future to enhance the quality of experience. It emphasises, the fact that a visitor will not be able to tell, at a first glance, if the artefact that he/she is looking at is real or not and it is expected to draw the visitors' interest. True AR is not limited to artefacts but extends even to buildings or life-sized character simulations of statues. It provides the best visual quality possible so that the users will not be able to tell the real objects from the augmented ones. Such applications can be beneficial for future museums, as with True AR, 3D models of various exhibits, monuments, statues, characters and buildings can be reconstructed and presented to the visitors in a realistic and innovative way. We also propose our Virtual Reality Sample application, a True AR playground featuring basic components and tools for generating interactive Virtual Museum applications, alongside a 3D reconstructed character (the priest of Asinou church) facilitating the storyteller of the augmented experience.
△ Less
Submitted 21 October, 2019; v1 submitted 20 September, 2019;
originally announced September 2019.
-
Scenior: An Immersive Visual Scripting system based on VR Software Design Patterns for Experiential Training
Authors:
Paul Zikas,
George Papagiannakis,
Nick Lydatakis,
Steve Kateros,
Stavroula Ntoa,
Ilia Adami,
Constantine Stephanidis
Abstract:
Virtual reality (VR) has re-emerged as a low-cost, highly accessible consumer product, and training on simulators is rapidly becoming standard in many industrial sectors. However, the available systems are either focusing on gaming context, featuring limited capabilities or they support only content creation of virtual environments without any rapid prototy** and modification. In this project, w…
▽ More
Virtual reality (VR) has re-emerged as a low-cost, highly accessible consumer product, and training on simulators is rapidly becoming standard in many industrial sectors. However, the available systems are either focusing on gaming context, featuring limited capabilities or they support only content creation of virtual environments without any rapid prototy** and modification. In this project, we propose a code-free, visual scripting platform to replicate gamified training scenarios through rapid prototy** and VR software design patterns. We implemented and compared two authoring tools: a) visual scripting and b) VR editor for the rapid reconstruction of VR training scenarios. Our visual scripting module is capable to generate training applications utilizing a node-based scripting system whereas the VR editor gives user/developer the ability to customize and populate new VR training scenarios directly from the virtual environment. We also introduce action prototypes, a new software design pattern suitable to replicate behavioral tasks for VR experiences. In addition, we present the training scenegraph architecture as the main model to represent training scenarios on a modular, dynamic and highly adaptive acyclic graph based on a structured educational curriculum. Finally, a user-based evaluation of the proposed solution indicated that users - regardless of their programming expertise - can effectively use the tools to create and modify training scenarios in VR.
△ Less
Submitted 3 May, 2020; v1 submitted 12 September, 2019;
originally announced September 2019.
