-
UniSG^GA: A 3D scenegraph powered by Geometric Algebra unifying geometry, behavior and GNNs towards generative AI
Authors:
Manos Kamarianakis,
Antonis Protopsaltis,
Dimitris Angelis,
Paul Zikas,
Mike Kentros,
George Papagiannakis
Abstract:
This work presents the introduction of UniSG^GA, a novel integrated scenegraph structure, that to incorporates behavior and geometry data on a 3D scene. It is specifically designed to seamlessly integrate Graph Neural Networks (GNNs) and address the challenges associated with transforming a 3D scenegraph (3D-SG) during generative tasks. To effectively capture and preserve the topological relations…
▽ More
This work presents the introduction of UniSG^GA, a novel integrated scenegraph structure, that to incorporates behavior and geometry data on a 3D scene. It is specifically designed to seamlessly integrate Graph Neural Networks (GNNs) and address the challenges associated with transforming a 3D scenegraph (3D-SG) during generative tasks. To effectively capture and preserve the topological relationships between objects in a simplified way, within the graph representation, we propose UniSG^GA, that seamlessly integrates Geometric Algebra (GA) forms. This novel approach enhances the overall performance and capability of GNNs in handling generative and predictive tasks, opening up new possibilities and aiming to lay the foundation for further exploration and development of graph-based generative AI models that can effectively incorporate behavior data for enhanced scene generation and synthesis.
△ Less
Submitted 18 June, 2023;
originally announced June 2023.
-
MAGES 4.0: Accelerating the world's transition to VR training and democratizing the authoring of the medical metaverse
Authors:
Paul Zikas,
Antonis Protopsaltis,
Nick Lydatakis,
Mike Kentros,
Stratos Geronikolakis,
Steve Kateros,
Manos Kamarianakis,
Giannis Evangelou,
Achilleas Filippidis,
Eleni Grigoriou,
Dimitris Angelis,
Michail Tamiolakis,
Michael Dodis,
George Kokiadis,
John Petropoulos,
Maria Pateraki,
George Papagiannakis
Abstract:
In this work, we propose MAGES 4.0, a novel Software Development Kit (SDK) to accelerate the creation of collaborative medical training applications in VR/AR. Our solution is essentially a low-code metaverse authoring platform for developers to rapidly prototype high-fidelity and high-complexity medical simulations. MAGES breaks the authoring boundaries across extended reality, since networked par…
▽ More
In this work, we propose MAGES 4.0, a novel Software Development Kit (SDK) to accelerate the creation of collaborative medical training applications in VR/AR. Our solution is essentially a low-code metaverse authoring platform for developers to rapidly prototype high-fidelity and high-complexity medical simulations. MAGES breaks the authoring boundaries across extended reality, since networked participants can also collaborate using different virtual/augmented reality as well as mobile and desktop devices, in the same metaverse world. With MAGES we propose an upgrade to the outdated 150-year-old master-apprentice medical training model. Our platform incorporates, in a nutsell, the following novelties: a) 5G edge-cloud remote rendering and physics dissection layer, b) realistic real-time simulation of organic tissues as soft-bodies under 10ms, c) a highly realistic cutting and tearing algorithm, d) neural network assessment for user profiling and, e) a VR recorder to record and replay or debrief the training simulation from any perspective.
△ Less
Submitted 19 January, 2023; v1 submitted 19 September, 2022;
originally announced September 2022.
-
Assessing unconstrained surgical cuttings in VR using CNNs
Authors:
Ilias Chrysovergis,
Manos Kamarianakis,
Mike Kentros,
Dimitris Angelis,
Antonis Protopsaltis,
George Papagiannakis
Abstract:
We present a Convolutional Neural Network (CNN) suitable to assess unconstrained surgical cuttings, trained on a dataset created with a data augmentation technique.
We present a Convolutional Neural Network (CNN) suitable to assess unconstrained surgical cuttings, trained on a dataset created with a data augmentation technique.
△ Less
Submitted 2 May, 2022;
originally announced May 2022.
-
Recording and replaying psychomotor user actions in VR
Authors:
Manos Kamarianakis,
Ilias Chrysovergis,
Mike Kentros,
George Papagiannakis
Abstract:
We introduce a novel method that describes the functionality and characteristics of an efficient VR recorder with replay capabilities, implemented in a modern game engine, publicly available for free.
We introduce a novel method that describes the functionality and characteristics of an efficient VR recorder with replay capabilities, implemented in a modern game engine, publicly available for free.
△ Less
Submitted 2 May, 2022;
originally announced May 2022.
-
Less Is More: Efficient Networked VR Transformation Handling Using Geometric Algebra
Authors:
Manos Kamarianakis,
Ilias Chrysovergis,
Nick Lydatakis,
Mike Kentros,
George Papagiannakis
Abstract:
As shared, collaborative, networked, virtual environments become increasingly popular, various challenges arise regarding the efficient transmission of model and scene transformation data over the network. As user immersion and real-time interactions heavily depend on VR stream synchronization, transmitting the entire data sat does not seem a suitable approach, especially for sessions involving a…
▽ More
As shared, collaborative, networked, virtual environments become increasingly popular, various challenges arise regarding the efficient transmission of model and scene transformation data over the network. As user immersion and real-time interactions heavily depend on VR stream synchronization, transmitting the entire data sat does not seem a suitable approach, especially for sessions involving a large number of users. Session recording is another momentum-gaining feature of VR applications that also faces the same challenge. The selection of a suitable data format can reduce the occupied volume, while it may also allow effective replication of the VR session and optimized post-processing for analytics and deep-learning algorithms. In this work, we propose two algorithms that can be applied in the context of a networked multiplayer VR session, to efficiently transmit the displacement and orientation data from the users' hand-based VR HMDs. Moreover, we present a novel method describing effective VR recording of the data exchanged in such a session. Our algorithms, based on the use of dual-quaternions and multivectors, impact the network consumption rate and are highly effective in scenarios involving multiple users. By sending less data over the network and interpolating the in-between frames locally, we manage to obtain better visual results than current state-of-the-art methods. Lastly, we prove that, for recording purposes, storing less data and interpolating them on-demand yields a data set quantitatively close to the original one.
△ Less
Submitted 25 July, 2022; v1 submitted 21 March, 2022;
originally announced March 2022.
-
MAGES 3.0: Tying the knot of medical VR
Authors:
George Papagiannakis,
Paul Zikas,
Nick Lydatakis,
Steve Kateros,
Mike Kentros,
Efstratios Geronikolakis,
Manos Kamarianakis,
Ioanna Kartsonaki,
Giannis Evangelou
Abstract:
In this work, we present MAGES 3.0, a novel Virtual Reality (VR)-based authoring SDK platform for accelerated surgical training and assessment. The MAGES Software Development Kit (SDK) allows code-free prototy** of any VR psychomotor simulation of medical operations by medical professionals, who urgently need a tool to solve the issue of outdated medical training. Our platform encapsulates the f…
▽ More
In this work, we present MAGES 3.0, a novel Virtual Reality (VR)-based authoring SDK platform for accelerated surgical training and assessment. The MAGES Software Development Kit (SDK) allows code-free prototy** of any VR psychomotor simulation of medical operations by medical professionals, who urgently need a tool to solve the issue of outdated medical training. Our platform encapsulates the following novel algorithmic techniques: a) collaborative networking layer with Geometric Algebra (GA) interpolation engine b) supervised machine learning analytics module for real-time recommendations and user profiling c) GA deformable cutting and tearing algorithm d) on-the-go configurable soft body simulation for deformable surfaces.
△ Less
Submitted 5 May, 2020; v1 submitted 3 May, 2020;
originally announced May 2020.
-
A True AR Authoring Tool for Interactive Virtual Museums
Authors:
Efstratios Geronikolakis,
Paul Zikas,
Steve Kateros,
Nick Lydatakis,
Stelios Georgiou,
Mike Kentros,
George Papagiannakis
Abstract:
In this work, a new and innovative way of spatial computing that appeared recently in the bibliography called True Augmented Reality (AR), is employed in cultural heritage preservation. This innovation could be adapted by the Virtual Museums of the future to enhance the quality of experience. It emphasises, the fact that a visitor will not be able to tell, at a first glance, if the artefact that h…
▽ More
In this work, a new and innovative way of spatial computing that appeared recently in the bibliography called True Augmented Reality (AR), is employed in cultural heritage preservation. This innovation could be adapted by the Virtual Museums of the future to enhance the quality of experience. It emphasises, the fact that a visitor will not be able to tell, at a first glance, if the artefact that he/she is looking at is real or not and it is expected to draw the visitors' interest. True AR is not limited to artefacts but extends even to buildings or life-sized character simulations of statues. It provides the best visual quality possible so that the users will not be able to tell the real objects from the augmented ones. Such applications can be beneficial for future museums, as with True AR, 3D models of various exhibits, monuments, statues, characters and buildings can be reconstructed and presented to the visitors in a realistic and innovative way. We also propose our Virtual Reality Sample application, a True AR playground featuring basic components and tools for generating interactive Virtual Museum applications, alongside a 3D reconstructed character (the priest of Asinou church) facilitating the storyteller of the augmented experience.
△ Less
Submitted 21 October, 2019; v1 submitted 20 September, 2019;
originally announced September 2019.
