-
SIM2VR: Towards Automated Biomechanical Testing in VR
Authors:
Florian Fischer,
Aleksi Ikkala,
Markus Klar,
Arthur Fleig,
Miroslav Bachinski,
Roderick Murray-Smith,
Perttu Hämäläinen,
Antti Oulasvirta,
Jörg Müller
Abstract:
Automated biomechanical testing has great potential for the development of VR applications, as initial insights into user behaviour can be gained in silico early in the design process. In particular, it allows prediction of user movements and ergonomic variables, such as fatigue, prior to conducting user studies. However, there is a fundamental disconnect between simulators hosting state-of-the-ar…
▽ More
Automated biomechanical testing has great potential for the development of VR applications, as initial insights into user behaviour can be gained in silico early in the design process. In particular, it allows prediction of user movements and ergonomic variables, such as fatigue, prior to conducting user studies. However, there is a fundamental disconnect between simulators hosting state-of-the-art biomechanical user models and simulators used to develop and run VR applications. Existing user simulators often struggle to capture the intricacies and nuances of real-world VR applications, reducing ecological validity of user predictions. In this paper, we introduce SIM2VR, a system that aligns user simulation with a given VR application by establishing a continuous closed loop between the two processes. This, for the first time, enables training simulated users directly in the same VR application that real users interact with. We demonstrate that SIM2VR can predict differences in user performance, ergonomics and strategies in a fast-paced, dynamic arcade game. In order to expand the scope of automated biomechanical testing beyond simple visuomotor tasks, advances in cognitive models and reward function design will be needed.
△ Less
Submitted 26 April, 2024;
originally announced April 2024.
-
Multi-objective Quantum Annealing approach for solving flexible job shop scheduling in manufacturing
Authors:
Philipp Schworm,
Xiangquian Wu,
Matthias Klar,
Moritz Glatt,
Jan C. Aurich
Abstract:
Flexible Job Shop Scheduling (FJSSP) is a complex optimization problem crucial for real-world process scheduling in manufacturing. Efficiently solving such problems is vital for maintaining competitiveness. This paper introduces Quantum Annealing-based solving algorithm (QASA) to address FJSSP, utilizing quantum annealing and classical techniques. QASA optimizes multi-criterial FJSSP considering m…
▽ More
Flexible Job Shop Scheduling (FJSSP) is a complex optimization problem crucial for real-world process scheduling in manufacturing. Efficiently solving such problems is vital for maintaining competitiveness. This paper introduces Quantum Annealing-based solving algorithm (QASA) to address FJSSP, utilizing quantum annealing and classical techniques. QASA optimizes multi-criterial FJSSP considering makespan, total workload, and job priority concurrently. It employs Hamiltonian formulation with Lagrange parameters to integrate constraints and objectives, allowing objective prioritization through weight assignment. To manage computational complexity, large instances are decomposed into subproblems, and a decision logic based on bottleneck factors is used. Experiments on benchmark problems show QASA, combining tabu search, simulated annealing, and Quantum Annealing, outperforms a classical solving algorithm (CSA) in solution quality (set coverage and hypervolume ratio metrics). Computational efficiency analysis indicates QASA achieves superior Pareto solutions with a reasonable increase in computation time compared to CSA.
△ Less
Submitted 16 November, 2023;
originally announced November 2023.
-
A scalable domain decomposition method for FEM discretizations of nonlocal equations of integrable and fractional type
Authors:
Manuel Klar,
Giacomo Capodaglio,
Marta D'Elia,
Christian Glusa,
Max Gunzburger,
Christian Vollmann
Abstract:
Nonlocal models allow for the description of phenomena which cannot be captured by classical partial differential equations. The availability of efficient solvers is one of the main concerns for the use of nonlocal models in real world engineering applications. We present a domain decomposition solver that is inspired by substructuring methods for classical local equations. In numerical experiment…
▽ More
Nonlocal models allow for the description of phenomena which cannot be captured by classical partial differential equations. The availability of efficient solvers is one of the main concerns for the use of nonlocal models in real world engineering applications. We present a domain decomposition solver that is inspired by substructuring methods for classical local equations. In numerical experiments involving finite element discretizations of scalar and vectorial nonlocal equations of integrable and fractional type, we observe improvements in solution time of up to 14.6x compared to commonly used solver strategies.
△ Less
Submitted 31 May, 2023;
originally announced June 2023.
-
Towards a Deep(er) Understanding of Interaction through Modeling, Simulation, and Optimization
Authors:
Florian Fischer,
Arthur Fleig,
Markus Klar,
Viktorija Paneva,
Jörg Müller
Abstract:
The traditional user-centered design process can hardly keep up with the ever faster technical development and increasingly diverse user preferences. As a solution, we propose to augment the tried-and-tested approach of conducting user studies with simulation and optimization of the entire human-computer interaction loop. This approach allows to better understand phenomena through explicit modelin…
▽ More
The traditional user-centered design process can hardly keep up with the ever faster technical development and increasingly diverse user preferences. As a solution, we propose to augment the tried-and-tested approach of conducting user studies with simulation and optimization of the entire human-computer interaction loop. This approach allows to better understand phenomena through explicit modeling, build virtual prototypes through simulation, and improve interaction techniques through optimization. Building predictive user models also supports the creation and validation of HCI theories, and constitutes a decisive step towards new, intelligent, and adaptive user interfaces. We report our experience in virtually develo** new interaction techniques on the example of acoustic levitation, and present our optimization-based framework for HCI. With this, we strive to gain a better understanding of interaction and at the same time feed the discussion on questions such as which tools and tutorials are necessary to make virtual prototy** more accessible to different audiences.
△ Less
Submitted 22 February, 2023;
originally announced February 2023.
-
nlfem: A flexible 2d Fem Code for Nonlocal Convection-Diffusion and Mechanics
Authors:
Manuel Klar,
Christian Vollmann,
Volker Schulz
Abstract:
In this work we present the mathematical foundation of an assembly code for finite element approximations of nonlocal models with compactly supported, weakly singular kernels. We demonstrate the code on a nonlocal diffusion model in various configurations and on a two-dimensional bond-based peridynamics model. The code nlfem is published under the MIT License and can be freely downloaded.
In this work we present the mathematical foundation of an assembly code for finite element approximations of nonlocal models with compactly supported, weakly singular kernels. We demonstrate the code on a nonlocal diffusion model in various configurations and on a two-dimensional bond-based peridynamics model. The code nlfem is published under the MIT License and can be freely downloaded.
△ Less
Submitted 8 July, 2022;
originally announced July 2022.
-
Simulating Interaction Movements via Model Predictive Control
Authors:
Markus Klar,
Florian Fischer,
Arthur Fleig,
Miroslav Bachinski,
Jörg Müller
Abstract:
We present a method to simulate movement in interaction with computers, using Model Predictive Control (MPC). The method starts from understanding interaction from an Optimal Feedback Control (OFC) perspective. We assume that users aim to minimize an internalized cost function, subject to the constraints imposed by the human body and the interactive system. In contrast to previous linear approache…
▽ More
We present a method to simulate movement in interaction with computers, using Model Predictive Control (MPC). The method starts from understanding interaction from an Optimal Feedback Control (OFC) perspective. We assume that users aim to minimize an internalized cost function, subject to the constraints imposed by the human body and the interactive system. In contrast to previous linear approaches used in HCI, MPC can compute optimal controls for nonlinear systems. This allows us to use state-of-the-art biomechanical models and handle nonlinearities that occur in almost any interactive system. Instead of torque actuation, our model employs second-order muscles acting directly at the joints. We compare three different cost functions and evaluate the simulated trajectories against user movements in a Fitts' Law type pointing study with four different interaction techniques. Our results show that the combination of distance, control, and joint acceleration cost matches individual users' movements best, and predicts movements with an accuracy that is within the between-user variance. To aid HCI researchers and designers, we introduce CFAT, a novel method to identify maximum voluntary torques in joint-actuated models based on experimental data, and give practical advice on how to simulate human movement for different users, interaction techniques, and tasks.
△ Less
Submitted 19 April, 2022;
originally announced April 2022.
-
Optimal Feedback Control for Modeling Human-Computer Interaction
Authors:
Florian Fischer,
Arthur Fleig,
Markus Klar,
Jörg Müller
Abstract:
Optimal feedback control (OFC) is a theory from the motor control literature that explains how humans move their body to achieve a certain goal, e.g., pointing with the finger. OFC is based on the assumption that humans aim to control their body optimally, within the constraints imposed by body, environment, and task. In this paper, we explain how this theory can be applied to understanding Human-…
▽ More
Optimal feedback control (OFC) is a theory from the motor control literature that explains how humans move their body to achieve a certain goal, e.g., pointing with the finger. OFC is based on the assumption that humans aim to control their body optimally, within the constraints imposed by body, environment, and task. In this paper, we explain how this theory can be applied to understanding Human-Computer Interaction (HCI) in the case of pointing. We propose that the human body and computer dynamics can be interpreted as a single dynamical system. The system state is controlled by the user via muscle control signals, and estimated from observations. Between-trial variability arises from signal-dependent control noise and observation noise. We compare four different models from optimal control theory and evaluate to what degree these models can replicate movements in the case of mouse pointing. We introduce a procedure to identify parameters that best explain observed user behavior. To support HCI researchers in simulating, analyzing, and optimizing interaction movements, we provide the Python toolbox OFC4HCI. We conclude that OFC presents a powerful framework for HCI to understand and simulate motion of the human body and of the interface on a moment by moment basis.
△ Less
Submitted 20 April, 2022; v1 submitted 1 October, 2021;
originally announced October 2021.
-
Second-order fast-slow dynamics of non-ergodic Hamiltonian systems: Thermodynamic interpretation and simulation
Authors:
Matthias Klar,
Karsten Matthies,
Celia Reina,
Johannes Zimmer
Abstract:
A class of fast-slow Hamiltonian systems with potential $U_\varepsilon$ describing the interaction of non-ergodic fast and slow degrees of freedom is studied. The parameter $\varepsilon$ indicates the typical timescale ratio of the fast and slow degrees of freedom. It is known that the Hamiltonian system converges for $\varepsilon\to0$ to a homogenised Hamiltonian system. We study the situation wh…
▽ More
A class of fast-slow Hamiltonian systems with potential $U_\varepsilon$ describing the interaction of non-ergodic fast and slow degrees of freedom is studied. The parameter $\varepsilon$ indicates the typical timescale ratio of the fast and slow degrees of freedom. It is known that the Hamiltonian system converges for $\varepsilon\to0$ to a homogenised Hamiltonian system. We study the situation where $\varepsilon$ is small but positive. First, we rigorously derive the second-order corrections to the homogenised (slow) degrees of freedom. They can be decomposed into explicitly given terms that oscillate rapidly around zero and terms that trace the average motion of the corrections, which are given as the solution to an inhomogeneous linear system of differential equations. Then, we analyse the energy of the fast degrees of freedom expanded to second-order from a thermodynamic point of view. In particular, we define and expand to second-order a temperature, an entropy and external forces and show that they satisfy to leading-order, as well as on average to second-order, thermodynamic energy relations akin to the first and second law of thermodynamics. Finally, we analyse for a specific fast-slow Hamiltonian system the second-order asymptotic expansion of the slow degrees of freedom from a numerical point of view. Their approximation quality for short and long time frames and their total computation time are compared with those of the solution to the original fast-slow Hamiltonian system of similar accuracy.
△ Less
Submitted 12 November, 2021; v1 submitted 9 March, 2021;
originally announced March 2021.
-
Reinforcement Learning Control of a Biomechanical Model of the Upper Extremity
Authors:
Florian Fischer,
Miroslav Bachinski,
Markus Klar,
Arthur Fleig,
Jörg Müller
Abstract:
Among the infinite number of possible movements that can be produced, humans are commonly assumed to choose those that optimize criteria such as minimizing movement time, subject to certain movement constraints like signal-dependent and constant motor noise. While so far these assumptions have only been evaluated for simplified point-mass or planar models, we address the question of whether they c…
▽ More
Among the infinite number of possible movements that can be produced, humans are commonly assumed to choose those that optimize criteria such as minimizing movement time, subject to certain movement constraints like signal-dependent and constant motor noise. While so far these assumptions have only been evaluated for simplified point-mass or planar models, we address the question of whether they can predict reaching movements in a full skeletal model of the human upper extremity. We learn a control policy using a motor babbling approach as implemented in reinforcement learning, using aimed movements of the tip of the right index finger towards randomly placed 3D targets of varying size. We use a state-of-the-art biomechanical model, which includes seven actuated degrees of freedom. To deal with the curse of dimensionality, we use a simplified second-order muscle model, acting at each degree of freedom instead of individual muscles. The results confirm that the assumptions of signal-dependent and constant motor noise, together with the objective of movement time minimization, are sufficient for a state-of-the-art skeletal model of the human upper extremity to reproduce complex phenomena of human movement, in particular Fitts' Law and the 2/3 Power Law. This result supports the notion that control of the complex human biomechanical system can plausibly be determined by a set of simple assumptions and can easily be learned.
△ Less
Submitted 20 April, 2022; v1 submitted 13 November, 2020;
originally announced November 2020.
-
Second-order asymptotic expansion and thermodynamic interpretation of a fast-slow Hamiltonian system
Authors:
Matthias Klar,
Karsten Matthies,
Johannes Zimmer
Abstract:
This article includes a short survey of selected averaging and dimension reduction techniques for deterministic fast-slow systems. This survey includes, among others, classical techniques, such as the WKB approximation or the averaging method, as well as modern techniques, such as the GENERIC formalism. The main part of this article combines ideas of some of these techniques and addresses the prob…
▽ More
This article includes a short survey of selected averaging and dimension reduction techniques for deterministic fast-slow systems. This survey includes, among others, classical techniques, such as the WKB approximation or the averaging method, as well as modern techniques, such as the GENERIC formalism. The main part of this article combines ideas of some of these techniques and addresses the problem of deriving a reduced system for the slow degrees of freedom (DOF) of a fast-slow Hamiltonian system. In the first part, we derive an asymptotic expansion of the averaged evolution of the fast-slow system up to second-order, using weak convergence techniques and two-scale convergence. In the second part, we determine quantities which can be interpreted as temperature and entropy of the system and expand these quantities up to second-order, using results from the first part. The results give new insights into the thermodynamic interpretation of the fast-slow system at different scales.
△ Less
Submitted 18 November, 2022; v1 submitted 21 October, 2020;
originally announced October 2020.
-
A general framework for substructuring-based domain decomposition methods for models having nonlocal interactions
Authors:
Giacomo Capodaglio,
Marta D'Elia,
Max Gunzburger,
Pavel Bochev,
Manuel Klar,
Christian Vollmann
Abstract:
A rigorous mathematical framework is provided for a substructuring-based domain-decomposition approach for nonlocal problems that feature interactions between points separated by a finite distance. Here, by substructuring it is meant that a traditional geometric configuration for local partial differential equation problems is used in which a computational domain is subdivided into non-overlap**…
▽ More
A rigorous mathematical framework is provided for a substructuring-based domain-decomposition approach for nonlocal problems that feature interactions between points separated by a finite distance. Here, by substructuring it is meant that a traditional geometric configuration for local partial differential equation problems is used in which a computational domain is subdivided into non-overlap** subdomains. In the nonlocal setting, this approach is substructuring-based in the sense that those subdomains interact with neighboring domains over interface regions having finite volume, in contrast to the local PDE setting in which interfaces are lower dimensional manifolds separating abutting subdomains. Key results include the equivalence between the global, single-domain nonlocal problem and its multi-domain reformulation, both at the continuous and discrete levels. These results provide the rigorous foundation necessary for the development of efficient solution strategies for nonlocal domain-decomposition methods.
△ Less
Submitted 26 August, 2020;
originally announced August 2020.
-
An Optimal Control Model of Mouse Pointing Using the LQR
Authors:
Florian Fischer,
Arthur Fleig,
Markus Klar,
Lars Gruene,
Joerg Mueller
Abstract:
In this paper we explore the Linear-Quadratic Regulator (LQR) to model movement of the mouse pointer. We propose a model in which users are assumed to behave optimally with respect to a certain cost function. Users try to minimize the distance of the mouse pointer to the target smoothly and with minimal effort, by simultaneously minimizing the jerk of the movement. We identify parameters of our mo…
▽ More
In this paper we explore the Linear-Quadratic Regulator (LQR) to model movement of the mouse pointer. We propose a model in which users are assumed to behave optimally with respect to a certain cost function. Users try to minimize the distance of the mouse pointer to the target smoothly and with minimal effort, by simultaneously minimizing the jerk of the movement. We identify parameters of our model from a dataset of reciprocal pointing with the mouse. We compare our model to the classical minimum-jerk and second-order lag models on data from 12 users with a total of 7702 movements. Our results show that our approach explains the data significantly better than either of these previous models.
△ Less
Submitted 26 February, 2020;
originally announced February 2020.
-
Robust Learning Under Label Noise With Iterative Noise-Filtering
Authors:
Duc Tam Nguyen,
Thi-Phuong-Nhung Ngo,
Zhongyu Lou,
Michael Klar,
Laura Beggel,
Thomas Brox
Abstract:
We consider the problem of training a model under the presence of label noise. Current approaches identify samples with potentially incorrect labels and reduce their influence on the learning process by either assigning lower weights to them or completely removing them from the training set. In the first case the model however still learns from noisy labels; in the latter approach, good training d…
▽ More
We consider the problem of training a model under the presence of label noise. Current approaches identify samples with potentially incorrect labels and reduce their influence on the learning process by either assigning lower weights to them or completely removing them from the training set. In the first case the model however still learns from noisy labels; in the latter approach, good training data can be lost. In this paper, we propose an iterative semi-supervised mechanism for robust learning which excludes noisy labels but is still able to learn from the corresponding samples. To this end, we add an unsupervised loss term that also serves as a regularizer against the remaining label noise. We evaluate our approach on common classification tasks with different noise ratios. Our robust models outperform the state-of-the-art methods by a large margin. Especially for very large noise ratios, we achieve up to 20 % absolute improvement compared to the previous best model.
△ Less
Submitted 1 June, 2019;
originally announced June 2019.
-
Anomaly Detection With Multiple-Hypotheses Predictions
Authors:
Duc Tam Nguyen,
Zhongyu Lou,
Michael Klar,
Thomas Brox
Abstract:
In one-class-learning tasks, only the normal case (foreground) can be modeled with data, whereas the variation of all possible anomalies is too erratic to be described by samples. Thus, due to the lack of representative data, the wide-spread discriminative approaches cannot cover such learning tasks, and rather generative models, which attempt to learn the input density of the foreground, are used…
▽ More
In one-class-learning tasks, only the normal case (foreground) can be modeled with data, whereas the variation of all possible anomalies is too erratic to be described by samples. Thus, due to the lack of representative data, the wide-spread discriminative approaches cannot cover such learning tasks, and rather generative models, which attempt to learn the input density of the foreground, are used. However, generative models suffer from a large input dimensionality (as in images) and are typically inefficient learners. We propose to learn the data distribution of the foreground more efficiently with a multi-hypotheses autoencoder. Moreover, the model is criticized by a discriminator, which prevents artificial data modes not supported by data, and enforces diversity across hypotheses. Our multiple-hypothesesbased anomaly detection framework allows the reliable identification of out-of-distribution samples. For anomaly detection on CIFAR-10, it yields up to 3.9% points improvement over previously reported results. On a real anomaly detection task, the approach reduces the error of the baseline models from 6.8% to 1.5%.
△ Less
Submitted 31 May, 2019; v1 submitted 31 October, 2018;
originally announced October 2018.
