-
On the Generalization of PINNs outside the training domain and the Hyperparameters influencing it
Authors:
Andrea Bonfanti,
Roberto Santana,
Marco Ellero,
Babak Gholami
Abstract:
Physics-Informed Neural Networks (PINNs) are Neural Network architectures trained to emulate solutions of differential equations without the necessity of solution data. They are currently ubiquitous in the scientific literature due to their flexible and promising settings. However, very little of the available research provides practical studies that aim for a better quantitative understanding of…
▽ More
Physics-Informed Neural Networks (PINNs) are Neural Network architectures trained to emulate solutions of differential equations without the necessity of solution data. They are currently ubiquitous in the scientific literature due to their flexible and promising settings. However, very little of the available research provides practical studies that aim for a better quantitative understanding of such architecture and its functioning. In this paper, we perform an empirical analysis of the behavior of PINN predictions outside their training domain. The primary goal is to investigate the scenarios in which a PINN can provide consistent predictions outside the training area. Thereinafter, we assess whether the algorithmic setup of PINNs can influence their potential for generalization and showcase the respective effect on the prediction. The results obtained in this study returns insightful and at times counterintuitive perspectives which can be highly relevant for architectures which combines PINNs with domain decomposition and/or adaptive training strategies.
△ Less
Submitted 24 August, 2023; v1 submitted 15 February, 2023;
originally announced February 2023.
-
Task-Discriminative Domain Alignment for Unsupervised Domain Adaptation
Authors:
Behnam Gholami,
Pritish Sahu,
Minyoung Kim,
Vladimir Pavlovic
Abstract:
Domain Adaptation (DA), the process of effectively adapting task models learned on one domain, the source, to other related but distinct domains, the targets, with no or minimal retraining, is typically accomplished using the process of source-to-target manifold alignment. However, this process often leads to unsatisfactory adaptation performance, in part because it ignores the task-specific struc…
▽ More
Domain Adaptation (DA), the process of effectively adapting task models learned on one domain, the source, to other related but distinct domains, the targets, with no or minimal retraining, is typically accomplished using the process of source-to-target manifold alignment. However, this process often leads to unsatisfactory adaptation performance, in part because it ignores the task-specific structure of the data. In this paper, we improve the performance of DA by introducing a discriminative discrepancy measure which takes advantage of auxiliary information available in the source and the target domains to better align the source and target distributions. Specifically, we leverage the cohesive clustering structure within individual data manifolds, associated with different tasks, to improve the alignment. This structure is explicit in the source, where the task labels are available, but is implicit in the target, making the problem challenging. We address the challenge by devising a deep DA framework, which combines a new task-driven domain alignment discriminator with domain regularizers that encourage the shared features as task-specific and domain invariant, and prompt the task model to be data structure preserving, guiding its decision boundaries through the low density data regions. We validate our framework on standard benchmarks, including Digits (MNIST, USPS, SVHN, MNIST-M), PACS, and VisDA. Our results show that our proposed model consistently outperforms the state-of-the-art in unsupervised domain adaptation.
△ Less
Submitted 26 September, 2019;
originally announced September 2019.
-
Unsupervised Visual Domain Adaptation: A Deep Max-Margin Gaussian Process Approach
Authors:
Minyoung Kim,
Pritish Sahu,
Behnam Gholami,
Vladimir Pavlovic
Abstract:
In unsupervised domain adaptation, it is widely known that the target domain error can be provably reduced by having a shared input representation that makes the source and target domains indistinguishable from each other. Very recently it has been studied that not just matching the marginal input distributions, but the alignment of output (class) distributions is also critical. The latter can be…
▽ More
In unsupervised domain adaptation, it is widely known that the target domain error can be provably reduced by having a shared input representation that makes the source and target domains indistinguishable from each other. Very recently it has been studied that not just matching the marginal input distributions, but the alignment of output (class) distributions is also critical. The latter can be achieved by minimizing the maximum discrepancy of predictors (classifiers). In this paper, we adopt this principle, but propose a more systematic and effective way to achieve hypothesis consistency via Gaussian processes (GP). The GP allows us to define/induce a hypothesis space of the classifiers from the posterior distribution of the latent random functions, turning the learning into a simple large-margin posterior separation problem, far easier to solve than previous approaches based on adversarial minimax optimization. We formulate a learning objective that effectively pushes the posterior to minimize the maximum discrepancy. This is further shown to be equivalent to maximizing margins and minimizing uncertainty of the class predictions in the target domain, a well-established principle in classical (semi-)supervised learning. Empirical results demonstrate that our approach is comparable or superior to the existing methods on several benchmark domain adaptation datasets.
△ Less
Submitted 22 February, 2019;
originally announced February 2019.
-
Unsupervised Multi-Target Domain Adaptation: An Information Theoretic Approach
Authors:
Behnam Gholami,
Pritish Sahu,
Ognjen Rudovic,
Konstantinos Bousmalis,
Vladimir Pavlovic
Abstract:
Unsupervised domain adaptation (uDA) models focus on pairwise adaptation settings where there is a single, labeled, source and a single target domain. However, in many real-world settings one seeks to adapt to multiple, but somewhat similar, target domains. Applying pairwise adaptation approaches to this setting may be suboptimal, as they fail to leverage shared information among multiple domains.…
▽ More
Unsupervised domain adaptation (uDA) models focus on pairwise adaptation settings where there is a single, labeled, source and a single target domain. However, in many real-world settings one seeks to adapt to multiple, but somewhat similar, target domains. Applying pairwise adaptation approaches to this setting may be suboptimal, as they fail to leverage shared information among multiple domains. In this work we propose an information theoretic approach for domain adaptation in the novel context of multiple target domains with unlabeled instances and one source domain with labeled instances. Our model aims to find a shared latent space common to all domains, while simultaneously accounting for the remaining private, domain-specific factors. Disentanglement of shared and private information is accomplished using a unified information-theoretic approach, which also serves to establish a stronger link between the latent representations and the observed data. The resulting model, accompanied by an efficient optimization algorithm, allows simultaneous adaptation from a single source to multiple target domains. We test our approach on three challenging publicly-available datasets, showing that it outperforms several popular domain adaptation methods.
△ Less
Submitted 26 October, 2018;
originally announced October 2018.
-
Gait Assessment for Multiple Sclerosis Patients Using Microsoft Kinect
Authors:
Farnood Gholami,
Daria A. Trojan,
Jozsef Kovecses,
Wassim M. Haddad,
Behnood Gholami
Abstract:
Gait analysis of patients with neurological disorders, including multiple sclerosis (MS), is important for rehabilitation and treatment. The Mircrosoft Kinect sensor, which was developed for motion recognition in gaming applications, is an ideal candidate for an inexpensive system providing the capability for human gait analysis. In this research, we develop a framework to quantify the gait abnorm…
▽ More
Gait analysis of patients with neurological disorders, including multiple sclerosis (MS), is important for rehabilitation and treatment. The Mircrosoft Kinect sensor, which was developed for motion recognition in gaming applications, is an ideal candidate for an inexpensive system providing the capability for human gait analysis. In this research, we develop a framework to quantify the gait abnormality of MS patients using a Kinect for Windows camera. In addition to the previously introduced gait indices, a novel set of MS gait indices based on the concept of dynamic time war** is introduced. The newly introduced indices can characterize a patient's gait pattern as a whole and quantify a subject's gait distance from the healthy population. We will investigate the correlation of gait indices with the multiple sclerosis walking scale (MSWS) and the clinical ambulation score. This work establishes the feasibility of using the Kinect sensor for clinical gait assessment for MS patients.
△ Less
Submitted 10 August, 2015;
originally announced August 2015.
-
Segmentation of Facial Expressions Using Semi-Definite Programming and Generalized Principal Component Analysis
Authors:
Behnood Gholami,
Allen R. Tannenbaum,
Wassim M. Haddad
Abstract:
In this paper, we use semi-definite programming and generalized principal component analysis (GPCA) to distinguish between two or more different facial expressions. In the first step, semi-definite programming is used to reduce the dimension of the image data and "unfold" the manifold which the data points (corresponding to facial expressions) reside on. Next, GPCA is used to fit a series of sub…
▽ More
In this paper, we use semi-definite programming and generalized principal component analysis (GPCA) to distinguish between two or more different facial expressions. In the first step, semi-definite programming is used to reduce the dimension of the image data and "unfold" the manifold which the data points (corresponding to facial expressions) reside on. Next, GPCA is used to fit a series of subspaces to the data points and associate each data point with a subspace. Data points that belong to the same subspace are claimed to belong to the same facial expression category. An example is provided.
△ Less
Submitted 10 June, 2009; v1 submitted 9 June, 2009;
originally announced June 2009.
