Showing 1–50 of 54 results for author: Frahm, J

Explicit Hilbert spaces for the unitary dual of rank one orthogonal groups and applications

Authors: Christian Arends, Frederik Bang-Jensen, Jan Frahm

Abstract: We realize all irreducible unitary representations of the group $\mathrm{SO}_0(n+1,1)$ on explicit Hilbert spaces of vector-valued $L^2$-functions on $\mathbb{R}^n\setminus\{0\}$. The key ingredient in our construction is an explicit expression for the standard Knapp-Stein intertwining operators between arbitrary principal series representations in terms of the Euclidean Fourier transform on a max… ▽ More We realize all irreducible unitary representations of the group $\mathrm{SO}_0(n+1,1)$ on explicit Hilbert spaces of vector-valued $L^2$-functions on $\mathbb{R}^n\setminus\{0\}$. The key ingredient in our construction is an explicit expression for the standard Knapp-Stein intertwining operators between arbitrary principal series representations in terms of the Euclidean Fourier transform on a maximal unipotent subgroup isomorphic to $\mathbb{R}^n$. As an application, we describe the space of Whittaker vectors on all irreducible Casselman-Wallach representations. Moreover, the new realizations of the irreducible unitary representations immediately reveal their decomposition into irreducible representations of a parabolic subgroup, thus providing a simple proof of a recent result of Liu-Oshima-Yu. △ Less

Submitted 17 June, 2024; originally announced June 2024.

Comments: 27 pages

MSC Class: 22E45

Construction and analysis of symmetry breaking operators for the pair $(\operatorname{GL}(n+1,\mathbb{R}),\operatorname{GL}(n,\mathbb{R}))$

Authors: Jonathan Ditlevsen, Jan Frahm

Abstract: The pair of real reductive groups $(G,H)=(\operatorname{GL}(n+1,\mathbb{R}),\operatorname{GL}(n,\mathbb{R}))$ is a strong Gelfand pair, i.e. the multiplicities $\dim\operatorname{Hom}_H(π|_H,τ)$ are either $0$ or $1$ for all irreducible Casselman-Wallach representations $π$ of $G$ and $τ$ of $H$. This paper is concerned with the construction of explicit intertwining operators in… ▽ More The pair of real reductive groups $(G,H)=(\operatorname{GL}(n+1,\mathbb{R}),\operatorname{GL}(n,\mathbb{R}))$ is a strong Gelfand pair, i.e. the multiplicities $\dim\operatorname{Hom}_H(π|_H,τ)$ are either $0$ or $1$ for all irreducible Casselman-Wallach representations $π$ of $G$ and $τ$ of $H$. This paper is concerned with the construction of explicit intertwining operators in $\operatorname{Hom}_H(π|_H,τ)$, so-called symmetry breaking operators, in the case where both $π$ and $τ$ are principal series representations. Such operators come in families that depend meromorphically on the induction parameters, and we show how to normalize them in order to make the parameter dependence holomorphic. This is done by establishing explicit Bernstein-Sato identities for their distribution kernels as well as explicit functional identities for the composition of symmetry breaking operators with standard Knapp-Stein intertwining operators for $G$ and $H$. We also show that the obtained normalization is optimal and identify a subset of parameters for which the family of operators vanishes. Finally, we relate the operators to the local archimedean Rankin-Selberg integrals and use this relation to evaluate them on the spherical vectors. △ Less

Submitted 21 March, 2024; originally announced March 2024.

Comments: 33 pages

MSC Class: 22E45

Realization of unitary representations of the Lorentz group on de Sitter space

Authors: Jan Frahm, Karl-Hermann Neeb, Gestur Olafsson

Abstract: This paper builds on our previous work in which we showed that, for all connected semisimple linear Lie groups $G$ acting on a non-compactly causal symmetric space $M = G/H$, every irreducible unitary representation of $G$ can be realized by boundary value maps of holomorphic extensions in distributional sections of a vector bundle over $M$. In the present paper we discuss this procedure for the c… ▽ More This paper builds on our previous work in which we showed that, for all connected semisimple linear Lie groups $G$ acting on a non-compactly causal symmetric space $M = G/H$, every irreducible unitary representation of $G$ can be realized by boundary value maps of holomorphic extensions in distributional sections of a vector bundle over $M$. In the present paper we discuss this procedure for the connected Lorentz group $G = SO_{1,d}(R)_e$ acting on de Sitter space $M = dS^d$. We show in particular that the previously constructed nets of real subspaces satisfy the locality condition. Following ideas of Bros and Moschella from the 1990's, we show that the matrix-valued spherical function that corresponds to our extension process extends analytically to a large domain $G_C^{cut}$ in the complexified group $G_C = \SO_{1,d}(C)$, which for $d = 1$ specializes to the complex cut plane $C \setminus (-\infinity, 0]$. A number of special situations is discussed specifically: (a) The case $d = 1$, which closely corresponds to standard subspaces in Hilbert spaces, (b) the case of scalar-valued functions, which for $d > 2$ is the case of spherical representations, for which we also describe the jump singularities of the holomorphic extensions on the cut in de Sitter space, (c) the case $d = 3$, where we obtain rather explicit formulas for the matrix-valued spherical functions. △ Less

Submitted 30 January, 2024; originally announced January 2024.

Comments: 53pp

MSC Class: 22E45; 81R05; 81T05

The holomorphic discrete series contribution to the generalized Whittaker Plancherel formula II. Non-tube type groups

Authors: Jan Frahm, Gestur Ólafsson, Bent Ørsted

Abstract: For every simple Hermitian Lie group $G$, we consider a certain maximal parabolic subgroup whose unipotent radical $N$ is either abelian (if $G$ is of tube type) or two-step nilpotent (if $G$ is of non-tube type). By the generalized Whittaker Plancherel formula we mean the Plancherel decomposition of $L^2(G/N,ω)$, the space of square-integrable sections of the homogeneous vector bundle over $G/N$… ▽ More For every simple Hermitian Lie group $G$, we consider a certain maximal parabolic subgroup whose unipotent radical $N$ is either abelian (if $G$ is of tube type) or two-step nilpotent (if $G$ is of non-tube type). By the generalized Whittaker Plancherel formula we mean the Plancherel decomposition of $L^2(G/N,ω)$, the space of square-integrable sections of the homogeneous vector bundle over $G/N$ associated with an irreducible unitary representation $ω$ of $N$. Assuming that the central character of $ω$ is contained in a certain cone, we construct embeddings of all holomorphic discrete series representations of $G$ into $L^2(G/N,ω)$ and show that the multiplicities are equal to the dimensions of the lowest $K$-types. The construction is in terms of a kernel function which can be explicitly defined using certain projections inside a complexification of $G$. This kernel function carries all information about the holomorphic discrete series embedding, the lowest $K$-type as functions on $G/N$, as well as the associated Whittaker vectors. △ Less

Submitted 12 January, 2024; originally announced January 2024.

Comments: 18 pages

MSC Class: Primary 22E46; Secondary 43A85

A Pairing Formula for Resonant States on Finite Regular Graphs

Authors: Christian Arends, Jan Frahm, Joachim Hilgert

Abstract: On a finite regular graph, (co)resonant states are eigendistributions of the transfer operator associated to the shift on one-sided infinite non-backtracking paths. We introduce two pairings of resonant and coresonant states, the vertex pairing which involves only the dependence on the initial/terminal vertex of the path, and the geodesic pairing which is given by integrating over all geodesics th… ▽ More On a finite regular graph, (co)resonant states are eigendistributions of the transfer operator associated to the shift on one-sided infinite non-backtracking paths. We introduce two pairings of resonant and coresonant states, the vertex pairing which involves only the dependence on the initial/terminal vertex of the path, and the geodesic pairing which is given by integrating over all geodesics the evaluation of the coresonant state on the first half of the geodesic times the resonant state on the second half. The main result is that these two pairings coincide up to a constant which depends on the resonance, i.e. the corresponding eigenvalue of the transfer operator. △ Less

Submitted 16 December, 2023; originally announced December 2023.

Comments: 41 pages

MSC Class: 37E25; 20E08; 47B93; 05C81

Edge Laplacians and Edge Poisson Transforms for Graphs

Authors: Christian Arends, Jan Frahm, Joachim Hilgert

Abstract: For a finite graph, we establish natural isomorphisms between eigenspaces of a Laplace operator acting on functions on the edges and eigenspaces of a transfer operator acting on functions on one-sided infinite non-backtracking paths. Interpreting the transfer operator as a classical dynamical system and the Laplace operator as its quantization, this result can be viewed as a quantum-classical corr… ▽ More For a finite graph, we establish natural isomorphisms between eigenspaces of a Laplace operator acting on functions on the edges and eigenspaces of a transfer operator acting on functions on one-sided infinite non-backtracking paths. Interpreting the transfer operator as a classical dynamical system and the Laplace operator as its quantization, this result can be viewed as a quantum-classical correspondence. In contrast to previously established quantum-classical correspondences for the vertex Laplacian which exclude certain exceptional spectral parameters, our correspondence is valid for all parameters. This allows us to relate certain spectral quantities to topological properties of the graph such as the cyclomatic number and the 2-colorability. The quantum-classical correspondence for the edge Laplacian is induced by an edge Poisson transform on the universal covering of the graph which is a tree of bounded degree. In the special case of regular trees, we relate both the vertex and the edge Poisson transform to the representation theory of the automorphism group of the tree and study associated operator valued Hecke algebras. △ Less

Submitted 14 December, 2023; originally announced December 2023.

Comments: 46 pages

MSC Class: 37E25; 47A11; 47B93; 37D40; 20C08

Symmetry breaking for $\operatorname{PGL}(2)$ over non-archimedean local fields

Authors: Corina Ciobotaru, Jan Frahm

Abstract: For a quadratic extension $\mathbb{E}/\mathbb{F}$ of non-archimedean local fields we construct explicit holomorphic families of intertwining operators between principal series representations of $\operatorname{PGL}(2,\mathbb{E})$ and $\operatorname{PGL}(2,\mathbb{F})$, also referred to as symmetry breaking operators. These families are given in terms of their distribution kernels which can be view… ▽ More For a quadratic extension $\mathbb{E}/\mathbb{F}$ of non-archimedean local fields we construct explicit holomorphic families of intertwining operators between principal series representations of $\operatorname{PGL}(2,\mathbb{E})$ and $\operatorname{PGL}(2,\mathbb{F})$, also referred to as symmetry breaking operators. These families are given in terms of their distribution kernels which can be viewed as distributions on $\mathbb{E}$ depending holomorphically on the principal series parameters. For all such parameters we determine the support of these distributions, and we study their map** properties. This leads to a classification of all intertwining operators between principal series representations, not necessarily irreducible. As an application, we show that every Steinberg representation of $\operatorname{PGL}(2,\mathbb{E})$ contains a Steinberg representation of $\operatorname{PGL}(2,\mathbb{F})$ as a direct summand of Hilbert spaces. △ Less

Submitted 26 September, 2023; originally announced September 2023.

Comments: 42 pages

MSC Class: Primary 22E35; Secondary 22E50

MVPSNet: Fast Generalizable Multi-view Photometric Stereo

Authors: Dongxu Zhao, Daniel Lichy, Pierre-Nicolas Perrin, Jan-Michael Frahm, Soumyadip Sengupta

Abstract: We propose a fast and generalizable solution to Multi-view Photometric Stereo (MVPS), called MVPSNet. The key to our approach is a feature extraction network that effectively combines images from the same view captured under multiple lighting conditions to extract geometric features from shading cues for stereo matching. We demonstrate these features, termed `Light Aggregated Feature Maps' (LAFM),… ▽ More We propose a fast and generalizable solution to Multi-view Photometric Stereo (MVPS), called MVPSNet. The key to our approach is a feature extraction network that effectively combines images from the same view captured under multiple lighting conditions to extract geometric features from shading cues for stereo matching. We demonstrate these features, termed `Light Aggregated Feature Maps' (LAFM), are effective for feature matching even in textureless regions, where traditional multi-view stereo methods fail. Our method produces similar reconstruction results to PS-NeRF, a state-of-the-art MVPS method that optimizes a neural network per-scene, while being 411$\times$ faster (105 seconds vs. 12 hours) in inference. Additionally, we introduce a new synthetic dataset for MVPS, sMVPS, which is shown to be effective to train a generalizable MVPS method. △ Less

Submitted 18 May, 2023; originally announced May 2023.

Heisenberg parabolically induced representations of Hermitian Lie groups, Part II: Next-to-minimal representations and branching rules

Authors: Jan Frahm, Clemens Weiske, Genkai Zhang

Abstract: Every simple Hermitian Lie group has a unique family of spherical representations induced from a maximal parabolic subgroup whose unipotent radical is a Heisenberg group. For most Hermitian groups, this family contains a complementary series, and at its endpoint sits a proper unitarizable subrepresentation. We show that this subrepresentation is next-to-minimal in the sense that its associated var… ▽ More Every simple Hermitian Lie group has a unique family of spherical representations induced from a maximal parabolic subgroup whose unipotent radical is a Heisenberg group. For most Hermitian groups, this family contains a complementary series, and at its endpoint sits a proper unitarizable subrepresentation. We show that this subrepresentation is next-to-minimal in the sense that its associated variety is a next-to-minimal nilpotent coadjoint orbit. Moreover, for the Hermitian groups $\operatorname{SO}_0(2,n)$ and $E_{6(-14)}$ we study some branching problems of these next-to-minimal representations. △ Less

Submitted 12 April, 2023; originally announced April 2023.

Comments: 21 pages

Restricting holomorphic discrete series representations to a compact dual pair

Authors: Jan Frahm, Quentin Labriet

Abstract: The goal of this article is to study the branching problem for a holomorphic discrete series representation of the conformal group of a simple Euclidean Jordan algebra $V$ restricted to the subgroup $\operatorname{PSL}_2(\mathbb{R})\times\operatorname{Aut}(V)$ where $\operatorname{Aut}(V)$ denotes the compact group of automorphisms of $V$. We use a realization of the holomorphic discrete series on… ▽ More The goal of this article is to study the branching problem for a holomorphic discrete series representation of the conformal group of a simple Euclidean Jordan algebra $V$ restricted to the subgroup $\operatorname{PSL}_2(\mathbb{R})\times\operatorname{Aut}(V)$ where $\operatorname{Aut}(V)$ denotes the compact group of automorphisms of $V$. We use a realization of the holomorphic discrete series on a space of vector-values $L^2$-functions as well as the stratified model developed by the second author to relate the branching problem to the decomposition of certain representations of the compact group $\operatorname{Aut}(V)$ and to vector-valued orthogonal polynomials. △ Less

Submitted 31 March, 2023; originally announced March 2023.

Comments: 16 pages

MSC Class: 22E46

Supervision Interpolation via LossMix: Generalizing Mixup for Object Detection and Beyond

Authors: Thanh Vu, Baochen Sun, Bodi Yuan, Alex Ngai, Yueqi Li, Jan-Michael Frahm

Abstract: The success of data mixing augmentations in image classification tasks has been well-received. However, these techniques cannot be readily applied to object detection due to challenges such as spatial misalignment, foreground/background distinction, and plurality of instances. To tackle these issues, we first introduce a novel conceptual framework called Supervision Interpolation (SI), which offer… ▽ More The success of data mixing augmentations in image classification tasks has been well-received. However, these techniques cannot be readily applied to object detection due to challenges such as spatial misalignment, foreground/background distinction, and plurality of instances. To tackle these issues, we first introduce a novel conceptual framework called Supervision Interpolation (SI), which offers a fresh perspective on interpolation-based augmentations by relaxing and generalizing Mixup. Based on SI, we propose LossMix, a simple yet versatile and effective regularization that enhances the performance and robustness of object detectors and more. Our key insight is that we can effectively regularize the training on mixed data by interpolating their loss errors instead of ground truth labels. Empirical results on the PASCAL VOC and MS COCO datasets demonstrate that LossMix can consistently outperform state-of-the-art methods widely adopted for detection. Furthermore, by jointly leveraging LossMix with unsupervised domain adaptation, we successfully improve existing approaches and set a new state of the art for cross-domain object detection. △ Less

Submitted 19 December, 2023; v1 submitted 18 March, 2023; originally announced March 2023.

Comments: AAAI-24 Camera Ready Version, with supplementary material, 15 pages

Nets of standard subspaces on non-compactly causal symmetric spaces

Authors: Jan Frahm, Karl-Hermann Neeb, Gestur Olafsson

Abstract: Let G be a connected simple linear Lie group and H in G a symmetric subgroup such that the corresponding symmetric space G/H is non-compactly causal. We show that any irreducible unitary representation of G leads naturally to a net of standard subspaces on G/H that is isotone, covariant and has the Reeh--Schlieder and the Bisognano--Wichmann property. We also show that this result extends to the u… ▽ More Let G be a connected simple linear Lie group and H in G a symmetric subgroup such that the corresponding symmetric space G/H is non-compactly causal. We show that any irreducible unitary representation of G leads naturally to a net of standard subspaces on G/H that is isotone, covariant and has the Reeh--Schlieder and the Bisognano--Wichmann property. We also show that this result extends to the universal covering group of SL(2,R) which has some interesting application to intersections of standard subspaces associated to representations of such groups. For this a detailed study of hyperfunction and distribution vectors is needed. In particular we show that every H-finite hyperfunction vector is in fact a distribution vector. △ Less

Submitted 21 September, 2023; v1 submitted 17 March, 2023; originally announced March 2023.

Comments: Final version, 75 pages

MSC Class: 22E45; 81R05; 81T05

A Surface-normal Based Neural Framework for Colonoscopy Reconstruction

Authors: Shuxian Wang, Yubo Zhang, Sarah K. McGill, Julian G. Rosenman, Jan-Michael Frahm, Soumyadip Sengupta, Stephen M. Pizer

Abstract: Reconstructing a 3D surface from colonoscopy video is challenging due to illumination and reflectivity variation in the video frame that can cause defective shape predictions. Aiming to overcome this challenge, we utilize the characteristics of surface normal vectors and develop a two-step neural framework that significantly improves the colonoscopy reconstruction quality. The normal-based depth i… ▽ More Reconstructing a 3D surface from colonoscopy video is challenging due to illumination and reflectivity variation in the video frame that can cause defective shape predictions. Aiming to overcome this challenge, we utilize the characteristics of surface normal vectors and develop a two-step neural framework that significantly improves the colonoscopy reconstruction quality. The normal-based depth initialization network trained with self-supervised normal consistency loss provides depth map initialization to the normal-depth refinement module, which utilizes the relationship between illumination and surface normals to refine the frame-wise normal and depth predictions recursively. Our framework's depth accuracy performance on phantom colonoscopy data demonstrates the value of exploiting the surface normals in colonoscopy reconstruction, especially on en face views. Due to its low depth error, the prediction result from our framework will require limited post-processing to be clinically applicable for real-time colonoscopy reconstruction. △ Less

Submitted 13 March, 2023; originally announced March 2023.

Comments: Accepted at IPMI 2023; first two authors contributed equally

Resonances and residue operators for pseudo-Riemannian hyperbolic spaces

Authors: Jan Frahm, Polyxeni Spilioti

Abstract: For any pseudo-Riemannian hyperbolic space $X$ over $\mathbb{R},\mathbb{C},\mathbb{H}$ or $\mathbb{O}$, we show that the resolvent $R(z)=(\Box-z\operatorname{Id})^{-1}$ of the Laplace-Beltrami operator $-\Box$ on $X$ can be extended meromorphically across the spectrum of $\Box$ as a family of operators $C_c^\infty(X)\to \mathcal{D}'(X)$. Its poles are called resonances and we determine them explic… ▽ More For any pseudo-Riemannian hyperbolic space $X$ over $\mathbb{R},\mathbb{C},\mathbb{H}$ or $\mathbb{O}$, we show that the resolvent $R(z)=(\Box-z\operatorname{Id})^{-1}$ of the Laplace-Beltrami operator $-\Box$ on $X$ can be extended meromorphically across the spectrum of $\Box$ as a family of operators $C_c^\infty(X)\to \mathcal{D}'(X)$. Its poles are called resonances and we determine them explicitly in all cases. For each resonance, the image of the corresponding residue operator in $\mathcal{D}'(X)$ forms a representation of the isometry group of $X$, which we identify with a subrepresentation of a degenerate principal series. Our study includes in particular the case of even functions on de Sitter and Anti-de Sitter spaces. For Riemannian symmetric spaces analogous results were obtained by Miatello-Will and Hilgert-Pasquale. The main qualitative differences between the Riemannian and the non-Riemannian setting are that for non-Riemannian spaces the resolvent can have poles of order two, it can have a pole at the branching point of the covering to which $R(z)$ extends, and the residue representations can be infinite-dimensional. △ Less

Submitted 18 August, 2023; v1 submitted 8 March, 2023; originally announced March 2023.

Comments: 23 pages, final published version

MSC Class: Primary: 43A85; Secondary: 22E46; 58J50

Journal ref: J. Math. Pures Appl. 177 (2023), 178-197

A Practical Stereo Depth System for Smart Glasses

Authors: Jialiang Wang, Daniel Scharstein, Akash Bapat, Kevin Blackburn-Matzen, Matthew Yu, Jonathan Lehman, Suhib Alsisan, Yanghan Wang, Sam Tsai, Jan-Michael Frahm, Zijian He, Peter Vajda, Michael F. Cohen, Matt Uyttendaele

Abstract: We present the design of a productionized end-to-end stereo depth sensing system that does pre-processing, online stereo rectification, and stereo depth estimation with a fallback to monocular depth estimation when rectification is unreliable. The output of our depth sensing system is then used in a novel view generation pipeline to create 3D computational photography effects using point-of-view i… ▽ More We present the design of a productionized end-to-end stereo depth sensing system that does pre-processing, online stereo rectification, and stereo depth estimation with a fallback to monocular depth estimation when rectification is unreliable. The output of our depth sensing system is then used in a novel view generation pipeline to create 3D computational photography effects using point-of-view images captured by smart glasses. All these steps are executed on-device on the stringent compute budget of a mobile phone, and because we expect the users can use a wide range of smartphones, our design needs to be general and cannot be dependent on a particular hardware or ML accelerator such as a smartphone GPU. Although each of these steps is well studied, a description of a practical system is still lacking. For such a system, all these steps need to work in tandem with one another and fallback gracefully on failures within the system or less than ideal input data. We show how we handle unforeseen changes to calibration, e.g., due to heat, robustly support depth estimation in the wild, and still abide by the memory and latency constraints required for a smooth user experience. We show that our trained models are fast, and run in less than 1s on a six-year-old Samsung Galaxy S8 phone's CPU. Our models generalize well to unseen data and achieve good results on Middlebury and in-the-wild images captured from the smart glasses. △ Less

Submitted 31 March, 2023; v1 submitted 18 November, 2022; originally announced November 2022.

Comments: Accepted at CVPR2023

Toward Edge-Efficient Dense Predictions with Synergistic Multi-Task Neural Architecture Search

Authors: Thanh Vu, Yanqi Zhou, Chunfeng Wen, Yueqi Li, Jan-Michael Frahm

Abstract: In this work, we propose a novel and scalable solution to address the challenges of develo** efficient dense predictions on edge platforms. Our first key insight is that MultiTask Learning (MTL) and hardware-aware Neural Architecture Search (NAS) can work in synergy to greatly benefit on-device Dense Predictions (DP). Empirical results reveal that the joint learning of the two paradigms is surpr… ▽ More In this work, we propose a novel and scalable solution to address the challenges of develo** efficient dense predictions on edge platforms. Our first key insight is that MultiTask Learning (MTL) and hardware-aware Neural Architecture Search (NAS) can work in synergy to greatly benefit on-device Dense Predictions (DP). Empirical results reveal that the joint learning of the two paradigms is surprisingly effective at improving DP accuracy, achieving superior performance over both the transfer learning of single-task NAS and prior state-of-the-art approaches in MTL, all with just 1/10th of the computation. To the best of our knowledge, our framework, named EDNAS, is the first to successfully leverage the synergistic relationship of NAS and MTL for DP. Our second key insight is that the standard depth training for multi-task DP can cause significant instability and noise to MTL evaluation. Instead, we propose JAReD, an improved, easy-to-adopt Joint Absolute-Relative Depth loss, that reduces up to 88% of the undesired noise while simultaneously boosting accuracy. We conduct extensive evaluations on standard datasets, benchmark against strong baselines and state-of-the-art approaches, as well as provide an analysis of the discovered optimal architectures. △ Less

Submitted 4 October, 2022; originally announced October 2022.

Comments: WACV 2023. 14 pages, 5 figures

Heisenberg parabolically induced representations of Hermitian Lie groups, Part I: Unitarity and subrepresentations

Authors: Jan Frahm, Clemens Weiske, Genkai Zhang

Abstract: For a Hermitian Lie group $G$, we study the family of representations induced from a character of the maximal parabolic subgroup $P=MAN$ whose unipotent radical $N$ is a Heisenberg group. Realizing these representations in the non-compact picture on a space $I(ν)$ of functions on the opposite unipotent radical $\bar{N}$, we apply the Heisenberg group Fourier transform map** functions on… ▽ More For a Hermitian Lie group $G$, we study the family of representations induced from a character of the maximal parabolic subgroup $P=MAN$ whose unipotent radical $N$ is a Heisenberg group. Realizing these representations in the non-compact picture on a space $I(ν)$ of functions on the opposite unipotent radical $\bar{N}$, we apply the Heisenberg group Fourier transform map** functions on $\bar N$ to operators on Fock spaces. The main result is an explicit expression for the Knapp-Stein intertwining operators $I(ν)\to I(-ν)$ on the Fourier transformed side. This gives a new construction of the complementary series and of certain unitarizable subrepresentations at points of reducibility. Further auxiliary results are a Bernstein-Sato identity for the Knapp-Stein kernel on $\bar{N}$ and the decomposition of the metaplectic representation under the non-compact group $M$. △ Less

Submitted 14 April, 2023; v1 submitted 9 September, 2022; originally announced September 2022.

Comments: 44 pages, v2: final published version

MSC Class: 17B15; 17B60; 22D30; 43A80; 43A85

Journal ref: Adv. Math. 422 (2023), 109001

Generalized Laguerre functions and Whittaker vectors for holomorphic discrete series

Authors: Jan Frahm, Gestur Ólafsson, Bent Ørsted

Abstract: We study degenerate Whittaker vectors in scalar type holomorphic discrete series representations of tube type Hermitian Lie groups and their analytic continuation. In four different realizations, the bounded domain picture, the tube domain picture, the $L^2$-model and the Fock model, we find their explicit $K$-type expansions. The coefficients are expressed in terms of the generalized Laguerre fun… ▽ More We study degenerate Whittaker vectors in scalar type holomorphic discrete series representations of tube type Hermitian Lie groups and their analytic continuation. In four different realizations, the bounded domain picture, the tube domain picture, the $L^2$-model and the Fock model, we find their explicit $K$-type expansions. The coefficients are expressed in terms of the generalized Laguerre functions on the corresponding symmetric cone, and we relate the $K$-type expansions to the formula for the generating function of the Laguerre polynomials and to their recurrence relations. △ Less

Submitted 18 August, 2023; v1 submitted 13 July, 2022; originally announced July 2022.

Comments: 19 pages, v2: final published version

MSC Class: Primary 22E46; Secondary 43A85

Journal ref: J. Lie Theory 33 (2023), no. 1, 253-270

Leveraging Disentangled Representations to Improve Vision-Based Keystroke Inference Attacks Under Low Data

Authors: John Lim, Jan-Michael Frahm, Fabian Monrose

Abstract: Keystroke inference attacks are a form of side-channel attacks in which an attacker leverages various techniques to recover a user's keystrokes as she inputs information into some display (e.g., while sending a text message or entering her pin). Typically, these attacks leverage machine learning approaches, but assessing the realism of the threat space has lagged behind the pace of machine learnin… ▽ More Keystroke inference attacks are a form of side-channel attacks in which an attacker leverages various techniques to recover a user's keystrokes as she inputs information into some display (e.g., while sending a text message or entering her pin). Typically, these attacks leverage machine learning approaches, but assessing the realism of the threat space has lagged behind the pace of machine learning advancements, due in-part, to the challenges in curating large real-life datasets. We aim to overcome the challenge of having limited number of real data by introducing a video domain adaptation technique that is able to leverage synthetic data through supervised disentangled learning. Specifically, for a given domain, we decompose the observed data into two factors of variation: Style and Content. Doing so provides four learned representations: real-life style, synthetic style, real-life content and synthetic content. Then, we combine them into feature representations from all combinations of style-content pairings across domains, and train a model on these combined representations to classify the content (i.e., labels) of a given datapoint in the style of another domain. We evaluate our method on real-life data using a variety of metrics to quantify the amount of information an attacker is able to recover. We show that our method prevents our model from overfitting to a small real-life training set, indicating that our method is an effective form of data augmentation, thereby making keystroke inference attacks more practical. △ Less

Submitted 5 April, 2022; originally announced April 2022.

The holomorphic discrete series contribution to the generalized Whittaker Plancherel formula

Authors: Jan Frahm, Gestur Ólafsson, Bent Ørsted

Abstract: For a Hermitian Lie group $G$ of tube type we find the contribution of the holomorphic discrete series to the Plancherel decomposition of the Whittaker space $L^2(G/N,ψ)$, where $N$ is the unipotent radical of the Siegel parabolic subgroup and $ψ$ is a certain non-degenerate unitary character on $N$. The holomorphic discrete series embeddings are constructed in terms of generalized Whittaker vecto… ▽ More For a Hermitian Lie group $G$ of tube type we find the contribution of the holomorphic discrete series to the Plancherel decomposition of the Whittaker space $L^2(G/N,ψ)$, where $N$ is the unipotent radical of the Siegel parabolic subgroup and $ψ$ is a certain non-degenerate unitary character on $N$. The holomorphic discrete series embeddings are constructed in terms of generalized Whittaker vectors for which we find explicit formulas in the bounded domain realization, the tube domain realization and the $L^2$-model of the holomorphic discrete series. Although $L^2(G/N,ψ)$ does not have finite multiplicities in general, the holomorphic discrete series contribution does. Moreover, we obtain an explicit formula for the formal dimensions of the holomorphic discrete series embeddings, and we interpret the holomorphic discrete series contribution to $L^2(G/N,ψ)$ as boundary values of holomorphic functions on a domain $Ξ$ in a complexification $G_{\mathbb{C}}$ of $G$ forming a Hardy type space $\mathcal{H}_2(Ξ,ψ)$. △ Less

Submitted 24 April, 2024; v1 submitted 28 March, 2022; originally announced March 2022.

Comments: 36 pages, final published version

MSC Class: Primary 22E46; Secondary 43A85

Journal ref: J. Funct. Anal. 286 (2024), no. 6, Paper No. 110333, 41 pp

VPFusion: Joint 3D Volume and Pixel-Aligned Feature Fusion for Single and Multi-view 3D Reconstruction

Authors: Jisan Mahmud, Jan-Michael Frahm

Abstract: We introduce a unified single and multi-view neural implicit 3D reconstruction framework VPFusion. VPFusion attains high-quality reconstruction using both - 3D feature volume to capture 3D-structure-aware context, and pixel-aligned image features to capture fine local detail. Existing approaches use RNN, feature pooling, or attention computed independently in each view for multi-view fusion. RNNs… ▽ More We introduce a unified single and multi-view neural implicit 3D reconstruction framework VPFusion. VPFusion attains high-quality reconstruction using both - 3D feature volume to capture 3D-structure-aware context, and pixel-aligned image features to capture fine local detail. Existing approaches use RNN, feature pooling, or attention computed independently in each view for multi-view fusion. RNNs suffer from long-term memory loss and permutation variance, while feature pooling or independently computed attention leads to representation in each view being unaware of other views before the final pooling step. In contrast, we show improved multi-view feature fusion by establishing transformer-based pairwise view association. In particular, we propose a novel interleaved 3D reasoning and pairwise view association architecture for feature volume fusion across different views. Using this structure-aware and multi-view-aware feature volume, we show improved 3D reconstruction performance compared to existing methods. VPFusion improves the reconstruction quality further by also incorporating pixel-aligned local image features to capture fine detail. We verify the effectiveness of VPFusion on the ShapeNet and ModelNet datasets, where we outperform or perform on-par the state-of-the-art single and multi-view 3D shape reconstruction methods. △ Less

Submitted 16 July, 2022; v1 submitted 14 March, 2022; originally announced March 2022.

ColDE: A Depth Estimation Framework for Colonoscopy Reconstruction

Authors: Yubo Zhang, Jan-Michael Frahm, Samuel Ehrenstein, Sarah K. McGill, Julian G. Rosenman, Shuxian Wang, Stephen M. Pizer

Abstract: One of the key elements of reconstructing a 3D mesh from a monocular video is generating every frame's depth map. However, in the application of colonoscopy video reconstruction, producing good-quality depth estimation is challenging. Neural networks can be easily fooled by photometric distractions or fail to capture the complex shape of the colon surface, predicting defective shapes that result i… ▽ More One of the key elements of reconstructing a 3D mesh from a monocular video is generating every frame's depth map. However, in the application of colonoscopy video reconstruction, producing good-quality depth estimation is challenging. Neural networks can be easily fooled by photometric distractions or fail to capture the complex shape of the colon surface, predicting defective shapes that result in broken meshes. Aiming to fundamentally improve the depth estimation quality for colonoscopy 3D reconstruction, in this work we have designed a set of training losses to deal with the special challenges of colonoscopy data. For better training, a set of geometric consistency objectives was developed, using both depth and surface normal information. Also, the classic photometric loss was extended with feature matching to compensate for illumination noise. With the training losses powerful enough, our self-supervised framework named ColDE is able to produce better depth maps of colonoscopy data as compared to the previous work utilizing prior depth knowledge. Used in reconstruction, our network is able to reconstruct good-quality colon meshes in real-time without any post-processing, making it the first to be clinically applicable. △ Less

Submitted 18 November, 2021; originally announced November 2021.

Comments: 13 pages, 5 figures

The twisted Ruelle zeta function on compact hyperbolic orbisurfaces and Reidemeister-Turaev torsion

Authors: Léo Bénard, Jan Frahm, Polyxeni Spilioti

Abstract: Let $X$ be a compact hyperbolic surface with finite order singularities, $X_1$ its unit tangent bundle. We consider the Ruelle zeta function $R(s;ρ)$ associated to a representation $ρ\colonπ_1(X_1)\to\operatorname{GL}(V_ρ)$. If $ρ$ does not factor through $π_1(X)$, we show that the value at $0$ of the Ruelle zeta function equals the sign-refined Reidemeister-Turaev torsion of $(X_1, ρ)$ with respe… ▽ More Let $X$ be a compact hyperbolic surface with finite order singularities, $X_1$ its unit tangent bundle. We consider the Ruelle zeta function $R(s;ρ)$ associated to a representation $ρ\colonπ_1(X_1)\to\operatorname{GL}(V_ρ)$. If $ρ$ does not factor through $π_1(X)$, we show that the value at $0$ of the Ruelle zeta function equals the sign-refined Reidemeister-Turaev torsion of $(X_1, ρ)$ with respect to the Euler structure induced by the geodesic flow and to the natural homology orientation of $X_1$. It generalizes Fried's conjecture to non-unitary representations, and solves the phase and sign ambiguity in the unitary case. We also compute the vanishing order and the leading coefficient of the Ruelle zeta function at $s=0$ when $ρ$ factors through $π_1(X)$. △ Less

Submitted 17 November, 2023; v1 submitted 13 October, 2021; originally announced October 2021.

Comments: 48 pages, final published version

MSC Class: Primary: 11M36; Secondary: 11F72; 37C30; 57Q10

Journal ref: J. Ec. polytech. Math. 10 (2023), 1391-1439

Twisted Ruelle zeta function at zero for compact hyperbolic surfaces

Authors: Jan Frahm, Polyxeni Spilioti

Abstract: Let $X$ be a compact, hyperbolic surface of genus $g\geq 2$. In this paper, we prove that the twisted Selberg and Ruelle zeta functions, associated with an arbitrary, finite-dimensional, complex representation $χ$ of $π_1(X)$ admit a meromorphic continuation to $\mathbb{C}$. Moreover, we study the behaviour of the twisted Ruelle zeta function at $s=0$ and prove that at this point, it has a zero of… ▽ More Let $X$ be a compact, hyperbolic surface of genus $g\geq 2$. In this paper, we prove that the twisted Selberg and Ruelle zeta functions, associated with an arbitrary, finite-dimensional, complex representation $χ$ of $π_1(X)$ admit a meromorphic continuation to $\mathbb{C}$. Moreover, we study the behaviour of the twisted Ruelle zeta function at $s=0$ and prove that at this point, it has a zero of order $\dim(χ)(2g-2)$. △ Less

Submitted 5 October, 2022; v1 submitted 27 May, 2021; originally announced May 2021.

Comments: v2: The functional equation for the twisted Ruelle zeta function is made more explicit

Journal ref: J. Number Theory 243 (2023), 38-61

On the direct integral decomposition in branching laws for real reductive groups

Authors: Jan Frahm

Abstract: The restriction of an irreducible unitary representation $π$ of a real reductive group $G$ to a reductive subgroup $H$ decomposes into a direct integral of irreducible unitary representations $τ$ of $H$ with multiplicities $m(π,τ)\in\mathbb{N}\cup\{\infty\}$. We show that on the smooth vectors of $π$, the direct integral is pointwise defined. This implies that $m(π,τ)$ is bounded above by the dime… ▽ More The restriction of an irreducible unitary representation $π$ of a real reductive group $G$ to a reductive subgroup $H$ decomposes into a direct integral of irreducible unitary representations $τ$ of $H$ with multiplicities $m(π,τ)\in\mathbb{N}\cup\{\infty\}$. We show that on the smooth vectors of $π$, the direct integral is pointwise defined. This implies that $m(π,τ)$ is bounded above by the dimension of the space $\operatorname{Hom}_H(π^\infty|_H,τ^\infty)$ of intertwining operators between the smooth vectors, also called symmetry breaking operators, and provides a precise relation between these two concepts of multiplicity. △ Less

Submitted 26 November, 2021; v1 submitted 16 December, 2020; originally announced December 2020.

Comments: 5 pages

MSC Class: Primary 22E45; Secondary 22E46

Journal ref: J. Lie Theory 32 (2022), no. 1, 191-196

Conformally invariant differential operators on Heisenberg groups and minimal representations

Authors: Jan Frahm

Abstract: For a simple real Lie group $G$ with Heisenberg parabolic subgroup $P$, we study the corresponding degenerate principal series representations. For a certain induction parameter the kernel of the conformally invariant system of second order differential operators constructed by Barchini, Kable and Zierau is a subrepresentation which turns out to be the minimal representation. To study this subrepr… ▽ More For a simple real Lie group $G$ with Heisenberg parabolic subgroup $P$, we study the corresponding degenerate principal series representations. For a certain induction parameter the kernel of the conformally invariant system of second order differential operators constructed by Barchini, Kable and Zierau is a subrepresentation which turns out to be the minimal representation. To study this subrepresentation, we take the Heisenberg group Fourier transform in the non-compact picture and show that it yields a new realization of the minimal representation on a space of $L^2$-functions. The Lie algebra action is given by differential operators of order $\leq3$ and we find explicit formulas for the functions constituting the lowest $K$-type. These $L^2$-models were previously known for the groups $\operatorname{SO}(n,n)$, $E_{6(6)}$, $E_{7(7)}$ and $E_{8(8)}$ by Kazhdan and Savin, for the group $G_{2(2)}$ by Gelfand, and for the group $\widetilde{\operatorname{SL}}(3,\mathbb{R})$ by Torasso, using different methods. Our new approach provides a uniform and systematic treatment of these cases and also constructs new $L^2$-models for $E_{6(2)}$, $E_{7(-5)}$ and $E_{8(-24)}$ for which the minimal representation is a continuation of the quaternionic discrete series, and for the groups $\widetilde{\operatorname{SO}}(p,q)$ with either $p\geq q=3$ or $p,q\geq4$ and $p+q$ even. As a byproduct of our construction, we find an explicit formula for the group action of a non-trivial Weyl group element that, together with the simple action of a parabolic subgroup, generates $G$. △ Less

Submitted 24 April, 2024; v1 submitted 10 December, 2020; originally announced December 2020.

Comments: 140 pages, final published version

MSC Class: Primary 22E45; Secondary 22E46; 35R03; 43A30

Journal ref: Mem. Soc. Math. Fr. (N.S.) (2024), no.180, x+139 pp

Any-Width Networks

Authors: Thanh Vu, Marc Eder, True Price, Jan-Michael Frahm

Abstract: Despite remarkable improvements in speed and accuracy, convolutional neural networks (CNNs) still typically operate as monolithic entities at inference time. This poses a challenge for resource-constrained practical applications, where both computational budgets and performance needs can vary with the situation. To address these constraints, we propose the Any-Width Network (AWN), an adjustable-wi… ▽ More Despite remarkable improvements in speed and accuracy, convolutional neural networks (CNNs) still typically operate as monolithic entities at inference time. This poses a challenge for resource-constrained practical applications, where both computational budgets and performance needs can vary with the situation. To address these constraints, we propose the Any-Width Network (AWN), an adjustable-width CNN architecture and associated training routine that allow for fine-grained control over speed and accuracy during inference. Our key innovation is the use of lower-triangular weight matrices which explicitly address width-varying batch statistics while being naturally suited for multi-width operations. We also show that this design facilitates an efficient training routine based on random width sampling. We empirically demonstrate that our proposed AWNs compare favorably to existing methods while providing maximally granular control during inference. △ Less

Submitted 5 December, 2020; originally announced December 2020.

Comments: 8 pages. Published at CVPR 2020 Workshop on Efficient Deep Learning in Computer Vision. Code at https://github.com/thanhmvu/awn

Revisiting the Threat Space for Vision-based Keystroke Inference Attacks

Authors: John Lim, True Price, Fabian Monrose, Jan-Michael Frahm

Abstract: A vision-based keystroke inference attack is a side-channel attack in which an attacker uses an optical device to record users on their mobile devices and infer their keystrokes. The threat space for these attacks has been studied in the past, but we argue that the defining characteristics for this threat space, namely the strength of the attacker, are outdated. Previous works do not study adversa… ▽ More A vision-based keystroke inference attack is a side-channel attack in which an attacker uses an optical device to record users on their mobile devices and infer their keystrokes. The threat space for these attacks has been studied in the past, but we argue that the defining characteristics for this threat space, namely the strength of the attacker, are outdated. Previous works do not study adversaries with vision systems that have been trained with deep neural networks because these models require large amounts of training data and curating such a dataset is expensive. To address this, we create a large-scale synthetic dataset to simulate the attack scenario for a keystroke inference attack. We show that first pre-training on synthetic data, followed by adopting transfer learning techniques on real-life data, increases the performance of our deep learning models. This indicates that these models are able to learn rich, meaningful representations from our synthetic data and that training on the synthetic data can help overcome the issue of having small, real-life datasets for vision-based key stroke inference attacks. For this work, we focus on single keypress classification where the input is a frame of a keypress and the output is a predicted key. We are able to get an accuracy of 95.6% after pre-training a CNN on our synthetic data and training on a small set of real-life data in an adversarial domain adaptation framework. Source Code for Simulator: https://github.com/jlim13/keystroke-inference-attack-synthetic-dataset-generator- △ Less

Submitted 12 September, 2020; originally announced September 2020.

One Shot 3D Photography

Authors: Johannes Kopf, Kevin Matzen, Suhib Alsisan, Ocean Quigley, Francis Ge, Yangming Chong, Josh Patterson, Jan-Michael Frahm, Shu Wu, Matthew Yu, Peizhao Zhang, Zijian He, Peter Vajda, Ayush Saraf, Michael Cohen

Abstract: 3D photography is a new medium that allows viewers to more fully experience a captured moment. In this work, we refer to a 3D photo as one that displays parallax induced by moving the viewpoint (as opposed to a stereo pair with a fixed viewpoint). 3D photos are static in time, like traditional photos, but are displayed with interactive parallax on mobile or desktop screens, as well as on Virtual R… ▽ More 3D photography is a new medium that allows viewers to more fully experience a captured moment. In this work, we refer to a 3D photo as one that displays parallax induced by moving the viewpoint (as opposed to a stereo pair with a fixed viewpoint). 3D photos are static in time, like traditional photos, but are displayed with interactive parallax on mobile or desktop screens, as well as on Virtual Reality devices, where viewing it also includes stereo. We present an end-to-end system for creating and viewing 3D photos, and the algorithmic and design choices therein. Our 3D photos are captured in a single shot and processed directly on a mobile device. The method starts by estimating depth from the 2D input image using a new monocular depth estimation network that is optimized for mobile devices. It performs competitively to the state-of-the-art, but has lower latency and peak memory consumption and uses an order of magnitude fewer parameters. The resulting depth is lifted to a layered depth image, and new geometry is synthesized in parallax regions. We synthesize color texture and structures in the parallax regions as well, using an inpainting network, also optimized for mobile devices, on the LDI directly. Finally, we convert the result into a mesh-based representation that can be efficiently transmitted and rendered even on low-end devices and over poor network connections. Altogether, the processing takes just a few seconds on a mobile device, and the result can be instantly viewed and shared. We perform extensive quantitative evaluation to validate our system and compare its new components against the current state-of-the-art. △ Less

Submitted 1 September, 2020; v1 submitted 27 August, 2020; originally announced August 2020.

Comments: Project page: https://facebookresearch.github.io/one_shot_3d_photography/ Code: https://github.com/facebookresearch/one_shot_3d_photography

Journal ref: ACM Transactions on Graphics (Proceedings of SIGGRAPH 2020), Volume 39, Number 4, 2020

Reducing Drift in Structure From Motion Using Extended Features

Authors: Aleksander Holynski, David Geraghty, Jan-Michael Frahm, Chris Sweeney, Richard Szeliski

Abstract: Low-frequency long-range errors (drift) are an endemic problem in 3D structure from motion, and can often hamper reasonable reconstructions of the scene. In this paper, we present a method to dramatically reduce scale and positional drift by using extended structural features such as planes and vanishing points. Unlike traditional feature matches, our extended features are able to span non-overlap… ▽ More Low-frequency long-range errors (drift) are an endemic problem in 3D structure from motion, and can often hamper reasonable reconstructions of the scene. In this paper, we present a method to dramatically reduce scale and positional drift by using extended structural features such as planes and vanishing points. Unlike traditional feature matches, our extended features are able to span non-overlap** input images, and hence provide long-range constraints on the scale and shape of the reconstruction. We add these features as additional constraints to a state-of-the-art global structure from motion algorithm and demonstrate that the added constraints enable the reconstruction of particularly drift-prone sequences such as long, low field-of-view videos without inertial measurements. Additionally, we provide an analysis of the drift-reducing capabilities of these constraints by evaluating on a synthetic dataset. Our structural features are able to significantly reduce drift for scenes that contain long-spanning man-made structures, such as aligned rows of windows or planar building facades. △ Less

Submitted 13 October, 2020; v1 submitted 27 August, 2020; originally announced August 2020.

Comments: 3DV 2020

Tangent Images for Mitigating Spherical Distortion

Authors: Marc Eder, Mykhailo Shvets, John Lim, Jan-Michael Frahm

Abstract: In this work, we propose "tangent images," a spherical image representation that facilitates transferable and scalable $360^\circ$ computer vision. Inspired by techniques in cartography and computer graphics, we render a spherical image to a set of distortion-mitigated, locally-planar image grids tangent to a subdivided icosahedron. By varying the resolution of these grids independently of the sub… ▽ More In this work, we propose "tangent images," a spherical image representation that facilitates transferable and scalable $360^\circ$ computer vision. Inspired by techniques in cartography and computer graphics, we render a spherical image to a set of distortion-mitigated, locally-planar image grids tangent to a subdivided icosahedron. By varying the resolution of these grids independently of the subdivision level, we can effectively represent high resolution spherical images while still benefiting from the low-distortion icosahedral spherical approximation. We show that training standard convolutional neural networks on tangent images compares favorably to the many specialized spherical convolutional kernels that have been developed, while also scaling efficiently to handle significantly higher spherical resolutions. Furthermore, because our approach does not require specialized kernels, we show that we can transfer networks trained on perspective images to spherical data without fine-tuning and with limited performance drop-off. Finally, we demonstrate that tangent images can be used to improve the quality of sparse feature detection on spherical images, illustrating its usefulness for traditional computer vision tasks like structure-from-motion and SLAM. △ Less

Submitted 22 May, 2020; v1 submitted 19 December, 2019; originally announced December 2019.

Comments: Updated version of CVPR 2020 publication (9 pages, 13 pages supplementary). Code: https://github.com/meder411/Tangent-Images

ViewSynth: Learning Local Features from Depth using View Synthesis

Authors: Jisan Mahmud, Rajat Vikram Singh, Peri Akiva, Spondon Kundu, Kuan-Chuan Peng, Jan-Michael Frahm

Abstract: The rapid development of inexpensive commodity depth sensors has made keypoint detection and matching in the depth image modality an important problem in computer vision. Despite great improvements in recent RGB local feature learning methods, adapting them directly in the depth modality leads to unsatisfactory performance. Most of these methods do not explicitly reason beyond the visible pixels i… ▽ More The rapid development of inexpensive commodity depth sensors has made keypoint detection and matching in the depth image modality an important problem in computer vision. Despite great improvements in recent RGB local feature learning methods, adapting them directly in the depth modality leads to unsatisfactory performance. Most of these methods do not explicitly reason beyond the visible pixels in the images. To address the limitations of these methods, we propose a framework ViewSynth, to jointly learn: (1) viewpoint invariant keypoint-descriptor from depth images using a proposed Contrastive Matching Loss, and (2) view synthesis of depth images from different viewpoints using the proposed View Synthesis Module and View Synthesis Loss. By learning view synthesis, we explicitly encourage the feature extractor to encode information about not only the visible, but also the occluded parts of the scene. We demonstrate that in the depth modality, ViewSynth outperforms the state-of-the-art depth and RGB local feature extraction techniques in the 3D keypoint matching and camera localization tasks on the RGB-D datasets 7-Scenes, TUM RGBD and CoRBS in most scenarios. We also show the generalizability of ViewSynth in 3D keypoint matching across different datasets. △ Less

Submitted 1 September, 2020; v1 submitted 22 November, 2019; originally announced November 2019.

Comments: Accepted to BMVC 2020

Mapped Convolutions

Authors: Marc Eder, True Price, Thanh Vu, Akash Bapat, Jan-Michael Frahm

Abstract: We present a versatile formulation of the convolution operation that we term a "mapped convolution." The standard convolution operation implicitly samples the pixel grid and computes a weighted sum. Our mapped convolution decouples these two components, freeing the operation from the confines of the image grid and allowing the kernel to process any type of structured data. As a test case, we demon… ▽ More We present a versatile formulation of the convolution operation that we term a "mapped convolution." The standard convolution operation implicitly samples the pixel grid and computes a weighted sum. Our mapped convolution decouples these two components, freeing the operation from the confines of the image grid and allowing the kernel to process any type of structured data. As a test case, we demonstrate its use by applying it to dense inference on spherical data. We perform an in-depth study of existing spherical image convolution methods and propose an improved sampling method for equirectangular images. Then, we discuss the impact of data discretization when deriving a sampling function, highlighting drawbacks of the cube map representation for spherical data. Finally, we illustrate how mapped convolutions enable us to convolve directly on a mesh by projecting the spherical image onto a geodesic grid and training on the textured mesh. This method exceeds the state of the art for spherical depth estimation by nearly 17%. Our findings suggest that mapped convolutions can be instrumental in expanding the application scope of convolutional neural networks. △ Less

Submitted 26 June, 2019; originally announced June 2019.

Convolutions on Spherical Images

Authors: Marc Eder, Jan-Michael Frahm

Abstract: Applying convolutional neural networks to spherical images requires particular considerations. We look to the millennia of work on cartographic map projections to provide the tools to define an optimal representation of spherical images for the convolution operation. We propose a representation for deep spherical image inference based on the icosahedral Snyder equal-area (ISEA) projection, a proje… ▽ More Applying convolutional neural networks to spherical images requires particular considerations. We look to the millennia of work on cartographic map projections to provide the tools to define an optimal representation of spherical images for the convolution operation. We propose a representation for deep spherical image inference based on the icosahedral Snyder equal-area (ISEA) projection, a projection onto a geodesic grid, and show that it vastly exceeds the state-of-the-art for convolution on spherical images, improving semantic segmentation results by 12.6%. △ Less

Submitted 20 May, 2019; originally announced May 2019.

Comments: Oral presentation at the 2019 SUMO Workshop on 360° Indoor Scene Understanding and Modeling at CVPR2019, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2019

Recurrent Neural Network for (Un-)supervised Learning of Monocular VideoVisual Odometry and Depth

Authors: Rui Wang, Stephen M. Pizer, Jan-Michael Frahm

Abstract: Deep learning-based, single-view depth estimation methods have recently shown highly promising results. However, such methods ignore one of the most important features for determining depth in the human vision system, which is motion. We propose a learning-based, multi-view dense depth map and odometry estimation method that uses Recurrent Neural Networks (RNN) and trains utilizing multi-view imag… ▽ More Deep learning-based, single-view depth estimation methods have recently shown highly promising results. However, such methods ignore one of the most important features for determining depth in the human vision system, which is motion. We propose a learning-based, multi-view dense depth map and odometry estimation method that uses Recurrent Neural Networks (RNN) and trains utilizing multi-view image reprojection and forward-backward flow-consistency losses. Our model can be trained in a supervised or even unsupervised mode. It is designed for depth and visual odometry estimation from video where the input frames are temporally correlated. However, it also generalizes to single-view depth estimation. Our method produces superior results to the state-of-the-art approaches for single-view and multi-view learning-based depth estimation on the KITTI driving dataset. △ Less

Submitted 15 April, 2019; originally announced April 2019.

An extension problem related to the fractional Branson-Gover operators

Authors: Jan Frahm, Bent Ørsted, Genkai Zhang

Abstract: The Branson-Gover operators are conformally invariant differential operators of even degree acting on differential forms. They can be interpolated by a holomorphic family of conformally invariant integral operators called fractional Branson-Gover operators. For Euclidean spaces we show that the fractional Branson-Gover operators can be obtained as Dirichlet-to-Neumann operators of certain conforma… ▽ More The Branson-Gover operators are conformally invariant differential operators of even degree acting on differential forms. They can be interpolated by a holomorphic family of conformally invariant integral operators called fractional Branson-Gover operators. For Euclidean spaces we show that the fractional Branson-Gover operators can be obtained as Dirichlet-to-Neumann operators of certain conformally invariant boundary value problems, generalizing the work of Caffarelli-Silvestre for the fractional Laplacians to differential forms. The relevant boundary value problems are studied in detail and we find appropriate Sobolev type spaces in which there exist unique solutions and obtain the explicit integral kernels of the solution operators as well as some of its properties. △ Less

Submitted 5 April, 2019; originally announced April 2019.

Comments: 25 pages

Journal ref: J. Funct. Anal. 278 (2020), no. 5, 108395

Joint T1 and T2 Map** with Tiny Dictionaries and Subspace-Constrained Reconstruction

Authors: Volkert Roeloffs, Martin Uecker, Jens Frahm

Abstract: Purpose: To develop a method that adaptively generates tiny dictionaries for joint T1-T2 map**. Theory: This work breaks the bond between dictionary size and representation accuracy (i) by approximating the Bloch-response manifold by piece-wise linear functions and (ii) by adaptively refining the sampling grid depending on the locally-linear approximation error. Methods: Data acquisition was… ▽ More Purpose: To develop a method that adaptively generates tiny dictionaries for joint T1-T2 map**. Theory: This work breaks the bond between dictionary size and representation accuracy (i) by approximating the Bloch-response manifold by piece-wise linear functions and (ii) by adaptively refining the sampling grid depending on the locally-linear approximation error. Methods: Data acquisition was accomplished with use of an 2D radially sampled Inversion-Recovery Hybrid-State Free Precession sequence. Adaptive dictionaries are generated with different error tolerances and compared to a heuristically designed dictionary. Based on simulation results, tiny dictionaries were used for T1-T2 map** in phantom and in vivo studies. Reconstruction and parameter map** were performed entirely in subspace. Results: All experiments demonstrated excellent agreement between the proposed map** technique and template matching using heuristic dictionaries. Conclusion: Adaptive dictionaries in combination with manifold projection allow to reduce the necessary dictionary sizes by one to two orders of magnitude. △ Less

Submitted 22 December, 2018; originally announced December 2018.

Journal ref: IEEE Transactions on Medical Imaging 2020;39:1008-1014

Symmetry breaking operators for real reductive groups of rank one

Authors: Jan Frahm, Clemens Weiske

Abstract: For a pair of real reductive groups $G'\subset G$ we consider the space ${\rm Hom}_{G'}(π|_{G'},τ)$ of intertwining operators between spherical principal series representations $π$ of $G$ and $τ$ of $G'$, also called \emph{symmetry breaking operators}. Restricting to those pairs $(G,G')$ where ${\rm dim\,Hom}_{G'}(π|_{G'},τ)<\infty$ and $G$ and $G'$ are of real rank one, we classify all symmetry b… ▽ More For a pair of real reductive groups $G'\subset G$ we consider the space ${\rm Hom}_{G'}(π|_{G'},τ)$ of intertwining operators between spherical principal series representations $π$ of $G$ and $τ$ of $G'$, also called \emph{symmetry breaking operators}. Restricting to those pairs $(G,G')$ where ${\rm dim\,Hom}_{G'}(π|_{G'},τ)<\infty$ and $G$ and $G'$ are of real rank one, we classify all symmetry breaking operators explicitly in terms of their distribution kernels. This generalizes previous work by Kobayashi--Speh for $(G,G')=({\rm O}(1,n+1),{\rm O}(1,n))$ to the reductive pairs $$ (G,G') = ({\rm U}(1,n+1;\mathbb{F}),{\rm U}(1,m+1;\mathbb{F})\times F) \qquad \mbox{with $\mathbb{F}=\mathbb{C},\mathbb{H},\mathbb{O}$ and $F<{\rm U}(n-m;\mathbb{F})$.} $$ In most cases, all symmetry breaking operators can be constructed using one meromorphic family of distributions whose poles and residues we describe in detail. In addition to this family, there may occur some sporadic symmetry breaking operators which we determine explicitly. △ Less

Submitted 12 May, 2020; v1 submitted 3 December, 2018; originally announced December 2018.

Comments: 56 pages

MSC Class: 22E45

Journal ref: J. Funct. Anal. 279 (2020), no. 5, 108568

Recurrent Neural Network for Learning DenseDepth and Ego-Motion from Video

Authors: Rui Wang, Jan-Michael Frahm, Stephen M. Pizer

Abstract: Learning-based, single-view depth estimation often generalizes poorly to unseen datasets. While learning-based, two-frame depth estimation solves this problem to some extent by learning to match features across frames, it performs poorly at large depth where the uncertainty is high. There exists few learning-based, multi-view depth estimation methods. In this paper, we present a learning-based, mu… ▽ More Learning-based, single-view depth estimation often generalizes poorly to unseen datasets. While learning-based, two-frame depth estimation solves this problem to some extent by learning to match features across frames, it performs poorly at large depth where the uncertainty is high. There exists few learning-based, multi-view depth estimation methods. In this paper, we present a learning-based, multi-view dense depth map and ego-motion estimation method that uses Recurrent Neural Networks (RNN). Our model is designed for 3D reconstruction from video where the input frames are temporally correlated. It is generalizable to single- or two-view dense depth estimation. Compared to recent single- or two-view CNN-based depth estimation methods, our model leverages more views and achieves more accurate results, especially at large distances. Our method produces superior results to the state-of-the-art learning-based, single- or two-view depth estimation methods on both indoor and outdoor benchmark datasets. We also demonstrate that our method can even work on extremely difficult sequences, such as endoscopic video, where none of the assumptions (static scene, constant lighting, Lambertian reflection, etc.) from traditional 3D reconstruction methods hold. △ Less

Submitted 16 May, 2018; originally announced May 2018.

The Domain Transform Solver

Authors: Akash Bapat, Jan-Michael Frahm

Abstract: We present a framework for edge-aware optimization that is an order of magnitude faster than the state of the art while having comparable performance. Our key insight is that the optimization can be formulated by leveraging properties of the domain transform, a method for edge-aware filtering that defines a distance-preserving 1D map** of the input space. This enables our method to improve perfo… ▽ More We present a framework for edge-aware optimization that is an order of magnitude faster than the state of the art while having comparable performance. Our key insight is that the optimization can be formulated by leveraging properties of the domain transform, a method for edge-aware filtering that defines a distance-preserving 1D map** of the input space. This enables our method to improve performance for a variety of problems including stereo, depth super-resolution, and render from defocus, while kee** the computational complexity linear in the number of pixels. Our method is highly parallelizable and adaptable, and it has demonstrable scalability with respect to image resolution. △ Less

Submitted 11 May, 2018; originally announced May 2018.

Frequency-modulated SSFP with radial sampling and subspace reconstruction: A time-efficient alternative to phase-cycled bSSFP

Authors: Volkert Roeloffs, Sebastian Rosenzweig, H. Christian M. Holme, Martin Uecker, Jens Frahm

Abstract: Purpose: A novel subspace-based reconstruction method for frequency-modulated balanced steady-state free precession (fmSSFP) MRI is presented. In this work, suitable data acquisition schemes, subspace sizes, and efficiencies for banding removal are investigated. Theory and Methods: By combining a fmSSFP MRI sequence with a 3D stack-of-stars trajectory, scan efficiency is maximized as spectral in… ▽ More Purpose: A novel subspace-based reconstruction method for frequency-modulated balanced steady-state free precession (fmSSFP) MRI is presented. In this work, suitable data acquisition schemes, subspace sizes, and efficiencies for banding removal are investigated. Theory and Methods: By combining a fmSSFP MRI sequence with a 3D stack-of-stars trajectory, scan efficiency is maximized as spectral information is obtained without intermediate preparation phases. A memory-efficient reconstruction routine is implemented by introducing the low-frequency Fourier transform as a subspace which allows for the formulation of a convex reconstruction problem. The removal of banding artifacts is investigated by comparing the proposed acquisition and reconstruction technique to phase-cycled bSSFP MRI. Aliasing properties of different undersampling schemes are analyzed and water/fat separation is demonstrated by reweighting the reconstructed subspace coefficients to generate virtual spectral responses in a post-processing step. Results: A simple root-of-sum-of-squares combination of the reconstructed subspace coefficients yields high-SNR images with the characteristic bSSFP contrast but without banding artifacts. Compared to Golden-Angle trajectories, turn-based sampling schemes were superior in minimizing aliasing across reconstructed subspace coefficients. Water/fat separated images of the human knee were obtained by reweighting subspace coefficients. Conclusion: The novel subspace-based fmSSFP MRI technique emerges as a time-efficient alternative to phase-cycled bSSFP. The method does not need intermediate preparation phases, offers high SNR and avoids banding artifacts. Reweighting of the reconstructed subspace coefficients allows for generating virtual spectral responses with applications to water/fat separation. △ Less

Submitted 25 October, 2018; v1 submitted 16 March, 2018; originally announced March 2018.

Journal ref: Magnetic Resonance in Medicine 81:1566-1579 (2019)

Holomorphic torsion and geometric zeta functions for certain Hermitian locally symmetric manifolds

Authors: Henri Moscovici, Robert J. Stanton, Jan Frahm

Abstract: We give a dynamical description, in terms of a Weil-type zeta function, to the holomorphic torsion with coefficients for certain compact Hermitian locally symmetric manifolds, whose connected group G of isometries of the universal cover has only one conjugacy class of cuspidal maximal parabolic subgroup and satisfies a technical Ansatz relative to the given coefficients. A distinguishing feature o… ▽ More We give a dynamical description, in terms of a Weil-type zeta function, to the holomorphic torsion with coefficients for certain compact Hermitian locally symmetric manifolds, whose connected group G of isometries of the universal cover has only one conjugacy class of cuspidal maximal parabolic subgroup and satisfies a technical Ansatz relative to the given coefficients. A distinguishing feature of our zeta function is that its construction involves in an essential way the geometry of a standard compactification of the universal cover. The two senior authors are indebted to their junior colleague, Jan Frahm, for his laborious work shedding light on the scope of the validity of the Ansatz, and for writing up the attached Appendix. The results therein show that for real rank one groups G the Ansatz is satisfied with respect to any coefficients, for some rank two groups G it is satisfied with respect to certain coefficients, and also that there are groups G which do not obey the Ansatz. △ Less

Submitted 3 April, 2021; v1 submitted 24 February, 2018; originally announced February 2018.

Comments: Appendix by Jan Frahm. Revisions were made in Sections 2 and 3; clarifying material was added to Sections 4--9; Section 10 is new

Symmetry breaking operators for line bundles over real projective spaces

Authors: Jan Frahm, Clemens Weiske

Abstract: The space of smooth sections of an equivariant line bundle over the real projective space $\mathbb{R}{\rm P}^n$ forms a natural representation of the group ${\rm GL}(n+1,\mathbb{R})$. We explicitly construct and classify all intertwining operators between such representations of ${\rm GL}(n+1,\mathbb{R})$ and its subgroup ${\rm GL}(n,\mathbb{R})$, intertwining for the subgroup. Intertwining operat… ▽ More The space of smooth sections of an equivariant line bundle over the real projective space $\mathbb{R}{\rm P}^n$ forms a natural representation of the group ${\rm GL}(n+1,\mathbb{R})$. We explicitly construct and classify all intertwining operators between such representations of ${\rm GL}(n+1,\mathbb{R})$ and its subgroup ${\rm GL}(n,\mathbb{R})$, intertwining for the subgroup. Intertwining operators of this form are called symmetry breaking operators, and they describe the occurrence of a representation of ${\rm GL}(n,\mathbb{R})$ inside the restriction of a representation of ${\rm GL}(n+1,\mathbb{R})$. In this way, our results contribute to the study of branching problems for the real reductive pair $({\rm GL}(n+1,\mathbb{R}),{\rm GL}(n,\mathbb{R}))$. The analogous classification is carried out for intertwining operators between algebraic sections of line bundles, where the Lie group action of ${\rm GL}(n,\mathbb{R})$ is replaced by the action of its Lie algebra $\mathfrak{gl}(n,\mathbb{R})$, and it turns out that all intertwining operators arise as restrictions of operators between smooth sections. △ Less

Submitted 18 December, 2017; originally announced December 2017.

Comments: 44 pages

MSC Class: Primary 22E46; Secondary 17B15; 46F12

Journal ref: J. Lie Theory 29 (2019), no. 2, 511-558

A minimal representation of the orthosymplectic Lie supergroup

Authors: Sigiswald Barbier, Jan Frahm

Abstract: We construct a minimal representation of the orthosymplectic Lie supergroup $OSp(p,q|2n)$, generalising the Schrödinger model of the minimal representation of $O(p,q)$ to the super case. The underlying Lie algebra representation is realized on functions on the minimal orbit inside the Jordan superalgebra associated with $\mathfrak{osp}(p,q|2n)$, so that our construction is in line with the orbit p… ▽ More We construct a minimal representation of the orthosymplectic Lie supergroup $OSp(p,q|2n)$, generalising the Schrödinger model of the minimal representation of $O(p,q)$ to the super case. The underlying Lie algebra representation is realized on functions on the minimal orbit inside the Jordan superalgebra associated with $\mathfrak{osp}(p,q|2n)$, so that our construction is in line with the orbit philosophy. Its annihilator is given by a Joseph-like ideal for $\mathfrak{osp}(p,q|2n)$, and therefore the representation is a natural generalization of a minimal representations to the context of Lie superalgebras. We also calculate its Gelfand--Kirillov dimension and construct a non-degenerate sesquilinear form for which the representation is skew-symmetric and which is the analogue of an $L^2$-inner product in the supercase. △ Less

Submitted 3 September, 2018; v1 submitted 19 October, 2017; originally announced October 2017.

Comments: 45 pages

MSC Class: 17B10; 17B60; 22E46; 58C50

Journal ref: Int. Math. Res. Not. IMRN 2017 (2017), no. 21, 16357-16420

Rankin-Selberg periods for spherical principal series

Authors: Jan Frahm, Feng Su

Abstract: By the unfolding method, Rankin-Selberg L-functions for ${\rm GL}(n)\times{\rm GL}(m)$ can be expressed in terms of period integrals. These period integrals actually define invariant forms on tensor products of the relevant automorphic representations. By the multiplicity-one theorems due to Sun-Zhu and Chen-Sun such invariant forms are unique up to scalar multiples and can therefore be related to… ▽ More By the unfolding method, Rankin-Selberg L-functions for ${\rm GL}(n)\times{\rm GL}(m)$ can be expressed in terms of period integrals. These period integrals actually define invariant forms on tensor products of the relevant automorphic representations. By the multiplicity-one theorems due to Sun-Zhu and Chen-Sun such invariant forms are unique up to scalar multiples and can therefore be related to invariant forms on equivalent principal series representations. We construct meromorphic families of such invariant forms for spherical principal series representations of ${\rm GL}(n,\mathbb{R})$ and conjecture that their special values at the spherical vectors agree in absolute value with the archimedean local L-factors of the corresponding L-functions. We verify this conjecture in several cases. This work can be viewed as the first of two steps in a technique due to Bernstein-Reznikov for estimating L-functions using their period integral expressions. △ Less

Submitted 5 October, 2022; v1 submitted 16 June, 2017; originally announced June 2017.

Comments: 25 pages. v4 is the final published version

MSC Class: Primary 11F70; Secondary 22E46; 53C35

Journal ref: Manuscripta Math. 168 (2022), no. 1-2, 1-33

Symmetry breaking operators for strongly spherical reductive pairs

Authors: Jan Frahm

Abstract: A real reductive pair $(G,H)$ is called strongly spherical if the homogeneous space $(G\times H)/{\rm diag}(H)$ is real spherical. This geometric condition is equivalent to the representation theoretic property that ${\rm dim\,Hom}_H(π|_H,τ)<\infty$ for all smooth admissible representations $π$ of $G$ and $τ$ of $H$. In this paper we explicitly construct for all strongly spherical pairs $(G,H)$ in… ▽ More A real reductive pair $(G,H)$ is called strongly spherical if the homogeneous space $(G\times H)/{\rm diag}(H)$ is real spherical. This geometric condition is equivalent to the representation theoretic property that ${\rm dim\,Hom}_H(π|_H,τ)<\infty$ for all smooth admissible representations $π$ of $G$ and $τ$ of $H$. In this paper we explicitly construct for all strongly spherical pairs $(G,H)$ intertwining operators in ${\rm Hom}_H(π|_H,τ)$ for $π$ and $τ$ spherical principal series representations of $G$ and $H$. These so-called symmetry breaking operators depend holomorphically on the induction parameters and we further show that they generically span the space ${\rm Hom}_H(π|_H,τ)$. In the special case of multiplicity one pairs we extend our construction to vector-valued principal series representations and obtain generic formulas for the multiplicities between arbitrary principal series. As an application, we prove an early version of the Gross-Prasad conjecture for complex orthogonal groups, and also provide lower bounds for the dimension of the space of Shintani functions. △ Less

Submitted 11 October, 2023; v1 submitted 17 May, 2017; originally announced May 2017.

Comments: 58 pages, v2: final published version

MSC Class: 22E46 (Primary); 11F70; 53C30 (Secondary)

Journal ref: Publ. Res. Inst. Math. Sci. 59 (2023), no. 2, 259-337

Simultaneous Multi-Slice MRI using Cartesian and Radial FLASH and Regularized Nonlinear Inversion: SMS-NLINV

Authors: Sebastian Rosenzweig, H. Christian M. Holme, Robin N. Wilke, Dirk Voit, Jens Frahm, Martin Uecker

Abstract: Purpose: The development of a calibrationless parallel imaging method for accelerated simultaneous multi-slice (SMS) MRI based on Regularized Nonlinear Inversion (NLINV), evaluated using Cartesian and radial FLASH. Theory and Methods: NLINV is a parallel imaging method that jointly estimates image content and coil sensitivities using a Newton-type method with regularization. Here, NLINV is extende… ▽ More Purpose: The development of a calibrationless parallel imaging method for accelerated simultaneous multi-slice (SMS) MRI based on Regularized Nonlinear Inversion (NLINV), evaluated using Cartesian and radial FLASH. Theory and Methods: NLINV is a parallel imaging method that jointly estimates image content and coil sensitivities using a Newton-type method with regularization. Here, NLINV is extended to SMS-NLINV for reconstruction and separation of all simultaneously acquired slices. The performance of the extended method is evaluated for different sampling schemes using phantom and in-vivo experiments based on Cartesian and radial SMS-FLASH sequences. Results: The basic algorithm was validated in Cartesian experiments by comparison with ESPIRiT. For Cartesian and radial sampling, improved results are demonstrated compared to single-slice experiments, and it is further shown that sampling schemes using complementary samples outperform schemes with the same samples in each partition. Conclusion: The extension of the NLINV algorithm for SMS data was implemented and successfully demonstrated in combination with a Cartesian and radial SMS-FLASH sequence. △ Less

Submitted 3 August, 2017; v1 submitted 11 May, 2017; originally announced May 2017.

Comments: Part of this work has been presented at the ISMRM Annual Conference 2016 (Singapore) and 2017 (Honolulu). 25 pages, 8+4 figures

Journal ref: Magnetic Resonance in Medicine 79:2057-2066 (2018)

Knapp-Stein Type Intertwining Operators for Symmetric Pairs II. -- The Translation Principle and Intertwining Operators for Spinors

Authors: Jan Frahm, Bent Ørsted

Abstract: For a symmetric pair $(G,H)$ of reductive groups we extend to a large class of generalized principal series representations our previous construction of meromorphic families of symmetry breaking operators. These operators intertwine between a possibly vector-valued principal series of $G$ and one for $H$ and are given explicitly in terms of their integral kernels. As an application we give a compl… ▽ More For a symmetric pair $(G,H)$ of reductive groups we extend to a large class of generalized principal series representations our previous construction of meromorphic families of symmetry breaking operators. These operators intertwine between a possibly vector-valued principal series of $G$ and one for $H$ and are given explicitly in terms of their integral kernels. As an application we give a complete classification of symmetry breaking operators from spinors on a Euclidean space to spinors on a hyperplane, intertwining for a double cover of the conformal group of the hyperplane. △ Less

Submitted 11 November, 2019; v1 submitted 8 February, 2017; originally announced February 2017.

MSC Class: 22E45 (Primary); 47G10 (Secondary)

Journal ref: SIGMA 15 (2019), 084, 50 pages

Accelerated Computing in Magnetic Resonance Imaging -- Real-Time Imaging Using Non-Linear Inverse Reconstruction

Authors: Sebastian Schaetz, Dirk Voit, Jens Frahm, Martin Uecker

Abstract: Purpose: To develop generic optimization strategies for image reconstruction using graphical processing units (GPUs) in magnetic resonance imaging (MRI) and to exemplarily report about our experience with a highly accelerated implementation of the non-linear inversion algorithm (NLINV) for dynamic MRI with high frame rates. Methods: The NLINV algorithm is optimized and ported to run on an a multi-… ▽ More Purpose: To develop generic optimization strategies for image reconstruction using graphical processing units (GPUs) in magnetic resonance imaging (MRI) and to exemplarily report about our experience with a highly accelerated implementation of the non-linear inversion algorithm (NLINV) for dynamic MRI with high frame rates. Methods: The NLINV algorithm is optimized and ported to run on an a multi-GPU single-node server. The algorithm is mapped to multiple GPUs by decomposing the data domain along the channel dimension. Furthermore, the algorithm is decomposed along the temporal domain by relaxing a temporal regularization constraint, allowing the algorithm to work on multiple frames in parallel. Finally, an autotuning method is presented that is capable of combining different decomposition variants to achieve optimal algorithm performance in different imaging scenarios. Results: The algorithm is successfully ported to a multi-GPU system and allows online image reconstruction with high frame rates. Real-time reconstruction with low latency and frame rates up to 30 frames per second is demonstrated. Conclusion: Novel parallel decomposition methods are presented which are applicable to many iterative algorithms for dynamic MRI. Using these methods to parallelize the NLINV algorithm on multiple GPUs it is possible to achieve online image reconstruction with high frame rates. △ Less

Submitted 8 November, 2017; v1 submitted 29 January, 2017; originally announced January 2017.

Comments: 22 pages, 8 figures, 6 tables

Journal ref: Computational and Mathematical Methods in Medicine, 2017:3527269 (2017)

Self-expressive Dictionary Learning for Dynamic 3D Reconstruction

Authors: Enliang Zheng, Dinghuang Ji, Enrique Dunn, Jan-Michael Frahm

Abstract: We target the problem of sparse 3D reconstruction of dynamic objects observed by multiple unsynchronized video cameras with unknown temporal overlap. To this end, we develop a framework to recover the unknown structure without sequencing information across video sequences. Our proposed compressed sensing framework poses the estimation of 3D structure as the problem of dictionary learning, where th… ▽ More We target the problem of sparse 3D reconstruction of dynamic objects observed by multiple unsynchronized video cameras with unknown temporal overlap. To this end, we develop a framework to recover the unknown structure without sequencing information across video sequences. Our proposed compressed sensing framework poses the estimation of 3D structure as the problem of dictionary learning, where the dictionary is defined as an aggregation of the temporally varying 3D structures. Given the smooth motion of dynamic objects, we observe any element in the dictionary can be well approximated by a sparse linear combination of other elements in the same dictionary (i. e. self-expression). Moreover, the sparse coefficients describing a locally linear 3D structural interpolation reveal the local sequencing information. Our formulation optimizes a biconvex cost function that leverages a compressed sensing formulation and enforces both structural dependency coherence across video streams, as well as motion smoothness across estimates from common video sources. We further analyze the reconstructability of our approach under different capture scenarios, and its comparison and relation to existing methods. Experimental results on large amounts of synthetic data as well as real imagery demonstrate the effectiveness of our approach. △ Less

Submitted 22 May, 2016; originally announced May 2016.

Comments: 15 pages, journal