Skip to main content

Showing 1–26 of 26 results for author: Saratchandran, H

.
  1. arXiv:2403.19243  [pdf, other

    cs.LG cs.CV cs.NE

    Sine Activated Low-Rank Matrices for Parameter Efficient Learning

    Authors: Yi** Ji, Hemanth Saratchandran, Cameron Gordon, Zeyu Zhang, Simon Lucey

    Abstract: Low-rank decomposition has emerged as a vital tool for enhancing parameter efficiency in neural network architectures, gaining traction across diverse applications in machine learning. These techniques significantly lower the number of parameters, striking a balance between compactness and performance. However, a common challenge has been the compromise between parameter efficiency and the accurac… ▽ More

    Submitted 28 March, 2024; originally announced March 2024.

    Comments: The first two authors contributed equally

  2. arXiv:2403.19205  [pdf, other

    cs.CV cs.LG

    From Activation to Initialization: Scaling Insights for Optimizing Neural Fields

    Authors: Hemanth Saratchandran, Sameera Ramasinghe, Simon Lucey

    Abstract: In the realm of computer vision, Neural Fields have gained prominence as a contemporary tool harnessing neural networks for signal representation. Despite the remarkable progress in adapting these networks to solve a variety of problems, the field still lacks a comprehensive theoretical framework. This article aims to address this gap by delving into the intricate interplay between initialization… ▽ More

    Submitted 28 March, 2024; originally announced March 2024.

    Comments: CVPR 2024

  3. arXiv:2403.19163  [pdf, other

    cs.LG cs.CV

    D'OH: Decoder-Only random Hypernetworks for Implicit Neural Representations

    Authors: Cameron Gordon, Lachlan Ewen MacDonald, Hemanth Saratchandran, Simon Lucey

    Abstract: Deep implicit functions have been found to be an effective tool for efficiently encoding all manner of natural signals. Their attractiveness stems from their ability to compactly represent signals with little to no off-line training data. Instead, they leverage the implicit bias of deep networks to decouple hidden redundancies within the signal. In this paper, we explore the hypothesis that additi… ▽ More

    Submitted 28 March, 2024; originally announced March 2024.

    Comments: 29 pages, 17 figures

  4. arXiv:2402.08784  [pdf, other

    cs.CV cs.LG

    Preconditioners for the Stochastic Training of Implicit Neural Representations

    Authors: Shin-Fang Chng, Hemanth Saratchandran, Simon Lucey

    Abstract: Implicit neural representations have emerged as a powerful technique for encoding complex continuous multidimensional signals as neural networks, enabling a wide range of applications in computer vision, robotics, and geometry. While Adam is commonly used for training due to its stochastic proficiency, it entails lengthy training durations. To address this, we explore alternative optimization tech… ▽ More

    Submitted 13 February, 2024; originally announced February 2024.

    Comments: The first two authors contributed equally

  5. arXiv:2402.05427  [pdf, other

    cs.LG

    A Sampling Theory Perspective on Activations for Implicit Neural Representations

    Authors: Hemanth Saratchandran, Sameera Ramasinghe, Violetta Shevchenko, Alexander Long, Simon Lucey

    Abstract: Implicit Neural Representations (INRs) have gained popularity for encoding signals as compact, differentiable entities. While commonly using techniques like Fourier positional encodings or non-traditional activation functions (e.g., Gaussian, sinusoid, or wavelets) to capture high-frequency content, their properties lack exploration within a unified theoretical framework. Addressing this gap, we c… ▽ More

    Submitted 8 February, 2024; originally announced February 2024.

  6. arXiv:2402.04783  [pdf, other

    cs.LG

    Analyzing the Neural Tangent Kernel of Periodically Activated Coordinate Networks

    Authors: Hemanth Saratchandran, Shin-Fang Chng, Simon Lucey

    Abstract: Recently, neural networks utilizing periodic activation functions have been proven to demonstrate superior performance in vision tasks compared to traditional ReLU-activated networks. However, there is still a limited understanding of the underlying reasons for this improved performance. In this paper, we aim to address this gap by providing a theoretical understanding of periodically activated ne… ▽ More

    Submitted 7 February, 2024; originally announced February 2024.

    Comments: arXiv admin note: substantial text overlap with arXiv:2402.02711

  7. arXiv:2402.02711  [pdf, other

    cs.LG

    Architectural Strategies for the optimization of Physics-Informed Neural Networks

    Authors: Hemanth Saratchandran, Shin-Fang Chng, Simon Lucey

    Abstract: Physics-informed neural networks (PINNs) offer a promising avenue for tackling both forward and inverse problems in partial differential equations (PDEs) by incorporating deep learning with fundamental physics principles. Despite their remarkable empirical success, PINNs have garnered a reputation for their notorious training challenges across a spectrum of PDEs. In this work, we delve into the in… ▽ More

    Submitted 4 February, 2024; originally announced February 2024.

  8. arXiv:2305.08552  [pdf, other

    cs.CV

    Curvature-Aware Training for Coordinate Networks

    Authors: Hemanth Saratchandran, Shin-Fang Chng, Sameera Ramasinghe, Lachlan MacDonald, Simon Lucey

    Abstract: Coordinate networks are widely used in computer vision due to their ability to represent signals as compressed, continuous entities. However, training these networks with first-order optimizers can be slow, hindering their use in real-time applications. Recent works have opted for shallow voxel-based representations to achieve faster training, but this sacrifices memory efficiency. This work propo… ▽ More

    Submitted 15 May, 2023; originally announced May 2023.

  9. arXiv:2303.05728  [pdf, other

    cs.LG eess.SP

    On the effectiveness of neural priors in modeling dynamical systems

    Authors: Sameera Ramasinghe, Hemanth Saratchandran, Violetta Shevchenko, Simon Lucey

    Abstract: Modelling dynamical systems is an integral component for understanding the natural world. To this end, neural networks are becoming an increasingly popular candidate owing to their ability to learn complex functions from large amounts of data. Despite this recent progress, there has not been an adequate discussion on the architectural regularization that neural networks offer when learning such sy… ▽ More

    Submitted 10 March, 2023; originally announced March 2023.

  10. arXiv:2303.00312  [pdf, ps, other

    math.DG

    A Ruelle dynamical zeta function for equivariant flows

    Authors: Peter Hochs, Hemanth Saratchandran

    Abstract: For proper group actions on smooth manifolds, with compact quotients, we define an equivariant version of the Ruelle dynamical $ζ$-function for equivariant flows satisfying a nondegeneracy condition. The construction is guided by an equivariant generalisation of Guillemin's trace formula. This involves an equivariant generalisation of the distributional flat trace appearing in Guillemin's formula.… ▽ More

    Submitted 15 March, 2023; v1 submitted 1 March, 2023; originally announced March 2023.

  11. arXiv:2210.05371  [pdf, other

    cs.LG

    On skip connections and normalisation layers in deep optimisation

    Authors: Lachlan Ewen MacDonald, Jack Valmadre, Hemanth Saratchandran, Simon Lucey

    Abstract: We introduce a general theoretical framework, designed for the study of gradient optimisation of deep neural networks, that encompasses ubiquitous architecture choices including batch normalisation, weight normalisation and skip connections. Our framework determines the curvature and regularity properties of multilayer loss landscapes in terms of their constituent layers, thereby elucidating the r… ▽ More

    Submitted 4 December, 2023; v1 submitted 10 October, 2022; originally announced October 2022.

    Comments: NeurIPS 2023

  12. arXiv:2206.08558  [pdf, other

    cs.LG

    How You Start Matters for Generalization

    Authors: Sameera Ramasinghe, Lachlan MacDonald, Moshiur Farazi, Hemanth Saratchandran, Simon Lucey

    Abstract: Characterizing the remarkable generalization properties of over-parameterized neural networks remains an open problem. In this paper, we promote a shift of focus towards initialization rather than neural architecture or (stochastic) gradient descent to explain this implicit regularization. Through a Fourier lens, we derive a general result for the spectral bias of neural networks and show that the… ▽ More

    Submitted 10 July, 2022; v1 submitted 17 June, 2022; originally announced June 2022.

  13. arXiv:2205.04117  [pdf, ps, other

    math.DG

    Equivariant analytic torsion for proper actions

    Authors: Peter Hochs, Hemanth Saratchandran

    Abstract: We construct an equivariant version of Ray-Singer analytic torsion for proper, isometric actions by locally compact groups on Riemannian manifolds, with compact quotients. We obtain results on convergence, metric independence, vanishing for even-dimensional manifolds, a product formula, and a decomposition of classical Ray-Singer analytic torsion as a sum over conjugacy classes of equivariant tors… ▽ More

    Submitted 29 June, 2023; v1 submitted 9 May, 2022; originally announced May 2022.

    Comments: 62 pages; minor corrections

  14. arXiv:2201.02826  [pdf, ps, other

    math.DG

    A new higher order Yang--Mills--Higgs flow in Riemannian $4$-manifold

    Authors: Hemanth Saratchandran, Jiaogen Zhang, Pan Zhang

    Abstract: Let $(M,g)$ be a closed Riemannian $4$-manifold and let $E$ be a vector bundle over $M$ with structure group $G$, where $G$ is a compact Lie group. In this paper, we consider a new higher order Yang--Mills--Higgs functional, in which the Higgs field is a section of $Ω^0(\textmd{ad}E)$. We show that, under suitable conditions, solutions to the gradient flow do not hit any finite time singularities.… ▽ More

    Submitted 10 May, 2022; v1 submitted 8 January, 2022; originally announced January 2022.

    Comments: arXiv admin note: substantial text overlap with arXiv:2004.00420

  15. arXiv:2107.03301  [pdf, ps, other

    math.AP math.DG math.FA

    Generalized Ornstein--Uhlenbeck Semigroups in weighted $L^p$-spaces on Riemannian Manifolds

    Authors: Ognjen Milatovic, Hemanth Saratchandran

    Abstract: Let $\mathcal{E}$ be a Hermitian vector bundle over a Riemannian manifold $M$ with metric $g$, let $\nabla$ be a metric covariant derivative on $\mathcal{E}$. We study the generalized Ornstein-Uhlenbeck differential expression $P^{\nabla}=\nabla^{\dagger}\nabla u+\nabla_{(dφ)^{\sharp}}u-\nabla_{X}u+Vu$, where $\nabla^{\dagger}$ is the formal adjoint of $\nabla$, $(dφ)^{\sharp}$ is the vector field… ▽ More

    Submitted 5 July, 2021; originally announced July 2021.

  16. arXiv:2104.01919  [pdf, ps, other

    math.AP math.DG math.FA math.SP

    Realisations of elliptic operators on compact manifolds with boundary

    Authors: Lashi Bandara, Magnus Goffeng, Hemanth Saratchandran

    Abstract: This paper investigates realisations of elliptic differential operators of general order on manifolds with boundary following the approach of Bär-Ballmann to first order elliptic operators. The space of possible boundary values of elements in the maximal domain is described as a Hilbert space densely sandwiched between two mixed order Sobolev spaces. The description uses Calderón projectors which,… ▽ More

    Submitted 22 July, 2021; v1 submitted 5 April, 2021; originally announced April 2021.

    MSC Class: 35J58; 35J56; 58J05; 58J32

    Journal ref: Advances in Mathematics 420 (2023)

  17. arXiv:2007.02507  [pdf, other

    math.DG hep-th math-ph math.KT

    On the Chern character in Higher Twisted K-theory and spherical T-duality

    Authors: Lachlan Macdonald, Varghese Mathai, Hemanth Saratchandran

    Abstract: In this paper, we construct for higher twists that arise from cohomotopy classes, the Chern character in higher twisted K-theory, that maps into higher twisted cohomology. We show that it gives rise to an isomorphism between higher twisted K-theory and higher twisted cohomology over the reals. Finally we compute spherical T-duality in higher twisted K-theory and higher twisted cohomology in very g… ▽ More

    Submitted 19 February, 2021; v1 submitted 5 July, 2020; originally announced July 2020.

    Comments: 40 pages, revised

    MSC Class: 81T30

    Journal ref: Commun. Math. Phys. 385, 331-368 (2021)

  18. arXiv:2003.07697  [pdf, ps, other

    math.AP

    Essential self-adjointness of perturbed biharmonic operators via conformally transformed metrics

    Authors: Ognjen Milatovic, Hemanth Saratchandran

    Abstract: We give sufficient conditions for the essential self-adjointness of perturbed biharmonic operators acting on sections of a Hermitian vector bundle over a Riemannian manifold with additional assumptions, such as lower semi-bounded Ricci curvature or bounded sectional curvature. In the case of lower semi-bounded Ricci curvature, we formulate our results in terms of the completeness of the metric tha… ▽ More

    Submitted 14 March, 2020; originally announced March 2020.

  19. arXiv:1904.07210  [pdf, ps, other

    math.DG math.AP math.FA math.SP

    Essential self-adjointness of perturbed quadharmonic operators on Riemannian manifolds with an application to the separation problem

    Authors: Hemanth Saratchandran

    Abstract: We consider perturbed quadharmonic operators, $Δ^4 + V$, acting on sections of a Hermitian vector bundle over a complete Riemannian manifold, with the potential $V$ satisfying a bound from below by a non-positive function depending on the distance from a point. Under a bounded geometry assumption on the Hermitian vector bundle and the underlying Riemannian manifold, we give a sufficient condition… ▽ More

    Submitted 11 April, 2019; originally announced April 2019.

  20. Inequalities and separation for covariant Schrödinger operators

    Authors: Ognjen Milatovic, Hemanth Saratchandran

    Abstract: We consider a differential expression $L^{\nabla}_{V}=\nabla^{\dagger}\nabla+V$, where $\nabla$ is a metric covariant derivative on a Hermitian bundle $E$ over a geodesically complete Riemannian manifold $(M,g)$ with metric $g$, and $V$ is a linear self-adjoint bundle map on $E$. In the language of Everitt and Giertz, the differential expression $L^{\nabla}_{V}$ is said to be separated in… ▽ More

    Submitted 15 May, 2018; originally announced May 2018.

  21. arXiv:1802.08573  [pdf, ps, other

    math.DG

    Higher order Seiberg-Witten functionals and their associated gradient flows

    Authors: Hemanth Saratchandran

    Abstract: We define functionals generalising the Seiberg-Witten functional on closed $spin^c$ manifolds, involving higher order derivatives of the curvature form and spinor field. We then consider their associated gradient flows and, using a gauge fixing technique, are able to prove short time existence for the flows. We then prove energy estimates along the flow, and establish local $L^2$-derivative estima… ▽ More

    Submitted 22 February, 2018; originally announced February 2018.

  22. arXiv:1606.06190  [pdf, other

    math.FA math.AP math.DG math.SP

    Essential self-adjointness of powers of first-order differential operators on non-compact manifolds with low-regularity metrics

    Authors: Lashi Bandara, Hemanth Saratchandran

    Abstract: We consider first-order differential operators with locally bounded measurable coefficients on vector bundles with measurable coefficient metrics. Under a mild set of assumptions, we demonstrate the equivalence between the essential self-adjointness of such operators to a negligible boundary property. When the operator possesses higher regularity coefficients, we show that higher powers are essent… ▽ More

    Submitted 13 September, 2017; v1 submitted 20 June, 2016; originally announced June 2016.

    MSC Class: 58J60; 47F05; 46E40; 58J05

    Journal ref: J. Funct. Anal., Volume 273, Issue 12, 2017, Pages 3719-3758

  23. arXiv:1504.06703  [pdf, other

    math.GT

    Complements of tori in $\#_{2k}S^2 \times S^2$ that admit a hyperbolic structure

    Authors: Hemanth Saratchandran

    Abstract: We construct examples of codimension two hyperbolic link complements in closed smooth 4-manifolds with homeomorphism type $\#_{2k}S^2 \times S^2$. All our examples are based on a construction of J. Ratcliffe and S. Tschantz, who constructed 1171 non-compact finite volume hyperbolic 4-manifolds of minimal volume. We then give necessary conditions for a closed smooth simply connected 4-manifold to c… ▽ More

    Submitted 25 April, 2015; originally announced April 2015.

    Comments: 26 pages

  24. arXiv:1504.01366  [pdf, other

    math.GT

    A four dimensional hyperbolic link complement in a standard $S^2 \times S^2$

    Authors: Hemanth Saratchandran

    Abstract: Using techniques from the theory of Kirby calculus we give an explicit construction of a four dimensional hyperbolic link complement in a 4-manifold that is diffeomorphic to a standard $S^2 \times S^2$.

    Submitted 2 April, 2015; originally announced April 2015.

    Comments: 43 pages

  25. arXiv:1503.07778  [pdf, other

    math.GT

    A four dimensional hyperbolic link complement in a standard $S^4$

    Authors: Hemanth Saratchandran

    Abstract: Using techniques from the theory of Kirby calculus we give an explicit construction of a four dimensional hyperbolic link complement in a 4-manifold that is diffeomorphic to the standard 4-sphere.

    Submitted 26 March, 2015; originally announced March 2015.

    Comments: 76 pages

  26. arXiv:1503.06722  [pdf, other

    math.GT

    Kirby diagrams and the Ratcliffe-Tschantz hyperbolic 4-manifolds

    Authors: Hemanth Saratchandran

    Abstract: We show how to construct a Kirby diagram for a large class of finite volume hyperbolic 4-manifolds constructed by J. Ratcliffe and S. Tschantz.

    Submitted 23 March, 2015; originally announced March 2015.