-
Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Review
Authors:
Thang-Anh-Quan Nguyen,
Amine Bourki,
Mátyás Macudzinski,
Anthony Brunel,
Mohammed Bennamoun
Abstract:
This review thoroughly examines the role of semantically-aware Neural Radiance Fields (NeRFs) in visual scene understanding, covering an analysis of over 250 scholarly papers. It explores how NeRFs adeptly infer 3D representations for both stationary and dynamic objects in a scene. This capability is pivotal for generating high-quality new viewpoints, completing missing scene details (inpainting),…
▽ More
This review thoroughly examines the role of semantically-aware Neural Radiance Fields (NeRFs) in visual scene understanding, covering an analysis of over 250 scholarly papers. It explores how NeRFs adeptly infer 3D representations for both stationary and dynamic objects in a scene. This capability is pivotal for generating high-quality new viewpoints, completing missing scene details (inpainting), conducting comprehensive scene segmentation (panoptic segmentation), predicting 3D bounding boxes, editing 3D scenes, and extracting object-centric 3D models. A significant aspect of this study is the application of semantic labels as viewpoint-invariant functions, which effectively map spatial coordinates to a spectrum of semantic labels, thus facilitating the recognition of distinct objects within the scene. Overall, this survey highlights the progression and diverse applications of semantically-aware neural radiance fields in the context of visual scene interpretation.
△ Less
Submitted 16 February, 2024;
originally announced February 2024.
-
A MUSCL-like finite volumes approximation of the momentum convection operator for low-order nonconforming face-centred discretizations
Authors:
Aubin Brunel,
Raphaele Herbin,
Jean-Claude Latché
Abstract:
We propose in this paper a discretization of the momentum convection operator for fluid flow simulations on quadrangular or hexahedral meshes. The space discretization is performed by the loworder nonconforming Rannacher-Turek finite element: the scalar unknowns are associated to the cells of the mesh, while the velocities unknowns are associated to the edges or faces. The momentum convection oper…
▽ More
We propose in this paper a discretization of the momentum convection operator for fluid flow simulations on quadrangular or hexahedral meshes. The space discretization is performed by the loworder nonconforming Rannacher-Turek finite element: the scalar unknowns are associated to the cells of the mesh, while the velocities unknowns are associated to the edges or faces. The momentum convection operator is of finite volume type, and its almost second order expression is derived by a MUSCL-like technique. The latter is of algebraic type, in the sense that the limitation procedure does not invoke any slope reconstruction, and is independent from the geometry of the cells. The derived discrete convection operator applies both to constant or variable density flows, and may thus be implemented in a scheme for incompressible or compressible flows. To achieve this goal, we derive a discrete analogue of the computation ui ($\partial$t($ρ$ui)+div($ρ$uiu) = 1 2 $\partial$t($ρ$u 2 i)+ 1 2 div($ρ$u 2 i u) (with u the velocity, ui one of its component, $ρ$ the density, and assuming that the mass balance holds) and discuss two applications of this result: firstly, we obtain stability results for a semi-implicit in time scheme for incompressible and barotropic compressible flows; secondly, we build a consistent, semi-implicit in time scheme that is based on the discretization of the internal energy balance rather than the total energy. The performance of the proposed discrete convection operator is assessed by numerical tests on the incompressible Navier-Stokes equations, the barotropic and the full compressible Navier-Stokes and the compressible Euler equations.
△ Less
Submitted 5 January, 2023;
originally announced January 2023.
-
A staggered scheme for the compressible Euler equations on general 3D meshes
Authors:
Aubin Brunel,
Raphaèle Herbin,
Jean-Claude Latché
Abstract:
We address here the discretization of the momentum convection operator for fluid flow simulations on 2D triangular and quadrangular meshes and 3D polyhedral meshes containing hexahedra, tetrahedra, prisms and pyramids. The finite volume scheme that we use for the full Euler equations is based on a staggered discretization: the density unknowns are associated with a primal mesh, whereas the velocit…
▽ More
We address here the discretization of the momentum convection operator for fluid flow simulations on 2D triangular and quadrangular meshes and 3D polyhedral meshes containing hexahedra, tetrahedra, prisms and pyramids. The finite volume scheme that we use for the full Euler equations is based on a staggered discretization: the density unknowns are associated with a primal mesh, whereas the velocity unknowns are associated with a "fictive" dual mesh. Accordingly, the convection operator of the mass balance equation is derived on the primal mesh, while the the convection operator of the momentum balance equation is discretized on the dual mesh. To avoid any hazardous interpolation of the unknowns on a possibly ill-defined dual mesh, the mass fluxes of the momentum convection operator are computed from the mass fluxes of the mass balance equation, so as to ensure the stability of the resulting operator. A coherent reconstruction of these dual fluxes is possible, based only on the kind of considered polygonal or polyhedral cell, and not on each cell itself. Moreover, we show that this process still yields a consistent convection operator in the Lax-Wendroff sense, that is, if a sequence of piecewise constant functions is supposed to converge to a a given limit, then the weak form of the corresponding discrete convection operator converges to the weak form of the continuous operator applied to this limit. The derived discrete convection operator applies to both constant and variable density flows and may thus be implemented in a scheme for incompressible or compressible flows. Numerical tests are performed for the Euler equations on several types of mesh, including hybrid meshes, and show the excellent performance of the method.
△ Less
Submitted 14 September, 2022;
originally announced September 2022.
-
Targeted Sub-attomole Cancer Biomarker Detection based on Phase Singularity 2D Nanomaterial-enhanced Plasmonic Biosensor
Authors:
Yuye Wang,
Shuwen Zeng,
Aurelian Crunteanu,
Zhenming Xie,
Georges Humbert,
Libo Ma,
Yuanyuan Wei,
Aude Brunel,
Barbara Bessette,
Jean-Christophe Orlianges,
Fabrice Lalloué,
Oliver G Schmidt,
Nanfang Yu,
Ho-Pui Ho
Abstract:
Detection of small cancer biomarkers with low molecular weight and a low concentration range has always been challenging yet urgent in many clinical applications such as diagnosing early-stage cancer, monitoring treatment and detecting relapse. Here, a highly enhanced plasmonic biosensor that can overcome this challenge using atomically thin two-dimensional (2D) phase change nanomaterial is develo…
▽ More
Detection of small cancer biomarkers with low molecular weight and a low concentration range has always been challenging yet urgent in many clinical applications such as diagnosing early-stage cancer, monitoring treatment and detecting relapse. Here, a highly enhanced plasmonic biosensor that can overcome this challenge using atomically thin two-dimensional (2D) phase change nanomaterial is developed. By precisely engineering the configuration with atomically thin materials, the phase singularity has been successfully achieved with a significantly enhanced lateral position shift effect. Based on our knowledge, it is the first experimental demonstration of a lateral position signal change > 340 μm at a sensing interface from all optical techniques. With this enhanced plasmonic effect, the detection limit has been experimentally demonstrated to be 10-15 mol L-1 for TNF-α cancer marker, which has been found in various human diseases including inflammatory diseases and different kinds of cancer. The as-reported novel integration of atomically thin Ge2Sb2Te5 (GST) with plasmonic substrate, which results in a phase singularity and thus a giant lateral position shift, enables the detection of cancer markers with low molecular weight at femtomolar level. These results will definitely hold promising potential in biomedical application and clinical diagnostics.
△ Less
Submitted 23 March, 2021; v1 submitted 6 December, 2020;
originally announced December 2020.
-
Backpropagation in the Simply Typed Lambda-calculus with Linear Negation
Authors:
Alois Brunel,
Damiano Mazza,
Michele Pagani
Abstract:
Backpropagation is a classic automatic differentiation algorithm computing the gradient of functions specified by a certain class of simple, first-order programs, called computational graphs. It is a fundamental tool in several fields, most notably machine learning, where it is the key for efficiently training (deep) neural networks. Recent years have witnessed the quick growth of a research field…
▽ More
Backpropagation is a classic automatic differentiation algorithm computing the gradient of functions specified by a certain class of simple, first-order programs, called computational graphs. It is a fundamental tool in several fields, most notably machine learning, where it is the key for efficiently training (deep) neural networks. Recent years have witnessed the quick growth of a research field called differentiable programming, the aim of which is to express computational graphs more synthetically and modularly by resorting to actual programming languages endowed with control flow operators and higher-order combinators, such as map and fold. In this paper, we extend the backpropagation algorithm to a paradigmatic example of such a programming language: we define a compositional program transformation from the simply-typed lambda-calculus to itself augmented with a notion of linear negation, and prove that this computes the gradient of the source program with the same efficiency as first-order backpropagation. The transformation is completely effect-free and thus provides a purely logical understanding of the dynamics of backpropagation.
△ Less
Submitted 6 November, 2019; v1 submitted 27 September, 2019;
originally announced September 2019.
-
A CNN adapted to time series for the classification of Supernovae
Authors:
Anthony Brunel,
Johanna Pasquet,
Jérôme Pasquet,
Nancy Rodriguez,
Frédéric Comby,
Dominique Fouchez,
Marc Chaumont
Abstract:
Cosmologists are facing the problem of the analysis of a huge quantity of data when observing the sky. The methods used in cosmology are, for the most of them, relying on astrophysical models, and thus, for the classification, they usually use a machine learning approach in two-steps, which consists in, first, extracting features, and second, using a classifier. In this paper, we are specifically…
▽ More
Cosmologists are facing the problem of the analysis of a huge quantity of data when observing the sky. The methods used in cosmology are, for the most of them, relying on astrophysical models, and thus, for the classification, they usually use a machine learning approach in two-steps, which consists in, first, extracting features, and second, using a classifier. In this paper, we are specifically studying the supernovae phenomenon and especially the binary classification "I.a supernovae versus not-I.a supernovae". We present two Convolutional Neural Networks (CNNs) defeating the current state-of-the-art. The first one is adapted to time series and thus to the treatment of supernovae light-curves. The second one is based on a Siamese CNN and is suited to the nature of data, i.e. their sparsity and their weak quantity (small learning database).
△ Less
Submitted 2 January, 2019;
originally announced January 2019.
-
Indexed realizability for bounded-time programming with references and type fixpoints
Authors:
Aloïs Brunel,
Antoine Madet
Abstract:
The field of implicit complexity has recently produced several bounded-complexity programming languages. This kind of language allows to implement exactly the functions belonging to a certain complexity class. We here present a realizability semantics for a higher-order functional language based on a fragment of linear logic called LAL which characterizes the complexity class PTIME. This language…
▽ More
The field of implicit complexity has recently produced several bounded-complexity programming languages. This kind of language allows to implement exactly the functions belonging to a certain complexity class. We here present a realizability semantics for a higher-order functional language based on a fragment of linear logic called LAL which characterizes the complexity class PTIME. This language features recursive types and higher-order store. Our realizability is based on biorthogonality, step-indexing and is moreover quantitative. This last feature enables us not only to derive a semantical proof of termination, but also to give bounds on the number of computational steps needed by typed programs to terminate.
△ Less
Submitted 21 June, 2012;
originally announced June 2012.
-
Quantitative classical realizability
Authors:
Aloïs Brunel
Abstract:
Introduced by Dal Lago and Hofmann, quantitative realizability is a technique used to define models for logics based on Multiplicative Linear Logic. A particularity is that functions are interpreted as bounded time computable functions. It has been used to give new and uniform proofs of soundness of several type systems with respect to certain time complexity classes. We propose a reformulation of…
▽ More
Introduced by Dal Lago and Hofmann, quantitative realizability is a technique used to define models for logics based on Multiplicative Linear Logic. A particularity is that functions are interpreted as bounded time computable functions. It has been used to give new and uniform proofs of soundness of several type systems with respect to certain time complexity classes. We propose a reformulation of their ideas in the setting of Krivine's classical realizability. The framework obtained generalizes Dal Lago and Hofmann's realizability, and reveals deep connections between quantitative realizability and a linear variant of Cohen's forcing.
△ Less
Submitted 19 November, 2012; v1 submitted 20 January, 2012;
originally announced January 2012.
-
Church => Scott = Ptime: an application of resource sensitive realizability
Authors:
Aloïs Brunel,
Kazushige Terui
Abstract:
We introduce a variant of linear logic with second order quantifiers and type fixpoints, both restricted to purely linear formulas. The Church encodings of binary words are typed by a standard non-linear type `Church,' while the Scott encodings (purely linear representations of words) are by a linear type `Scott.' We give a characterization of polynomial time functions, which is derived from (…
▽ More
We introduce a variant of linear logic with second order quantifiers and type fixpoints, both restricted to purely linear formulas. The Church encodings of binary words are typed by a standard non-linear type `Church,' while the Scott encodings (purely linear representations of words) are by a linear type `Scott.' We give a characterization of polynomial time functions, which is derived from (Leivant and Marion 93): a function is computable in polynomial time if and only if it can be represented by a term of type Church => Scott.
To prove soundness, we employ a resource sensitive realizability technique developed by Hofmann and Dal Lago.
△ Less
Submitted 4 May, 2010;
originally announced May 2010.
