-
Self-Supervised Geometry-Guided Initialization for Robust Monocular Visual Odometry
Authors:
Takayuki Kanai,
Igor Vasiljevic,
Vitor Guizilini,
Kazuhiro Shintani
Abstract:
Monocular visual odometry is a key technology in a wide variety of autonomous systems. Relative to traditional feature-based methods, that suffer from failures due to poor lighting, insufficient texture, large motions, etc., recent learning-based SLAM methods exploit iterative dense bundle adjustment to address such failure cases and achieve robust accurate localization in a wide variety of real e…
▽ More
Monocular visual odometry is a key technology in a wide variety of autonomous systems. Relative to traditional feature-based methods, that suffer from failures due to poor lighting, insufficient texture, large motions, etc., recent learning-based SLAM methods exploit iterative dense bundle adjustment to address such failure cases and achieve robust accurate localization in a wide variety of real environments, without depending on domain-specific training data. However, despite its potential, learning-based SLAM still struggles with scenarios involving large motion and object dynamics. In this paper, we diagnose key weaknesses in a popular learning-based SLAM model (DROID-SLAM) by analyzing major failure cases on outdoor benchmarks and exposing various shortcomings of its optimization process. We then propose the use of self-supervised priors leveraging a frozen large-scale pre-trained monocular depth estimation to initialize the dense bundle adjustment process, leading to robust visual odometry without the need to fine-tune the SLAM backbone. Despite its simplicity, our proposed method demonstrates significant improvements on KITTI odometry, as well as the challenging DDAD benchmark. Code and pre-trained models will be released upon publication.
△ Less
Submitted 2 June, 2024;
originally announced June 2024.
-
CLIP-Loc: Multi-modal Landmark Association for Global Localization in Object-based Maps
Authors:
Shigemichi Matsuzaki,
Takuma Sugino,
Kazuhito Tanaka,
Zijun Sha,
Shintaro Nakaoka,
Shintaro Yoshizawa,
Kazuhiro Shintani
Abstract:
This paper describes a multi-modal data association method for global localization using object-based maps and camera images. In global localization, or relocalization, using object-based maps, existing methods typically resort to matching all possible combinations of detected objects and landmarks with the same object category, followed by inlier extraction using RANSAC or brute-force search. Thi…
▽ More
This paper describes a multi-modal data association method for global localization using object-based maps and camera images. In global localization, or relocalization, using object-based maps, existing methods typically resort to matching all possible combinations of detected objects and landmarks with the same object category, followed by inlier extraction using RANSAC or brute-force search. This approach becomes infeasible as the number of landmarks increases due to the exponential growth of correspondence candidates. In this paper, we propose labeling landmarks with natural language descriptions and extracting correspondences based on conceptual similarity with image observations using a Vision Language Model (VLM). By leveraging detailed text information, our approach efficiently extracts correspondences compared to methods using only object categories. Through experiments, we demonstrate that the proposed method enables more accurate global localization with fewer iterations compared to baseline methods, exhibiting its efficiency.
△ Less
Submitted 8 February, 2024;
originally announced February 2024.
-
Eco-Friendly Sensing for Human Activity Recognition
Authors:
Kaede Shintani,
Hamada Rizk,
Hirozumi Yamaguchi
Abstract:
With the increasing number of IoT devices, there is a growing demand for energy-free sensors. Human activity recognition holds immense value in numerous daily healthcare applications. However, the majority of current sensing modalities consume energy, thus limiting their sustainable adoption. In this paper, we present a novel activity recognition system that not only operates without requiring ene…
▽ More
With the increasing number of IoT devices, there is a growing demand for energy-free sensors. Human activity recognition holds immense value in numerous daily healthcare applications. However, the majority of current sensing modalities consume energy, thus limiting their sustainable adoption. In this paper, we present a novel activity recognition system that not only operates without requiring energy for sensing but also harvests energy. Our proposed system utilizes photovoltaic cells, attached to the wrist and shoes, as eco-friendly sensing devices for activity recognition. By capturing photovoltaic readings and employing a deep transformer model with powerful learning capabilities, the system effectively recognizes user activities. To ensure robust performance across various subjects, time periods, and lighting conditions, the system incorporates feature extraction and different processing modules. The evaluation of the proposed system on realistic indoor and outdoor environments demonstrated its ability to recognize activities with an accuracy of 91.7%.
△ Less
Submitted 9 August, 2023; v1 submitted 30 July, 2023;
originally announced July 2023.
-
Compositional Confluence Criteria
Authors:
Kiraku Shintani,
Nao Hirokawa
Abstract:
We show how confluence criteria based on decreasing diagrams are generalized to ones composable with other criteria. For demonstration of the method, the confluence criteria of orthogonality, rule labeling, and critical pair systems for term rewriting are recast into composable forms. We also show how such a criterion can be used for a reduction method that removes rewrite rules unnecessary for co…
▽ More
We show how confluence criteria based on decreasing diagrams are generalized to ones composable with other criteria. For demonstration of the method, the confluence criteria of orthogonality, rule labeling, and critical pair systems for term rewriting are recast into composable forms. We also show how such a criterion can be used for a reduction method that removes rewrite rules unnecessary for confluence analysis. In addition to them, we prove that Toyama's parallel closedness result based on parallel critical pairs subsumes his almost parallel closedness theorem.
△ Less
Submitted 22 January, 2024; v1 submitted 7 March, 2023;
originally announced March 2023.
-
A Simons type condition for instability of $F$-Yang-Mills connections
Authors:
Kurando Baba,
Kazuto Shintani
Abstract:
$F$-Yang-Mills connections are critical points of $F$-Yang Mills functional on the space of connections of a principal fiber bundle, which is a generalization of Yang-Mills connections, $p$-Yang-Mills connections and exponential Yang-Mills connections and so on. Here, $F$ is a strictly increasing $C^{2}…
▽ More
$F$-Yang-Mills connections are critical points of $F$-Yang Mills functional on the space of connections of a principal fiber bundle, which is a generalization of Yang-Mills connections, $p$-Yang-Mills connections and exponential Yang-Mills connections and so on. Here, $F$ is a strictly increasing $C^{2}$-function. In this paper, we extend Simons theorem for an instability of Yang-Mills connections to $F$-Yang-Mills connections. We derive a sufficient condition that any non-flat, $F$-Yang-Mills connection over convex hypersurfaces in a Euclidean space is instable. In the sphere case, this condition is expressed by an inequality with respect to its dimension and a degree of the differential of the function $F$. The proofs of the results are given by extending Kobayashi-Ohnita-Takeuchi's calculation to $F$-Yang-Mills connections.
△ Less
Submitted 10 January, 2023;
originally announced January 2023.
-
Spin and charge transport induced by a twisted light beam on a surface of a topological insulator
Authors:
Kunitaka Shintani,
Katsuhisa Taguchi,
Yukio Tanaka,
Yuki Kawaguchi
Abstract:
We theoretically study spin and charge transport induced by a twisted light beam irradiated on a disordered surface of a doped three dimensional topological insulator (TI). We find that various types of spin vortices are imprinted on the surface of the TI depending on the spin and orbital angular momentum of the incident light. The key mechanism for the appearance of the unconventional spin struct…
▽ More
We theoretically study spin and charge transport induced by a twisted light beam irradiated on a disordered surface of a doped three dimensional topological insulator (TI). We find that various types of spin vortices are imprinted on the surface of the TI depending on the spin and orbital angular momentum of the incident light. The key mechanism for the appearance of the unconventional spin structure is the spin-momentum locking in the surface state of the TI. Besides, the diffusive transport of electrons under an inhomogeneous electric field causes a gradient of the charge density, which then induces nonlocal charge current and spin density as well as the spin current. We discuss the relation between these quantities within the linear response to the applied electric field using the Keldysh-Green's function method.
△ Less
Submitted 29 December, 2015;
originally announced December 2015.
-
Spin-charge transport driven by magnetization dynamics on disordered surface of doped topological insulators
Authors:
Katsuhisa Taguchi,
Kunitaka Shintani,
Yukio Tanaka
Abstract:
We theoretically study the spin and charge generation along with the electron transport on a disordered surface of a doped three-dimensional topological insulator/magnetic insulator junction by using Green's function techniques. We find that the spin and charge current are induced by not only local but also nonlocal magnetization dynamics through nonmagnetic impurity scattering on the disordered s…
▽ More
We theoretically study the spin and charge generation along with the electron transport on a disordered surface of a doped three-dimensional topological insulator/magnetic insulator junction by using Green's function techniques. We find that the spin and charge current are induced by not only local but also nonlocal magnetization dynamics through nonmagnetic impurity scattering on the disordered surface of the doped topological insulator. We also clarify that the spin current as well as charge density are induced by spatially inhomogeneous magnetization dynamics, and the spin current diffusively propagates on the disordered surface. Using these results, we discuss both local and nonlocal spin torques before and after the spin and spin current generation on the surface, and provide a procedure to detect the spin current.
△ Less
Submitted 1 May, 2015;
originally announced May 2015.
