-
CLIP-Loc: Multi-modal Landmark Association for Global Localization in Object-based Maps
Authors:
Shigemichi Matsuzaki,
Takuma Sugino,
Kazuhito Tanaka,
Zijun Sha,
Shintaro Nakaoka,
Shintaro Yoshizawa,
Kazuhiro Shintani
Abstract:
This paper describes a multi-modal data association method for global localization using object-based maps and camera images. In global localization, or relocalization, using object-based maps, existing methods typically resort to matching all possible combinations of detected objects and landmarks with the same object category, followed by inlier extraction using RANSAC or brute-force search. Thi…
▽ More
This paper describes a multi-modal data association method for global localization using object-based maps and camera images. In global localization, or relocalization, using object-based maps, existing methods typically resort to matching all possible combinations of detected objects and landmarks with the same object category, followed by inlier extraction using RANSAC or brute-force search. This approach becomes infeasible as the number of landmarks increases due to the exponential growth of correspondence candidates. In this paper, we propose labeling landmarks with natural language descriptions and extracting correspondences based on conceptual similarity with image observations using a Vision Language Model (VLM). By leveraging detailed text information, our approach efficiently extracts correspondences compared to methods using only object categories. Through experiments, we demonstrate that the proposed method enables more accurate global localization with fewer iterations compared to baseline methods, exhibiting its efficiency.
△ Less
Submitted 8 February, 2024;
originally announced February 2024.
-
Super-resolution of clinical CT volumes with modified CycleGAN using micro CT volumes
Authors:
Tong ZHENG,
Hirohisa ODA,
Takayasu MORIYA,
Takaaki SUGINO,
Shota NAKAMURA,
Masahiro ODA,
Masaki MORI,
Hirotsugu TAKABATAKE,
Hiroshi NATORI,
Kensaku MORI
Abstract:
This paper presents a super-resolution (SR) method with unpaired training dataset of clinical CT and micro CT volumes. For obtaining very detailed information such as cancer invasion from pre-operative clinical CT volumes of lung cancer patients, SR of clinical CT volumes to $\m$}CT level is desired. While most SR methods require paired low- and high- resolution images for training, it is infeasib…
▽ More
This paper presents a super-resolution (SR) method with unpaired training dataset of clinical CT and micro CT volumes. For obtaining very detailed information such as cancer invasion from pre-operative clinical CT volumes of lung cancer patients, SR of clinical CT volumes to $\m$}CT level is desired. While most SR methods require paired low- and high- resolution images for training, it is infeasible to obtain paired clinical CT and μCT volumes. We propose a SR approach based on CycleGAN, which could perform SR on clinical CT into $μ$CT level. We proposed new loss functions to keep cycle consistency, while training without paired volumes. Experimental results demonstrated that our proposed method successfully performed SR of clinical CT volume of lung cancer patients into $μ$CT level.
△ Less
Submitted 7 April, 2020;
originally announced April 2020.
-
A multi-scale pyramid of 3D fully convolutional networks for abdominal multi-organ segmentation
Authors:
Holger R. Roth,
Chen Shen,
Hirohisa Oda,
Takaaki Sugino,
Masahiro Oda,
Yuichiro Hayashi,
Kazunari Misawa,
Kensaku Mori
Abstract:
Recent advances in deep learning, like 3D fully convolutional networks (FCNs), have improved the state-of-the-art in dense semantic segmentation of medical images. However, most network architectures require severely downsampling or crop** the images to meet the memory limitations of today's GPU cards while still considering enough context in the images for accurate segmentation. In this work, w…
▽ More
Recent advances in deep learning, like 3D fully convolutional networks (FCNs), have improved the state-of-the-art in dense semantic segmentation of medical images. However, most network architectures require severely downsampling or crop** the images to meet the memory limitations of today's GPU cards while still considering enough context in the images for accurate segmentation. In this work, we propose a novel approach that utilizes auto-context to perform semantic segmentation at higher resolutions in a multi-scale pyramid of stacked 3D FCNs. We train and validate our models on a dataset of manually annotated abdominal organs and vessels from 377 clinical CT images used in gastric surgery, and achieve promising results with close to 90% Dice score on average. For additional evaluation, we perform separate testing on datasets from different sources and achieve competitive results, illustrating the robustness of the model and approach.
△ Less
Submitted 6 June, 2018;
originally announced June 2018.
