-
LivePose: Online 3D Reconstruction from Monocular Video with Dynamic Camera Poses
Authors:
Noah Stier,
Baptiste Angles,
Liang Yang,
Yajie Yan,
Alex Colburn,
Ming Chuang
Abstract:
Dense 3D reconstruction from RGB images traditionally assumes static camera pose estimates. This assumption has endured, even as recent works have increasingly focused on real-time methods for mobile devices. However, the assumption of a fixed pose for each image does not hold for online execution: poses from real-time SLAM are dynamic and may be updated following events such as bundle adjustment…
▽ More
Dense 3D reconstruction from RGB images traditionally assumes static camera pose estimates. This assumption has endured, even as recent works have increasingly focused on real-time methods for mobile devices. However, the assumption of a fixed pose for each image does not hold for online execution: poses from real-time SLAM are dynamic and may be updated following events such as bundle adjustment and loop closure. This has been addressed in the RGB-D setting, by de-integrating past views and re-integrating them with updated poses, but it remains largely untreated in the RGB-only setting. We formalize this problem to define the new task of dense online reconstruction from dynamically-posed images. To support further research, we introduce a dataset called LivePose containing the dynamic poses from a SLAM system running on ScanNet. We select three recent reconstruction systems and apply a framework based on de-integration to adapt each one to the dynamic-pose setting. In addition, we propose a novel, non-linear de-integration module that learns to remove stale scene content. We show that responding to pose updates is critical for high-quality reconstruction, and that our de-integration framework is an effective solution.
△ Less
Submitted 18 August, 2023; v1 submitted 31 March, 2023;
originally announced April 2023.
-
A Feature Learning and Object Recognition Framework for Underwater Fish Images
Authors:
Meng-Che Chuang,
Jenq-Neng Hwang,
Kresimir Williams
Abstract:
Live fish recognition is one of the most crucial elements of fisheries survey applications where vast amount of data are rapidly acquired. Different from general scenarios, challenges to underwater image recognition are posted by poor image quality, uncontrolled objects and environment, as well as difficulty in acquiring representative samples. Also, most existing feature extraction techniques are…
▽ More
Live fish recognition is one of the most crucial elements of fisheries survey applications where vast amount of data are rapidly acquired. Different from general scenarios, challenges to underwater image recognition are posted by poor image quality, uncontrolled objects and environment, as well as difficulty in acquiring representative samples. Also, most existing feature extraction techniques are hindered from automation due to involving human supervision. Toward this end, we propose an underwater fish recognition framework that consists of a fully unsupervised feature learning technique and an error-resilient classifier. Object parts are initialized based on saliency and relaxation labeling to match object parts correctly. A non-rigid part model is then learned based on fitness, separation and discrimination criteria. For the classifier, an unsupervised clustering approach generates a binary class hierarchy, where each node is a classifier. To exploit information from ambiguous images, the notion of partial classification is introduced to assign coarse labels by optimizing the "benefit" of indecision made by the classifier. Experiments show that the proposed framework achieves high accuracy on both public and self-collected underwater fish images with high uncertainty and class imbalance.
△ Less
Submitted 5 March, 2016;
originally announced March 2016.
-
Underwater Fish Tracking for Moving Cameras based on Deformable Multiple Kernels
Authors:
Meng-Che Chuang,
Jenq-Neng Hwang,
Jian-Hui Ye,
Shih-Chia Huang,
Kresimir Williams
Abstract:
Fishery surveys that call for the use of single or multiple underwater cameras have been an emerging technology as a non-extractive mean to estimate the abundance of fish stocks. Tracking live fish in an open aquatic environment posts challenges that are different from general pedestrian or vehicle tracking in surveillance applications. In many rough habitats fish are monitored by cameras installe…
▽ More
Fishery surveys that call for the use of single or multiple underwater cameras have been an emerging technology as a non-extractive mean to estimate the abundance of fish stocks. Tracking live fish in an open aquatic environment posts challenges that are different from general pedestrian or vehicle tracking in surveillance applications. In many rough habitats fish are monitored by cameras installed on moving platforms, where tracking is even more challenging due to inapplicability of background models. In this paper, a novel tracking algorithm based on the deformable multiple kernels (DMK) is proposed to address these challenges. Inspired by the deformable part model (DPM) technique, a set of kernels is defined to represent the holistic object and several parts that are arranged in a deformable configuration. Color histogram, texture histogram and the histogram of oriented gradients (HOG) are extracted and serve as object features. Kernel motion is efficiently estimated by the mean-shift algorithm on color and texture features to realize tracking. Furthermore, the HOG-feature deformation costs are adopted as soft constraints on kernel positions to maintain the part configuration. Experimental results on practical video set from underwater moving cameras show the reliable performance of the proposed method with much less computational cost comparing with state-of-the-art techniques.
△ Less
Submitted 5 March, 2016;
originally announced March 2016.
-
A Connectivity-Aware Multi-level Finite-Element System for Solving Laplace-Beltrami Equations
Authors:
Ming Chuang,
Michael Kazhdan
Abstract:
Recent work on octree-based finite-element systems has developed a multigrid solver for Poisson equations on meshes. While the idea of defining a regularly indexed function space has been successfully used in a number of applications, it has also been noted that the richness of the function space is limited because the function values can be coupled across locally disconnected regions. In this wor…
▽ More
Recent work on octree-based finite-element systems has developed a multigrid solver for Poisson equations on meshes. While the idea of defining a regularly indexed function space has been successfully used in a number of applications, it has also been noted that the richness of the function space is limited because the function values can be coupled across locally disconnected regions. In this work, we show how to enrich the function space by introducing functions that resolve the coupling while still preserving the nesting hierarchy that supports multigrid. A spectral analysis reveals the superior quality of the resulting Laplace-Beltrami operator and applications to surface flow demonstrate that our new solver more efficiently converges to the correct solution.
△ Less
Submitted 14 May, 2015;
originally announced May 2015.
-
Tracking Live Fish from Low-Contrast and Low-Frame-Rate Stereo Videos
Authors:
Meng-Che Chuang,
Jenq-Neng Hwang,
Kresimir Williams,
Richard Towler
Abstract:
Non-extractive fish abundance estimation with the aid of visual analysis has drawn increasing attention. Unstable illumination, ubiquitous noise and low frame rate video capturing in the underwater environment, however, make conventional tracking methods unreliable. In this paper, we present a multiple fish tracking system for low-contrast and low-frame-rate stereo videos with the use of a trawl-b…
▽ More
Non-extractive fish abundance estimation with the aid of visual analysis has drawn increasing attention. Unstable illumination, ubiquitous noise and low frame rate video capturing in the underwater environment, however, make conventional tracking methods unreliable. In this paper, we present a multiple fish tracking system for low-contrast and low-frame-rate stereo videos with the use of a trawl-based underwater camera system. An automatic fish segmentation algorithm overcomes the low-contrast issues by adopting a histogram backprojection approach on double local-thresholded images to ensure an accurate segmentation on the fish shape boundaries. Built upon a reliable feature-based object matching method, a multiple-target tracking algorithm via a modified Viterbi data association is proposed to overcome the poor motion continuity and frequent entrance/exit of fish targets under low-frame-rate scenarios. In addition, a computationally efficient block-matching approach performs successful stereo matching, which enables an automatic fish-body tail compensation to greatly reduce segmentation error and allows for an accurate fish length measurement. Experimental results show that an effective and reliable tracking performance for multiple live fish with underwater stereo cameras is achieved.
△ Less
Submitted 15 April, 2015;
originally announced April 2015.
-
Automatic Annotation of Axoplasmic Reticula in Pursuit of Connectomes
Authors:
Ayushi Sinha,
William Gray Roncal,
Narayanan Kasthuri,
Ming Chuang,
Priya Manavalan,
Dean M. Kleissas,
Joshua T. Vogelstein,
R. Jacob Vogelstein,
Randal Burns,
Jeff W. Lichtman,
Michael Kazhdan
Abstract:
In this paper, we present a new pipeline which automatically identifies and annotates axoplasmic reticula, which are small subcellular structures present only in axons. We run our algorithm on the Kasthuri11 dataset, which was color corrected using gradient-domain techniques to adjust contrast. We use a bilateral filter to smooth out the noise in this data while preserving edges, which highlights…
▽ More
In this paper, we present a new pipeline which automatically identifies and annotates axoplasmic reticula, which are small subcellular structures present only in axons. We run our algorithm on the Kasthuri11 dataset, which was color corrected using gradient-domain techniques to adjust contrast. We use a bilateral filter to smooth out the noise in this data while preserving edges, which highlights axoplasmic reticula. These axoplasmic reticula are then annotated using a morphological region growing algorithm. Additionally, we perform Laplacian sharpening on the bilaterally filtered data to enhance edges, and repeat the morphological region growing algorithm to annotate more axoplasmic reticula. We track our annotations through the slices to improve precision, and to create long objects to aid in segment merging. This method annotates axoplasmic reticula with high precision. Our algorithm can easily be adapted to annotate axoplasmic reticula in different sets of brain data by changing a few thresholds. The contribution of this work is the introduction of a straightforward and robust pipeline which annotates axoplasmic reticula with high precision, contributing towards advancements in automatic feature annotations in neural EM data.
△ Less
Submitted 16 April, 2014;
originally announced April 2014.
