-
SEVA: Leveraging sketches to evaluate alignment between human and machine visual abstraction
Authors:
Kushin Mukherjee,
Holly Huey,
Xuanchen Lu,
Yael Vinker,
Rio Aguina-Kang,
Ariel Shamir,
Judith E. Fan
Abstract:
Sketching is a powerful tool for creating abstract images that are sparse but meaningful. Sketch understanding poses fundamental challenges for general-purpose vision algorithms because it requires robustness to the sparsity of sketches relative to natural visual inputs and because it demands tolerance for semantic ambiguity, as sketches can reliably evoke multiple meanings. While current vision a…
▽ More
Sketching is a powerful tool for creating abstract images that are sparse but meaningful. Sketch understanding poses fundamental challenges for general-purpose vision algorithms because it requires robustness to the sparsity of sketches relative to natural visual inputs and because it demands tolerance for semantic ambiguity, as sketches can reliably evoke multiple meanings. While current vision algorithms have achieved high performance on a variety of visual tasks, it remains unclear to what extent they understand sketches in a human-like way. Here we introduce SEVA, a new benchmark dataset containing approximately 90K human-generated sketches of 128 object concepts produced under different time constraints, and thus systematically varying in sparsity. We evaluated a suite of state-of-the-art vision algorithms on their ability to correctly identify the target concept depicted in these sketches and to generate responses that are strongly aligned with human response patterns on the same sketch recognition task. We found that vision algorithms that better predicted human sketch recognition performance also better approximated human uncertainty about sketch meaning, but there remains a sizable gap between model and human response patterns. To explore the potential of models that emulate human visual abstraction in generative tasks, we conducted further evaluations of a recently developed sketch generation algorithm (Vinker et al., 2022) capable of generating sketches that vary in sparsity. We hope that public release of this dataset and evaluation protocol will catalyze progress towards algorithms with enhanced capacities for human-like visual abstraction.
△ Less
Submitted 5 December, 2023;
originally announced December 2023.
-
Echocardiography video synthesis from end diastolic semantic map via diffusion model
Authors:
Phi Nguyen Van,
Duc Tran Minh,
Hieu Pham Huy,
Long Tran Quoc
Abstract:
Denoising Diffusion Probabilistic Models (DDPMs) have demonstrated significant achievements in various image and video generation tasks, including the domain of medical imaging. However, generating echocardiography videos based on semantic anatomical information remains an unexplored area of research. This is mostly due to the constraints imposed by the currently available datasets, which lack suf…
▽ More
Denoising Diffusion Probabilistic Models (DDPMs) have demonstrated significant achievements in various image and video generation tasks, including the domain of medical imaging. However, generating echocardiography videos based on semantic anatomical information remains an unexplored area of research. This is mostly due to the constraints imposed by the currently available datasets, which lack sufficient scale and comprehensive frame-wise annotations for every cardiac cycle. This paper aims to tackle the aforementioned challenges by expanding upon existing video diffusion models for the purpose of cardiac video synthesis. More specifically, our focus lies in generating video using semantic maps of the initial frame during the cardiac cycle, commonly referred to as end diastole. To further improve the synthesis process, we integrate spatial adaptive normalization into multiscale feature maps. This enables the inclusion of semantic guidance during synthesis, resulting in enhanced realism and coherence of the resultant video sequences. Experiments are conducted on the CAMUS dataset, which is a highly used dataset in the field of echocardiography. Our model exhibits better performance compared to the standard diffusion technique in terms of multiple metrics, including FID, FVD, and SSMI.
△ Less
Submitted 10 October, 2023;
originally announced October 2023.
-
Fully Decentralized Peer-to-Peer Community Grid with Dynamic and Congestion Pricing
Authors:
Hien Thanh Doan,
Truong Hoang Bao Huy,
Daehee Kim,
Hongseok Kim
Abstract:
Peer-to-peer (P2P) electricity markets enable prosumers to minimize their costs, which has been extensively studied in recent research. However, there are several challenges with P2P trading when physical network constraints are also included. Moreover, most studies use fixed prices for grid power prices without considering dynamic grid pricing, and equity for all participants. This policy may neg…
▽ More
Peer-to-peer (P2P) electricity markets enable prosumers to minimize their costs, which has been extensively studied in recent research. However, there are several challenges with P2P trading when physical network constraints are also included. Moreover, most studies use fixed prices for grid power prices without considering dynamic grid pricing, and equity for all participants. This policy may negatively affect the long-term development of the market if prosumers with low demand are not treated fairly. An initial step towards addressing these problems is the design of a new decentralized P2P electricity market with two dynamic grid pricing schemes that are determined by consumer demand. Futhermore, we consider a decentralized system with physical constraints for optimizing power flow in networks without compromising privacy. We propose a dynamic congestion price to effectively address congestion and then prove the convergence and global optimality of the proposed method. Our experiments show that P2P energy trade decreases generation cost of main grid by 56.9% compared with previous works. Consumers reduce grid trading by 57.3% while the social welfare of consumers is barely affected by the increase of grid price.
△ Less
Submitted 9 August, 2023;
originally announced August 2023.
-
Echocardiography Segmentation Using Neural ODE-based Diffeomorphic Registration Field
Authors:
Phi Nguyen Van,
Hieu Pham Huy,
Long Tran Quoc
Abstract:
Convolutional neural networks (CNNs) have recently proven their excellent ability to segment 2D cardiac ultrasound images. However, the majority of attempts to perform full-sequence segmentation of cardiac ultrasound videos either rely on models trained only on keyframe images or fail to maintain the topology over time. To address these issues, in this work, we consider segmentation of ultrasound…
▽ More
Convolutional neural networks (CNNs) have recently proven their excellent ability to segment 2D cardiac ultrasound images. However, the majority of attempts to perform full-sequence segmentation of cardiac ultrasound videos either rely on models trained only on keyframe images or fail to maintain the topology over time. To address these issues, in this work, we consider segmentation of ultrasound video as a registration estimation problem and present a novel method for diffeomorphic image registration using neural ordinary differential equations (Neural ODE). In particular, we consider the registration field vector field between frames as a continuous trajectory ODE. The estimated registration field is then applied to the segmentation mask of the first frame to obtain a segment for the whole cardiac cycle. The proposed method, Echo-ODE, introduces several key improvements compared to the previous state-of-the-art. Firstly, by solving a continuous ODE, the proposed method achieves smoother segmentation, preserving the topology of segmentation maps over the whole sequence (Hausdorff distance: 3.7-4.4). Secondly, it maintains temporal consistency between frames without explicitly optimizing for temporal consistency attributes, achieving temporal consistency in 91% of the videos in the dataset. Lastly, the proposed method is able to maintain the clinical accuracy of the segmentation maps (MAE of the LVEF: 2.7-3.1). The results show that our method surpasses the previous state-of-the-art in multiple aspects, demonstrating the importance of spatial-temporal data processing for the implementation of Neural ODEs in medical imaging applications. These findings open up new research directions for solving echocardiography segmentation tasks.
△ Less
Submitted 16 June, 2023;
originally announced June 2023.
-
Forward and Pullback Dynamics of Nonautonomous Integrodifference Equations: Basic Constructions
Authors:
Huy Huy,
Peter E. Kloeden,
Christian Pötzsche
Abstract:
In theoretical ecology, models describing the spatial dispersal and the temporal evolution of species having non-overlap** generations are often based on integrodifference equations. For various such applications the environment has an aperiodic influence on the models leading to nonautonomous integrodifference equations. In order to capture their long-term behaviour comprehensively, both pullba…
▽ More
In theoretical ecology, models describing the spatial dispersal and the temporal evolution of species having non-overlap** generations are often based on integrodifference equations. For various such applications the environment has an aperiodic influence on the models leading to nonautonomous integrodifference equations. In order to capture their long-term behaviour comprehensively, both pullback and forward attractors, as well as forward limit sets are constructed for general infinite-dimensional nonautonomous dynamical systems in discrete time. While the theory of pullback attractors, but not their application to integrodifference equations, is meanwhile well-established, the present novel approach is needed in order to understand their future behaviour.
△ Less
Submitted 11 May, 2022;
originally announced May 2022.
-
Shear-induced mixing of granular materials featuring broad granule size distributions
Authors:
Joyjit Chattoraj,
Nguyen Hoang Huy,
Saurabh Aggarwal,
Mohamed Salahuddin Habibullah,
Farzam Farbiz
Abstract:
Granular flows during a shear-induced mixing process are studied using Discrete Element Methods. The aim is to understand the underlying elementary mechanisms of transition from unmixed to mixed phases for a granular material featuring a broad distribution of particles, which we investigate systematically by varying the strain rate and system size. Here the strain rate varies over four orders of m…
▽ More
Granular flows during a shear-induced mixing process are studied using Discrete Element Methods. The aim is to understand the underlying elementary mechanisms of transition from unmixed to mixed phases for a granular material featuring a broad distribution of particles, which we investigate systematically by varying the strain rate and system size. Here the strain rate varies over four orders of magnitude and the system size varies from ten thousand to more than a million granules. A strain rate-dependent transition from quasistatic to purely inertial flow is observed. At the macroscopic scale, the contact stresses drop due to the formation of shear-induced instabilities that serves as an onset of granular flows and initiates mixing between the granules. The stress-drop displays a profound system size dependence. At the granular scale, mixing dynamics are correlated with the formation of shear bands, which result in significantly different timescales of mixing, especially for those regions that are close to the system walls and the bulk. Overall, our results reveal that although the transient dynamics display a generic behavior these have a significant finite-size effect. In contrast, macroscopic behaviors at steady states have negligible system size dependence.
△ Less
Submitted 13 February, 2022;
originally announced February 2022.
