Showing 1–50 of 230 results for author: Jackson, B

Autonomous navigation of catheters and guidewires in mechanical thrombectomy using inverse reinforcement learning

Authors: Harry Robertshaw, Lennart Karstensen, Benjamin Jackson, Alejandro Granados, Thomas C. Booth

Abstract: Purpose: Autonomous navigation of catheters and guidewires can enhance endovascular surgery safety and efficacy, reducing procedure times and operator radiation exposure. Integrating tele-operated robotics could widen access to time-sensitive emergency procedures like mechanical thrombectomy (MT). Reinforcement learning (RL) shows potential in endovascular navigation, yet its application encounter… ▽ More Purpose: Autonomous navigation of catheters and guidewires can enhance endovascular surgery safety and efficacy, reducing procedure times and operator radiation exposure. Integrating tele-operated robotics could widen access to time-sensitive emergency procedures like mechanical thrombectomy (MT). Reinforcement learning (RL) shows potential in endovascular navigation, yet its application encounters challenges without a reward signal. This study explores the viability of autonomous navigation in MT vasculature using inverse RL (IRL) to leverage expert demonstrations. Methods: This study established a simulation-based training and evaluation environment for MT navigation. We used IRL to infer reward functions from expert behaviour when navigating a guidewire and catheter. We utilized soft actor-critic to train models with various reward functions and compared their performance in silico. Results: We demonstrated feasibility of navigation using IRL. When evaluating single versus dual device (i.e. guidewire versus catheter and guidewire) tracking, both methods achieved high success rates of 95% and 96%, respectively. Dual-tracking, however, utilized both devices mimicking an expert. A success rate of 100% and procedure time of 22.6 s were obtained when training with a reward function obtained through reward sha**. This outperformed a dense reward function (96%, 24.9 s) and an IRL-derived reward function (48%, 59.2 s). Conclusions: We have contributed to the advancement of autonomous endovascular intervention navigation, particularly MT, by employing IRL. The results underscore the potential of using reward sha** to train models, offering a promising avenue for enhancing the accessibility and precision of MT. We envisage that future research can extend our methodology to diverse anatomical structures to enhance generalizability. △ Less

Submitted 18 June, 2024; originally announced June 2024.

Comments: Abstract shortened for arXiv character limit

Journal ref: Int J CARS (2024)

An Effective-Efficient Approach for Dense Multi-Label Action Detection

Authors: Faegheh Sardari, Armin Mustafa, Philip J. B. Jackson, Adrian Hilton

Abstract: Unlike the sparse label action detection task, where a single action occurs in each timestamp of a video, in a dense multi-label scenario, actions can overlap. To address this challenging task, it is necessary to simultaneously learn (i) temporal dependencies and (ii) co-occurrence action relationships. Recent approaches model temporal information by extracting multi-scale features through hierarc… ▽ More Unlike the sparse label action detection task, where a single action occurs in each timestamp of a video, in a dense multi-label scenario, actions can overlap. To address this challenging task, it is necessary to simultaneously learn (i) temporal dependencies and (ii) co-occurrence action relationships. Recent approaches model temporal information by extracting multi-scale features through hierarchical transformer-based networks. However, the self-attention mechanism in transformers inherently loses temporal positional information. We argue that combining this with multiple sub-sampling processes in hierarchical designs can lead to further loss of positional information. Preserving this information is essential for accurate action detection. In this paper, we address this issue by proposing a novel transformer-based network that (a) employs a non-hierarchical structure when modelling different ranges of temporal dependencies and (b) embeds relative positional encoding in its transformer layers. Furthermore, to model co-occurrence action relationships, current methods explicitly embed class relations into the transformer network. However, these approaches are not computationally efficient, as the network needs to compute all possible pair action class relations. We also overcome this challenge by introducing a novel learning paradigm that allows the network to benefit from explicitly modelling temporal co-occurrence action dependencies without imposing their additional computational costs during inference. We evaluate the performance of our proposed approach on two challenging dense multi-label benchmark datasets and show that our method improves the current state-of-the-art results. △ Less

Submitted 10 June, 2024; originally announced June 2024.

Comments: 14 pages. arXiv admin note: substantial text overlap with arXiv:2308.05051

The TEMPO Survey II: Science Cases Leveraged from a Proposed 30-Day Time Domain Survey of the Orion Nebula with the Nancy Grace Roman Space Telescope

Authors: Melinda Soares-Furtado, Mary Anne Limbach, Andrew Vanderburg, John Bally, Juliette Becker, Anna L. Rosen, Luke G. Bouma, Johanna M. Vos, Steve B. Howell, Thomas G. Beatty, William M. J. Best, Anne Marie Cody, Adam Distler, Elena D'Onghia, René Heller, Brandon S. Hensley, Natalie R. Hinkel, Brian Jackson, Marina Kounkel, Adam Kraus, Andrew W. Mann, Nicholas T. Marston, Massimo Robberto, Joseph E. Rodriguez, Jason H. Steffen , et al. (4 additional authors not shown)

Abstract: The TEMPO (Transiting Exosatellites, Moons, and Planets in Orion) Survey is a proposed 30-day observational campaign using the Nancy Grace Roman Space Telescope. By providing deep, high-resolution, short-cadence infrared photometry of a dynamic star-forming region, TEMPO will investigate the demographics of exosatellites orbiting free-floating planets and brown dwarfs -- a largely unexplored disco… ▽ More The TEMPO (Transiting Exosatellites, Moons, and Planets in Orion) Survey is a proposed 30-day observational campaign using the Nancy Grace Roman Space Telescope. By providing deep, high-resolution, short-cadence infrared photometry of a dynamic star-forming region, TEMPO will investigate the demographics of exosatellites orbiting free-floating planets and brown dwarfs -- a largely unexplored discovery space. Here, we present the simulated detection yields of three populations: extrasolar moon analogs orbiting free-floating planets, exosatellites orbiting brown dwarfs, and exoplanets orbiting young stars. Additionally, we outline a comprehensive range of anticipated scientific outcomes accompanying such a survey. These science drivers include: obtaining observational constraints to test prevailing theories of moon, planet, and star formation; directly detecting widely separated exoplanets orbiting young stars; investigating the variability of young stars and brown dwarfs; constraining the low-mass end of the stellar initial mass function; constructing the distribution of dust in the Orion Nebula and map** evolution in the near-infrared extinction law; map** emission features that trace the shocked gas in the region; constructing a dynamical map of Orion members using proper motions; and searching for extragalactic sources and transients via deep extragalactic observations reaching a limiting magnitude of $m_{AB}=29.7$\,mag (F146 filter). △ Less

Submitted 3 June, 2024; originally announced June 2024.

Comments: 15 pages, 6 figures, submitted to OJAp

Audio-Visual Talker Localization in Video for Spatial Sound Reproduction

Authors: Davide Berghi, Philip J. B. Jackson

Abstract: Object-based audio production requires the positional metadata to be defined for each point-source object, including the key elements in the foreground of the sound scene. In many media production use cases, both cameras and microphones are employed to make recordings, and the human voice is often a key element. In this research, we detect and locate the active speaker in the video, facilitating t… ▽ More Object-based audio production requires the positional metadata to be defined for each point-source object, including the key elements in the foreground of the sound scene. In many media production use cases, both cameras and microphones are employed to make recordings, and the human voice is often a key element. In this research, we detect and locate the active speaker in the video, facilitating the automatic extraction of the positional metadata of the talker relative to the camera's reference frame. With the integration of the visual modality, this study expands upon our previous investigation focused solely on audio-based active speaker detection and localization. Our experiments compare conventional audio-visual approaches for active speaker detection that leverage monaural audio, our previous audio-only method that leverages multichannel recordings from a microphone array, and a novel audio-visual approach integrating vision and multichannel audio. We found the role of the two modalities to complement each other. Multichannel audio, overcoming the problem of visual occlusions, provides a double-digit reduction in detection error compared to audio-visual methods with single-channel audio. The combination of multichannel audio and vision further enhances spatial accuracy, leading to a four-percentage point increase in F1 score on the Tragic Talkers dataset. Future investigations will assess the robustness of the model in noisy and highly reverberant environments, as well as tackle the problem of off-screen speakers. △ Less

Submitted 1 June, 2024; originally announced June 2024.

CoLeaF: A Contrastive-Collaborative Learning Framework for Weakly Supervised Audio-Visual Video Parsing

Authors: Faegheh Sardari, Armin Mustafa, Philip J. B. Jackson, Adrian Hilton

Abstract: Weakly supervised audio-visual video parsing (AVVP) methods aim to detect audible-only, visible-only, and audible-visible events using only video-level labels. Existing approaches tackle this by leveraging unimodal and cross-modal contexts. However, we argue that while cross-modal learning is beneficial for detecting audible-visible events, in the weakly supervised scenario, it negatively impacts… ▽ More Weakly supervised audio-visual video parsing (AVVP) methods aim to detect audible-only, visible-only, and audible-visible events using only video-level labels. Existing approaches tackle this by leveraging unimodal and cross-modal contexts. However, we argue that while cross-modal learning is beneficial for detecting audible-visible events, in the weakly supervised scenario, it negatively impacts unaligned audible or visible events by introducing irrelevant modality information. In this paper, we propose CoLeaF, a novel learning framework that optimizes the integration of cross-modal context in the embedding space such that the network explicitly learns to combine cross-modal information for audible-visible events while filtering them out for unaligned events. Additionally, as videos often involve complex class relationships, modelling them improves performance. However, this introduces extra computational costs into the network. Our framework is designed to leverage cross-class relationships during training without incurring additional computations at inference. Furthermore, we propose new metrics to better evaluate a method's capabilities in performing AVVP. Our extensive experiments demonstrate that CoLeaF significantly improves the state-of-the-art results by an average of 1.9% and 2.4% F-score on the LLP and UnAV-100 datasets, respectively. △ Less

Submitted 20 May, 2024; v1 submitted 17 May, 2024; originally announced May 2024.

Artificial Intelligence in the Autonomous Navigation of Endovascular Interventions: A Systematic Review

Authors: Harry Robertshaw, Lennart Karstensen, Benjamin Jackson, Hadi Sadati, Kawal Rhode, Sebastien Ourselin, Alejandro Granados, Thomas C Booth

Abstract: Purpose: Autonomous navigation of devices in endovascular interventions can decrease operation times, improve decision-making during surgery, and reduce operator radiation exposure while increasing access to treatment. This systematic review explores recent literature to assess the impact, challenges, and opportunities artificial intelligence (AI) has for the autonomous endovascular intervention n… ▽ More Purpose: Autonomous navigation of devices in endovascular interventions can decrease operation times, improve decision-making during surgery, and reduce operator radiation exposure while increasing access to treatment. This systematic review explores recent literature to assess the impact, challenges, and opportunities artificial intelligence (AI) has for the autonomous endovascular intervention navigation. Methods: PubMed and IEEEXplore databases were queried. Eligibility criteria included studies investigating the use of AI in enabling the autonomous navigation of catheters/guidewires in endovascular interventions. Following PRISMA, articles were assessed using QUADAS-2. PROSPERO: CRD42023392259. Results: Among 462 studies, fourteen met inclusion criteria. Reinforcement learning (9/14, 64%) and learning from demonstration (7/14, 50%) were used as data-driven models for autonomous navigation. Studies predominantly utilised physical phantoms (10/14, 71%) and in silico (4/14, 29%) models. Experiments within or around the blood vessels of the heart were reported by the majority of studies (10/14, 71%), while simple non-anatomical vessel platforms were used in three studies (3/14, 21%), and the porcine liver venous system in one study. We observed that risk of bias and poor generalisability were present across studies. No procedures were performed on patients in any of the studies reviewed. Studies lacked patient selection criteria, reference standards, and reproducibility, resulting in low clinical evidence levels. Conclusions: AI's potential in autonomous endovascular navigation is promising, but in an experimental proof-of-concept stage, with a technology readiness level of 3. We highlight that reference standards with well-identified performance metrics are crucial to allow for comparisons of data-driven algorithms proposed in the years to come. △ Less

Submitted 6 May, 2024; originally announced May 2024.

Comments: Abstract shortened for arXiv character limit

Journal ref: (2023) Front. Hum. Neurosci. 17:1239374

Exploring Convergence in Relation using Association Rules Mining: A Case Study in Collaborative Knowledge Production

Authors: Jiahe Ling, Corey B. Jackson

Abstract: This study delves into the pivotal role played by non-experts in knowledge production on open collaboration platforms, with a particular focus on the intricate process of tag development that culminates in the proposal of new glitch classes. Leveraging the power of Association Rule Mining (ARM), this research endeavors to unravel the underlying dynamics of collaboration among citizen scientists. B… ▽ More This study delves into the pivotal role played by non-experts in knowledge production on open collaboration platforms, with a particular focus on the intricate process of tag development that culminates in the proposal of new glitch classes. Leveraging the power of Association Rule Mining (ARM), this research endeavors to unravel the underlying dynamics of collaboration among citizen scientists. By meticulously quantifying tag associations and scrutinizing their temporal dynamics, the study provides a comprehensive and nuanced understanding of how non-experts collaborate to generate valuable scientific insights. Furthermore, this investigation extends its purview to examine the phenomenon of ideological convergence within online citizen science knowledge production. To accomplish this, a novel measurement algorithm, based on the Mann-Kendall Trend Test, is introduced. This innovative approach sheds illuminating light on the dynamics of collaborative knowledge production, revealing both the vast opportunities and daunting challenges inherent in leveraging non-expert contributions for scientific research endeavors. Notably, the study uncovers a robust pattern of convergence in ideology, employing both the newly proposed convergence testing method and the traditional approach based on the stationarity of time series data. This groundbreaking discovery holds significant implications for understanding the dynamics of online citizen science communities and underscores the crucial role played by non-experts in sha** the scientific landscape of the digital age. Ultimately, this study contributes significantly to our understanding of online citizen science communities, highlighting their potential to harness collective intelligence for tackling complex scientific tasks and enriching our comprehension of collaborative knowledge production processes in the digital age. △ Less

Submitted 13 May, 2024; v1 submitted 23 April, 2024; originally announced April 2024.

Doomed Worlds I: No new evidence for orbital decay in a long-term survey of 43 ultra-hot Jupiters

Authors: Elisabeth R. Adams, Brian Jackson, Amanda A. Sickafoose, Jeffrey P. Morgenthaler, Hannah Worters, Hailey Stubbers, Dallon Carson, Sakhee Bhure, Stijn Dekeyser, Chelsea Huang, Nevin N. Weinberg

Abstract: Ultra-hot Jupiters are likely doomed by tidal forces to undergo orbital decay and eventual disruption by their stars, but the timescale over which this process unfolds is unknown. We present results from a long-term project to monitor ultra-hot Jupiters transits. We recovered WASP-12 b's orbital decay rate of dP/dt = -29.8 +/- 1.6 ms yr-1, in agreement with prior work. Five other systems initially… ▽ More Ultra-hot Jupiters are likely doomed by tidal forces to undergo orbital decay and eventual disruption by their stars, but the timescale over which this process unfolds is unknown. We present results from a long-term project to monitor ultra-hot Jupiters transits. We recovered WASP-12 b's orbital decay rate of dP/dt = -29.8 +/- 1.6 ms yr-1, in agreement with prior work. Five other systems initially had promising non-linear transit ephemerides. However, a closer examination of two -- WASP-19 b and CoRoT-2 b, both with prior tentative detections -- revealed several independent errors with the literature timing data; after correction neither planet shows signs of orbital decay. Meanwhile, a potential decreasing period for TrES-1 b, dP/dt = -16 +/- 5 ms yr-1, corresponds to a tidal quality factor Q*' = 160 and likely does not result from orbital decay, if driven by dissipation within the host star. Nominal period increases in two systems, WASP-121 b and WASP-46 b, rest on a small handful of points. Only 1/43 planets (WASP-12 b) in our sample is experiencing detectable orbital decay. For nearly half (20/42) we can rule out dP/dt as high as observed for WASP-12 b. Thus while many ultra-hot Jupiters could still be experiencing rapid decay that we cannot yet detect, a sizeable sub-population of UHJs are decaying at least an order of magnitude more slowly than WASP-12 b. Our reanalysis of Kepler-1658 b with no new data finds that it remains a promising orbital decay candidate. Finally, we recommend that the scientific community take steps to avoid spurious detections through better management of the multi-decade-spanning datasets needed to search for and study planetary orbital decay. △ Less

Submitted 10 April, 2024; originally announced April 2024.

Comments: Accepted to PSJ

DPA-Net: Structured 3D Abstraction from Sparse Views via Differentiable Primitive Assembly

Authors: Fenggen Yu, Yiming Qian, Xu Zhang, Francisca Gil-Ureta, Brian Jackson, Eric Bennett, Hao Zhang

Abstract: We present a differentiable rendering framework to learn structured 3D abstractions in the form of primitive assemblies from sparse RGB images capturing a 3D object. By leveraging differentiable volume rendering, our method does not require 3D supervision. Architecturally, our network follows the general pipeline of an image-conditioned neural radiance field (NeRF) exemplified by pixelNeRF for col… ▽ More We present a differentiable rendering framework to learn structured 3D abstractions in the form of primitive assemblies from sparse RGB images capturing a 3D object. By leveraging differentiable volume rendering, our method does not require 3D supervision. Architecturally, our network follows the general pipeline of an image-conditioned neural radiance field (NeRF) exemplified by pixelNeRF for color prediction. As our core contribution, we introduce differential primitive assembly (DPA) into NeRF to output a 3D occupancy field in place of density prediction, where the predicted occupancies serve as opacity values for volume rendering. Our network, coined DPA-Net, produces a union of convexes, each as an intersection of convex quadric primitives, to approximate the target 3D object, subject to an abstraction loss and a masking loss, both defined in the image space upon volume rendering. With test-time adaptation and additional sampling and loss designs aimed at improving the accuracy and compactness of the obtained assemblies, our method demonstrates superior performance over state-of-the-art alternatives for 3D primitive abstraction from sparse views. △ Less

Submitted 2 April, 2024; v1 submitted 31 March, 2024; originally announced April 2024.

Comments: 14 pages

A topology on the poset of quiver mutation classes

Authors: Tucker J. Ervin, Blake Jackson

Abstract: To better understand mutation-invariant and hereditary properties of quivers (and more generally skew-symmetrizable matrices), we have constructed a topology on the set of all mutation classes of quivers which we call the mutation class topology. This topology is the Alexandrov topology induced by the poset structure on the set of mutation classes of quivers from the partial order of quiver embedd… ▽ More To better understand mutation-invariant and hereditary properties of quivers (and more generally skew-symmetrizable matrices), we have constructed a topology on the set of all mutation classes of quivers which we call the mutation class topology. This topology is the Alexandrov topology induced by the poset structure on the set of mutation classes of quivers from the partial order of quiver embedding. The closed sets of our topology -- equivalently, the lower sets of the poset -- are in bijective correspondence with mutation-invariant and hereditary properties of quivers. We show that this space is strictly $T_0$, connected, non-Noetherian, and that every open set is dense. We close by providing open questions from cluster algebra theory in the setting of the mutation class topology and some directions for future research. △ Less

Submitted 11 April, 2024; v1 submitted 29 March, 2024; originally announced March 2024.

Comments: 15 pages, 1 figure

MSC Class: 13F60 (Primary) 06A06; 54H99 (Secondary)

System performance of a TDM test-bed with long flex harness towards the new X-IFU FPA-DM

Authors: D. Vaccaro, M. de Wit, J. van der Kuur, L. Gottardi, K. Ravensberg, E. Taralli, J. Adams, S. R. Bandler, J. A. Chervenak, W. B. Doriese, M. Durkin, C. Reintsema, K. Sakai, S. J. Smith, N. A. Wakeham, B. Jackson, P. Khosropanah, J. R. Gao, J. W. A. den Herder, P. Roelfsema

Abstract: SRON (Netherlands Institute for Space Research) is develo** the Focal Plane Assembly (FPA) for Athena X-IFU, whose Demonstration Model (DM) will use for the first time a time domain multiplexing (TDM)-based readout system for the on-board transition-edge sensors (TES). We report on the characterization activities on a TDM setup provided by NASA Goddard Space Flight Center (GSFC) and National Ins… ▽ More SRON (Netherlands Institute for Space Research) is develo** the Focal Plane Assembly (FPA) for Athena X-IFU, whose Demonstration Model (DM) will use for the first time a time domain multiplexing (TDM)-based readout system for the on-board transition-edge sensors (TES). We report on the characterization activities on a TDM setup provided by NASA Goddard Space Flight Center (GSFC) and National Institute for Standards and Technology (NIST) and tested in SRON cryogenic test facilities. The goal of these activities is to study the impact of the longer harness, closer to X-IFU specs, in a different EMI environment and switching from a single-ended to a differential readout scheme. In this contribution we describe the advancement in the debugging of the system in the SRON cryostat, which led to the demonstration of the nominal spectral performance of 2.8 eV at 5.9~keV with 16-row multiplexing, as well as an outlook for the future endeavours for the TDM readout integration on X-IFU's FPA-DM at SRON. △ Less

Submitted 5 March, 2024; originally announced March 2024.

Comments: Under publication in Journal of Low Temperature Physics

Advancing Glitch Classification in Gravity Spy: Multi-view Fusion with Attention-based Machine Learning for Advanced LIGO's Fourth Observing Run

Authors: Yunan Wu, Michael Zevin, Christopher P. L. Berry, Kevin Crowston, Carsten Østerlund, Zoheyr Doctor, Sharan Banagiri, Corey B. Jackson, Vicky Kalogera, Aggelos K. Katsaggelos

Abstract: The first successful detection of gravitational waves by ground-based observatories, such as the Laser Interferometer Gravitational-Wave Observatory (LIGO), marked a revolutionary breakthrough in our comprehension of the Universe. However, due to the unprecedented sensitivity required to make such observations, gravitational-wave detectors also capture disruptive noise sources called glitches, pot… ▽ More The first successful detection of gravitational waves by ground-based observatories, such as the Laser Interferometer Gravitational-Wave Observatory (LIGO), marked a revolutionary breakthrough in our comprehension of the Universe. However, due to the unprecedented sensitivity required to make such observations, gravitational-wave detectors also capture disruptive noise sources called glitches, potentially masking or appearing as gravitational-wave signals themselves. To address this problem, a community-science project, Gravity Spy, incorporates human insight and machine learning to classify glitches in LIGO data. The machine learning classifier, integrated into the project since 2017, has evolved over time to accommodate increasing numbers of glitch classes. Despite its success, limitations have arisen in the ongoing LIGO fourth observing run (O4) due to its architecture's simplicity, which led to poor generalization and inability to handle multi-time window inputs effectively. We propose an advanced classifier for O4 glitches. Our contributions include evaluating fusion strategies for multi-time window inputs, using label smoothing to counter noisy labels, and enhancing interpretability through attention module-generated weights. This development seeks to enhance glitch classification, aiding in the ongoing exploration of gravitational-wave phenomena. △ Less

Submitted 23 January, 2024; originally announced January 2024.

Leveraging Visual Supervision for Array-based Active Speaker Detection and Localization

Authors: Davide Berghi, Philip J. B. Jackson

Abstract: Conventional audio-visual approaches for active speaker detection (ASD) typically rely on visually pre-extracted face tracks and the corresponding single-channel audio to find the speaker in a video. Therefore, they tend to fail every time the face of the speaker is not visible. We demonstrate that a simple audio convolutional recurrent neural network (CRNN) trained with spatial input features ext… ▽ More Conventional audio-visual approaches for active speaker detection (ASD) typically rely on visually pre-extracted face tracks and the corresponding single-channel audio to find the speaker in a video. Therefore, they tend to fail every time the face of the speaker is not visible. We demonstrate that a simple audio convolutional recurrent neural network (CRNN) trained with spatial input features extracted from multichannel audio can perform simultaneous horizontal active speaker detection and localization (ASDL), independently of the visual modality. To address the time and cost of generating ground truth labels to train such a system, we propose a new self-supervised training pipeline that embraces a ``student-teacher'' learning approach. A conventional pre-trained active speaker detector is adopted as a ``teacher'' network to provide the position of the speakers as pseudo-labels. The multichannel audio ``student'' network is trained to generate the same results. At inference, the student network can generalize and locate also the occluded speakers that the teacher network is not able to detect visually, yielding considerable improvements in recall rate. Experiments on the TragicTalkers dataset show that an audio network trained with the proposed self-supervised learning approach can exceed the performance of the typical audio-visual methods and produce results competitive with the costly conventional supervised training. We demonstrate that improvements can be achieved when minimal manual supervision is introduced in the learning pipeline. Further gains may be sought with larger training sets and integrating vision with the multichannel audio system. △ Less

Submitted 21 December, 2023; originally announced December 2023.

Fusion of Audio and Visual Embeddings for Sound Event Localization and Detection

Authors: Davide Berghi, Peipei Wu, **zheng Zhao, Wenwu Wang, Philip J. B. Jackson

Abstract: Sound event localization and detection (SELD) combines two subtasks: sound event detection (SED) and direction of arrival (DOA) estimation. SELD is usually tackled as an audio-only problem, but visual information has been recently included. Few audio-visual (AV)-SELD works have been published and most employ vision via face/object bounding boxes, or human pose keypoints. In contrast, we explore th… ▽ More Sound event localization and detection (SELD) combines two subtasks: sound event detection (SED) and direction of arrival (DOA) estimation. SELD is usually tackled as an audio-only problem, but visual information has been recently included. Few audio-visual (AV)-SELD works have been published and most employ vision via face/object bounding boxes, or human pose keypoints. In contrast, we explore the integration of audio and visual feature embeddings extracted with pre-trained deep networks. For the visual modality, we tested ResNet50 and Inflated 3D ConvNet (I3D). Our comparison of AV fusion methods includes the AV-Conformer and Cross-Modal Attentive Fusion (CMAF) model. Our best models outperform the DCASE 2023 Task3 audio-only and AV baselines by a wide margin on the development set of the STARSS23 dataset, making them competitive amongst state-of-the-art results of the AV challenge, without model ensembling, heavy data augmentation, or prediction post-processing. Such techniques and further pre-training could be applied as next steps to improve performance. △ Less

Submitted 14 December, 2023; originally announced December 2023.

Comments: ICASSP 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Audio-Visual Speaker Tracking: Progress, Challenges, and Future Directions

Authors: **zheng Zhao, Yong Xu, Xinyuan Qian, Davide Berghi, Peipei Wu, Meng Cui, Jianyuan Sun, Philip J. B. Jackson, Wenwu Wang

Abstract: Audio-visual speaker tracking has drawn increasing attention over the past few years due to its academic values and wide application. Audio and visual modalities can provide complementary information for localization and tracking. With audio and visual information, the Bayesian-based filter can solve the problem of data association, audio-visual fusion and track management. In this paper, we condu… ▽ More Audio-visual speaker tracking has drawn increasing attention over the past few years due to its academic values and wide application. Audio and visual modalities can provide complementary information for localization and tracking. With audio and visual information, the Bayesian-based filter can solve the problem of data association, audio-visual fusion and track management. In this paper, we conduct a comprehensive overview of audio-visual speaker tracking. To our knowledge, this is the first extensive survey over the past five years. We introduce the family of Bayesian filters and summarize the methods for obtaining audio-visual measurements. In addition, the existing trackers and their performance on AV16.3 dataset are summarized. In the past few years, deep learning techniques have thrived, which also boosts the development of audio visual speaker tracking. The influence of deep learning techniques in terms of measurement extraction and state estimation is also discussed. At last, we discuss the connections between audio-visual speaker tracking and other areas such as speech separation and distributed speaker tracking. △ Less

Submitted 17 December, 2023; v1 submitted 23 October, 2023; originally announced October 2023.

Gravity Spy: Lessons Learned and a Path Forward

Authors: Michael Zevin, Corey B. Jackson, Zoheyr Doctor, Yunan Wu, Carsten Østerlund, L. Clifton Johnson, Christopher P. L. Berry, Kevin Crowston, Scott B. Coughlin, Vicky Kalogera, Sharan Banagiri, Derek Davis, Jane Glanzer, Renzhi Hao, Aggelos K. Katsaggelos, Oli Patane, Jennifer Sanchez, Joshua Smith, Siddharth Soni, Laura Trouille, Marissa Walker, Irina Aerith, Wilfried Domainko, Victor-Georges Baranowski, Gerhard Niklasch , et al. (1 additional authors not shown)

Abstract: The Gravity Spy project aims to uncover the origins of glitches, transient bursts of noise that hamper analysis of gravitational-wave data. By using both the work of citizen-science volunteers and machine-learning algorithms, the Gravity Spy project enables reliable classification of glitches. Citizen science and machine learning are intrinsically coupled within the Gravity Spy framework, with mac… ▽ More The Gravity Spy project aims to uncover the origins of glitches, transient bursts of noise that hamper analysis of gravitational-wave data. By using both the work of citizen-science volunteers and machine-learning algorithms, the Gravity Spy project enables reliable classification of glitches. Citizen science and machine learning are intrinsically coupled within the Gravity Spy framework, with machine-learning classifications providing a rapid first-pass classification of the dataset and enabling tiered volunteer training, and volunteer-based classifications verifying the machine classifications, bolstering the machine-learning training set and identifying new morphological classes of glitches. These classifications are now routinely used in studies characterizing the performance of the LIGO gravitational-wave detectors. Providing the volunteers with a training framework that teaches them to classify a wide range of glitches, as well as additional tools to aid their investigations of interesting glitches, empowers them to make discoveries of new classes of glitches. This demonstrates that, when giving suitable support, volunteers can go beyond simple classification tasks to identify new features in data at a level comparable to domain experts. The Gravity Spy project is now providing volunteers with more complicated data that includes auxiliary monitors of the detector to identify the root cause of glitches. △ Less

Submitted 31 January, 2024; v1 submitted 29 August, 2023; originally announced August 2023.

Comments: 33 pages, 5 figures, published in European Physical Journal Plus for focus issue on "Citizen science for physics: From Education and Outreach to Crowdsourcing fundamental research"

Journal ref: The European Physical Journal Plus, 139, 100 (2024)

PAT: Position-Aware Transformer for Dense Multi-Label Action Detection

Authors: Faegheh Sardari, Armin Mustafa, Philip J. B. Jackson, Adrian Hilton

Abstract: We present PAT, a transformer-based network that learns complex temporal co-occurrence action dependencies in a video by exploiting multi-scale temporal features. In existing methods, the self-attention mechanism in transformers loses the temporal positional information, which is essential for robust action detection. To address this issue, we (i) embed relative positional encoding in the self-att… ▽ More We present PAT, a transformer-based network that learns complex temporal co-occurrence action dependencies in a video by exploiting multi-scale temporal features. In existing methods, the self-attention mechanism in transformers loses the temporal positional information, which is essential for robust action detection. To address this issue, we (i) embed relative positional encoding in the self-attention mechanism and (ii) exploit multi-scale temporal relationships by designing a novel non hierarchical network, in contrast to the recent transformer-based approaches that use a hierarchical structure. We argue that joining the self-attention mechanism with multiple sub-sampling processes in the hierarchical approaches results in increased loss of positional information. We evaluate the performance of our proposed approach on two challenging dense multi-label benchmark datasets, and show that PAT improves the current state-of-the-art result by 1.1% and 0.6% mAP on the Charades and MultiTHUMOS datasets, respectively, thereby achieving the new state-of-the-art mAP at 26.5% and 44.6%, respectively. We also perform extensive ablation studies to examine the impact of the different components of our proposed network. △ Less

Submitted 9 August, 2023; originally announced August 2023.

CME Propagation Through the Heliosphere: Status and Future of Observations and Model Development

Authors: M. Temmer, C. Scolini, I. G. Richardson, S. G. Heinemann, E. Paouris, A. Vourlidas, M. M. Bisi, writing teams, :, N. Al-Haddad, T. Amerstorfer, L. Barnard, D. Buresova, S. J. Hofmeister, K. Iwai, B. V. Jackson, R. Jarolim, L. K. Jian, J. A. Linker, N. Lugaz, P. K. Manoharan, M. L. Mays, W. Mishra, M. J. Owens, E. Palmerio , et al. (9 additional authors not shown)

Abstract: The ISWAT clusters H1+H2 have a focus on interplanetary space and its characteristics, especially on the large-scale co-rotating and transient structures impacting Earth. SIRs, generated by the interaction between high-speed solar wind originating in large-scale open coronal magnetic fields and slower solar wind from closed magnetic fields, are regions of compressed plasma and magnetic field follo… ▽ More The ISWAT clusters H1+H2 have a focus on interplanetary space and its characteristics, especially on the large-scale co-rotating and transient structures impacting Earth. SIRs, generated by the interaction between high-speed solar wind originating in large-scale open coronal magnetic fields and slower solar wind from closed magnetic fields, are regions of compressed plasma and magnetic field followed by high-speed streams that recur at the ca. 27 day solar rotation period. Short-term reconfigurations of the lower coronal magnetic field generate flare emissions and provide the energy to accelerate enormous amounts of magnetised plasma and particles in the form of CMEs into interplanetary space. The dynamic interplay between these phenomena changes the configuration of interplanetary space on various temporal and spatial scales which in turn influences the propagation of individual structures. While considerable efforts have been made to model the solar wind, we outline the limitations arising from the rather large uncertainties in parameters inferred from observations that make reliable predictions of the structures impacting Earth difficult. Moreover, the increased complexity of interplanetary space as solar activity rises in cycle 25 is likely to pose a challenge to these models. Combining observational and modeling expertise will extend our knowledge of the relationship between these different phenomena and the underlying physical processes, leading to improved models and scientific understanding and more-reliable space-weather forecasting. The current paper summarizes the efforts and progress achieved in recent years, identifies open questions, and gives an outlook for the next 5-10 years. It acts as basis for updating the existing COSPAR roadmap by Schrijver+ (2015), as well as providing a useful and practical guide for peer-users and the next generation of space weather scientists. △ Less

Submitted 9 August, 2023; originally announced August 2023.

Comments: Accepted for publication in Advances in Space Research

Metrics for Optimizing Searches for Tidally Decaying Exoplanets

Authors: Brian Jackson, Elisabeth R. Adams, Jeffrey P. Morgenthaler

Abstract: Tidal interactions between short-period exoplanets and their host stars drive orbital decay and have likely led to engulfment of planets by their stars. Precise transit timing surveys, with baselines now spanning decades for some planets, are directly detecting orbital decay for a handful of planets, with corroboration for planetary engulfment coming from independent lines of evidence. More than t… ▽ More Tidal interactions between short-period exoplanets and their host stars drive orbital decay and have likely led to engulfment of planets by their stars. Precise transit timing surveys, with baselines now spanning decades for some planets, are directly detecting orbital decay for a handful of planets, with corroboration for planetary engulfment coming from independent lines of evidence. More than that, recent observations have perhaps even caught the moment of engulfment for one unfortunate planet. These portentous signs bolster prospects for ongoing surveys, but optimizing such a survey requires considering the astrophysical parameters that give rise to robust timing constraints and large tidal decay rates, as well as how best to schedule observations conducted over many years. The large number of possible targets means it is not feasible to continually observe all planets that might exhibit detectable tidal decay. In this study, we explore astrophysical and observational properties for a short-period exoplanet system that can maximize the likelihood for observing tidally driven transit-timing variations. We consider several fiducial observational strategies and real exoplanet systems reported to exhibit decay. We show that moderately frequent (a few transits per year) observations may suffice to detect tidal decay within just a few years. Tidally driven timing variations take time to grow to detectable levels, and so we estimate how long that growth takes as a function of timing uncertainties and tidal decay rate and provide thresholds for deciding that tidal decay has been detected. △ Less

Submitted 8 August, 2023; originally announced August 2023.

Comments: Accepted to AJ; 17 pages, 8 figures

Audio Inputs for Active Speaker Detection and Localization via Microphone Array

Authors: Davide Berghi, Philip J. B. Jackson

Abstract: This study considers the problem of detecting and locating an active talker's horizontal position from multichannel audio captured by a microphone array. We refer to this as active speaker detection and localization (ASDL). Our goal was to investigate the performance of spatial acoustic features extracted from the multichannel audio as the input of a convolutional recurrent neural network (CRNN),… ▽ More This study considers the problem of detecting and locating an active talker's horizontal position from multichannel audio captured by a microphone array. We refer to this as active speaker detection and localization (ASDL). Our goal was to investigate the performance of spatial acoustic features extracted from the multichannel audio as the input of a convolutional recurrent neural network (CRNN), in relation to the number of channels employed and additive noise. To this end, experiments were conducted to compare the generalized cross-correlation with phase transform (GCC-PHAT), the spatial cue-augmented log-spectrogram (SALSA) features, and a recently-proposed beamforming method, evaluating their robustness to various noise intensities. The array aperture and sampling density were tested by taking subsets from the 16-microphone array. Results and tests of statistical significance demonstrate the microphones' contribution to performance on the TragicTalkers dataset, which offers opportunities to investigate audio-visual approaches in the future. △ Less

Submitted 27 July, 2023; originally announced July 2023.

Validation of heliospheric modeling algorithms through pulsar observations I: Interplanetary scintillation-based tomography

Authors: C. Tiburzi, B. V. Jackson, L. Cota, G. M. Shaifullah, R. A. Fallows, M. Tokumaru, P. Zucca

Abstract: Solar-wind 3-D reconstruction tomography based on interplanetary scintillation (IPS) studies provides fundamental information for space-weather forecasting models, and gives the possibility to determine heliospheric column densities. Here we compare the time series of Solar-wind column densities derived from long-term observations of pulsars, and the Solar-wind reconstruction provided by the UCSD… ▽ More Solar-wind 3-D reconstruction tomography based on interplanetary scintillation (IPS) studies provides fundamental information for space-weather forecasting models, and gives the possibility to determine heliospheric column densities. Here we compare the time series of Solar-wind column densities derived from long-term observations of pulsars, and the Solar-wind reconstruction provided by the UCSD IPS tomography. This work represents a completely independent comparison and validation of these techniques to provide this measurement, and it strengthens confidence in the use of both in space-weather analyses applications. △ Less

Submitted 12 June, 2023; originally announced June 2023.

Comments: Published in Journal of Advances in Space Research

Journal ref: AdSpR (2022)

Answering Two OPAC Problems Involving Banff Quivers

Authors: Tucker J. Ervin, Blake Jackson

Abstract: In a post on the Open Problems in Algebraic Combinatorics (OPAC) blog, E. Bucher and J. Machacek posed three open problems: OPAC-033, OPAC-034, and OPAC-035. These three problems deal with the relationships between three infinite classes of quivers: the Banff, Louise, and $\mathcal{P}$ quivers. OPAC-034 asks whether or not every Banff quiver can be verified to be Banff by only considering sources… ▽ More In a post on the Open Problems in Algebraic Combinatorics (OPAC) blog, E. Bucher and J. Machacek posed three open problems: OPAC-033, OPAC-034, and OPAC-035. These three problems deal with the relationships between three infinite classes of quivers: the Banff, Louise, and $\mathcal{P}$ quivers. OPAC-034 asks whether or not every Banff quiver can be verified to be Banff by only considering sources and sinks, and OPAC-035 asks whether or not every Banff quiver is contained in the class $\mathcal{P}$. We give an answer to both questions, showing that every Banff quiver can be verified to be Banff by using sources and sinks, and therefore that every Banff quiver lives in the class $\mathcal{P}$. We also make some progress on OPAC-033, showing a result similar to our result OPAC-034 for Louise quivers. △ Less

Submitted 26 May, 2023; originally announced May 2023.

Comments: 10 pages, 1 figure

MSC Class: 05E40 (Primary) 13F60; 05E99 (Secondary)

An Intersection-Dimension Formula for Preprojective Modules of Type $\widetilde{D}_n$

Authors: Blake Jackson

Abstract: This paper proves the existence of an intersection-dimension formula for preprojective modules over path algebras of type $\widetilde{D}_n$. Identical intersection-dimension formulas have previously been provided for modules over path algebras of type $A_n, D_n,$ and $\widetilde{A}_n$ due to Schiffler as well as He, Zhou, and Zhu. These modules can be represented geometrically by some set of curve… ▽ More This paper proves the existence of an intersection-dimension formula for preprojective modules over path algebras of type $\widetilde{D}_n$. Identical intersection-dimension formulas have previously been provided for modules over path algebras of type $A_n, D_n,$ and $\widetilde{A}_n$ due to Schiffler as well as He, Zhou, and Zhu. These modules can be represented geometrically by some set of curves on special surfaces. The intersection-dimension formula is an equality of the intersection number between two curves and the dimensions of the first extension spaces between the two modules they represent. This paper takes a direct approach to proving the formula utilizing the known structure of the Auslander-Reiten quiver of type $\widetilde{D}_n$. Future work will extend the formula to the entire module category (not just the preprojective modules) over path algebras of type $\widetilde{D}_n$. △ Less

Submitted 23 May, 2023; originally announced May 2023.

Comments: 27 pages, 14 figures

MSC Class: 16G70 (Primary) 13F60; 16G20 (Secondary)

RoSI: Recovering 3D Shape Interiors from Few Articulation Images

Authors: Akshay Gadi Patil, Yiming Qian, Shan Yang, Brian Jackson, Eric Bennett, Hao Zhang

Abstract: The dominant majority of 3D models that appear in gaming, VR/AR, and those we use to train geometric deep learning algorithms are incomplete, since they are modeled as surface meshes and missing their interior structures. We present a learning framework to recover the shape interiors (RoSI) of existing 3D models with only their exteriors from multi-view and multi-articulation images. Given a set o… ▽ More The dominant majority of 3D models that appear in gaming, VR/AR, and those we use to train geometric deep learning algorithms are incomplete, since they are modeled as surface meshes and missing their interior structures. We present a learning framework to recover the shape interiors (RoSI) of existing 3D models with only their exteriors from multi-view and multi-articulation images. Given a set of RGB images that capture a target 3D object in different articulated poses, possibly from only few views, our method infers the interior planes that are observable in the input images. Our neural architecture is trained in a category-agnostic manner and it consists of a motion-aware multi-view analysis phase including pose, depth, and motion estimations, followed by interior plane detection in images and 3D space, and finally multi-view plane fusion. In addition, our method also predicts part articulations and is able to realize and even extrapolate the captured motions on the target 3D object. We evaluate our method by quantitative and qualitative comparisons to baselines and alternative solutions, as well as testing on untrained object categories and real image inputs to assess its generalization capabilities. △ Less

Submitted 13 April, 2023; originally announced April 2023.

Rigidity of Symmetric Simplicial Complexes and the Lower Bound Theorem

Authors: James Cruickshank, Bill Jackson, Shinichi Tanigawa

Abstract: We show that, if $Γ$ is a point group of $\mathbb{R}^{k+1}$ of order two for some $k\geq 2$ and $\mathcal S$ is a $k$-pseudomanifold which has a free automorphism of order two, then either $\mathcal S$ has a $Γ$-symmetric infinitesimally rigid realisation in $\mathbb{R}^{k+1}$ or $k=2$ and $Γ$ is a half-turn rotation group.This verifies a conjecture made by Klee, Nevo, Novik and Zhang for the case… ▽ More We show that, if $Γ$ is a point group of $\mathbb{R}^{k+1}$ of order two for some $k\geq 2$ and $\mathcal S$ is a $k$-pseudomanifold which has a free automorphism of order two, then either $\mathcal S$ has a $Γ$-symmetric infinitesimally rigid realisation in $\mathbb{R}^{k+1}$ or $k=2$ and $Γ$ is a half-turn rotation group.This verifies a conjecture made by Klee, Nevo, Novik and Zhang for the case when $Γ$ is a point-inversion group. Our result implies that Stanley's lower bound theorem for centrally symmetric polytopes extends to pseudomanifolds with a free simplicial involution, thus verifying (the inequality part) of another conjecture of Klee, Nevo, Novik and Zheng. Both results actually apply to a much larger class of simplicial complexes, namely the circuits of the simplicial matroid. The proof of our rigidity result adapts earlier ideas of Fogelsanger to the setting of symmetric simplicial complexes. △ Less

Submitted 10 April, 2023; originally announced April 2023.

Comments: 22 pages, 2 figures

MSC Class: 52C25 (Primary); 52B05; 13F55 (Secondary)

The Multiview Observatory for Solar Terrestrial Science (MOST)

Authors: N. Gopalswamy, S. Christe, S. F. Fung, Q. Gong, J. R. Gruesbeck, L. K. Jian, S. G. Kanekal, C. Kay, T. A. Kucera, J. E. Leake, L. Li, P. Makela, P. Nikulla, N. L. Reginald, A. Shih, S. K. Tadikonda, N. Viall, L. B. Wilson III, S. Yashiro, L. Golub, E. DeLuca, K. Reeves, A. C. Sterling, A. R. Winebarger, C. DeForest , et al. (32 additional authors not shown)

Abstract: We report on a study of the Multiview Observatory for Solar Terrestrial Science (MOST) mission that will provide comprehensive imagery and time series data needed to understand the magnetic connection between the solar interior and the solar atmosphere/inner heliosphere. MOST will build upon the successes of SOHO and STEREO missions with new views of the Sun and enhanced instrument capabilities. T… ▽ More We report on a study of the Multiview Observatory for Solar Terrestrial Science (MOST) mission that will provide comprehensive imagery and time series data needed to understand the magnetic connection between the solar interior and the solar atmosphere/inner heliosphere. MOST will build upon the successes of SOHO and STEREO missions with new views of the Sun and enhanced instrument capabilities. This article is based on a study conducted at NASA Goddard Space Flight Center that determined the required instrument refinement, spacecraft accommodation, launch configuration, and flight dynamics for mission success. MOST is envisioned as the next generation great observatory positioned to obtain three-dimensional information of large-scale heliospheric structures such as coronal mass ejections, stream interaction regions, and the solar wind itself. The MOST mission consists of 2 pairs of spacecraft located in the vicinity of Sun-Earth Lagrange points L4 (MOST1, MOST3) and L5 (MOST2 and MOST4). The spacecraft stationed at L4 (MOST1) and L5 (MOST2) will each carry seven remote-sensing and three in-situ instrument suites, including a novel radio package known as the Faraday Effect Tracker of Coronal and Heliospheric structures (FETCH). MOST3 and MOST4 will carry only the FETCH instruments and are positioned at variable locations along the Earth orbit up to 20° ahead of L4 and 20° behind L5, respectively. FETCH will have polarized radio transmitters and receivers on all four spacecraft to measure the magnetic content of solar wind structures propagating from the Sun to Earth using the Faraday rotation technique. The MOST mission will be able to sample the magnetized plasma throughout the Sun-Earth connected space during the mission lifetime over a solar cycle. △ Less

Submitted 10 December, 2023; v1 submitted 6 March, 2023; originally announced March 2023.

Comments: 42 pages, 19 figures, 8 tables, to appear in J. Atmospheric and Solar Terrestrial Physics

HAL3D: Hierarchical Active Learning for Fine-Grained 3D Part Labeling

Authors: Fenggen Yu, Yiming Qian, Francisca Gil-Ureta, Brian Jackson, Eric Bennett, Hao Zhang

Abstract: We present the first active learning tool for fine-grained 3D part labeling, a problem which challenges even the most advanced deep learning (DL) methods due to the significant structural variations among the small and intricate parts. For the same reason, the necessary data annotation effort is tremendous, motivating approaches to minimize human involvement. Our labeling tool iteratively verifies… ▽ More We present the first active learning tool for fine-grained 3D part labeling, a problem which challenges even the most advanced deep learning (DL) methods due to the significant structural variations among the small and intricate parts. For the same reason, the necessary data annotation effort is tremendous, motivating approaches to minimize human involvement. Our labeling tool iteratively verifies or modifies part labels predicted by a deep neural network, with human feedback continually improving the network prediction. To effectively reduce human efforts, we develop two novel features in our tool, hierarchical and symmetry-aware active labeling. Our human-in-the-loop approach, coined HAL3D, achieves 100% accuracy (barring human errors) on any test set with pre-defined hierarchical part labels, with 80% time-saving over manual effort. △ Less

Submitted 1 April, 2024; v1 submitted 25 January, 2023; originally announced January 2023.

Comments: Accepted to ICCV 2023

Data-Efficient Model Learning for Control with Jacobian-Regularized Dynamic-Mode Decomposition}

Authors: Brian E. Jackson, Jeong Hun Lee, Kevin Tracy, Zachary Manchester

Abstract: We present a data-efficient algorithm for learning models for model-predictive control (MPC). Our approach, Jacobian-Regularized Dynamic-Mode Decomposition (JDMD), offers improved sample efficiency over traditional Koopman approaches based on Dynamic-Mode Decomposition (DMD) by leveraging Jacobian information from an approximate prior model of the system, and improved tracking performance over tra… ▽ More We present a data-efficient algorithm for learning models for model-predictive control (MPC). Our approach, Jacobian-Regularized Dynamic-Mode Decomposition (JDMD), offers improved sample efficiency over traditional Koopman approaches based on Dynamic-Mode Decomposition (DMD) by leveraging Jacobian information from an approximate prior model of the system, and improved tracking performance over traditional model-based MPC. We demonstrate JDMD's ability to quickly learn bilinear Koopman dynamics representations across several realistic examples in simulation, including a perching maneuver for a fixed-wing aircraft with an empirically derived high-fidelity physics model. In all cases, we show that the models learned by JDMD provide superior tracking and generalization performance within a model-predictive control framework, even in the presence of significant model mismatch, when compared to approximate prior models and models learned by standard Extended DMD (EDMD). △ Less

Submitted 28 January, 2023; v1 submitted 25 October, 2022; originally announced December 2022.

Journal ref: Conference on Robot Learning (CoRL) 2022, Auckland, New Zealand

Tragic Talkers: A Shakespearean Sound- and Light-Field Dataset for Audio-Visual Machine Learning Research

Authors: Davide Berghi, Marco Volino, Philip J. B. Jackson

Abstract: 3D audio-visual production aims to deliver immersive and interactive experiences to the consumer. Yet, faithfully reproducing real-world 3D scenes remains a challenging task. This is partly due to the lack of available datasets enabling audio-visual research in this direction. In most of the existing multi-view datasets, the accompanying audio is neglected. Similarly, datasets for spatial audio re… ▽ More 3D audio-visual production aims to deliver immersive and interactive experiences to the consumer. Yet, faithfully reproducing real-world 3D scenes remains a challenging task. This is partly due to the lack of available datasets enabling audio-visual research in this direction. In most of the existing multi-view datasets, the accompanying audio is neglected. Similarly, datasets for spatial audio research primarily offer unimodal content, and when visual data is included, the quality is far from meeting the standard production needs. We present "Tragic Talkers", an audio-visual dataset consisting of excerpts from the "Romeo and Juliet" drama captured with microphone arrays and multiple co-located cameras for light-field video. Tragic Talkers provides ideal content for object-based media (OBM) production. It is designed to cover various conventional talking scenarios, such as monologues, two-people conversations, and interactions with considerable movement and occlusion, yielding 30 sequences captured from a total of 22 different points of view and two 16-element microphone arrays. Additionally, we provide voice activity labels, 2D face bounding boxes for each camera view, 2D pose detection keypoints, 3D tracking data of the mouth of the actors, and dialogue transcriptions. We believe the community will benefit from this dataset as it can assist multidisciplinary research. Possible uses of the dataset are discussed. △ Less

Submitted 4 December, 2022; originally announced December 2022.

Application of Novel Interplanetary Scintillation Visualisations using LOFAR: A Case Study of Merged CMEs from September 2017

Authors: R. A. Fallows, K. Iwai, B. V. Jackson, P. Zhang, M. M. Bisi, P. Zucca

Abstract: Observations of interplanetary scintillation (IPS - the scintillation of compact radio sources due to density variations in the solar wind) enable the velocity of the solar wind to be determined, and its bulk density to be estimated, throughout the inner heliosphere. A series of observations using the Low Frequency Array (LOFAR - a radio telescope centred on the Netherlands with stations across Eu… ▽ More Observations of interplanetary scintillation (IPS - the scintillation of compact radio sources due to density variations in the solar wind) enable the velocity of the solar wind to be determined, and its bulk density to be estimated, throughout the inner heliosphere. A series of observations using the Low Frequency Array (LOFAR - a radio telescope centred on the Netherlands with stations across Europe) were undertaken using this technique to observe the passage of an ultra-fast CME which launched from the Sun following the X-class flare of 10 September 2017. LOFAR observed the strong radio source 3C147 at an elongation of 82 degrees from the Sun over a period of more than 30 hours and observed a strong increase in speed to 900km/s followed two hours later by a strong increase in the level of scintillation, interpreted as a strong increase in density. Both speed and density remained enhanced for a period of more than seven hours, to beyond the period of observation. Further analysis of these data demonstrates a view of magnetic-field rotation due to the passage of the CME, using advanced IPS techniques only available to a unique instrument such as LOFAR. △ Less

Submitted 5 October, 2022; originally announced October 2022.

Comments: Accepted for publication in Advances in Space Research, 17 pages, 18 figures

Magnetohydrodynamic simulation of coronal mass ejections using interplanetary scintillation data observed from radio sites ISEE and LOFAR

Authors: Kazumasa Iwai, Richard A. Fallows, Mario M. Bisi, Daikou Shiota, Bernard V. Jackson, Munetoshi Tokumaru, Ken'ichi Fujiki

Abstract: Interplanetary scintillation (IPS) is a useful tool for detecting coronal mass ejections (CMEs) throughout interplanetary space. Global magnetohydrodynamic (MHD) simulations of the heliosphere, which are usually used to predict the arrival and geo-effectiveness of CMEs, can be improved using IPS data. In this study, we demonstrate an MHD simulation that includes IPS data from multiple stations to… ▽ More Interplanetary scintillation (IPS) is a useful tool for detecting coronal mass ejections (CMEs) throughout interplanetary space. Global magnetohydrodynamic (MHD) simulations of the heliosphere, which are usually used to predict the arrival and geo-effectiveness of CMEs, can be improved using IPS data. In this study, we demonstrate an MHD simulation that includes IPS data from multiple stations to improve CME modelling. The CMEs, which occurred on 09-10 September 2017, were observed over the period 10-12 September 2017 using the Low-Frequency Array (LOFAR) and IPS array of the Institute for Space-Earth Environmental Research (ISEE), Nagoya University, as they tracked through the inner heliosphere. We simulated CME propagation using a global MHD simulation, SUSANOO-CME, in which CMEs were modeled as spheromaks, and the IPS data were synthesised from the simulation results. The MHD simulation suggests that the CMEs merged in interplanetary space, forming complicated IPS g-level distributions in the sky map. We found that the MHD simulation that best fits both LOFAR and ISEE data provided a better reconstruction of the CMEs and a better forecast of their arrival at Earth than from measurements when these simulations were fit from the ISEE site alone. More IPS data observed from multiple stations at different local times in this study can help reconstruct the global structure of the CME, thus improving and evaluating the CME modelling. △ Less

Submitted 26 September, 2022; originally announced September 2022.

Comments: 17 pages, 9 figures, and 2 tables, accepted for publication in Advances in Space Research

Signal readout for Transition-Edge Sensor X-ray imaging spectrometers

Authors: H. Akamatsu, W. B. Doriese, J. A. B. Mates, B. D. Jackson

Abstract: Arrays of low-temperature microcalorimeters provide a promising technology for X-ray astrophysics: the imaging spectrometer. A camera with at least several thousand pixels, each of which has an energy-resolving power ($E/ΔE\urss{FWHM}$) of a few thousand across a broad energy range (200~eV to 10~keV or higher), would be a revolutionary instrument for the study of energetic astrophysical objects an… ▽ More Arrays of low-temperature microcalorimeters provide a promising technology for X-ray astrophysics: the imaging spectrometer. A camera with at least several thousand pixels, each of which has an energy-resolving power ($E/ΔE\urss{FWHM}$) of a few thousand across a broad energy range (200~eV to 10~keV or higher), would be a revolutionary instrument for the study of energetic astrophysical objects and phenomena. Signal readout is a critical enabling technology. Multiplexed readout, in which signals from multiple pixels are combined into a single amplifier channel, allows a kilo pixel-scale microcalorimeter array to meet the stringent requirements for power consumption, mass, volume, and cooling capacity in orbit. This chapter describes three different multiplexed-readout technologies for transition-edge-sensor microcalorimeters: time-division multiplexing, frequency-domain multiplexing, and microwave-SQUID multiplexing. For each multiplexing technique, we present the basic method, discuss some design considerations and parameters, and show the state of the art. The chapter concludes with a brief discussion of future prospects. △ Less

Submitted 12 September, 2022; originally announced September 2022.

Comments: 50 pages, 23 figures. This Chapter will appear in the Section "Detectors for X-ray Astrophysics" (Section Editors: J-W. den Harder, M. Feroci, N. Meidinger) of the "Handbook of X-ray and Gamma-ray Astrophysics" (Editors in chief: C. Bambi and A. Santangelo)

The Athena X-ray Integral Field Unit: a consolidated design for the system requirement review of the preliminary definition phase

Authors: Didier Barret, Vincent Albouys, Jan-Willem den Herder, Luigi Piro, Massimo Cappi, Juhani Huovelin, Richard Kelley, J. Miguel Mas-Hesse, Stéphane Paltani, Gregor Rauw, Agata Rozanska, Jiri Svoboda, Joern Wilms, Noriko Yamasaki, Marc Audard, Simon Bandler, Marco Barbera, Xavier Barcons, Enrico Bozzo, Maria Teresa Ceballos, Ivan Charles, Elisa Costantini, Thomas Dauser, Anne Decourchelle, Lionel Duband , et al. (274 additional authors not shown)

Abstract: The Athena X-ray Integral Unit (X-IFU) is the high resolution X-ray spectrometer, studied since 2015 for flying in the mid-30s on the Athena space X-ray Observatory, a versatile observatory designed to address the Hot and Energetic Universe science theme, selected in November 2013 by the Survey Science Committee. Based on a large format array of Transition Edge Sensors (TES), it aims to provide sp… ▽ More The Athena X-ray Integral Unit (X-IFU) is the high resolution X-ray spectrometer, studied since 2015 for flying in the mid-30s on the Athena space X-ray Observatory, a versatile observatory designed to address the Hot and Energetic Universe science theme, selected in November 2013 by the Survey Science Committee. Based on a large format array of Transition Edge Sensors (TES), it aims to provide spatially resolved X-ray spectroscopy, with a spectral resolution of 2.5 eV (up to 7 keV) over an hexagonal field of view of 5 arc minutes (equivalent diameter). The X-IFU entered its System Requirement Review (SRR) in June 2022, at about the same time when ESA called for an overall X-IFU redesign (including the X-IFU cryostat and the cooling chain), due to an unanticipated cost overrun of Athena. In this paper, after illustrating the breakthrough capabilities of the X-IFU, we describe the instrument as presented at its SRR, browsing through all the subsystems and associated requirements. We then show the instrument budgets, with a particular emphasis on the anticipated budgets of some of its key performance parameters. Finally we briefly discuss on the ongoing key technology demonstration activities, the calibration and the activities foreseen in the X-IFU Instrument Science Center, and touch on communication and outreach activities, the consortium organisation, and finally on the life cycle assessment of X-IFU aiming at minimising the environmental footprint, associated with the development of the instrument. Thanks to the studies conducted so far on X-IFU, it is expected that along the design-to-cost exercise requested by ESA, the X-IFU will maintain flagship capabilities in spatially resolved high resolution X-ray spectroscopy, enabling most of the original X-IFU related scientific objectives of the Athena mission to be retained. (abridged). △ Less

Submitted 28 November, 2022; v1 submitted 30 August, 2022; originally announced August 2022.

Comments: 48 pages, 29 figures, Accepted for publication in Experimental Astronomy with minor editing

Susceptibility study of TES micro-calorimeters for X-ray spectroscopy under FDM readout

Authors: Davide Vaccaro, Hiroki Akamatsu, Luciano Gottardi, Jan van der Kuur, Emanuele Taralli, Martin de Wit, Marcel P. Bruijn, Roland den Hartog, Mikko Kiviranta, Anton J. van der Linden, Kenichiro Nagayoshi, Kevin Ravensberg, Marcel L. Ridder, Sven Visser, Brian D. Jackson, Jian-Rong Gao, Ruud W. M. Hoogeveen, Jan-Willem A. den Herder

Abstract: We present a characterization of the sensitivity of TES X-ray micro-calorimeters to environmental conditions under frequency-domain multiplexing (FDM) readout. In the FDM scheme, each TES in a readout chain is in series with a LC band-pass filter and AC biased with an independent carrier at MHz range. Using TES arrays, cold readout circuitry and warm electronics fabricated at SRON and SQUIDs produ… ▽ More We present a characterization of the sensitivity of TES X-ray micro-calorimeters to environmental conditions under frequency-domain multiplexing (FDM) readout. In the FDM scheme, each TES in a readout chain is in series with a LC band-pass filter and AC biased with an independent carrier at MHz range. Using TES arrays, cold readout circuitry and warm electronics fabricated at SRON and SQUIDs produced at VTT Finland, we characterize the sensitivity of the detectors to bias voltage, bath temperature and magnetic field. We compare our results with the requirements for the Athena X-IFU instrument, showing the compliance of the measured sensitivities. We find in particular that FDM is intrinsically insensitive to the magnetic field because of TES design and AC readout. △ Less

Submitted 23 August, 2022; originally announced August 2022.

Comments: This paper has been accepted for publication in Journal of Low Temperature Physics

Journal ref: Journal of Low Temperature Physics 2022

Worldwide scaling of waste generation in urban systems

Authors: Mingzhen Lu, Chuanbin Zhou, Chenghao Wang, Robert B. Jackson, Christopher P. Kempes

Abstract: The production of waste as a consequence of human activities is one of the most fundamental challenges facing our society and global ecological systems. Waste generation is rapidly increasing, with corresponding shifts in the structure of our societies where almost all nations are moving from rural agrarian societies to urban and technological ones. However, the connections between these radical s… ▽ More The production of waste as a consequence of human activities is one of the most fundamental challenges facing our society and global ecological systems. Waste generation is rapidly increasing, with corresponding shifts in the structure of our societies where almost all nations are moving from rural agrarian societies to urban and technological ones. However, the connections between these radical societal shifts and waste generation have not yet been described. Here we apply scaling theory to establish a new understanding of waste in urban systems. We identify universal scaling laws of waste generation across diverse urban systems worldwide for three forms of waste: wastewater, municipal solid waste, and greenhouse gasses. We show that wastewater generation scales superlinearly, municipal solid waste scales linearly, and greenhouse gasses scales sublinearly with city size. In specific cases production can be understood in terms of city size coupled with financial and natural resources. For example, wastewater generation can be understood in terms of the increased economic activity of larger cities, and the deviations around the scaling relationship - indicating relative efficiency - depend on GDP per person and local rainfall. We also show how the temporal evolution of these scaling relationships reveals a loss of economies of scale and the general increase in waste production, where sublinear scaling relationships become linear. Our findings suggest general mechanisms controlling waste generation across diverse cities and global urban systems. Our approach offers a systematic approach to uncover these underlying mechanisms that might be key to reducing waste and pursing a more sustainable future. △ Less

Submitted 16 August, 2022; originally announced August 2022.

METER-ML: A Multi-Sensor Earth Observation Benchmark for Automated Methane Source Map**

Authors: Bryan Zhu, Nicholas Lui, Jeremy Irvin, Jimmy Le, Sahil Tadwalkar, Chenghao Wang, Zutao Ouyang, Frankie Y. Liu, Andrew Y. Ng, Robert B. Jackson

Abstract: Reducing methane emissions is essential for mitigating global warming. To attribute methane emissions to their sources, a comprehensive dataset of methane source infrastructure is necessary. Recent advancements with deep learning on remotely sensed imagery have the potential to identify the locations and characteristics of methane sources, but there is a substantial lack of publicly available data… ▽ More Reducing methane emissions is essential for mitigating global warming. To attribute methane emissions to their sources, a comprehensive dataset of methane source infrastructure is necessary. Recent advancements with deep learning on remotely sensed imagery have the potential to identify the locations and characteristics of methane sources, but there is a substantial lack of publicly available data to enable machine learning researchers and practitioners to build automated map** approaches. To help fill this gap, we construct a multi-sensor dataset called METER-ML containing 86,599 georeferenced NAIP, Sentinel-1, and Sentinel-2 images in the U.S. labeled for the presence or absence of methane source facilities including concentrated animal feeding operations, coal mines, landfills, natural gas processing plants, oil refineries and petroleum terminals, and wastewater treatment plants. We experiment with a variety of models that leverage different spatial resolutions, spatial footprints, image products, and spectral bands. We find that our best model achieves an area under the precision recall curve of 0.915 for identifying concentrated animal feeding operations and 0.821 for oil refineries and petroleum terminals on an expert-labeled test set, suggesting the potential for large-scale map**. We make METER-ML freely available at https://stanfordmlgroup.github.io/projects/meter-ml/ to support future work on automated methane source map**. △ Less

Submitted 15 August, 2022; v1 submitted 22 July, 2022; originally announced July 2022.

Comments: Workshop on Complex Data Challenges in Earth Observation at IJCAI-ECAI 2022

ATHENA X-IFU Demonstration Model: First joint operation of the main TES Array and its Cryogenic AntiCoincidence Detector (CryoAC)

Authors: M. D'Andrea, K. Ravensberg, A. Argan, D. Brienza, S. Lotti, C. Macculi, G. Minervini, L. Piro, G. Torrioli, F. Chiarello, L. Ferrari Barusso, M. Biasotti, G. Gallucci, F. Gatti, M. Rigano, H. Akamatsu, J. Dercksen, L. Gottardi, F. de Groote, R. den Hartog, J. -W. den Herder, R. Hoogeveen, B. Jackson, A. McCalden, S. Rosman , et al. (6 additional authors not shown)

Abstract: The X-IFU is the cryogenic spectrometer onboard the future ATHENA X-ray observatory. It is based on a large array of TES microcalorimeters, which works in combination with a Cryogenic AntiCoincidence detector (CryoAC). This is necessary to reduce the particle background level thus enabling part of the mission science goals. Here we present the first joint test of X-IFU TES array and CryoAC Demonst… ▽ More The X-IFU is the cryogenic spectrometer onboard the future ATHENA X-ray observatory. It is based on a large array of TES microcalorimeters, which works in combination with a Cryogenic AntiCoincidence detector (CryoAC). This is necessary to reduce the particle background level thus enabling part of the mission science goals. Here we present the first joint test of X-IFU TES array and CryoAC Demonstration Models, performed in a FDM setup. We show that it is possible to operate properly both detectors, and we provide a preliminary demonstration of the anti-coincidence capability of the system achieved by the simultaneous detection of cosmic muons. △ Less

Submitted 30 June, 2022; originally announced June 2022.

Comments: Accepted for publication in the Journal of Low Temperature Physics for LTD-19 special issue

Global Rigidity of Triangulated Manifolds

Authors: James Cruickshank, Bill Jackson, Shin-ichi Tanigawa

Abstract: We prove that if $G$ is the graph of a connected triangulated $(d-1)$-manifold, for $d\geq 3$, then $G$ is generically globally rigid in $\mathbb R^d$ if and only if it is $(d+1)$-connected and, if $d=3$, $G$ is not planar. The special case $d=3$ verifies a conjecture of Connelly. Our results actually apply to a much larger class of simplicial complexes, namely the circuits of the simplicial matro… ▽ More We prove that if $G$ is the graph of a connected triangulated $(d-1)$-manifold, for $d\geq 3$, then $G$ is generically globally rigid in $\mathbb R^d$ if and only if it is $(d+1)$-connected and, if $d=3$, $G$ is not planar. The special case $d=3$ verifies a conjecture of Connelly. Our results actually apply to a much larger class of simplicial complexes, namely the circuits of the simplicial matroid. We also give two significant applications of our main theorems. We show that that the characterisation of pseudomanifolds with extremal edge numbers given by the Lower Bound Theorem extends to circuits of the simplicial matroid. We also prove the generic case of a conjecture of Kalai concerning the reconstructability of a polytope from its space of stresses. The proofs of our main results adapt earlier ideas of Fogelsanger and Whiteley to the setting of global rigidity. In particular we verify a special case of Whiteley's vertex splitting conjecture for global rigidity. △ Less

Submitted 28 March, 2024; v1 submitted 5 April, 2022; originally announced April 2022.

Comments: Revised version subnitted to journal

MSC Class: 52C25; 05E45 (Primary) 52B05; 05C10 (Secondary)

Estimating the Heights of Martian Vortices from Mars 2020 MEDA Data

Authors: Brian Jackson

Abstract: Small convective vortices occur ubiquitously on Mars, frequently as dust devils, and they produce detectable signals in meteorological data -- in pressure, temperature, and wind speed and direction. In addition to being important contributors to the martian dust budget, convective vortices may serve as probes of the boundary layer, providing clues on convective instability, boundary layer diurnal… ▽ More Small convective vortices occur ubiquitously on Mars, frequently as dust devils, and they produce detectable signals in meteorological data -- in pressure, temperature, and wind speed and direction. In addition to being important contributors to the martian dust budget, convective vortices may serve as probes of the boundary layer, providing clues on convective instability, boundary layer diurnal evolution, and surface-atmosphere interactions. Using vortices as boundary layer probes requires a detailed understanding of the link between their properties and occurrence rates and the conditions that produce them. Fortunately, the growing cache of data from the Mars Environmental Dynamics Analyzer (MEDA) instrument suite onboard the Mars 2020 Perseverance rover promises to elucidate these relationships. In this study, we present a catalog of vortex detections from mission sols 90 through 179 to bolster our previous catalog based on sols 15 through 89. Consistent with predictions, we find more vortex encounters during this second half of the mission than from the first half. In addition to analyzing the pressure signals from these vortex encounters, we also use a Gaussian process analysis to recover contemporaneous temperature signals. By combining these signals with a long-established thermodynamics model, we estimate heights of the vortices and find some agreement with previous work and evidence for the diurnal growth and decay of the martian boundary layer. We also discuss prospects for additional boundary layer studies using martian vortex encounters. △ Less

Submitted 17 August, 2022; v1 submitted 1 April, 2022; originally announced April 2022.

Comments: 16 pages, 9 figures, accepted by PSJ. Full catalog of vortex detections available as MRT here - https://drive.google.com/file/d/10apMdeNC3i4Vi1OFnG76qWACvBRQoHW_/view?usp=sharing

Visually Supervised Speaker Detection and Localization via Microphone Array

Authors: Davide Berghi, Adrian Hilton, Philip J. B. Jackson

Abstract: Active speaker detection (ASD) is a multi-modal task that aims to identify who, if anyone, is speaking from a set of candidates. Current audio-visual approaches for ASD typically rely on visually pre-extracted face tracks (sequences of consecutive face crops) and the respective monaural audio. However, their recall rate is often low as only the visible faces are included in the set of candidates.… ▽ More Active speaker detection (ASD) is a multi-modal task that aims to identify who, if anyone, is speaking from a set of candidates. Current audio-visual approaches for ASD typically rely on visually pre-extracted face tracks (sequences of consecutive face crops) and the respective monaural audio. However, their recall rate is often low as only the visible faces are included in the set of candidates. Monaural audio may successfully detect the presence of speech activity but fails in localizing the speaker due to the lack of spatial cues. Our solution extends the audio front-end using a microphone array. We train an audio convolutional neural network (CNN) in combination with beamforming techniques to regress the speaker's horizontal position directly in the video frames. We propose to generate weak labels using a pre-trained active speaker detector on pre-extracted face tracks. Our pipeline embraces the "student-teacher" paradigm, where a trained "teacher" network is used to produce pseudo-labels visually. The "student" network is an audio network trained to generate the same results. At inference, the student network can independently localize the speaker in the visual frames directly from the audio input. Experimental results on newly collected data prove that our approach significantly outperforms a variety of other baselines as well as the teacher network itself. It results in an excellent speech activity detector too. △ Less

Submitted 7 March, 2022; originally announced March 2022.

Comments: Erratum: Due to a bug in the evaluation script, the correct average distance (aD) metric is here reported in yellow. The analysis remains unchanged from the original paper as the trend between the old and new measures are perfectly monotonic. The bug was caused by an incorrect normalization factor

Journal ref: IEEE 23rd International Workshop on Multimedia Signal Processing (MMSP), 2021

Investigating Mercury's Environment with the Two-Spacecraft BepiColombo Mission

Authors: A. Milillo, M. Fujimoto, G. Murakami, J. Benkhoff, J. Zender, S. Aizawa, M. Dósa, L. Griton, D. Heyner, G. Ho, S. M. Imber, X. Jia, T. Karlsson, R. M. Killen, M. Laurenza, S. T. Lindsay, S. McKenna-Lawlor, A. Mura, J. M. Raines, D. A. Rothery, N. André, W. Baumjohann, A. Berezhnoy, P. -A. Bourdin, E. J. Bunce , et al. (54 additional authors not shown)

Abstract: The ESA-JAXA BepiColombo mission will provide simultaneous measurements from two spacecraft, offering an unprecedented opportunity to investigate magnetospheric and exospheric dynamics at Mercury as well as their interactions with the solar wind, radiation, and interplanetary dust. Many scientific instruments onboard the two spacecraft will be completely, or partially devoted to study the near-spa… ▽ More The ESA-JAXA BepiColombo mission will provide simultaneous measurements from two spacecraft, offering an unprecedented opportunity to investigate magnetospheric and exospheric dynamics at Mercury as well as their interactions with the solar wind, radiation, and interplanetary dust. Many scientific instruments onboard the two spacecraft will be completely, or partially devoted to study the near-space environment of Mercury as well as the complex processes that govern it. Many issues remain unsolved even after the MESSENGER mission that ended in 2015. The specific orbits of the two spacecraft, MPO and Mio, and the comprehensive scientific payload allow a wider range of scientific questions to be addressed than those that could be achieved by the individual instruments acting alone, or by previous missions. These joint observations are of key importance because many phenomena in Mercury's environment are highly temporally and spatially variable. Examples of possible coordinated observations are described in this article, analysing the required geometrical conditions, pointing, resolutions and operation timing of different BepiColombo instruments sensors. △ Less

Submitted 26 February, 2022; originally announced February 2022.

Comments: 78 pages, 14 figures, published

Journal ref: Space Science Reviews (2020), Volume 216, Issue 5, article id.93

Demonstration of MHz frequency domain multiplexing readout of 37 transition edge sensors for high-resolution x-ray imaging spectrometers

Authors: H. Akamatsu, D. Vaccaro, L. Gottardi, J. van der Kuur, C. P. de Vries, M. Kiviranta, K. Ravensberg, M. D'Andrea, E. Taralli, M. de Wit, M. P. Bruijn, P. van der Hulst, R. H. den Hartog, B-J. van Leeuwen, A. J. van der Linden, A. J McCalden, K. Nagayoshi, A. C. T. Nieuwenhuizen, M. L. Ridder, S. Visser, P. van Winden, J. R. Gao, R. W. M. Hoogeveen, B. D. Jackson, J-W. A. den Herder

Abstract: We report on the development and demonstration of a MHz frequency domain multiplexing (FDM) technology to read out arrays of cryogenic transition edge sensor (TES) X-ray microcalorimeters. In our FDM scheme, TESs are AC-biased at different resonant frequencies in the low MHz range through an array of high-$Q$ LC resonators. The current signals of all TESs are summed at superconducting quantum inte… ▽ More We report on the development and demonstration of a MHz frequency domain multiplexing (FDM) technology to read out arrays of cryogenic transition edge sensor (TES) X-ray microcalorimeters. In our FDM scheme, TESs are AC-biased at different resonant frequencies in the low MHz range through an array of high-$Q$ LC resonators. The current signals of all TESs are summed at superconducting quantum interference devices (SQUIDs). We have demonstrated multiplexing for a readout of 31 pixels using room temperature electronics, high-$Q$ LC filters and TES arrays developed at SRON, and SQUID arrays from VTT. We repeated this on a second setup with 37 pixels. The summed X-ray spectral resolutions $@$ 5.9 keV are $ΔE_{\rm 31 pix ~MUX}=2.14\pm0.03$ eV and $ΔE_{\rm 37 pix ~MUX}=2.23\pm0.03$ eV. The demonstrated results are comparable with other multiplexing approaches. There is potential to further improve the spectral resolution and to increase the number of multiplexed TESs, and to open up applications for TES X-ray microcalorimeters. △ Less

Submitted 2 November, 2021; originally announced November 2021.

Comments: 6 pages, 3 figures, accepted for publication on Applied physics letters

Journal ref: Appl. Phys. Lett. 119, 182601 (2021)

Vortices and Dust Devils As Observed by the MEDA Instruments onboard Mars 2020 Perseverance Rover

Authors: Brian Jackson

Abstract: An important and perhaps dominant source of dust in the martian atmosphere, dust devils play a key role in Mars' climate. Datasets from previous landed missions have revealed dust devil activity, constrained their structures, and elucidated their dust-lifting capacities. However, each landing site and observational season exhibits unique meteorological properties that shape dust devil activity and… ▽ More An important and perhaps dominant source of dust in the martian atmosphere, dust devils play a key role in Mars' climate. Datasets from previous landed missions have revealed dust devil activity, constrained their structures, and elucidated their dust-lifting capacities. However, each landing site and observational season exhibits unique meteorological properties that shape dust devil activity and help illuminate their dependence on ambient conditions. The recent release of data from the Mars Environmental Dynamics Analyzer (MEDA) instrument suite onboard the Mars 2020 Perseverance rover promises a new treasure-trove for dust devil studies. In this study, we sift the time-series from MEDA's Pressure Sensor (PS) and Radiative and Dust Sensors (RDS) to look for the signals of passing vortices and dust devils. We detected 309 vortex encounters over the mission's first 89 sols. Consistent with predictions, these encounter rates exceed InSight and Curiosity's encounter rates. The RDS time-series also allows us to assess whether a passing vortex is likely to be dusty (and therefore is a true dust devil) or dustless. We find that about one quarter of vortices show signs of dust-lofting, although unfavorable encounter geometries may have prevented us from detecting dust for other vortices. In addition to these results, we discuss prospects for vortex studies as additional data from Mars 2020 are processed and made available. △ Less

Submitted 20 December, 2021; v1 submitted 9 September, 2021; originally announced September 2021.

Comments: Accepted by PSJ on 2021 Dec 20. Some corrections from the prior version

Permutations whose reverse shares the same recording tableau in the RSK correspondence

Authors: Tucker J. Ervin, Blake Jackson, Jay Lane, Kyungyong Lee, Son Dang Nguyen, Jack O'Donohue, Michael Vaughan

Abstract: The RSK correspondence is a bijection between permutations and pairs of standard Young tableaux with identical shape, where the tableaux are commonly denoted $P$ (insertion) and $Q$ (recording). It has been an open problem to demonstrate… ▽ More The RSK correspondence is a bijection between permutations and pairs of standard Young tableaux with identical shape, where the tableaux are commonly denoted $P$ (insertion) and $Q$ (recording). It has been an open problem to demonstrate $$ |\{w \in \mathfrak{S}_n | \, Q(w) = Q(w^r)\}| = \begin{cases} \displaystyle 2^{\frac{n-1}{2}}{n-1 \choose \frac{n-1}{2}} & n \text{ odd} \newline \displaystyle 0 & n \text{ even} \end{cases}, $$ where $w^r$ is the reverse permutation of $w$. First we show that for each $w$ where $Q(w) = Q(w^r)$ the recording tableau $Q(w)$ has a symmetric hook shape and satisfies a certain simple property. From these two results, we succeed in proving the desired identity. △ Less

Submitted 19 August, 2021; originally announced August 2021.

Comments: 14 pages, 4 figures

MSC Class: 05A15 (Primary) 05A05; 05A17; 05A19 (Secondary)

Journal ref: Séminaire Lotharingien de Combinatoire, B86a (2022), 15 pp

Mutations of reflections and existence of pseudo-acyclic orderings for type $A_n$

Authors: Tucker J. Ervin, Blake Jackson, Kyu-Hwan Lee, Kyungyong Lee

Abstract: In a recent paper by K.-H. Lee, K. Lee and M. Mills, a mutation of reflections in the universal Coxeter group is defined in association with a mutation of a quiver. A matrix representation of these reflections is determined by a linear ordering on the set of vertices of the quiver. It was conjectured that there exists an ordering (called a pseudo-acyclic ordering in this paper) such that whenever… ▽ More In a recent paper by K.-H. Lee, K. Lee and M. Mills, a mutation of reflections in the universal Coxeter group is defined in association with a mutation of a quiver. A matrix representation of these reflections is determined by a linear ordering on the set of vertices of the quiver. It was conjectured that there exists an ordering (called a pseudo-acyclic ordering in this paper) such that whenever two mutation sequences of a quiver lead to the same labeled seed, the representations of the associated reflections also coincide. In this paper, we prove this conjecture for every quiver mutation-equivalent to an orientation of a type $A_n$ Dynkin diagram by decomposing a mutation sequence into a product of elementary swaps and checking relations studied by Barot and Marsh. △ Less

Submitted 6 August, 2021; originally announced August 2021.

Comments: 43 Pages, 9 Figures

MSC Class: 13F60 (Primary)

Inferring Vortex and Dust Devil Statistics from InSight

Authors: Brian Jackson, Justin Crevier, Michelle Szurgot, Ryan Battin, Clément Perrin, Sébastien Rodriguez

Abstract: The InSight mission has operated on the surface of Mars for nearly two Earth years, returning detections of the first Marsquakes. The lander also deployed a meteorological instrument package and cameras to monitor local surface activity. These instruments have detected boundary layer phenomena, including small-scale vortices. These vortices register as short-lived, negative pressure excursions and… ▽ More The InSight mission has operated on the surface of Mars for nearly two Earth years, returning detections of the first Marsquakes. The lander also deployed a meteorological instrument package and cameras to monitor local surface activity. These instruments have detected boundary layer phenomena, including small-scale vortices. These vortices register as short-lived, negative pressure excursions and closely resemble those that could generate dust devils. Although our analysis shows InSight encountered more than 900 vortices and collected more than 1000 images of the martian surface, no active dust devils were imaged. In spite of the lack of dust devil detections, we can leverage the vortex detections and InSight's daily wind speed measurements to learn about the boundary layer processes that create dust devils. We discuss our analysis of InSight's meteorological data to assess the statistics of vortex and dust devil activity. We also infer encounter distances for the vortices and, therefrom, the maximum vortex wind speeds. Surveying the available imagery, we place upper limits on what fraction of vortices carry dust (i.e., how many are bonafide dust devils) and estimate threshold wind speeds for dust lifting. Comparing our results to detections of dust devil tracks seen in space-based observations of the InSight landing site, we can also infer thresholds and frequency of track formation by vortices. Comparing vortex encounters and parameters with advective wind speeds, we find evidence that high wind speeds at InSight may have suppressed the formation of dust devils, explaining the lack of imaged dust devils. △ Less

Submitted 10 September, 2021; v1 submitted 26 July, 2021; originally announced July 2021.

Comments: 23 pages, 15 figures, accepted for publication by Planetary Science Journal; This version involves a few minor revisions compared to the first version

A test platform for the detection and readout chain for the Athena X-IFU

Authors: Gabriele Betancourt-Martinez, François Pajot, Sophie Beaumont, Gilles Roudil, Joseph Adams, Hiroki Akamatsu, Simon Bandler, Bernard Bertrand, Marcel Bruijn, Florent Castellani, Edoardo Cucchetti, William Doriese, Michel Dupieux, Hervé Geoffray, Luciano Gottardi, Brian Jackson, Jan van der Kuur, Mikko Kiviranta, Antoine Miniussi, Phillipe Peille, Kevin Ravensberg, Laurent Ravera, Carl Reintsema, Kazuhiro Sakai, Stephen Smith , et al. (4 additional authors not shown)

Abstract: We present a test platform for the Athena X-IFU detection chain, which will serve as the first demonstration of the representative end-to-end detection and readout chain for the X-IFU, using prototypes of the future flight electronics and currently available subsystems. This test bench, housed in a commercial two-stage ADR cryostat, includes a focal plane array placed at the 50 mK cold stage of th… ▽ More We present a test platform for the Athena X-IFU detection chain, which will serve as the first demonstration of the representative end-to-end detection and readout chain for the X-IFU, using prototypes of the future flight electronics and currently available subsystems. This test bench, housed in a commercial two-stage ADR cryostat, includes a focal plane array placed at the 50 mK cold stage of the ADR with a kilopixel array of transition-edge sensor microcalorimeter spectrometers and associated cold readout electronics. Prototype room temperature electronics for the X-IFU provide the readout, and will evolve over time to become more representative of the X-IFU mission baseline. The test bench yields critical feedback on subsystem designs and interfaces, in particular the warm readout electronics, and will provide an in-house detection system for continued testing and development of the warm readout electronics and for the validation of X-ray calibration sources. In this paper, we describe the test bench subsystems and design, characterization of the cryostat, and current status of the project. △ Less

Submitted 7 July, 2021; originally announced July 2021.

Comments: 11 pages, 4 figures, to appear in Proc. SPIE Astronomical Telescopes and Instrumentation, 2020

Evidence for low kick velocities among high-mass X-ray binaries in the Small Magellanic Cloud from the spatial correlation function

Authors: Arash Bodaghee, Vallia Antoniou, Andreas Zezas, John A. Tomsick, Zachary Jordan, Eric Frechette, Brenton Jackson, Ryan Agnew, Ann E. Hornschemeier, Jerome Rodriguez

Abstract: We present the two-point cross-correlation function between high-mass X-ray binaries (HMXBs) in the Small Magellanic Cloud (SMC) and their likely birthplaces (OB Associations: OBAs). This function compares the spatial correlation between the observed HMXB and OBA populations against mock catalogs in which the members are distributed randomly across the sky. A significant correlation (15 sigma) is… ▽ More We present the two-point cross-correlation function between high-mass X-ray binaries (HMXBs) in the Small Magellanic Cloud (SMC) and their likely birthplaces (OB Associations: OBAs). This function compares the spatial correlation between the observed HMXB and OBA populations against mock catalogs in which the members are distributed randomly across the sky. A significant correlation (15 sigma) is found for the HMXB and OBA populations when compared with a randomized catalog in which the OBAs are distributed uniformly over the SMC. A less significant correlation (4 sigma) is found for a randomized catalog of OBAs built with a bootstrap method. However, no significant correlation is detected when the randomized catalogs assume the form of a Gaussian ellipsoid or a distribution that reflects the star-formation history from 40 Myr ago. Based on their observed distributions and assuming a range of migration timescales, we infer that the average value of the kick velocity inherited by an HMXB during the formation of its compact object is 2-34 km/s. This is considerably less than the value obtained for their counterparts in the Milky Way hinting that the galactic environment affecting stellar evolution plays a role in setting the average kick velocity of HMXBs. △ Less

Submitted 7 July, 2021; v1 submitted 6 July, 2021; originally announced July 2021.

Comments: 10 pages, 7 figures, accepted for publication in the Astrophysical Journal

Coincident Rigidity of 2-Dimensional Frameworks

Authors: Hakan Guler, Bill Jackson

Abstract: Fekete, Jordán and Kaszanitzky [4] characterised the graphs which can be realised as 2-dimensional, infinitesimally rigid, bar-joint frameworks in which two given vertices are coincident. We formulate a conjecture which would extend their characterisation to an arbitrary set T of vertices and verify our conjecture when |T| = 3. Fekete, Jordán and Kaszanitzky [4] characterised the graphs which can be realised as 2-dimensional, infinitesimally rigid, bar-joint frameworks in which two given vertices are coincident. We formulate a conjecture which would extend their characterisation to an arbitrary set T of vertices and verify our conjecture when |T| = 3. △ Less

Submitted 12 June, 2021; originally announced June 2021.

MSC Class: 52C25

Naturalistic audio-visual volumetric sequences dataset of sounding actions for six degree-of-freedom interaction

Authors: Hanne Stenzel, Davide Berghi, Marco Volino, Philip J. B. Jackson

Abstract: As audio-visual systems increasingly bring immersive and interactive capabilities into our work and leisure activities, so the need for naturalistic test material grows. New volumetric datasets have captured high-quality 3D video, but accompanying audio is often neglected, making it hard to test an integrated bimodal experience. Designed to cover diverse sound types and features, the presented vol… ▽ More As audio-visual systems increasingly bring immersive and interactive capabilities into our work and leisure activities, so the need for naturalistic test material grows. New volumetric datasets have captured high-quality 3D video, but accompanying audio is often neglected, making it hard to test an integrated bimodal experience. Designed to cover diverse sound types and features, the presented volumetric dataset was constructed from audio and video studio recordings of scenes to yield forty short action sequences. Potential uses in technical and scientific tests are discussed. △ Less

Submitted 3 May, 2021; originally announced May 2021.

Comments: for dataset visit cvssp.org/data/navvs; accepted as poster in IEEE VR 2021