-
Echo Chambers in the Age of Algorithms: An Audit of Twitter's Friend Recommender System
Authors:
Kayla Duskin,
Joseph S. Schafer,
Jevin D. West,
Emma S. Spiro
Abstract:
The presence of political misinformation and ideological echo chambers on social media platforms is concerning given the important role that these sites play in the public's exposure to news and current events. Algorithmic systems employed on these platforms are presumed to play a role in these phenomena, but little is known about their mechanisms and effects. In this work, we conduct an algorithm…
▽ More
The presence of political misinformation and ideological echo chambers on social media platforms is concerning given the important role that these sites play in the public's exposure to news and current events. Algorithmic systems employed on these platforms are presumed to play a role in these phenomena, but little is known about their mechanisms and effects. In this work, we conduct an algorithmic audit of Twitter's Who-To-Follow friend recommendation system, the first empirical audit that investigates the impact of this algorithm in-situ. We create automated Twitter accounts that initially follow left and right affiliated U.S. politicians during the 2022 U.S. midterm elections and then grow their information networks using the platform's recommender system. We pair the experiment with an observational study of Twitter users who already follow the same politicians. Broadly, we find that while following the recommendation algorithm leads accounts into dense and reciprocal neighborhoods that structurally resemble echo chambers, the recommender also results in less political homogeneity of a user's network compared to accounts growing their networks through social endorsement. Furthermore, accounts that exclusively followed users recommended by the algorithm had fewer opportunities to encounter content centered on false or misleading election narratives compared to choosing friends based on social endorsement.
△ Less
Submitted 9 April, 2024;
originally announced April 2024.
-
Online Tree Reconstruction and Forest Inventory on a Mobile Robotic System
Authors:
Leonard Freißmuth,
Matias Mattamala,
Nived Chebrolu,
Simon Schaefer,
Stefan Leutenegger,
Maurice Fallon
Abstract:
Terrestrial laser scanning (TLS) is the standard technique used to create accurate point clouds for digital forest inventories. However, the measurement process is demanding, requiring up to two days per hectare for data collection, significant data storage, as well as resource-heavy post-processing of 3D data. In this work, we present a real-time map** and analysis system that enables online ge…
▽ More
Terrestrial laser scanning (TLS) is the standard technique used to create accurate point clouds for digital forest inventories. However, the measurement process is demanding, requiring up to two days per hectare for data collection, significant data storage, as well as resource-heavy post-processing of 3D data. In this work, we present a real-time map** and analysis system that enables online generation of forest inventories using mobile laser scanners that can be mounted e.g. on mobile robots. Given incrementally created and locally accurate submaps-data payloads-our approach extracts tree candidates using a custom, Voronoi-inspired clustering algorithm. Tree candidates are reconstructed using an adapted Hough algorithm, which enables robust modeling of the tree stem. Further, we explicitly incorporate the incremental nature of the data collection by consistently updating the database using a pose graph LiDAR SLAM system. This enables us to refine our estimates of the tree traits if an area is revisited later during a mission. We demonstrate competitive accuracy to TLS or manual measurements using laser scanners that we mounted on backpacks or mobile robots operating in conifer, broad-leaf and mixed forests. Our results achieve RMSE of 1.93 cm, a bias of 0.65 cm and a standard deviation of 1.81 cm (averaged across these sequences)-with no post-processing required after the mission is complete.
△ Less
Submitted 26 March, 2024;
originally announced March 2024.
-
SC-Diff: 3D Shape Completion with Latent Diffusion Models
Authors:
Juan D. Galvis,
Xingxing Zuo,
Simon Schaefer,
Stefan Leutengger
Abstract:
This paper introduces a 3D shape completion approach using a 3D latent diffusion model optimized for completing shapes, represented as Truncated Signed Distance Functions (TSDFs), from partial 3D scans. Our method combines image-based conditioning through cross-attention and spatial conditioning through the integration of 3D features from captured partial scans. This dual guidance enables high-fid…
▽ More
This paper introduces a 3D shape completion approach using a 3D latent diffusion model optimized for completing shapes, represented as Truncated Signed Distance Functions (TSDFs), from partial 3D scans. Our method combines image-based conditioning through cross-attention and spatial conditioning through the integration of 3D features from captured partial scans. This dual guidance enables high-fidelity, realistic shape completions at superior resolutions. At the core of our approach is the compression of 3D data into a low-dimensional latent space using an auto-encoder inspired by 2D latent diffusion models. This compression facilitates the processing of higher-resolution shapes and allows us to apply our model across multiple object classes, a significant improvement over other existing diffusion-based shape completion methods, which often require a separate diffusion model for each class. We validated our approach against two common benchmarks in the field of shape completion, demonstrating competitive performance in terms of accuracy and realism and performing on par with state-of-the-art methods despite operating at a higher resolution with a single model for all object classes. We present a comprehensive evaluation of our model, showcasing its efficacy in handling diverse shape completion challenges, even on unseen object classes. The code will be released upon acceptance.
△ Less
Submitted 19 March, 2024;
originally announced March 2024.
-
DynamicGlue: Epipolar and Time-Informed Data Association in Dynamic Environments using Graph Neural Networks
Authors:
Theresa Huber,
Simon Schaefer,
Stefan Leutenegger
Abstract:
The assumption of a static environment is common in many geometric computer vision tasks like SLAM but limits their applicability in highly dynamic scenes. Since these tasks rely on identifying point correspondences between input images within the static part of the environment, we propose a graph neural network-based sparse feature matching network designed to perform robust matching under challe…
▽ More
The assumption of a static environment is common in many geometric computer vision tasks like SLAM but limits their applicability in highly dynamic scenes. Since these tasks rely on identifying point correspondences between input images within the static part of the environment, we propose a graph neural network-based sparse feature matching network designed to perform robust matching under challenging conditions while excluding keypoints on moving objects. We employ a similar scheme of attentional aggregation over graph edges to enhance keypoint representations as state-of-the-art feature-matching networks but augment the graph with epipolar and temporal information and vastly reduce the number of graph edges. Furthermore, we introduce a self-supervised training scheme to extract pseudo labels for image pairs in dynamic environments from exclusively unprocessed visual-inertial data. A series of experiments show the superior performance of our network as it excludes keypoints on moving objects compared to state-of-the-art feature matching networks while still achieving similar results regarding conventional matching metrics. When integrated into a SLAM system, our network significantly improves performance, especially in highly dynamic scenes.
△ Less
Submitted 1 July, 2024; v1 submitted 17 March, 2024;
originally announced March 2024.
-
Scalable Autonomous Drone Flight in the Forest with Visual-Inertial SLAM and Dense Submaps Built without LiDAR
Authors:
Sebastián Barbas Laina,
Simon Boche,
Sotiris Papatheodorou,
Dimos Tzoumanikas,
Simon Schaefer,
Hanzhi Chen,
Stefan Leutenegger
Abstract:
Forestry constitutes a key element for a sustainable future, while it is supremely challenging to introduce digital processes to improve efficiency. The main limitation is the difficulty of obtaining accurate maps at high temporal and spatial resolution as a basis for informed forestry decision-making, due to the vast area forests extend over and the sheer number of trees. To address this challeng…
▽ More
Forestry constitutes a key element for a sustainable future, while it is supremely challenging to introduce digital processes to improve efficiency. The main limitation is the difficulty of obtaining accurate maps at high temporal and spatial resolution as a basis for informed forestry decision-making, due to the vast area forests extend over and the sheer number of trees. To address this challenge, we present an autonomous Micro Aerial Vehicle (MAV) system which purely relies on cost-effective and light-weight passive visual and inertial sensors to perform under-canopy autonomous navigation. We leverage visual-inertial simultaneous localization and map** (VI-SLAM) for accurate MAV state estimates and couple it with a volumetric occupancy submap** system to achieve a scalable map** framework which can be directly used for path planning. As opposed to a monolithic map, submaps inherently deal with inevitable drift and corrections from VI-SLAM, since they move with pose estimates as they are updated. To ensure the safety of the MAV during navigation, we also propose a novel reference trajectory anchoring scheme that moves and deforms the reference trajectory the MAV is tracking upon state updates from the VI-SLAM system in a consistent way, even upon large changes in state estimates due to loop-closures. We thoroughly validate our system in both real and simulated forest environments with high tree densities in excess of 400 trees per hectare and at speeds up to 3 m/s - while not encountering a single collision or system failure. To the best of our knowledge this is the first system which achieves this level of performance in such unstructured environment using low-cost passive visual sensors and fully on-board computation including VI-SLAM.
△ Less
Submitted 14 March, 2024;
originally announced March 2024.
-
Viral Privacy: Contextual Integrity as a Lens to Understand Content Creators' Privacy Perceptions and Needs After Sudden Attention
Authors:
Joseph S. Schafer,
Annie Denton,
Chloe Seelhoff,
Jordyn Vo,
Kate Starbird
Abstract:
When designing multi-stakeholder privacy systems, it is important to consider how different groups of social media users have different goals and requirements for privacy. Additionally, we must acknowledge that it is important to keep in mind that even a single creator's needs can change as their online visibility and presence shifts, and that robust multi-stakeholder privacy systems should accoun…
▽ More
When designing multi-stakeholder privacy systems, it is important to consider how different groups of social media users have different goals and requirements for privacy. Additionally, we must acknowledge that it is important to keep in mind that even a single creator's needs can change as their online visibility and presence shifts, and that robust multi-stakeholder privacy systems should account for these shifts. Using the framework of contextual integrity, we explain a theoretical basis for how to evaluate the potential changing privacy needs of users as their profiles undergo a sudden rise in online attention, and ongoing projects to understand these potential shifts in perspectives.
△ Less
Submitted 18 December, 2023;
originally announced December 2023.
-
Towards Incorporating Researcher Safety into Information Integrity Research Ethics
Authors:
Joseph S. Schafer,
Kate Starbird
Abstract:
Traditional research ethics has mainly and rightly been focused on making sure that participants are treated safely, justly, and ethically, to avoid the violation of their rights or putting participants in harm's way. Information integrity research within CSCW has also correspondingly mainly focused on these issues, and the focus of internet research ethics has primarily focused on increasing prot…
▽ More
Traditional research ethics has mainly and rightly been focused on making sure that participants are treated safely, justly, and ethically, to avoid the violation of their rights or putting participants in harm's way. Information integrity research within CSCW has also correspondingly mainly focused on these issues, and the focus of internet research ethics has primarily focused on increasing protections of participant data. However, as branches of internet research focus on more fraught contexts such as information integrity and problematic information, more explicit consideration of other ethical frames and subjects is warranted. In this workshop paper, we argue that researcher protections should be more explicitly considered and acknowledged in these studies, and should be considered alongside more standard ethical considerations for participants and for broader society.
△ Less
Submitted 28 December, 2023; v1 submitted 14 December, 2023;
originally announced December 2023.
-
Assessing Large Language Models on Climate Information
Authors:
Jannis Bulian,
Mike S. Schäfer,
Afra Amini,
Heidi Lam,
Massimiliano Ciaramita,
Ben Gaiarin,
Michelle Chen Hübscher,
Christian Buck,
Niels G. Mede,
Markus Leippold,
Nadine Strauß
Abstract:
As Large Language Models (LLMs) rise in popularity, it is necessary to assess their capability in critically relevant domains. We present a comprehensive evaluation framework, grounded in science communication research, to assess LLM responses to questions about climate change. Our framework emphasizes both presentational and epistemological adequacy, offering a fine-grained analysis of LLM genera…
▽ More
As Large Language Models (LLMs) rise in popularity, it is necessary to assess their capability in critically relevant domains. We present a comprehensive evaluation framework, grounded in science communication research, to assess LLM responses to questions about climate change. Our framework emphasizes both presentational and epistemological adequacy, offering a fine-grained analysis of LLM generations spanning 8 dimensions and 30 issues. Our evaluation task is a real-world example of a growing number of challenging problems where AI can complement and lift human performance. We introduce a novel protocol for scalable oversight that relies on AI Assistance and raters with relevant education. We evaluate several recent LLMs on a set of diverse climate questions. Our results point to a significant gap between surface and epistemological qualities of LLMs in the realm of climate communication.
△ Less
Submitted 28 May, 2024; v1 submitted 4 October, 2023;
originally announced October 2023.
-
GloPro: Globally-Consistent Uncertainty-Aware 3D Human Pose Estimation & Tracking in the Wild
Authors:
Simon Schaefer,
Dorian F. Henning,
Stefan Leutenegger
Abstract:
An accurate and uncertainty-aware 3D human body pose estimation is key to enabling truly safe but efficient human-robot interactions. Current uncertainty-aware methods in 3D human pose estimation are limited to predicting the uncertainty of the body posture, while effectively neglecting the body shape and root pose. In this work, we present GloPro, which to the best of our knowledge the first fram…
▽ More
An accurate and uncertainty-aware 3D human body pose estimation is key to enabling truly safe but efficient human-robot interactions. Current uncertainty-aware methods in 3D human pose estimation are limited to predicting the uncertainty of the body posture, while effectively neglecting the body shape and root pose. In this work, we present GloPro, which to the best of our knowledge the first framework to predict an uncertainty distribution of a 3D body mesh including its shape, pose, and root pose, by efficiently fusing visual clues with a learned motion model. We demonstrate that it vastly outperforms state-of-the-art methods in terms of human trajectory accuracy in a world coordinate system (even in the presence of severe occlusions), yields consistent uncertainty distributions, and can run in real-time.
△ Less
Submitted 20 September, 2023; v1 submitted 19 September, 2023;
originally announced September 2023.
-
Neural Parametric Surfaces for Shape Modeling
Authors:
Lei Yang,
Yongqing Liang,
Xin Li,
Congyi Zhang,
Guying Lin,
Alla Sheffer,
Scott Schaefer,
John Keyser,
Wen** Wang
Abstract:
The recent surge of utilizing deep neural networks for geometric processing and shape modeling has opened up exciting avenues. However, there is a conspicuous lack of research efforts on using powerful neural representations to extend the capabilities of parametric surfaces, which are the prevalent surface representations in product design, CAD/CAM, and computer animation. We present Neural Parame…
▽ More
The recent surge of utilizing deep neural networks for geometric processing and shape modeling has opened up exciting avenues. However, there is a conspicuous lack of research efforts on using powerful neural representations to extend the capabilities of parametric surfaces, which are the prevalent surface representations in product design, CAD/CAM, and computer animation. We present Neural Parametric Surfaces, the first piecewise neural surface representation that allows coarse patch layouts of arbitrary $n$-sided surface patches to model complex surface geometries with high precision, offering greater flexibility over traditional parametric surfaces. By construction, this new surface representation guarantees $G^0$ continuity between adjacent patches and empirically achieves $G^1$ continuity, which cannot be attained by existing neural patch-based methods. The key ingredient of our neural parametric surface is a learnable feature complex $\mathcal{C}$ that is embedded in a high-dimensional space $\mathbb{R}^D$ and topologically equivalent to the patch layout of the surface; each face cell of the complex is defined by interpolating feature vectors at its vertices. The learned feature complex is mapped by an MLP-encoded function $f:\mathcal{C} \rightarrow \mathcal{S}$ to produce the neural parametric surface $\mathcal{S}$. We present a surface fitting algorithm that optimizes the feature complex $\mathcal{C}$ and trains the neural map** $f$ to reconstruct given target shapes with high accuracy. We further show that the proposed representation along with a compact-size neural net can learn a plausible shape space from a shape collection, which can be used for shape interpolation or shape completion from noisy and incomplete input data. Extensive experiments show that neural parametric surfaces offer greater modeling capabilities than traditional parametric surfaces.
△ Less
Submitted 18 September, 2023;
originally announced September 2023.
-
Surface Extraction from Neural Unsigned Distance Fields
Authors:
Congyi Zhang,
Guying Lin,
Lei Yang,
Xin Li,
Taku Komura,
Scott Schaefer,
John Keyser,
Wen** Wang
Abstract:
We propose a method, named DualMesh-UDF, to extract a surface from unsigned distance functions (UDFs), encoded by neural networks, or neural UDFs. Neural UDFs are becoming increasingly popular for surface representation because of their versatility in presenting surfaces with arbitrary topologies, as opposed to the signed distance function that is limited to representing a closed surface. However,…
▽ More
We propose a method, named DualMesh-UDF, to extract a surface from unsigned distance functions (UDFs), encoded by neural networks, or neural UDFs. Neural UDFs are becoming increasingly popular for surface representation because of their versatility in presenting surfaces with arbitrary topologies, as opposed to the signed distance function that is limited to representing a closed surface. However, the applications of neural UDFs are hindered by the notorious difficulty in extracting the target surfaces they represent. Recent methods for surface extraction from a neural UDF suffer from significant geometric errors or topological artifacts due to two main difficulties: (1) A UDF does not exhibit sign changes; and (2) A neural UDF typically has substantial approximation errors. DualMesh-UDF addresses these two difficulties. Specifically, given a neural UDF encoding a target surface $\bar{S}$ to be recovered, we first estimate the tangent planes of $\bar{S}$ at a set of sample points close to $\bar{S}$. Next, we organize these sample points into local clusters, and for each local cluster, solve a linear least squares problem to determine a final surface point. These surface points are then connected to create the output mesh surface, which approximates the target surface. The robust estimation of the tangent planes of the target surface and the subsequent minimization problem constitute our core strategy, which contributes to the favorable performance of DualMesh-UDF over other competing methods. To efficiently implement this strategy, we employ an adaptive Octree. Within this framework, we estimate the location of a surface point in each of the octree cells identified as containing part of the target surface. Extensive experiments show that our method outperforms existing methods in terms of surface reconstruction quality while maintaining comparable computational efficiency.
△ Less
Submitted 16 September, 2023;
originally announced September 2023.
-
BodySLAM++: Fast and Tightly-Coupled Visual-Inertial Camera and Human Motion Tracking
Authors:
Dorian F. Henning,
Christopher Choi,
Simon Schaefer,
Stefan Leutenegger
Abstract:
Robust, fast, and accurate human state - 6D pose and posture - estimation remains a challenging problem. For real-world applications, the ability to estimate the human state in real-time is highly desirable. In this paper, we present BodySLAM++, a fast, efficient, and accurate human and camera state estimation framework relying on visual-inertial data. BodySLAM++ extends an existing visual-inertia…
▽ More
Robust, fast, and accurate human state - 6D pose and posture - estimation remains a challenging problem. For real-world applications, the ability to estimate the human state in real-time is highly desirable. In this paper, we present BodySLAM++, a fast, efficient, and accurate human and camera state estimation framework relying on visual-inertial data. BodySLAM++ extends an existing visual-inertial state estimation framework, OKVIS2, to solve the dual task of estimating camera and human states simultaneously. Our system improves the accuracy of both human and camera state estimation with respect to baseline methods by 26% and 12%, respectively, and achieves real-time performance at 15+ frames per second on an Intel i7-model CPU. Experiments were conducted on a custom dataset containing both ground truth human and camera poses collected with an indoor motion tracking system.
△ Less
Submitted 3 September, 2023;
originally announced September 2023.
-
Int-HRL: Towards Intention-based Hierarchical Reinforcement Learning
Authors:
Anna Penzkofer,
Simon Schaefer,
Florian Strohm,
Mihai Bâce,
Stefan Leutenegger,
Andreas Bulling
Abstract:
While deep reinforcement learning (RL) agents outperform humans on an increasing number of tasks, training them requires data equivalent to decades of human gameplay. Recent hierarchical RL methods have increased sample efficiency by incorporating information inherent to the structure of the decision problem but at the cost of having to discover or use human-annotated sub-goals that guide the lear…
▽ More
While deep reinforcement learning (RL) agents outperform humans on an increasing number of tasks, training them requires data equivalent to decades of human gameplay. Recent hierarchical RL methods have increased sample efficiency by incorporating information inherent to the structure of the decision problem but at the cost of having to discover or use human-annotated sub-goals that guide the learning process. We show that intentions of human players, i.e. the precursor of goal-oriented decisions, can be robustly predicted from eye gaze even for the long-horizon sparse rewards task of Montezuma's Revenge - one of the most challenging RL tasks in the Atari2600 game suite. We propose Int-HRL: Hierarchical RL with intention-based sub-goals that are inferred from human eye gaze. Our novel sub-goal extraction pipeline is fully automatic and replaces the need for manual sub-goal annotation by human experts. Our evaluations show that replacing hand-crafted sub-goals with automatically extracted intentions leads to a HRL agent that is significantly more sample efficient than previous methods.
△ Less
Submitted 20 June, 2023;
originally announced June 2023.
-
Autonomous sputter synthesis of thin film nitrides with composition controlled by Bayesian optimization of optical plasma emission
Authors:
Davi M. Febba,
Kevin R. Talley,
Kendal Johnson,
Stephen Schaefer,
Sage R. Bauers,
John S. Mangum,
Rebecca W. Smaha,
Andriy Zakutayev
Abstract:
Autonomous experimentation has emerged as an efficient approach to accelerate the pace of materials discovery. Although instruments for autonomous synthesis have become popular in molecular and polymer science, solution processing of hybrid materials and nanoparticles, examples of autonomous tools for physical vapor deposition are scarce yet important for the semiconductor industry. Here, we repor…
▽ More
Autonomous experimentation has emerged as an efficient approach to accelerate the pace of materials discovery. Although instruments for autonomous synthesis have become popular in molecular and polymer science, solution processing of hybrid materials and nanoparticles, examples of autonomous tools for physical vapor deposition are scarce yet important for the semiconductor industry. Here, we report the design and implementation of an autonomous workflow for sputter deposition of thin films with controlled composition, leveraging a highly automated sputtering reactor custom-controlled by Python, optical emission spectroscopy (OES), and a Bayesian optimization algorithm. We modeled film composition, measured by x-ray fluorescence, as a linear function of emission lines monitored during the co-sputtering from elemental Zn and Ti targets in N$_2$ atmosphere. A Bayesian control algorithm, informed by OES, navigates the space of sputtering power to fabricate films with user-defined composition, by minimizing the absolute error between desired and measured emission signals. We validated our approach by autonomously fabricating Zn$_x$Ti$_{1-x}$N$_y$ films with deviations from the targeted cation composition within relative 3.5 %, even for 15 nm thin films, demonstrating that the proposed approach can reliably synthesize thin films with specific composition and minimal human interference. Moreover, the proposed method can be extended to more difficult synthesis experiments where plasma intensity depends non-linearly on pressure, or the elemental sticking coefficients strongly depend on the substrate temperature.
△ Less
Submitted 10 August, 2023; v1 submitted 18 May, 2023;
originally announced May 2023.
-
The Challenges of Studying Misinformation on Video-Sharing Platforms During Crises and Mass-Convergence Events
Authors:
Sukrit Venkatagiri,
Joseph S. Schafer,
Stephen Prochaska
Abstract:
Mis- and disinformation can spread rapidly on video-sharing platforms (VSPs). Despite the growing use of VSPs, there has not been a proportional increase in our ability to understand this medium and the messages conveyed through it. In this work, we draw on our prior experiences to outline three core challenges faced in studying VSPs in high-stakes and fast-paced settings: (1) navigating the uniqu…
▽ More
Mis- and disinformation can spread rapidly on video-sharing platforms (VSPs). Despite the growing use of VSPs, there has not been a proportional increase in our ability to understand this medium and the messages conveyed through it. In this work, we draw on our prior experiences to outline three core challenges faced in studying VSPs in high-stakes and fast-paced settings: (1) navigating the unique affordances of VSPs, (2) understanding VSP content and determining its authenticity, and (3) novel user behaviors on VSPs for spreading misinformation. By highlighting these challenges, we hope that researchers can reflect on how to adapt existing research methods and tools to these new contexts, or develop entirely new ones.
△ Less
Submitted 24 March, 2023;
originally announced March 2023.
-
Followback Clusters, Satellite Audiences, and Bridge Nodes: Coengagement Networks for the 2020 US Election
Authors:
Andrew Beers,
Joseph S. Schafer,
Ian Kennedy,
Morgan Wack,
Emma S. Spiro,
Kate Starbird
Abstract:
The 2020 United States presidential election was, and has continued to be, the focus of pervasive and persistent mis- and disinformation spreading through our media ecosystems, including social media. This event has driven the collection and analysis of large, directed social network datasets, but such datasets can resist intuitive understanding. In such large datasets, the overwhelming number of…
▽ More
The 2020 United States presidential election was, and has continued to be, the focus of pervasive and persistent mis- and disinformation spreading through our media ecosystems, including social media. This event has driven the collection and analysis of large, directed social network datasets, but such datasets can resist intuitive understanding. In such large datasets, the overwhelming number of nodes and edges present in typical representations create visual artifacts, such as densely overlap** edges and tightly-packed formations of low-degree nodes, which obscure many features of more practical interest. We apply a method, coengagement transformations, to convert such networks of social data into tractable images. Intuitively, this approach allows for parameterized network visualizations that make shared audiences of engaged viewers salient to viewers. Using the interpretative capabilities of this method, we perform an extensive case study of the 2020 United States presidential election on Twitter, contributing an empirical analysis of coengagement. By creating and contrasting different networks at different parameter sets, we define and characterize several structures in this discourse network, including bridging accounts, satellite audiences, and followback communities. We discuss the importance and implications of these empirical network features in this context. In addition, we release open-source code for creating coengagement networks from Twitter and other structured interaction data.
△ Less
Submitted 30 May, 2023; v1 submitted 28 February, 2023;
originally announced March 2023.
-
The Software Stack That Won the Formula Student Driverless Competition
Authors:
Andres Alvarez,
Nico Denner,
Zhe Feng,
David Fischer,
Yang Gao,
Lukas Harsch,
Sebastian Herz,
Nick Le Large,
Bach Nguyen,
Carlos Rosero,
Simon Schaefer,
Alexander Terletskiy,
Luca Wahl,
Shaoxiang Wang,
Jonona Yakupova,
Haocen Yu
Abstract:
This report describes our approach to design and evaluate a software stack for a race car capable of achieving competitive driving performance in the different disciplines of the Formula Student Driverless. By using a 360° LiDAR and optionally three cameras, we reliably recognize the plastic cones that mark the track boundaries at distances of around 35 m, enabling us to drive at the physical limi…
▽ More
This report describes our approach to design and evaluate a software stack for a race car capable of achieving competitive driving performance in the different disciplines of the Formula Student Driverless. By using a 360° LiDAR and optionally three cameras, we reliably recognize the plastic cones that mark the track boundaries at distances of around 35 m, enabling us to drive at the physical limits of the car. Using a GraphSLAM algorithm, we are able to map these cones with a root-mean-square error of less than 15 cm while driving at speeds of over 70 kph on a narrow track. The high-precision map is used in the trajectory planning to detect the lane boundaries using Delaunay triangulation and a parametric cubic spline. We calculate an optimized trajectory using a minimum curvature approach together with a GGS-diagram that takes the aerodynamics at different velocities into account. To track the target path with accelerations of up to 1.6 g, the control system is split into a PI controller for longitudinal control and model predictive controller for lateral control. Additionally, a low-level optimal control allocation is used. The software is realized in ROS C++ and tested in a custom simulation, as well as on the actual race track.
△ Less
Submitted 19 October, 2022;
originally announced October 2022.
-
A Pipeline for DNS-Based Software Fingerprinting
Authors:
Sebastian Schäfer,
Ulrike Meyer
Abstract:
In this paper, we present the modular design and implementation of DONUT, a novel tool for identifying software running on a device. Our tool uses a rule-based approach to detect software-specific DNS fingerprints (stored in an easily extendable database) in passively monitored DNS traffic. We automated the rule extraction process for DONUT with the help of ATLAS, a novel tool we developed for lab…
▽ More
In this paper, we present the modular design and implementation of DONUT, a novel tool for identifying software running on a device. Our tool uses a rule-based approach to detect software-specific DNS fingerprints (stored in an easily extendable database) in passively monitored DNS traffic. We automated the rule extraction process for DONUT with the help of ATLAS, a novel tool we developed for labeling network traffic by the software that created it. We demonstrate the functionality of our pipeline by generating rules for a number of applications, evaluate the performance as well as scalability of the analysis, and confirm the functional correctness of DONUT using an artificial data set for which the ground-truth is known. In addition, we evaluate DONUT's analysis results on a large real-world data set with unknown ground truth.
△ Less
Submitted 15 August, 2022;
originally announced August 2022.
-
Visual-Inertial SLAM with Tightly-Coupled Dropout-Tolerant GPS Fusion
Authors:
Simon Boche,
Xingxing Zuo,
Simon Schaefer,
Stefan Leutenegger
Abstract:
Robotic applications are continuously striving towards higher levels of autonomy. To achieve that goal, a highly robust and accurate state estimation is indispensable. Combining visual and inertial sensor modalities has proven to yield accurate and locally consistent results in short-term applications. Unfortunately, visual-inertial state estimators suffer from the accumulation of drift for long-t…
▽ More
Robotic applications are continuously striving towards higher levels of autonomy. To achieve that goal, a highly robust and accurate state estimation is indispensable. Combining visual and inertial sensor modalities has proven to yield accurate and locally consistent results in short-term applications. Unfortunately, visual-inertial state estimators suffer from the accumulation of drift for long-term trajectories. To eliminate this drift, global measurements can be fused into the state estimation pipeline. The most known and widely available source of global measurements is the Global Positioning System (GPS). In this paper, we propose a novel approach that fully combines stereo Visual-Inertial Simultaneous Localisation and Map** (SLAM), including visual loop closures, with the fusion of global sensor modalities in a tightly-coupled and optimisation-based framework. Incorporating measurement uncertainties, we provide a robust criterion to solve the global reference frame initialisation problem. Furthermore, we propose a loop-closure-like optimisation scheme to compensate drift accumulated during outages in receiving GPS signals. Experimental validation on datasets and in a real-world experiment demonstrates the robustness of our approach to GPS dropouts as well as its capability to estimate highly accurate and globally consistent trajectories compared to existing state-of-the-art methods.
△ Less
Submitted 1 August, 2022;
originally announced August 2022.
-
AEGNN: Asynchronous Event-based Graph Neural Networks
Authors:
Simon Schaefer,
Daniel Gehrig,
Davide Scaramuzza
Abstract:
The best performing learning algorithms devised for event cameras work by first converting events into dense representations that are then processed using standard CNNs. However, these steps discard both the sparsity and high temporal resolution of events, leading to high computational burden and latency. For this reason, recent works have adopted Graph Neural Networks (GNNs), which process events…
▽ More
The best performing learning algorithms devised for event cameras work by first converting events into dense representations that are then processed using standard CNNs. However, these steps discard both the sparsity and high temporal resolution of events, leading to high computational burden and latency. For this reason, recent works have adopted Graph Neural Networks (GNNs), which process events as ``static" spatio-temporal graphs, which are inherently "sparse". We take this trend one step further by introducing Asynchronous, Event-based Graph Neural Networks (AEGNNs), a novel event-processing paradigm that generalizes standard GNNs to process events as ``evolving" spatio-temporal graphs. AEGNNs follow efficient update rules that restrict recomputation of network activations only to the nodes affected by each new event, thereby significantly reducing both computation and latency for event-by-event processing. AEGNNs are easily trained on synchronous inputs and can be converted to efficient, "asynchronous" networks at test time. We thoroughly validate our method on object classification and detection tasks, where we show an up to a 11-fold reduction in computational complexity (FLOPs), with similar or even better performance than state-of-the-art asynchronous methods. This reduction in computation directly translates to an 8-fold reduction in computational latency when compared to standard GNNs, which opens the door to low-latency event-based processing.
△ Less
Submitted 1 November, 2022; v1 submitted 31 March, 2022;
originally announced March 2022.
-
High-Dimensional Stock Portfolio Trading with Deep Reinforcement Learning
Authors:
Uta Pigorsch,
Sebastian Schäfer
Abstract:
This paper proposes a Deep Reinforcement Learning algorithm for financial portfolio trading based on Deep Q-learning. The algorithm is capable of trading high-dimensional portfolios from cross-sectional datasets of any size which may include data gaps and non-unique history lengths in the assets. We sequentially set up environments by sampling one asset for each environment while rewarding investm…
▽ More
This paper proposes a Deep Reinforcement Learning algorithm for financial portfolio trading based on Deep Q-learning. The algorithm is capable of trading high-dimensional portfolios from cross-sectional datasets of any size which may include data gaps and non-unique history lengths in the assets. We sequentially set up environments by sampling one asset for each environment while rewarding investments with the resulting asset's return and cash reservation with the average return of the set of assets. This enforces the agent to strategically assign capital to assets that it predicts to perform above-average. We apply our methodology in an out-of-sample analysis to 48 US stock portfolio setups, varying in the number of stocks from ten up to 500 stocks, in the selection criteria and in the level of transaction costs. The algorithm on average outperforms all considered passive and active benchmark investment strategies by a large margin using only one hyperparameter setup for all portfolios.
△ Less
Submitted 9 December, 2021;
originally announced December 2021.
-
Leveraging Neural Network Gradients within Trajectory Optimization for Proactive Human-Robot Interactions
Authors:
Simon Schaefer,
Karen Leung,
Boris Ivanovic,
Marco Pavone
Abstract:
To achieve seamless human-robot interactions, robots need to intimately reason about complex interaction dynamics and future human behaviors within their motion planning process. However, there is a disconnect between state-of-the-art neural network-based human behavior models and robot motion planners -- either the behavior models are limited in their consideration of downstream planning or a sim…
▽ More
To achieve seamless human-robot interactions, robots need to intimately reason about complex interaction dynamics and future human behaviors within their motion planning process. However, there is a disconnect between state-of-the-art neural network-based human behavior models and robot motion planners -- either the behavior models are limited in their consideration of downstream planning or a simplified behavior model is used to ensure tractability of the planning problem. In this work, we present a framework that fuses together the interpretability and flexibility of trajectory optimization (TO) with the predictive power of state-of-the-art human trajectory prediction models. In particular, we leverage gradient information from data-driven prediction models to explicitly reason about human-robot interaction dynamics within a gradient-based TO problem. We demonstrate the efficacy of our approach in a multi-agent scenario whereby a robot is required to safely and efficiently navigate through a crowd of up to ten pedestrians. We compare against a variety of planning methods, and show that by explicitly accounting for interaction dynamics within the planner, our method offers safer and more efficient behaviors, even yielding proactive and nuanced behaviors such as waiting for a pedestrian to pass before moving.
△ Less
Submitted 2 December, 2020;
originally announced December 2020.
-
2020 NDSA Agenda for Digital Stewardship
Authors:
Micah Altman,
Karen Cariani,
Bradley Daigle,
Christie Moffatt,
Sibyl Schaefer,
Bethany Scott,
Lauren Work
Abstract:
The NDSA Agenda is a comprehensive overview of the state of global digital preservation. It casts its eye over current research trends, grants, projects, and various efforts spanning the preservation ecosystem. The agenda identifies successes and ongoing challenges in addition to providing some tangible recommendations to both researcher and practitioner alike. As both an overview and comprehensiv…
▽ More
The NDSA Agenda is a comprehensive overview of the state of global digital preservation. It casts its eye over current research trends, grants, projects, and various efforts spanning the preservation ecosystem. The agenda identifies successes and ongoing challenges in addition to providing some tangible recommendations to both researcher and practitioner alike. As both an overview and comprehensive dive into digital preservation issues, the audience ranges from high level to hands on experts. Funders can use this report as a signpost for the overall state of the profession.
△ Less
Submitted 11 May, 2020;
originally announced May 2020.
-
Low Rank Matrix Approximation for Geometry Filtering
Authors:
Xuequan Lu,
Scott Schaefer,
Jun Luo,
Lizhuang Ma,
Ying He
Abstract:
We propose a robust normal estimation method for both point clouds and meshes using a low rank matrix approximation algorithm. First, we compute a local feature descriptor for each point and find similar, non-local neighbors that we organize into a matrix. We then show that a low rank matrix approximation algorithm can robustly estimate normals for both point clouds and meshes. Furthermore, we pro…
▽ More
We propose a robust normal estimation method for both point clouds and meshes using a low rank matrix approximation algorithm. First, we compute a local feature descriptor for each point and find similar, non-local neighbors that we organize into a matrix. We then show that a low rank matrix approximation algorithm can robustly estimate normals for both point clouds and meshes. Furthermore, we provide a new filtering method for point cloud data to smooth the position data to fit the estimated normals. We show applications of our method to point cloud filtering, point set upsampling, surface reconstruction, mesh denoising, and geometric texture removal. Our experiments show that our method outperforms current methods in both visual quality and accuracy.
△ Less
Submitted 4 December, 2018; v1 submitted 18 March, 2018;
originally announced March 2018.
-
Cone Detection using a Combination of LiDAR and Vision-based Machine Learning
Authors:
Nico Messikommer,
Simon Schaefer,
Renaud Dubé,
Mark Pfeiffer
Abstract:
The classification and the position estimation of objects become more and more relevant as the field of robotics is expanding in diverse areas of society. In this Bachelor Thesis, we developed a cone detection algorithm for an autonomous car using a LiDAR sensor and a colour camera. By evaluating simple constraints, the LiDAR detection algorithm preselects cone candidates in the 3 dimensional spac…
▽ More
The classification and the position estimation of objects become more and more relevant as the field of robotics is expanding in diverse areas of society. In this Bachelor Thesis, we developed a cone detection algorithm for an autonomous car using a LiDAR sensor and a colour camera. By evaluating simple constraints, the LiDAR detection algorithm preselects cone candidates in the 3 dimensional space. The candidates are projected into the image plane of the colour camera and an image candidate is cropped out. A convolutional neural networks classifies the image candidates as cone or not a cone. With the fusion of the precise position estimation of the LiDAR sensor and the high classification accuracy of a neural network, a reliable cone detection algorithm was implemented. Furthermore, a path planning algorithm generates a path around the detected cones. The final system detects cones even at higher velocity and has the potential to drive fully autonomous around the cones.
△ Less
Submitted 6 November, 2017;
originally announced November 2017.
-
Autonomous Electric Race Car Design
Authors:
Niklas Funk,
Nikhilesh Alatur,
Robin Deuber,
Frederick Gonon,
Nico Messikommer,
Julian Nubert,
Moritz Patriarca,
Simon Schaefer,
Dominic Scotoni,
Nicholas Bünger,
Renaud Dube,
Raghav Khanna,
Mark Pfeiffer,
Erik Wilhelm,
Roland Siegwart
Abstract:
Autonomous driving and electric vehicles are nowadays very active research and development areas. In this paper we present the conversion of a standard Kyburz eRod into an autonomous vehicle that can be operated in challenging environments such as Swiss mountain passes. The overall hardware and software architectures are described in detail with a special emphasis on the sensor requirements for au…
▽ More
Autonomous driving and electric vehicles are nowadays very active research and development areas. In this paper we present the conversion of a standard Kyburz eRod into an autonomous vehicle that can be operated in challenging environments such as Swiss mountain passes. The overall hardware and software architectures are described in detail with a special emphasis on the sensor requirements for autonomous vehicles operating in partially structured environments. Furthermore, the design process itself and the finalized system architecture are presented. The work shows state of the art results in localization and controls for self-driving high-performance electric vehicles. Test results of the overall system are presented, which show the importance of generalizable state estimation algorithms to handle a plethora of conditions.
△ Less
Submitted 1 November, 2017;
originally announced November 2017.
