-
The Fellowship of the Dyson Ring: ACT&Friends' Results and Methods for GTOC 11
Authors:
Marcus Märtens,
Dario Izzo,
Emmanuel Blazquez,
Moritz von Looz,
Pablo Gómez,
Anne Mergy,
Giacomo Acciarini,
Chit Hong Yam,
Javier Hernando Ayuso,
Yuri Shimane
Abstract:
Dyson spheres are hypothetical megastructures encircling stars in order to harvest most of their energy output. During the 11th edition of the GTOC challenge, participants were tasked with a complex trajectory planning related to the construction of a precursor Dyson structure, a heliocentric ring made of twelve stations. To this purpose, we developed several new approaches that synthesize techniq…
▽ More
Dyson spheres are hypothetical megastructures encircling stars in order to harvest most of their energy output. During the 11th edition of the GTOC challenge, participants were tasked with a complex trajectory planning related to the construction of a precursor Dyson structure, a heliocentric ring made of twelve stations. To this purpose, we developed several new approaches that synthesize techniques from machine learning, combinatorial optimization, planning and scheduling, and evolutionary optimization effectively integrated into a fully automated pipeline. These include a machine learned transfer time estimator, improving the established Edelbaum approximation and thus better informing a Lazy Race Tree Search to identify and collect asteroids with high arrival mass for the stations; a series of optimally-phased low-thrust transfers to all stations computed by indirect optimization techniques, exploiting the synodic periodicity of the system; and a modified Hungarian scheduling algorithm, which utilizes evolutionary techniques to arrange a mass-balanced arrival schedule out of all transfer possibilities. We describe the steps of our pipeline in detail with a special focus on how our approaches mutually benefit from each other. Lastly, we outline and analyze the final solution of our team, ACT&Friends, which ranked second at the GTOC 11 challenge.
△ Less
Submitted 23 May, 2022; v1 submitted 13 May, 2022;
originally announced May 2022.
-
Guidelines for Experimental Algorithmics in Network Analysis
Authors:
Eugenio Angriman,
Alexander van der Grinten,
Moritz von Looz,
Henning Meyerhenke,
Martin Nöllenburg,
Maria Predari,
Charilaos Tzovas
Abstract:
The field of network science is a highly interdisciplinary area; for the empirical analysis of network data, it draws algorithmic methodologies from several research fields. Hence, research procedures and descriptions of the technical results often differ, sometimes widely. In this paper we focus on methodologies for the experimental part of algorithm engineering for network analysis -- an importa…
▽ More
The field of network science is a highly interdisciplinary area; for the empirical analysis of network data, it draws algorithmic methodologies from several research fields. Hence, research procedures and descriptions of the technical results often differ, sometimes widely. In this paper we focus on methodologies for the experimental part of algorithm engineering for network analysis -- an important ingredient for a research area with empirical focus. More precisely, we unify and adapt existing recommendations from different fields and propose universal guidelines -- including statistical analyses -- for the systematic evaluation of network analysis algorithms. This way, the behavior of newly proposed algorithms can be properly assessed and comparisons to existing solutions become meaningful. Moreover, as the main technical contribution, we provide SimexPal, a highly automated tool to perform and analyze experiments following our guidelines. To illustrate the merits of SimexPal and our guidelines, we apply them in a case study: we design, perform, visualize and evaluate experiments of a recent algorithm for approximating betweenness centrality, an important problem in network analysis. In summary, both our guidelines and SimexPal shall modernize and complement previous efforts in experimental algorithmics; they are not only useful for network analysis, but also in related contexts.
△ Less
Submitted 25 March, 2019;
originally announced April 2019.
-
Balanced k-means for Parallel Geometric Partitioning
Authors:
Moritz von Looz,
Charilaos Tzovas,
Henning Meyerhenke
Abstract:
Mesh partitioning is an indispensable tool for efficient parallel numerical simulations. Its goal is to minimize communication between the processes of a simulation while achieving load balance. Established graph-based partitioning tools yield a high solution quality; however, their scalability is limited. Geometric approaches usually scale better, but their solution quality may be unsatisfactory…
▽ More
Mesh partitioning is an indispensable tool for efficient parallel numerical simulations. Its goal is to minimize communication between the processes of a simulation while achieving load balance. Established graph-based partitioning tools yield a high solution quality; however, their scalability is limited. Geometric approaches usually scale better, but their solution quality may be unsatisfactory for `non-trivial' mesh topologies.
In this paper, we present a scalable version of $k$-means that is adapted to yield balanced clusters. Balanced $k$-means constitutes the core of our new partitioning algorithm Geographer. Bootstrap** of initial centers is performed with space-filling curves, leading to fast convergence of the subsequent balanced k-means algorithm.
Our experiments with up to 16384 MPI processes on numerous benchmark meshes show the following: (i) Geographer produces partitions with a lower communication volume than state-of-the-art geometric partitioners from the Zoltan package; (ii) Geographer scales well on large inputs; (iii) a Delaunay mesh with a few billion vertices and edges can be partitioned in a few seconds.
△ Less
Submitted 3 May, 2018;
originally announced May 2018.
-
Updating Dynamic Random Hyperbolic Graphs in Sublinear Time
Authors:
Moritz von Looz,
Henning Meyerhenke
Abstract:
Generative network models play an important role in algorithm development, scaling studies, network analysis, and realistic system benchmarks for graph data sets. A complex network model gaining considerable popularity builds random hyperbolic graphs, generated by distributing points within a disk in the hyperbolic plane and then adding edges between points with a probability depending on their hy…
▽ More
Generative network models play an important role in algorithm development, scaling studies, network analysis, and realistic system benchmarks for graph data sets. A complex network model gaining considerable popularity builds random hyperbolic graphs, generated by distributing points within a disk in the hyperbolic plane and then adding edges between points with a probability depending on their hyperbolic distance.
We present a dynamic extension to model gradual network change, while preserving at each step the point position probabilities. To process the dynamic changes efficiently, we formalize the concept of a probabilistic neighborhood: Let $P$ be a set of $n$ points in Euclidean or hyperbolic space, $q$ a query point, $\operatorname{dist}$ a distance metric, and $f : \mathbb{R}^+ \rightarrow [0,1]$ a monotonically decreasing function. Then, the probabilistic neighborhood $N(q, f)$ of $q$ with respect to $f$ is a random subset of $P$ and each point $p \in P$ belongs to $N(q,f)$ with probability $f(\operatorname{dist}(p,q))$. We present a fast, sublinear-time query algorithm to sample probabilistic neighborhoods from planar point sets. For certain distributions of planar $P$, we prove that our algorithm answers a query in $O((|N(q,f)| + \sqrt{n})\log n)$ time with high probability. This enables us to process a node movement in random hyperbolic graphs in sublinear time, resulting in a speedup of about one order of magnitude in practice compared to the fastest previous approach. Apart from that, our query algorithm is also applicable to Euclidean geometry, making it of independent interest for other sampling or probabilistic spreading scenarios.
△ Less
Submitted 8 February, 2018;
originally announced February 2018.
-
Communication-free Massively Distributed Graph Generation
Authors:
Daniel Funke,
Sebastian Lamm,
Ulrich Meyer,
Peter Sanders,
Manuel Penschuck,
Christian Schulz,
Darren Strash,
Moritz von Looz
Abstract:
Analyzing massive complex networks yields promising insights about our everyday lives. Building scalable algorithms to do so is a challenging task that requires a careful analysis and an extensive evaluation. However, engineering such algorithms is often hindered by the scarcity of publicly~available~datasets.
Network generators serve as a tool to alleviate this problem by providing synthetic in…
▽ More
Analyzing massive complex networks yields promising insights about our everyday lives. Building scalable algorithms to do so is a challenging task that requires a careful analysis and an extensive evaluation. However, engineering such algorithms is often hindered by the scarcity of publicly~available~datasets.
Network generators serve as a tool to alleviate this problem by providing synthetic instances with controllable parameters. However, many network generators fail to provide instances on a massive scale due to their sequential nature or resource constraints. Additionally, truly scalable network generators are few and often limited in their realism.
In this work, we present novel generators for a variety of network models that are frequently used as benchmarks. By making use of pseudorandomization and divide-and-conquer schemes, our generators follow a communication-free paradigm. The resulting generators are thus embarrassingly parallel and have a near optimal scaling behavior. This allows us to generate instances of up to $2^{43}$ vertices and $2^{47}$ edges in less than 22 minutes on 32768 cores. Therefore, our generators allow new graph families to be used on an unprecedented scale.
△ Less
Submitted 18 March, 2019; v1 submitted 20 October, 2017;
originally announced October 2017.
-
Generating massive complex networks with hyperbolic geometry faster in practice
Authors:
Moritz von Looz,
Mustafa Özdayi,
Sören Laue,
Henning Meyerhenke
Abstract:
Generative network models play an important role in algorithm development, scaling studies, network analysis, and realistic system benchmarks for graph data sets. The commonly used graph-based benchmark model R-MAT has some drawbacks concerning realism and the scaling behavior of network properties. A complex network model gaining considerable popularity builds random hyperbolic graphs, generated…
▽ More
Generative network models play an important role in algorithm development, scaling studies, network analysis, and realistic system benchmarks for graph data sets. The commonly used graph-based benchmark model R-MAT has some drawbacks concerning realism and the scaling behavior of network properties. A complex network model gaining considerable popularity builds random hyperbolic graphs, generated by distributing points within a disk in the hyperbolic plane and then adding edges between points whose hyperbolic distance is below a threshold.
We present in this paper a fast generation algorithm for such graphs. Our experiments show that our new generator achieves speedup factors of 3-60 over the best previous implementation. One billion edges can now be generated in under one minute on a shared-memory workstation. Furthermore, we present a dynamic extension to model gradual network change, while preserving at each step the point position probabilities.
△ Less
Submitted 30 June, 2016;
originally announced June 2016.
-
Better partitions of protein graphs for subsystem quantum chemistry
Authors:
Moritz von Looz,
Mario Wolter,
Christoph R. Jacob,
Henning Meyerhenke
Abstract:
Determining the interaction strength between proteins and small molecules is key to analyzing their biological function. Quantum-mechanical calculations such as \emph{Density Functional Theory} (DFT) give accurate and theoretically well-founded results. With common implementations the running time of DFT calculations increases quadratically with molecule size. Thus, numerous subsystem-based approa…
▽ More
Determining the interaction strength between proteins and small molecules is key to analyzing their biological function. Quantum-mechanical calculations such as \emph{Density Functional Theory} (DFT) give accurate and theoretically well-founded results. With common implementations the running time of DFT calculations increases quadratically with molecule size. Thus, numerous subsystem-based approaches have been developed to accelerate quantum-chemical calculations. These approaches partition the protein into different fragments, which are treated separately. Interactions between different fragments are approximated and introduce inaccuracies in the calculated interaction energies.
To minimize these inaccuracies, we represent the amino acids and their interactions as a weighted graph in order to apply graph partitioning. None of the existing graph partitioning work can be directly used, though, due to the unique constraints in partitioning such protein graphs. We therefore present and evaluate several algorithms, partially building upon established concepts, but adapted to handle the new constraints. For the special case of partitioning a protein along the main chain, we also present an efficient dynamic programming algorithm that yields provably optimal results. In the general scenario our algorithms usually improve the previous approach significantly and take at most a few seconds.
△ Less
Submitted 10 June, 2016;
originally announced June 2016.
-
Querying Probabilistic Neighborhoods in Spatial Data Sets Efficiently
Authors:
Moritz von Looz,
Henning Meyerhenke
Abstract:
$\newcommand{\dist}{\operatorname{dist}}$ In this paper we define the notion of a probabilistic neighborhood in spatial data: Let a set $P$ of $n$ points in $\mathbb{R}^d$, a query point $q \in \mathbb{R}^d$, a distance metric $\dist$, and a monotonically decreasing function $f : \mathbb{R}^+ \rightarrow [0,1]$ be given. Then a point $p \in P$ belongs to the probabilistic neighborhood $N(q, f)…
▽ More
$\newcommand{\dist}{\operatorname{dist}}$ In this paper we define the notion of a probabilistic neighborhood in spatial data: Let a set $P$ of $n$ points in $\mathbb{R}^d$, a query point $q \in \mathbb{R}^d$, a distance metric $\dist$, and a monotonically decreasing function $f : \mathbb{R}^+ \rightarrow [0,1]$ be given. Then a point $p \in P$ belongs to the probabilistic neighborhood $N(q, f)$ of $q$ with respect to $f$ with probability $f(\dist(p,q))$. We envision applications in facility location, sensor networks, and other scenarios where a connection between two entities becomes less likely with increasing distance. A straightforward query algorithm would determine a probabilistic neighborhood in $Θ(n\cdot d)$ time by probing each point in $P$.
To answer the query in sublinear time for the planar case, we augment a quadtree suitably and design a corresponding query algorithm. Our theoretical analysis shows that -- for certain distributions of planar $P$ -- our algorithm answers a query in $O((|N(q,f)| + \sqrt{n})\log n)$ time with high probability (whp). This matches up to a logarithmic factor the cost induced by quadtree-based algorithms for deterministic queries and is asymptotically faster than the straightforward approach whenever $|N(q,f)| \in o(n / \log n)$.
As practical proofs of concept we use two applications, one in the Euclidean and one in the hyperbolic plane. In particular, our results yield the first generator for random hyperbolic graphs with arbitrary temperatures in subquadratic time. Moreover, our experimental data show the usefulness of our algorithm even if the point distribution is unknown or not uniform: The running time savings over the pairwise probing approach constitute at least one order of magnitude already for a modest number of points and queries.
△ Less
Submitted 16 August, 2016; v1 submitted 7 September, 2015;
originally announced September 2015.
-
Fast generation of complex networks with underlying hyperbolic geometry
Authors:
Moritz von Looz,
Christian L. Staudt,
Henning Meyerhenke,
Roman Prutkin
Abstract:
Complex networks have become increasingly popular for modeling various real-world phenomena. Realistic generative network models are important in this context as they avoid privacy concerns of real data and simplify complex network research regarding data sharing, reproducibility, and scalability studies. \emph{Random hyperbolic graphs} are a well-analyzed family of geometric graphs. Previous work…
▽ More
Complex networks have become increasingly popular for modeling various real-world phenomena. Realistic generative network models are important in this context as they avoid privacy concerns of real data and simplify complex network research regarding data sharing, reproducibility, and scalability studies. \emph{Random hyperbolic graphs} are a well-analyzed family of geometric graphs. Previous work provided empirical and theoretical evidence that this generative graph model creates networks with non-vanishing clustering and other realistic features. However, the investigated networks in previous applied work were small, possibly due to the quadratic running time of a previous generator.
In this work we provide the first generation algorithm for these networks with subquadratic running time. We prove a time complexity of $O((n^{3/2}+m) \log n)$ with high probability for the generation process. This running time is confirmed by experimental data with our implementation. The acceleration stems primarily from the reduction of pairwise distance computations through a polar quadtree, which we adapt to hyperbolic space for this purpose. In practice we improve the running time of a previous implementation by at least two orders of magnitude this way. Networks with billions of edges can now be generated in a few minutes.
Finally, we evaluate the largest networks of this model published so far. Our empirical analysis shows that important features are retained over different graph densities and degree distributions.
△ Less
Submitted 23 April, 2015; v1 submitted 14 January, 2015;
originally announced January 2015.