-
Optimal Sample Complexity of Contrastive Learning
Authors:
Noga Alon,
Dmitrii Avdiukhin,
Dor Elboim,
Orr Fischer,
Grigory Yaroslavtsev
Abstract:
Contrastive learning is a highly successful technique for learning representations of data from labeled tuples, specifying the distance relations within the tuple. We study the sample complexity of contrastive learning, i.e. the minimum number of labeled tuples sufficient for getting high generalization accuracy. We give tight bounds on the sample complexity in a variety of settings, focusing on a…
▽ More
Contrastive learning is a highly successful technique for learning representations of data from labeled tuples, specifying the distance relations within the tuple. We study the sample complexity of contrastive learning, i.e. the minimum number of labeled tuples sufficient for getting high generalization accuracy. We give tight bounds on the sample complexity in a variety of settings, focusing on arbitrary distance functions, both general $\ell_p$-distances, and tree metrics. Our main result is an (almost) optimal bound on the sample complexity of learning $\ell_p$-distances for integer $p$. For any $p \ge 1$ we show that $\tilde Θ(\min(nd,n^2))$ labeled tuples are necessary and sufficient for learning $d$-dimensional representations of $n$-point datasets. Our results hold for an arbitrary distribution of the input samples and are based on giving the corresponding bounds on the Vapnik-Chervonenkis/Natarajan dimension of the associated problems. We further show that the theoretical bounds on sample complexity obtained via VC/Natarajan dimension can have strong predictive power for experimental results, in contrast with the folklore belief about a substantial gap between the statistical learning theory and the practice of deep learning.
△ Less
Submitted 1 December, 2023;
originally announced December 2023.
-
Distributed CONGEST Algorithms against Mobile Adversaries
Authors:
Orr Fischer,
Merav Parter
Abstract:
In their seminal PODC 1991 paper, Ostrovsky and Yung introduced the study of distributed computation in the presence of mobile adversaries which can dynamically appear throughout the network. Over the years, this setting has been studied mostly under the assumption that the communication graph is fully-connected. Resilient CONGEST algorithms for general graphs, on the other hand, are currently kno…
▽ More
In their seminal PODC 1991 paper, Ostrovsky and Yung introduced the study of distributed computation in the presence of mobile adversaries which can dynamically appear throughout the network. Over the years, this setting has been studied mostly under the assumption that the communication graph is fully-connected. Resilient CONGEST algorithms for general graphs, on the other hand, are currently known only for the classical static setting, i.e., where the set of corrupted edges (or nodes) is fixed throughout the entire computation.
We fill this gap by providing round-efficient simulations that translate given CONGEST algorithms into equivalent algorithms that are resilient against $f$-mobile edge adversaries. Our main results are:
-Perfect-Security with Mobile Eavesdroppers: A translation of any $r$-round $f$-static-secure algorithm into an equivalent $Θ(f)$-mobile-secure algorithm with $Θ(r)$ rounds. We also show that the $f$-static-secure algorithms of [Hitron, Parter and Yogev, DISC 2022 & ITCS 2023] can be modified into $f$-mobile-secure algorithms with the same number of rounds.
-Resilience with Mobile Byzantine Adversaries: An $f$-mobile-byzantine simulation which is based on a decomposition of the graph into low-diameter edge-disjoint spanning trees. This provides us with near-optimal CONGEST compilers for expander graphs. It also leads to near-optimal compilers in the congested-clique model against $Θ(n)$-mobile adversaries. For general $(2f+1)$ edge-connected graphs with $f$-mobile adversary, we almost match the bounds known for the $f$-static setting, when provided a trusted pre-processing phase.
Our results are based on a collection of tools from interactive coding [Gelles, Found. Trends Theor. Comput. Sci. 2017], linear sketches and low-congestion graph decomposition. The introduced toolkit might have further applications for resilient computation.
△ Less
Submitted 23 May, 2023;
originally announced May 2023.
-
Massively Parallel Computation in a Heterogeneous Regime
Authors:
Orr Fischer,
Adi Horowitz,
Rotem Oshman
Abstract:
Massively-parallel graph algorithms have received extensive attention over the past decade, with research focusing on three memory regimes: the superlinear regime, the near-linear regime, and the sublinear regime. The sublinear regime is the most desirable in practice, but conditional hardness results point towards its limitations.
In this work we study a \emph{heterogeneous} model, where the me…
▽ More
Massively-parallel graph algorithms have received extensive attention over the past decade, with research focusing on three memory regimes: the superlinear regime, the near-linear regime, and the sublinear regime. The sublinear regime is the most desirable in practice, but conditional hardness results point towards its limitations.
In this work we study a \emph{heterogeneous} model, where the memory of the machines varies in size. We focus mostly on the heterogeneous setting created by adding a single near-linear machine to the sublinear MPC regime, and show that even a single large machine suffices to circumvent most of the conditional hardness results for the sublinear regime: for graphs with $n$ vertices and $m$ edges, we give (a) an MST algorithm that runs in $O(\log\log(m/n))$ rounds; (b) an algorithm that constructs an $O(k)$-spanner of size $O(n^{1+1/k})$ in $O(1)$ rounds; and (c) a maximal-matching algorithm that runs in $O(\sqrt{\log(m/n)}\log\log(m/n))$ rounds.
We also observe that the best known near-linear MPC algorithms for several other graph problems which are conjectured to be hard in the sublinear regime (minimum cut, maximal independent set, and vertex coloring) can easily be transformed to work in the heterogeneous MPC model with a single near-linear machine, while retaining their original round complexity in the near-linear regime. If the large machine is allowed to have \emph{superlinear} memory, all of the problems above can be solved in $O(1)$ rounds.
△ Less
Submitted 28 February, 2023;
originally announced February 2023.
-
Tree Learning: Optimal Algorithms and Sample Complexity
Authors:
Dmitrii Avdiukhin,
Grigory Yaroslavtsev,
Danny Vainstein,
Orr Fischer,
Sauman Das,
Faraz Mirza
Abstract:
We study the problem of learning a hierarchical tree representation of data from labeled samples, taken from an arbitrary (and possibly adversarial) distribution. Consider a collection of data tuples labeled according to their hierarchical structure. The smallest number of such tuples required in order to be able to accurately label subsequent tuples is of interest for data collection in machine l…
▽ More
We study the problem of learning a hierarchical tree representation of data from labeled samples, taken from an arbitrary (and possibly adversarial) distribution. Consider a collection of data tuples labeled according to their hierarchical structure. The smallest number of such tuples required in order to be able to accurately label subsequent tuples is of interest for data collection in machine learning. We present optimal sample complexity bounds for this problem in several learning settings, including (agnostic) PAC learning and online learning. Our results are based on tight bounds of the Natarajan and Littlestone dimensions of the associated problem. The corresponding tree classifiers can be constructed efficiently in near-linear time.
△ Less
Submitted 9 February, 2023;
originally announced February 2023.
-
Prismatic Soft Actuator Augments the Workspace of Soft Continuum Robots
Authors:
Philipp Wand,
Oliver Fischer,
Robert K. Katzschmann
Abstract:
Soft robots are promising for manipulation tasks thanks to their compliance, safety, and high degree of freedom. However, the commonly used bidirectional continuum segment design means soft robotic manipulators only function in a limited hemispherical workspace. This work increases a soft robotic arm's workspace by designing, fabricating, and controlling an additional soft prismatic actuator at th…
▽ More
Soft robots are promising for manipulation tasks thanks to their compliance, safety, and high degree of freedom. However, the commonly used bidirectional continuum segment design means soft robotic manipulators only function in a limited hemispherical workspace. This work increases a soft robotic arm's workspace by designing, fabricating, and controlling an additional soft prismatic actuator at the base of the soft arm. This actuator consists of pneumatic artificial muscles and a piston, making the actuator back-driveable. We increase the task space volume by 116\%, and we are now able to perform manipulation tasks that were previously impossible for soft robots, such as picking and placing objects at different positions on a surface and grabbing an object out of a container. By combining a soft robotic arm with a prismatic joint, we greatly increase the usability of soft robots for object manipulation. This work promotes the use of integrated and modular soft robotic systems for practical manipulation applications in human-centered environments.
△ Less
Submitted 22 August, 2022; v1 submitted 15 April, 2022;
originally announced April 2022.
-
Quantum Distributed Algorithms for Detection of Cliques
Authors:
Keren Censor-Hillel,
Orr Fischer,
François Le Gall,
Dean Leitersdorf,
Rotem Oshman
Abstract:
The possibilities offered by quantum computing have drawn attention in the distributed computing community recently, with several breakthrough results showing quantum distributed algorithms that run faster than the fastest known classical counterparts, and even separations between the two models. A prime example is the result by Izumi, Le Gall, and Magniez [STACS 2020], who showed that triangle de…
▽ More
The possibilities offered by quantum computing have drawn attention in the distributed computing community recently, with several breakthrough results showing quantum distributed algorithms that run faster than the fastest known classical counterparts, and even separations between the two models. A prime example is the result by Izumi, Le Gall, and Magniez [STACS 2020], who showed that triangle detection by quantum distributed algorithms is easier than triangle listing, while an analogous result is not known in the classical case.
In this paper we present a framework for fast quantum distributed clique detection. This improves upon the state-of-the-art for the triangle case, and is also more general, applying to larger clique sizes.
Our main technical contribution is a new approach for detecting cliques by encapsulating this as a search task for nodes that can be added to smaller cliques. To extract the best complexities out of our approach, we develop a framework for nested distributed quantum searches, which employ checking procedures that are quantum themselves.
Moreover, we show a circuit-complexity barrier on proving a lower bound of the form $Ω(n^{3/5+ε})$ for $K_p$-detection for any $p \geq 4$, even in the classical (non-quantum) distributed CONGEST setting.
△ Less
Submitted 9 January, 2022;
originally announced January 2022.
-
Dynamic Task Space Control Enables Soft Manipulators to Perform Real-World Tasks
Authors:
Oliver Fischer,
Yasunori Toshimitsu,
Amirhossein Kazemipour,
Robert K. Katzschmann
Abstract:
Dynamic motions are a key feature of robotic arms, enabling them to perform tasks quickly and efficiently. Soft continuum manipulators do not currently consider dynamic parameters when operating in task space. This shortcoming makes existing soft robots slow and limits their ability to deal with external forces, especially during object manipulation. We address this issue by using dynamic operatio…
▽ More
Dynamic motions are a key feature of robotic arms, enabling them to perform tasks quickly and efficiently. Soft continuum manipulators do not currently consider dynamic parameters when operating in task space. This shortcoming makes existing soft robots slow and limits their ability to deal with external forces, especially during object manipulation. We address this issue by using dynamic operational space control. Our control approach takes into account the dynamic parameters of the 3D continuum arm and introduces new models that enable multi-segment soft manipulators to operate smoothly in task space. Advanced control methods, previously afforded only to rigid robots, are now adapted to soft robots; for example, potential field avoidance was previously only shown for rigid robots and is now extended to soft robots. Using our approach, a soft manipulator can now achieve a variety of tasks that were previously not possible: we evaluate the manipulator's performance in closed-loop controlled experiments such as pick-and-place, obstacle avoidance, throwing objects using an attached soft gripper, and deliberately applying forces to a surface by drawing with a grasped piece of chalk. Besides the newly enabled skills, our approach improves tracking accuracy by 59% and increases speed by a factor of 19.3 compared to state of the art for task space control. With these newfound abilities, soft robots can start to challenge rigid robots in the field of manipulation. Our inherently safe and compliant soft robot moves the future of robotic manipulation towards a cageless setup where humans and robots work in parallel.
△ Less
Submitted 18 October, 2022; v1 submitted 6 January, 2022;
originally announced January 2022.
-
Adaptive Dynamic Sliding Mode Control of Soft Continuum Manipulators
Authors:
Amirhossein Kazemipour,
Oliver Fischer,
Yasunori Toshimitsu,
Ki Wan Wong,
Robert K. Katzschmann
Abstract:
Soft robots are made of compliant materials and perform tasks that are challenging for rigid robots. However, their continuum nature makes it difficult to develop model-based control strategies. This work presents a robust model-based control scheme for soft continuum robots. Our dynamic model is based on the Euler-Lagrange approach, but it uses a more accurate description of the robot's inertia a…
▽ More
Soft robots are made of compliant materials and perform tasks that are challenging for rigid robots. However, their continuum nature makes it difficult to develop model-based control strategies. This work presents a robust model-based control scheme for soft continuum robots. Our dynamic model is based on the Euler-Lagrange approach, but it uses a more accurate description of the robot's inertia and does not include oversimplified assumptions. Based on this model, we introduce an adaptive sliding mode control scheme, which is robust against model parameter uncertainties and unknown input disturbances. We perform a series of experiments with a physical soft continuum arm to evaluate the effectiveness of our controller at tracking task-space trajectory under different payloads. The tracking performance of the controller is around 38\% more accurate than that of a state-of-the-art controller, i.e., the inverse dynamics method. Moreover, the proposed model-based control design is flexible and can be generalized to any continuum robotic arm with an arbitrary number of segments. With this control strategy, soft robotic object manipulation can become more accurate while remaining robust to disturbances.
△ Less
Submitted 26 February, 2022; v1 submitted 23 September, 2021;
originally announced September 2021.
-
Fast Distributed Algorithms for Girth, Cycles and Small Subgraphs
Authors:
Keren Censor-Hillel,
Orr Fischer,
Tzlil Gonen,
François Le Gall,
Dean Leitersdorf,
Rotem Oshman
Abstract:
In this paper we give fast distributed graph algorithms for detecting and listing small subgraphs, and for computing or approximating the girth. Our algorithms improve upon the state of the art by polynomial factors, and for girth, we obtain an constant-time algorithm for additive +1 approximation in the Congested Clique, and the first parametrized algorithm for exact computation in CONGEST.
In…
▽ More
In this paper we give fast distributed graph algorithms for detecting and listing small subgraphs, and for computing or approximating the girth. Our algorithms improve upon the state of the art by polynomial factors, and for girth, we obtain an constant-time algorithm for additive +1 approximation in the Congested Clique, and the first parametrized algorithm for exact computation in CONGEST.
In the Congested Clique, we develop a technique for learning small neighborhoods, and apply it to obtain an $O(1)$-round algorithm that computes the girth with only an additive +1 error. Next, we introduce a new technique (the partition tree technique) allowing for efficiently and deterministically listing all copies of any subgraph, improving upon the state-of the-art for non-dense graphs. We give two applications of this technique: First we show that for constant $k$, $C_{2k}$-detection can be solved in $O(1)$ rounds in the Congested Clique, improving on prior work which used matrix multiplication and had polynomial round complexity. Second, we show that in triangle-free graphs, the girth can be exactly computed in time polynomially faster than the best known bounds for general graphs.
In CONGEST, we describe a new approach for finding cycles, and apply it in two ways: first we show a fast parametrized algorithm for girth with round complexity $\tilde{O}(\min(g\cdot n^{1-1/Θ(g)},n))$ for any girth $g$; and second, we show how to find small even-length cycles $C_{2k}$ for $k = 3,4,5$ in $O(n^{1-1/k})$ rounds, which is a polynomial improvement upon the previous running times.
Finally, using our improved $C_6$-freeness algorithm and the barrier on proving lower bounds on triangle-freeness of Eden et al., we show that improving the current $\tildeΩ(\sqrt{n})$ lower bound for $C_6$-freeness of Korhonen et al. by any polynomial factor would imply strong circuit complexity lower bounds.
△ Less
Submitted 19 January, 2021;
originally announced January 2021.
-
Superlinear Lower Bounds for Distributed Subgraph Detection
Authors:
Orr Fischer,
Tzlil Gonen,
Rotem Oshman
Abstract:
In the distributed subgraph-freeness problem, we are given a graph $H$, and asked to determine whether the network graph contains $H$ as a subgraph or not. Subgraph-freeness is an extremely local problem: if the network had no bandwidth constraints, we could detect any subgraph $H$ in $|H|$ rounds, by having each node of the network learn its entire $|H|$-neighborhood. However, when bandwidth is l…
▽ More
In the distributed subgraph-freeness problem, we are given a graph $H$, and asked to determine whether the network graph contains $H$ as a subgraph or not. Subgraph-freeness is an extremely local problem: if the network had no bandwidth constraints, we could detect any subgraph $H$ in $|H|$ rounds, by having each node of the network learn its entire $|H|$-neighborhood. However, when bandwidth is limited, the problem becomes harder.
Upper and lower bounds in the presence of congestion have been established for several classes of subgraphs, including cycles, trees, and more complicated subgraphs. All bounds shown so far have been linear or sublinear. We show that the subgraph-freeness problem is not, in general, solvable in linear time: for any $k \geq 2$, there exists a subgraph $H_k$ such that $H_k$-freeness requires $Ω( n^{2-1/k} / (Bk) )$ rounds to solve. Here $B$ is the bandwidth of each communication link. The lower bound holds even for diameter-3 subgraphs and diameter-3 network graphs. In particular, taking $k = Θ(\log n)$, we obtain a lower bound of $Ω(n^2 / (B \log n))$.
△ Less
Submitted 18 November, 2017;
originally announced November 2017.
-
On The Multiparty Communication Complexity of Testing Triangle-Freeness
Authors:
Orr Fischer,
Shay Gershtein,
Rotem Oshman
Abstract:
In this paper we initiate the study of property testing in simultaneous and non-simultaneous multi-party communication complexity, focusing on testing triangle-freeness in graphs. We consider the $\textit{coordinator}$ model, where we have $k$ players receiving private inputs, and a coordinator who receives no input; the coordinator can communicate with all the players, but the players cannot comm…
▽ More
In this paper we initiate the study of property testing in simultaneous and non-simultaneous multi-party communication complexity, focusing on testing triangle-freeness in graphs. We consider the $\textit{coordinator}$ model, where we have $k$ players receiving private inputs, and a coordinator who receives no input; the coordinator can communicate with all the players, but the players cannot communicate with each other. In this model, we ask: if an input graph is divided between the players, with each player receiving some of the edges, how many bits do the players and the coordinator need to exchange to determine if the graph is triangle-free, or $\textit{far}$ from triangle-free?
For general communication protocols, we show that $\tilde{O}(k(nd)^{1/4}+k^2)$ bits are sufficient to test triangle-freeness in graphs of size $n$ with average degree $d$ (the degree need not be known in advance). For $\textit{simultaneous}$ protocols, where there is only one communication round, we give a protocol that uses $\tilde{O}(k \sqrt{n})$ bits when $d = O(\sqrt{n})$ and $\tilde{O}(k (nd)^{1/3})$ when $d = Ω(\sqrt{n})$; here, again, the average degree $d$ does not need to be known in advance. We show that for average degree $d = O(1)$, our simultaneous protocol is asymptotically optimal up to logarithmic factors. For higher degrees, we are not able to give lower bounds on testing triangle-freeness, but we give evidence that the problem is hard by showing that finding an edge that participates in a triangle is hard, even when promised that at least a constant fraction of the edges must be removed in order to make the graph triangle-free.
△ Less
Submitted 23 May, 2017;
originally announced May 2017.
-
Distributed Property Testing for Subgraph-Freeness Revisited
Authors:
Orr Fischer,
Tzlil Gonen,
Rotem Oshman
Abstract:
In the subgraph-freeness problem, we are given a constant-size graph $H$, and wish to determine whether the network contains $H$ as a subgraph or not. The \emph{property-testing} relaxation of the problem only requires us to distinguish graphs that are $H$-free from graphs that are $ε$-far from $H$-free, meaning an $ε$-fraction of their edges must be removed to obtain an $H$-free graph. Recently,…
▽ More
In the subgraph-freeness problem, we are given a constant-size graph $H$, and wish to determine whether the network contains $H$ as a subgraph or not. The \emph{property-testing} relaxation of the problem only requires us to distinguish graphs that are $H$-free from graphs that are $ε$-far from $H$-free, meaning an $ε$-fraction of their edges must be removed to obtain an $H$-free graph. Recently, Censor-Hillel et. al. and Fraigniaud et al. showed that in the property-testing regime it is possible to test $H$-freeness for any graph $H$ of size 4 in constant time, $O(1/ε^2)$ rounds, regardless of the network size. However, Fraigniaud et. al. also showed that their techniques for graphs $H$ of size 4 cannot test $5$-cycle-freeness in constant time.
In this paper we revisit the subgraph-freeness problem and show that $5$-cycle-freeness, and indeed $H$-freeness for many other graphs $H$ comprising more than 4 vertices, can be tested in constant time. We show that $C_k$-freeness can be tested in $O(1/ε)$ rounds for any cycle $C_k$, improving on the running time of $O(1/ε^2)$ of the previous algorithms for triangle-freeness and $C_4$-freeness. In the special case of triangles, we show that triangle-freeness can be solved in $O(1)$ rounds independently of $ε$, when $ε$ is not too small with respect to the number of nodes and edges. We also show that $T$-freeness for any constant-size tree $T$ can be tested in $O(1)$ rounds, even without the property-testing relaxation. Building on these results, we define a general class of graphs for which we can test subgraph-freeness in $O(1/ε)$ rounds. This class includes all graphs over 5 vertices except the 5-clique, $K_5$. For cliques $K_s$ over $s \geq 3$ nodes, we show that $K_s$-freeness can be tested in $O(m^{1/2-1/(s-2)}/ε^{1/2+1/(s-2)})$ rounds, where $m$ is the number of edges.
△ Less
Submitted 11 May, 2017;
originally announced May 2017.
-
A Lower Bound for the Distributed Lovász Local Lemma
Authors:
Sebastian Brandt,
Orr Fischer,
Juho Hirvonen,
Barbara Keller,
Tuomo Lempiäinen,
Joel Rybicki,
Jukka Suomela,
Jara Uitto
Abstract:
We show that any randomised Monte Carlo distributed algorithm for the Lovász local lemma requires $Ω(\log \log n)$ communication rounds, assuming that it finds a correct assignment with high probability. Our result holds even in the special case of $d = O(1)$, where $d$ is the maximum degree of the dependency graph. By prior work, there are distributed algorithms for the Lovász local lemma with a…
▽ More
We show that any randomised Monte Carlo distributed algorithm for the Lovász local lemma requires $Ω(\log \log n)$ communication rounds, assuming that it finds a correct assignment with high probability. Our result holds even in the special case of $d = O(1)$, where $d$ is the maximum degree of the dependency graph. By prior work, there are distributed algorithms for the Lovász local lemma with a running time of $O(\log n)$ rounds in bounded-degree graphs, and the best lower bound before our work was $Ω(\log^* n)$ rounds [Chung et al. 2014].
△ Less
Submitted 3 November, 2015;
originally announced November 2015.
