-
Robust Surgical Phase Recognition From Annotation Efficient Supervision
Authors:
Or Rubin,
Shlomi Laufer
Abstract:
Surgical phase recognition is a key task in computer-assisted surgery, aiming to automatically identify and categorize the different phases within a surgical procedure. Despite substantial advancements, most current approaches rely on fully supervised training, requiring expensive and time-consuming frame-level annotations. Timestamp supervision has recently emerged as a promising alternative, sig…
▽ More
Surgical phase recognition is a key task in computer-assisted surgery, aiming to automatically identify and categorize the different phases within a surgical procedure. Despite substantial advancements, most current approaches rely on fully supervised training, requiring expensive and time-consuming frame-level annotations. Timestamp supervision has recently emerged as a promising alternative, significantly reducing annotation costs while maintaining competitive performance. However, models trained on timestamp annotations can be negatively impacted by missing phase annotations, leading to a potential drawback in real-world scenarios. In this work, we address this issue by proposing a robust method for surgical phase recognition that can handle missing phase annotations effectively. Furthermore, we introduce the SkipTag@K annotation approach to the surgical domain, enabling a flexible balance between annotation effort and model performance. Our method achieves competitive results on two challenging datasets, demonstrating its efficacy in handling missing phase annotations and its potential for reducing annotation costs. Specifically, we achieve an accuracy of 85.1\% on the MultiBypass140 dataset using only 3 annotated frames per video, showcasing the effectiveness of our method and the potential of the SkipTag@K setup. We perform extensive experiments to validate the robustness of our method and provide valuable insights to guide future research in surgical phase recognition. Our work contributes to the advancement of surgical workflow recognition and paves the way for more efficient and reliable surgical phase recognition systems.
△ Less
Submitted 26 June, 2024;
originally announced June 2024.
-
Computing Chebyshev polynomials using the complex Remez algorithm
Authors:
Olof Rubin
Abstract:
We employ the generalized Remez algorithm, initially suggested by P. T. P. Tang, to perform an experimental study of Chebyshev polynomials in the complex plane. Our focus lies particularly on the examination of their norms and zeros. What sets our study apart is the breadth of examples considered, coupled with the fact that the degrees under investigation are substantially higher than those in pre…
▽ More
We employ the generalized Remez algorithm, initially suggested by P. T. P. Tang, to perform an experimental study of Chebyshev polynomials in the complex plane. Our focus lies particularly on the examination of their norms and zeros. What sets our study apart is the breadth of examples considered, coupled with the fact that the degrees under investigation are substantially higher than those in previous studies where other methods have been applied. These computations of Chebyshev polynomials of high degrees reveal discernible patterns which allow for conjectures to be formulated based on abundant experimental evidence. The use of Tang's algorithm allows for computations executed with precision, maintaining accuracy within quantifiable margins of error. Additionally, as a result of our experimental study, we propose what we believe to be a fundamental relationship between Chebyshev and Faber polynomials associated with a compact set.
△ Less
Submitted 16 May, 2024; v1 submitted 8 May, 2024;
originally announced May 2024.
-
CuVLER: Enhanced Unsupervised Object Discoveries through Exhaustive Self-Supervised Transformers
Authors:
Shahaf Arica,
Or Rubin,
Sapir Gershov,
Shlomi Laufer
Abstract:
In this paper, we introduce VoteCut, an innovative method for unsupervised object discovery that leverages feature representations from multiple self-supervised models. VoteCut employs normalized-cut based graph partitioning, clustering and a pixel voting approach. Additionally, We present CuVLER (Cut-Vote-and-LEaRn), a zero-shot model, trained using pseudo-labels, generated by VoteCut, and a nove…
▽ More
In this paper, we introduce VoteCut, an innovative method for unsupervised object discovery that leverages feature representations from multiple self-supervised models. VoteCut employs normalized-cut based graph partitioning, clustering and a pixel voting approach. Additionally, We present CuVLER (Cut-Vote-and-LEaRn), a zero-shot model, trained using pseudo-labels, generated by VoteCut, and a novel soft target loss to refine segmentation accuracy. Through rigorous evaluations across multiple datasets and several unsupervised setups, our methods demonstrate significant improvements in comparison to previous state-of-the-art models. Our ablation studies further highlight the contributions of each component, revealing the robustness and efficacy of our approach. Collectively, VoteCut and CuVLER pave the way for future advancements in image segmentation.
△ Less
Submitted 12 March, 2024;
originally announced March 2024.
-
Extremal polynomials and polynomial preimages
Authors:
Jacob S. Christiansen,
Benjamin Eichinger,
Olof Rubin
Abstract:
This article examines the asymptotic behavior of the Widom factors, denoted $\mathcal{W}_n$, for Chebyshev polynomials of finite unions of Jordan arcs. We prove that, in contrast to Widom's proposal, when dealing with a single smooth Jordan arc, $\mathcal{W}_n$ converges to 2 exclusively when the arc is a straight line segment. Our main focus is on analysing polynomial preimages of the interval…
▽ More
This article examines the asymptotic behavior of the Widom factors, denoted $\mathcal{W}_n$, for Chebyshev polynomials of finite unions of Jordan arcs. We prove that, in contrast to Widom's proposal, when dealing with a single smooth Jordan arc, $\mathcal{W}_n$ converges to 2 exclusively when the arc is a straight line segment. Our main focus is on analysing polynomial preimages of the interval $[-2,2]$, and we provide a complete description of the asymptotic behavior of $\mathcal{W}_n$ for symmetric star graphs and quadratic preimages of $[-2,2]$. We observe that in the case of star graphs, the Chebyshev polynomials and the polynomials orthogonal with respect to equilibrium measure share the same norm asymptotics, suggesting a potential extension of a conjecture posed by Christiansen, Simon and Zinchenko. Lastly, we propose a possible connection between the $S$-property and Widom factors converging to $2$.
△ Less
Submitted 20 December, 2023;
originally announced December 2023.
-
Chebyshev polynomials corresponding to a vanishing weight
Authors:
Alex Bergman,
Olof Rubin
Abstract:
We consider weighted Chebyshev polynomials on the unit circle corresponding to a weight of the form $(z-1)^s$ where $s>0$. For integer values of $s$ this corresponds to prescribing a zero of the polynomial on the boundary. As such, we extend findings of Lachance, Saff and Varga, to non-integer $s$. Using this generalisation, we are able to relate Chebyshev polynomials on lemniscates and other, mor…
▽ More
We consider weighted Chebyshev polynomials on the unit circle corresponding to a weight of the form $(z-1)^s$ where $s>0$. For integer values of $s$ this corresponds to prescribing a zero of the polynomial on the boundary. As such, we extend findings of Lachance, Saff and Varga, to non-integer $s$. Using this generalisation, we are able to relate Chebyshev polynomials on lemniscates and other, more established, categories of Chebyshev polynomials. An essential part of our proof involves the broadening of the Erdős--Lax inequality to encompass powers of polynomials. We believe that this particular result holds significance in its own right.
△ Less
Submitted 15 January, 2024; v1 submitted 5 September, 2023;
originally announced September 2023.
-
Long-range Language Modeling with Self-retrieval
Authors:
Ohad Rubin,
Jonathan Berant
Abstract:
Retrieval-augmented language models (LMs) have received much attention recently. However, typically the retriever is not trained jointly as a native component of the LM, but added to an already-pretrained LM, which limits the ability of the LM and the retriever to adapt to one another. In this work, we propose the Retrieval-Pretrained Transformer (RPT), an architecture and training procedure for j…
▽ More
Retrieval-augmented language models (LMs) have received much attention recently. However, typically the retriever is not trained jointly as a native component of the LM, but added to an already-pretrained LM, which limits the ability of the LM and the retriever to adapt to one another. In this work, we propose the Retrieval-Pretrained Transformer (RPT), an architecture and training procedure for jointly training a retrieval-augmented LM from scratch for the task of modeling long texts. Given a recently generated text chunk in a long document, the LM computes query representations, which are then used to retrieve earlier chunks in the document, located potentially tens of thousands of tokens before. Information from retrieved chunks is fused into the LM representations to predict the next target chunk. We train the retriever component with a semantic objective, where the goal is to retrieve chunks that increase the probability of the next chunk, according to a reference LM. We evaluate RPT on four long-range language modeling tasks, spanning books, code, and mathematical writing, and demonstrate that RPT improves retrieval quality and subsequently perplexity across the board compared to strong baselines.
△ Less
Submitted 23 June, 2023;
originally announced June 2023.
-
Kinematic Data-Based Action Segmentation for Surgical Applications
Authors:
Adam Goldbraikh,
Omer Shubi,
Or Rubin,
Carla M Pugh,
Shlomi Laufer
Abstract:
Action segmentation is a challenging task in high-level process analysis, typically performed on video or kinematic data obtained from various sensors. In the context of surgical procedures, action segmentation is critical for workflow analysis algorithms. This work presents two contributions related to action segmentation on kinematic data. Firstly, we introduce two multi-stage architectures, MS-…
▽ More
Action segmentation is a challenging task in high-level process analysis, typically performed on video or kinematic data obtained from various sensors. In the context of surgical procedures, action segmentation is critical for workflow analysis algorithms. This work presents two contributions related to action segmentation on kinematic data. Firstly, we introduce two multi-stage architectures, MS-TCN-BiLSTM and MS-TCN-BiGRU, specifically designed for kinematic data. The architectures consist of a prediction generator with intra-stage regularization and Bidirectional LSTM or GRU-based refinement stages. Secondly, we propose two new data augmentation techniques, World Frame Rotation and Horizontal-Flip, which utilize the strong geometric structure of kinematic data to improve algorithm performance and robustness. We evaluate our models on three datasets of surgical suturing tasks: the Variable Tissue Simulation (VTS) Dataset and the newly introduced Bowel Repair Simulation (BRS) Dataset, both of which are open surgery simulation datasets collected by us, as well as the JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS), a well-known benchmark in robotic surgery. Our methods achieve state-of-the-art performance on all benchmark datasets and establish a strong baseline for the BRS dataset.
△ Less
Submitted 14 March, 2023;
originally announced March 2023.
-
QAMPARI: An Open-domain Question Answering Benchmark for Questions with Many Answers from Multiple Paragraphs
Authors:
Samuel Joseph Amouyal,
Tomer Wolfson,
Ohad Rubin,
Ori Yoran,
Jonathan Herzig,
Jonathan Berant
Abstract:
Existing benchmarks for open-domain question answering (ODQA) typically focus on questions whose answers can be extracted from a single paragraph. By contrast, many natural questions, such as "What players were drafted by the Brooklyn Nets?" have a list of answers. Answering such questions requires retrieving and reading from many passages, in a large corpus. We introduce QAMPARI, an ODQA benchmar…
▽ More
Existing benchmarks for open-domain question answering (ODQA) typically focus on questions whose answers can be extracted from a single paragraph. By contrast, many natural questions, such as "What players were drafted by the Brooklyn Nets?" have a list of answers. Answering such questions requires retrieving and reading from many passages, in a large corpus. We introduce QAMPARI, an ODQA benchmark, where question answers are lists of entities, spread across many paragraphs. We created QAMPARI by (a) generating questions with multiple answers from Wikipedia's knowledge graph and tables, (b) automatically pairing answers with supporting evidence in Wikipedia paragraphs, and (c) manually paraphrasing questions and validating each answer. We train ODQA models from the retrieve-and-read family and find that QAMPARI is challenging in terms of both passage retrieval and answer generation, reaching an F1 score of 32.8 at best. Our results highlight the need for develo** ODQA models that handle a broad range of question types, including single and multi-answer questions.
△ Less
Submitted 29 May, 2023; v1 submitted 25 May, 2022;
originally announced May 2022.
-
Learning To Retrieve Prompts for In-Context Learning
Authors:
Ohad Rubin,
Jonathan Herzig,
Jonathan Berant
Abstract:
In-context learning is a recent paradigm in natural language understanding, where a large pre-trained language model (LM) observes a test instance and a few training examples as its input, and directly decodes the output without any update to its parameters. However, performance has been shown to strongly depend on the selected training examples (termed prompt). In this work, we propose an efficie…
▽ More
In-context learning is a recent paradigm in natural language understanding, where a large pre-trained language model (LM) observes a test instance and a few training examples as its input, and directly decodes the output without any update to its parameters. However, performance has been shown to strongly depend on the selected training examples (termed prompt). In this work, we propose an efficient method for retrieving prompts for in-context learning using annotated data and a LM. Given an input-output pair, we estimate the probability of the output given the input and a candidate training example as the prompt, and label training examples as positive or negative based on this probability. We then train an efficient dense retriever from this data, which is used to retrieve training examples as prompts at test time. We evaluate our approach on three sequence-to-sequence tasks where language utterances are mapped to meaning representations, and find that it substantially outperforms prior work and multiple baselines across the board.
△ Less
Submitted 8 May, 2022; v1 submitted 16 December, 2021;
originally announced December 2021.
-
SmBoP: Semi-autoregressive Bottom-up Semantic Parsing
Authors:
Ohad Rubin,
Jonathan Berant
Abstract:
The de-facto standard decoding method for semantic parsing in recent years has been to autoregressively decode the abstract syntax tree of the target program using a top-down depth-first traversal. In this work, we propose an alternative approach: a Semi-autoregressive Bottom-up Parser (SmBoP) that constructs at decoding step $t$ the top-$K$ sub-trees of height $\leq t$. Our parser enjoys several…
▽ More
The de-facto standard decoding method for semantic parsing in recent years has been to autoregressively decode the abstract syntax tree of the target program using a top-down depth-first traversal. In this work, we propose an alternative approach: a Semi-autoregressive Bottom-up Parser (SmBoP) that constructs at decoding step $t$ the top-$K$ sub-trees of height $\leq t$. Our parser enjoys several benefits compared to top-down autoregressive parsing. From an efficiency perspective, bottom-up parsing allows to decode all sub-trees of a certain height in parallel, leading to logarithmic runtime complexity rather than linear. From a modeling perspective, a bottom-up parser learns representations for meaningful semantic sub-programs at each step, rather than for semantically-vacuous partial trees. We apply SmBoP on Spider, a challenging zero-shot semantic parsing benchmark, and show that SmBoP leads to a 2.2x speed-up in decoding time and a $\sim$5x speed-up in training time, compared to a semantic parser that uses autoregressive decoding. SmBoP obtains 71.1 denotation accuracy on Spider, establishing a new state-of-the-art, and 69.5 exact match, comparable to the 69.6 exact match of the autoregressive RAT-SQL+GraPPa.
△ Less
Submitted 11 April, 2021; v1 submitted 23 October, 2020;
originally announced October 2020.
