Showing 1–50 of 54 results for author: Nikolic, M

Engineering an Efficient Object Tracker for Non-Linear Motion

Authors: Momir Adžemović, Predrag Tadić, Andrija Petrović, Mladen Nikolić

Abstract: The goal of multi-object tracking is to detect and track all objects in a scene while maintaining unique identifiers for each, by associating their bounding boxes across video frames. This association relies on matching motion and appearance patterns of detected objects. This task is especially hard in case of scenarios involving dynamic and non-linear motion patterns. In this paper, we introduce… ▽ More The goal of multi-object tracking is to detect and track all objects in a scene while maintaining unique identifiers for each, by associating their bounding boxes across video frames. This association relies on matching motion and appearance patterns of detected objects. This task is especially hard in case of scenarios involving dynamic and non-linear motion patterns. In this paper, we introduce DeepMoveSORT, a novel, carefully engineered multi-object tracker designed specifically for such scenarios. In addition to standard methods of appearance-based association, we improve motion-based association by employing deep learnable filters (instead of the most commonly used Kalman filter) and a rich set of newly proposed heuristics. Our improvements to motion-based association methods are severalfold. First, we propose a new transformer-based filter architecture, TransFilter, which uses an object's motion history for both motion prediction and noise filtering. We further enhance the filter's performance by careful handling of its motion history and accounting for camera motion. Second, we propose a set of heuristics that exploit cues from the position, shape, and confidence of detected bounding boxes to improve association performance. Our experimental evaluation demonstrates that DeepMoveSORT outperforms existing trackers in scenarios featuring non-linear motion, surpassing state-of-the-art results on three such datasets. We also perform a thorough ablation study to evaluate the contributions of different tracker components which we proposed. Based on our study, we conclude that using a learnable filter instead of the Kalman filter, along with appearance-based association is key to achieving strong general tracking performance. △ Less

Submitted 30 June, 2024; originally announced July 2024.

Comments: 3 figures, 20 tables

ACM Class: I.2.10; I.2.6

Generative AI as a metacognitive agent: A comparative mixed-method study with human participants on ICF-mimicking exam performance

Authors: Jelena Pavlovic, Jugoslav Krstic, Luka Mitrovic, Djordje Babic, Adrijana Milosavljevic, Milena Nikolic, Tijana Karaklic, Tijana Mitrovic

Abstract: This study investigates the metacognitive capabilities of Large Language Models relative to human metacognition in the context of the International Coaching Federation ICF mimicking exam, a situational judgment test related to coaching competencies. Using a mixed method approach, we assessed the metacognitive performance, including sensitivity, accuracy in probabilistic predictions, and bias, of h… ▽ More This study investigates the metacognitive capabilities of Large Language Models relative to human metacognition in the context of the International Coaching Federation ICF mimicking exam, a situational judgment test related to coaching competencies. Using a mixed method approach, we assessed the metacognitive performance, including sensitivity, accuracy in probabilistic predictions, and bias, of human participants and five advanced LLMs (GPT-4, Claude-3-Opus 3, Mistral Large, Llama 3, and Gemini 1.5 Pro). The results indicate that LLMs outperformed humans across all metacognitive metrics, particularly in terms of reduced overconfidence, compared to humans. However, both LLMs and humans showed less adaptability in ambiguous scenarios, adhering closely to predefined decision frameworks. The study suggests that Generative AI can effectively engage in human-like metacognitive processing without conscious awareness. Implications of the study are discussed in relation to development of AI simulators that scaffold cognitive and metacognitive aspects of mastering coaching competencies. More broadly, implications of these results are discussed in relation to development of metacognitive modules that lead towards more autonomous and intuitive AI systems. △ Less

Submitted 7 May, 2024; originally announced May 2024.

Tractable Conjunctive Queries over Static and Dynamic Relations

Authors: Ahmet Kara, Zheng Luo, Milos Nikolic, Dan Olteanu, Haozhe Zhang

Abstract: We investigate the evaluation of conjunctive queries over static and dynamic relations. While static relations are given as input and do not change, dynamic relations are subject to inserts and deletes. We characterise syntactically three classes of queries that admit constant update time and constant enumeration delay. We call such queries tractable. Depending on the class, the preprocessing ti… ▽ More We investigate the evaluation of conjunctive queries over static and dynamic relations. While static relations are given as input and do not change, dynamic relations are subject to inserts and deletes. We characterise syntactically three classes of queries that admit constant update time and constant enumeration delay. We call such queries tractable. Depending on the class, the preprocessing time is linear, polynomial, or exponential (under data complexity, so the query size is constant). To decide whether a query is tractable, it does not suffice to analyse separately the sub-query over the static relations and the sub-query over the dynamic relations. Instead, we need to take the interaction between the static and the dynamic relations into account. Even when the sub-query over the dynamic relations is not tractable, the overall query can become tractable if the dynamic relations are sufficiently constrained by the static ones. △ Less

Submitted 24 April, 2024; originally announced April 2024.

ACM Class: H.2.4

Beyond Kalman Filters: Deep Learning-Based Filters for Improved Object Tracking

Authors: Momir Adžemović, Predrag Tadić, Andrija Petrović, Mladen Nikolić

Abstract: Traditional tracking-by-detection systems typically employ Kalman filters (KF) for state estimation. However, the KF requires domain-specific design choices and it is ill-suited to handling non-linear motion patterns. To address these limitations, we propose two innovative data-driven filtering methods. Our first method employs a Bayesian filter with a trainable motion model to predict an object's… ▽ More Traditional tracking-by-detection systems typically employ Kalman filters (KF) for state estimation. However, the KF requires domain-specific design choices and it is ill-suited to handling non-linear motion patterns. To address these limitations, we propose two innovative data-driven filtering methods. Our first method employs a Bayesian filter with a trainable motion model to predict an object's future location and combines its predictions with observations gained from an object detector to enhance bounding box prediction accuracy. Moreover, it dispenses with most domain-specific design choices characteristic of the KF. The second method, an end-to-end trainable filter, goes a step further by learning to correct detector errors, further minimizing the need for domain expertise. Additionally, we introduce a range of motion model architectures based on Recurrent Neural Networks, Neural Ordinary Differential Equations, and Conditional Neural Processes, that are combined with the proposed filtering methods. Our extensive evaluation across multiple datasets demonstrates that our proposed filters outperform the traditional KF in object tracking, especially in the case of non-linear motion patterns -- the use case our filters are best suited to. We also conduct noise robustness analysis of our filters with convincing positive results. We further propose a new cost function for associating observations with tracks. Our tracker, which incorporates this new association cost with our proposed filters, outperforms the conventional SORT method and other motion-based trackers in multi-object tracking according to multiple metrics on motion-rich DanceTrack and SportsMOT datasets. △ Less

Submitted 15 February, 2024; originally announced February 2024.

Comments: 29 pages

Even SIMP miracles are possible

Authors: Xiaoyong Chu, Marco Nikolic, Josef Pradler

Abstract: Strongly interacting massive particles $π$ have been advocated as prominent dark matter candidates when they regulate their relic abundance through odd-numbered $3 π\to2π$ annihilation. We show that successful freeze-out may also be achieved through even-numbered interactions $X X \to ππ$ once bound states $X$ among the particles of the low-energy spectrum exist. In addition, $X$-formation hosts t… ▽ More Strongly interacting massive particles $π$ have been advocated as prominent dark matter candidates when they regulate their relic abundance through odd-numbered $3 π\to2π$ annihilation. We show that successful freeze-out may also be achieved through even-numbered interactions $X X \to ππ$ once bound states $X$ among the particles of the low-energy spectrum exist. In addition, $X$-formation hosts the potential of also catalyzing odd-numbered $3 π\to2π$ annihilation processes, turning them into effective two-body processes $πX \to ππ$. Bound states are often a natural consequence of strongly interacting theories. We calculate the dark matter freeze-out and comment on the cosmic viability and possible extensions. Candidate theories can encompass confining sectors without a mass gap, glueball dark matter, or $φ^3$ and $φ^4$ theories with strong Yukawa or self-interactions. △ Less

Submitted 22 January, 2024; originally announced January 2024.

Comments: 15 pages, 5 figures

In-Database Data Imputation

Authors: Massimo Perini, Milos Nikolic

Abstract: Missing data is a widespread problem in many domains, creating challenges in data analysis and decision making. Traditional techniques for dealing with missing data, such as excluding incomplete records or imputing simple estimates (e.g., mean), are computationally efficient but may introduce bias and disrupt variable relationships, leading to inaccurate analyses. Model-based imputation techniques… ▽ More Missing data is a widespread problem in many domains, creating challenges in data analysis and decision making. Traditional techniques for dealing with missing data, such as excluding incomplete records or imputing simple estimates (e.g., mean), are computationally efficient but may introduce bias and disrupt variable relationships, leading to inaccurate analyses. Model-based imputation techniques offer a more robust solution that preserves the variability and relationships in the data, but they demand significantly more computation time, limiting their applicability to small datasets. This work enables efficient, high-quality, and scalable data imputation within a database system using the widely used MICE method. We adapt this method to exploit computation sharing and a ring abstraction for faster model training. To impute both continuous and categorical values, we develop techniques for in-database learning of stochastic linear regression and Gaussian discriminant analysis models. Our MICE implementations in PostgreSQL and DuckDB outperform alternative MICE implementations and model-based imputation techniques by up to two orders of magnitude in terms of computation time, while maintaining high imputation quality. △ Less

Submitted 6 January, 2024; originally announced January 2024.

Comments: Published at SIGMOD 2024 (26 pages)

Scale invariance in early embryonic development

Authors: Miloš Nikolić, Victoria Antonetti, Feng Liu, Gentian Muhaxheri, Mariela D. Petkova, Martin Scheeler, Eric M. Smith, William Bialek, Thomas Gregor

Abstract: The body plan of the fruit fly is determined by the expression of just a handful of genes. We show that the spatial patterns of expression for several of these genes scale precisely with the size of the embryo. Concretely, discrete positional markers such as the peaks in striped patterns have absolute positions along the anterior-posterior axis that are proportional to embryo length, with better t… ▽ More The body plan of the fruit fly is determined by the expression of just a handful of genes. We show that the spatial patterns of expression for several of these genes scale precisely with the size of the embryo. Concretely, discrete positional markers such as the peaks in striped patterns have absolute positions along the anterior-posterior axis that are proportional to embryo length, with better than 1% accuracy. Further, the information (in bits) that graded patterns of expression provide about position can be decomposed into information about fractional or scaled position and information about absolute position or embryo length; all of the available information is about scaled position, again with ~1% accuracy. These observations suggest that the underlying genetic network exhibits scale invariance in a deeper mathematical sense. Taking this mathematical statement seriously requires that the network dynamics have a zero mode, which connects to many other observations on this system. △ Less

Submitted 29 December, 2023; originally announced December 2023.

Finding the last bits of positional information

Authors: Lauren McGough, Helena Casademunt, Miloš Nikolić, Mariela D. Petkova, Thomas Gregor, William Bialek

Abstract: In a develo** embryo, information about the position of cells is encoded in the concentrations of "morphogen" molecules. In the fruit fly, the local concentrations of just a handful of proteins encoded by the gap genes are sufficient to specify position with a precision comparable to the spacing between cells along the anterior--posterior axis. This matches the precision of downstream events suc… ▽ More In a develo** embryo, information about the position of cells is encoded in the concentrations of "morphogen" molecules. In the fruit fly, the local concentrations of just a handful of proteins encoded by the gap genes are sufficient to specify position with a precision comparable to the spacing between cells along the anterior--posterior axis. This matches the precision of downstream events such as the striped patterns of expression in the pair-rule genes, but is not quite sufficient to define unique identities for individual cells. We demonstrate theoretically that this information gap can be bridged if positional errors are spatially correlated, with relatively long correlation lengths. We then show experimentally that these correlations are present, with the required strength, in the fluctuating positions of the pair-rule stripes, and this can be traced back to the gap genes. Taking account of these correlations, the available information matches the information needed for unique cellular specification, within error bars of ~2%. These observation support a precisionist view of information flow through the underlying genetic networks, in which accurate signals are available from the start and preserved as they are transformed into the final spatial patterns. △ Less

Submitted 10 December, 2023; originally announced December 2023.

Incorporating LLM Priors into Tabular Learners

Authors: Max Zhu, Siniša Stanivuk, Andrija Petrovic, Mladen Nikolic, Pietro Lio

Abstract: We present a method to integrate Large Language Models (LLMs) and traditional tabular data classification techniques, addressing LLMs challenges like data serialization sensitivity and biases. We introduce two strategies utilizing LLMs for ranking categorical variables and generating priors on correlations between continuous variables and targets, enhancing performance in few-shot scenarios. We fo… ▽ More We present a method to integrate Large Language Models (LLMs) and traditional tabular data classification techniques, addressing LLMs challenges like data serialization sensitivity and biases. We introduce two strategies utilizing LLMs for ranking categorical variables and generating priors on correlations between continuous variables and targets, enhancing performance in few-shot scenarios. We focus on Logistic Regression, introducing MonotonicLR that employs a non-linear monotonic function for map** ordinals to cardinals while preserving LLM-determined orders. Validation against baseline models reveals the superior performance of our approach, especially in low-data scenarios, while remaining interpretable. △ Less

Submitted 20 November, 2023; originally announced November 2023.

Comments: Table Representation Learning Workshop at NeurIPS 2023

Tabular Few-Shot Generalization Across Heterogeneous Feature Spaces

Authors: Max Zhu, Katarzyna Kobalczyk, Andrija Petrovic, Mladen Nikolic, Mihaela van der Schaar, Boris Delibasic, Petro Lio

Abstract: Despite the prevalence of tabular datasets, few-shot learning remains under-explored within this domain. Existing few-shot methods are not directly applicable to tabular datasets due to varying column relationships, meanings, and permutational invariance. To address these challenges, we propose FLAT-a novel approach to tabular few-shot learning, encompassing knowledge sharing between datasets with… ▽ More Despite the prevalence of tabular datasets, few-shot learning remains under-explored within this domain. Existing few-shot methods are not directly applicable to tabular datasets due to varying column relationships, meanings, and permutational invariance. To address these challenges, we propose FLAT-a novel approach to tabular few-shot learning, encompassing knowledge sharing between datasets with heterogeneous feature spaces. Utilizing an encoder inspired by Dataset2Vec, FLAT learns low-dimensional embeddings of datasets and their individual columns, which facilitate knowledge transfer and generalization to previously unseen datasets. A decoder network parametrizes the predictive target network, implemented as a Graph Attention Network, to accommodate the heterogeneous nature of tabular datasets. Experiments on a diverse collection of 118 UCI datasets demonstrate FLAT's successful generalization to new tabular datasets and a considerable improvement over the baselines. △ Less

Submitted 16 November, 2023; originally announced November 2023.

Comments: Tabular learning, Deep learning, Few shot learning

The LSST AGN Data Challenge: Selection methods

Authors: Đorđe V. Savić, Isidora Jankov, Weixiang Yu, Vincenzo Petrecca, Matthew J. Temple, Qingling Ni, Raphael Shirley, Andjelka B. Kovacevic, Mladen Nikolic, Dragana Ilic, Luka C. Popovic, Maurizio Paolillo, Swayamtrupta Panda, Aleksandra Ciprijanovic, Gordon T. Richards

Abstract: Development of the Rubin Observatory Legacy Survey of Space and Time (LSST) includes a series of Data Challenges (DC) arranged by various LSST Scientific Collaborations (SC) that are taking place during the projects preoperational phase. The AGN Science Collaboration Data Challenge (AGNSCDC) is a partial prototype of the expected LSST AGN data, aimed at validating machine learning approaches for A… ▽ More Development of the Rubin Observatory Legacy Survey of Space and Time (LSST) includes a series of Data Challenges (DC) arranged by various LSST Scientific Collaborations (SC) that are taking place during the projects preoperational phase. The AGN Science Collaboration Data Challenge (AGNSCDC) is a partial prototype of the expected LSST AGN data, aimed at validating machine learning approaches for AGN selection and characterization in large surveys like LSST. The AGNSC-DC took part in 2021 focusing on accuracy, robustness, and scalability. The training and the blinded datasets were constructed to mimic the future LSST release catalogs using the data from the Sloan Digital Sky Survey Stripe 82 region and the XMM-Newton Large Scale Structure Survey region. Data features were divided into astrometry, photometry, color, morphology, redshift and class label with the addition of variability features and images. We present the results of four DC submitted solutions using both classical and machine learning methods. We systematically test the performance of supervised (support vector machine, random forest, extreme gradient boosting, artificial neural network, convolutional neural network) and unsupervised (deep embedding clustering) models when applied to the problem of classifying/clustering sources as stars, galaxies or AGNs. We obtained classification accuracy 97.5% for supervised and clustering accuracy 96.0% for unsupervised models and 95.0% with a classic approach for a blinded dataset. We find that variability features significantly improve the accuracy of the trained models and correlation analysis among different bands enables a fast and inexpensive first order selection of quasar candidates △ Less

Submitted 8 July, 2023; originally announced July 2023.

Comments: Accepted by ApJ. 21 pages, 14 figures, 5 tables

Report number: FERMILAB-PUB-22-735-SCD

Deep learning of quasar lightcurves in the LSST era

Authors: Andjelka B. Kovacevic, Dragana Ilic, Luka C. Popovic, Nikola Andric Mitrovic, Mladen Nikolic, Marina S. Pavlovic, Iva Cvorovic Hajdinjak, Miljan Knezevic, Djordje V. Savic

Abstract: Deep learning techniques are required for the analysis of synoptic (multi-band and multi-epoch) light curves in massive data of quasars, as expected from the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST). In this follow-up study, we introduced an upgraded version of a conditional neural process (CNP) embedded in a multistep approach for analysis of large data of quasars in the L… ▽ More Deep learning techniques are required for the analysis of synoptic (multi-band and multi-epoch) light curves in massive data of quasars, as expected from the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST). In this follow-up study, we introduced an upgraded version of a conditional neural process (CNP) embedded in a multistep approach for analysis of large data of quasars in the LSST Active Galactic Nuclei Scientific Collaboration data challenge database. We present a case study of a stratified set of the u-band light curves for 283 quasars with very low variability $\sim 0.03$. In this sample, CNP average mean square error is found to be $\sim 5\% $($\sim 0.5$ mag). Interestingly, beside similar level of variability there are indications that individual light curves show flare like features. According to preliminary structure function analysis, these occurrences may be associated to microlensing events with larger time scales $5-10$ years. △ Less

Submitted 12 June, 2023; originally announced June 2023.

Comments: accepted for publication by the MDPI Universe Special Issue A Multimessenger View of Supermassive Black Holes and the Quasar Main Sequence, Eds. Ascension Del Olmo and Paola Marziani

Interpretable Medical Diagnostics with Structured Data Extraction by Large Language Models

Authors: Aleksa Bisercic, Mladen Nikolic, Mihaela van der Schaar, Boris Delibasic, Pietro Lio, Andrija Petrovic

Abstract: Tabular data is often hidden in text, particularly in medical diagnostic reports. Traditional machine learning (ML) models designed to work with tabular data, cannot effectively process information in such form. On the other hand, large language models (LLMs) which excel at textual tasks, are probably not the best tool for modeling tabular data. Therefore, we propose a novel, simple, and effective… ▽ More Tabular data is often hidden in text, particularly in medical diagnostic reports. Traditional machine learning (ML) models designed to work with tabular data, cannot effectively process information in such form. On the other hand, large language models (LLMs) which excel at textual tasks, are probably not the best tool for modeling tabular data. Therefore, we propose a novel, simple, and effective methodology for extracting structured tabular data from textual medical reports, called TEMED-LLM. Drawing upon the reasoning capabilities of LLMs, TEMED-LLM goes beyond traditional extraction techniques, accurately inferring tabular features, even when their names are not explicitly mentioned in the text. This is achieved by combining domain-specific reasoning guidelines with a proposed data validation and reasoning correction feedback loop. By applying interpretable ML models such as decision trees and logistic regression over the extracted and validated data, we obtain end-to-end interpretable predictions. We demonstrate that our approach significantly outperforms state-of-the-art text classification models in medical diagnostics. Given its predictive performance, simplicity, and interpretability, TEMED-LLM underscores the potential of leveraging LLMs to improve the performance and trustworthiness of ML models in medical applications. △ Less

Submitted 8 June, 2023; originally announced June 2023.

F-IVM: Analytics over Relational Databases under Updates

Authors: Ahmet Kara, Milos Nikolic, Dan Olteanu, Haozhe Zhang

Abstract: This article describes F-IVM, a unified approach for maintaining analytics over changing relational data. We exemplify its versatility in four disciplines: processing queries with group-by aggregates and joins; learning linear regression models using the covariance matrix of the input features; building Chow-Liu trees using pairwise mutual information of the input features; and matrix chain multip… ▽ More This article describes F-IVM, a unified approach for maintaining analytics over changing relational data. We exemplify its versatility in four disciplines: processing queries with group-by aggregates and joins; learning linear regression models using the covariance matrix of the input features; building Chow-Liu trees using pairwise mutual information of the input features; and matrix chain multiplication. F-IVM has three main ingredients: higher-order incremental view maintenance; factorized computation; and ring abstraction. F-IVM reduces the maintenance of a task to that of a hierarchy of simple views. Such views are functions map** keys, which are tuples of input values, to payloads, which are elements from a ring. F-IVM also supports efficient factorized computation over keys, payloads, and updates. Finally, F-IVM treats uniformly seemingly disparate tasks. In the key space, all tasks require joins and variable marginalization. In the payload space, tasks differ in the definition of the sum and product ring operations. We implemented F-IVM on top of DBToaster and show that it can outperform classical first-order and fully recursive higher-order incremental view maintenance by orders of magnitude while using less memory. △ Less

Submitted 29 January, 2024; v1 submitted 15 March, 2023; originally announced March 2023.

Strongly Interacting Dark Matter from $Sp(4)$ Gauge Theory

Authors: Fabian Zierler, Suchita Kulkarni, Axel Maas, Seán Mee, Marco Nikolic, Josef Pradler

Abstract: The stable hadronic bound states in a hidden new non-Abelian gauge sector provide interesting candidates for strongly-interacting Dark Matter (DM). A particular example are theories in which DM is made up of dark pions which set the DM relic abundance through self-annihilation. One of the simplest realizations is $Sp(4)_c$ gauge theory with two Dirac fermions. We discuss its mesonic multiplets for… ▽ More The stable hadronic bound states in a hidden new non-Abelian gauge sector provide interesting candidates for strongly-interacting Dark Matter (DM). A particular example are theories in which DM is made up of dark pions which set the DM relic abundance through self-annihilation. One of the simplest realizations is $Sp(4)_c$ gauge theory with two Dirac fermions. We discuss its mesonic multiplets for degenerate and non-degenerate fermions, construct a low-energy effective theory and present lattice results for the pseudoscalar mesons and vector mesons. △ Less

Submitted 21 November, 2022; originally announced November 2022.

Comments: 7 pages, 2 figures, Contribution to the proceedings of the 15th Quark Confinement and the Hadron Spectrum conference (ConfXV), 1st-6th August 2022, Stavanger, Norway

From Data to Software to Science with the Rubin Observatory LSST

Authors: Katelyn Breivik, Andrew J. Connolly, K. E. Saavik Ford, Mario Jurić, Rachel Mandelbaum, Adam A. Miller, Dara Norman, Knut Olsen, William O'Mullane, Adrian Price-Whelan, Timothy Sacco, J. L. Sokoloski, Ashley Villar, Viviana Acquaviva, Tomas Ahumada, Yusra AlSayyad, Catarina S. Alves, Igor Andreoni, Timo Anguita, Henry J. Best, Federica B. Bianco, Rosaria Bonito, Andrew Bradshaw, Colin J. Burke, Andresa Rodrigues de Campos , et al. (75 additional authors not shown)

Abstract: The Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) dataset will dramatically alter our understanding of the Universe, from the origins of the Solar System to the nature of dark matter and dark energy. Much of this research will depend on the existence of robust, tested, and scalable algorithms, software, and services. Identifying and develo** such tools ahead of time has the po… ▽ More The Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) dataset will dramatically alter our understanding of the Universe, from the origins of the Solar System to the nature of dark matter and dark energy. Much of this research will depend on the existence of robust, tested, and scalable algorithms, software, and services. Identifying and develo** such tools ahead of time has the potential to significantly accelerate the delivery of early science from LSST. Develo** these collaboratively, and making them broadly available, can enable more inclusive and equitable collaboration on LSST science. To facilitate such opportunities, a community workshop entitled "From Data to Software to Science with the Rubin Observatory LSST" was organized by the LSST Interdisciplinary Network for Collaboration and Computing (LINCC) and partners, and held at the Flatiron Institute in New York, March 28-30th 2022. The workshop included over 50 in-person attendees invited from over 300 applications. It identified seven key software areas of need: (i) scalable cross-matching and distributed joining of catalogs, (ii) robust photometric redshift determination, (iii) software for determination of selection functions, (iv) frameworks for scalable time-series analyses, (v) services for image access and reprocessing at scale, (vi) object image access (cutouts) and analysis at scale, and (vii) scalable job execution systems. This white paper summarizes the discussions of this workshop. It considers the motivating science use cases, identified cross-cutting algorithms, software, and services, their high-level technical specifications, and the principles of inclusive collaborations needed to develop them. We provide it as a useful roadmap of needs, as well as to spur action and collaboration between groups and individuals looking to develop reusable software for early LSST science. △ Less

Submitted 4 August, 2022; originally announced August 2022.

Comments: White paper from "From Data to Software to Science with the Rubin Observatory LSST" workshop

Conjunctive Queries with Free Access Patterns under Updates

Authors: Ahmet Kara, Milos Nikolic, Dan Olteanu, Haozhe Zhang

Abstract: We study the problem of answering conjunctive queries with free access patterns (CQAP) under updates. A free access pattern is a partition of the free variables of the query into input and output. The query returns tuples over the output variables given a tuple of values over the input variables. We introduce a fully dynamic evaluation approach for CQAP queries. We also give a syntactic characte… ▽ More We study the problem of answering conjunctive queries with free access patterns (CQAP) under updates. A free access pattern is a partition of the free variables of the query into input and output. The query returns tuples over the output variables given a tuple of values over the input variables. We introduce a fully dynamic evaluation approach for CQAP queries. We also give a syntactic characterisation of those CQAP queries that admit constant time per single-tuple update and whose output tuples can be enumerated with constant delay given a tuple of values over the input variables. Finally, we chart the complexity trade-off between the preprocessing time, update time and enumeration delay for CQAP queries. For a class of CQAP queries, our approach achieves optimal, albeit non-constant, update time and delay. Their optimality is predicated on the Online Matrix-Vector Multiplication conjecture. Our results recover prior work on the dynamic evaluation of conjunctive queries without access patterns. We also illustrate an application of our dynamic evaluation approach to tractable CQAP queries over probabilistic databases. △ Less

Submitted 14 February, 2024; v1 submitted 17 June, 2022; originally announced June 2022.

Comments: Extended and polished version. Added new Section 11 on the dynamic evaluation of conjunctive queries with free access patterns over probabilistic databases

ACM Class: H.2.4

Intrinsically motivated option learning: a comparative study of recent methods

Authors: Djordje Božić, Predrag Tadić, Mladen Nikolić

Abstract: Options represent a framework for reasoning across multiple time scales in reinforcement learning (RL). With the recent active interest in the unsupervised learning paradigm in the RL research community, the option framework was adapted to utilize the concept of empowerment, which corresponds to the amount of influence the agent has on the environment and its ability to perceive this influence, an… ▽ More Options represent a framework for reasoning across multiple time scales in reinforcement learning (RL). With the recent active interest in the unsupervised learning paradigm in the RL research community, the option framework was adapted to utilize the concept of empowerment, which corresponds to the amount of influence the agent has on the environment and its ability to perceive this influence, and which can be optimized without any supervision provided by the environment's reward structure. Many recent papers modify this concept in various ways achieving commendable results. Through these various modifications, however, the initial context of empowerment is often lost. In this work we offer a comparative study of such papers through the lens of the original empowerment principle. △ Less

Submitted 13 June, 2022; originally announced June 2022.

Comments: Accepted for presentation at the 2021 29th Telecommunications Forum (TELFOR), Belgrade, Serbia

Schrödinger's FP: Dynamic Adaptation of Floating-Point Containers for Deep Learning Training

Authors: Miloš Nikolić, Enrique Torres Sanchez, Jiahui Wang, Ali Hadi Zadeh, Mostafa Mahmoud, Ameer Abdelhadi, Kareem Ibrahim, Andreas Moshovos

Abstract: The transfer of tensors from/to memory during neural network training dominates time and energy. To improve energy efficiency and performance, research has been exploring ways to use narrower data representations. So far, these attempts relied on user-directed trial-and-error to achieve convergence. We present methods that relieve users from this responsibility. Our methods dynamically adjust the… ▽ More The transfer of tensors from/to memory during neural network training dominates time and energy. To improve energy efficiency and performance, research has been exploring ways to use narrower data representations. So far, these attempts relied on user-directed trial-and-error to achieve convergence. We present methods that relieve users from this responsibility. Our methods dynamically adjust the size and format of the floating-point containers used for activations and weights during training, achieving adaptivity across three dimensions: i) which datatype to use, ii) on which tensor, and iii) how it changes over time. The different meanings and distributions of exponent and mantissas lead us to tailored approaches for each. We present two lossy pairs of methods to eliminate as many mantissa and exponent bits as possible without affecting accuracy. Quantum Mantissa and Quantum Exponent are machine learning compression methods that tap into the gradient descent algorithm to learn the minimal mantissa and exponent bitlengths on a per-layer granularity. They automatically learn that many tensors can use just 1 or 2 mantissa bits and 3 or 4 exponent bits. Overall, the two machine learning methods reduce the footprint by $4.74\times$. Alternatively, BitWave observes changes in the loss function during training to adjust mantissa and exponent bitlengths network-wide, yielding a $3.19\times$ reduction in footprint. Finally, we present an optional method, Gecko, to exploit the naturally emerging, lop-sided exponent distribution to losslessly compress resulting exponents from Quantum Exponent or BitWave and, on average, improve compression rates to $5.64\times$ and $4.56\times$. △ Less

Submitted 16 May, 2024; v1 submitted 28 April, 2022; originally announced April 2022.

Low-energy effective description of dark $Sp(4)$ theories

Authors: Suchita Kulkarni, Axel Maas, Seán Mee, Marco Nikolic, Josef Pradler, Fabian Zierler

Abstract: Strongly interacting massive particles are viable dark matter candidates. We consider a dark $Sp(4)$ gauge theory with $N_f=2$ fermions in the pseudo-real fundamental representation and construct the chiral low-energy effective theory. We determine the flavor multiplet structure and the chiral Lagrangian, including the Wess-Zumino-Witten term for mass-degenerate and non-degenerate flavors. We then… ▽ More Strongly interacting massive particles are viable dark matter candidates. We consider a dark $Sp(4)$ gauge theory with $N_f=2$ fermions in the pseudo-real fundamental representation and construct the chiral low-energy effective theory. We determine the flavor multiplet structure and the chiral Lagrangian, including the Wess-Zumino-Witten term for mass-degenerate and non-degenerate flavors. We then study the possible charge assignments under a $U(1)'$ gauge symmetry, emphasizing on dark state stability, and provide the full Lagrangian description for Goldstone bosons and vector resonances, including the Wess-Zumino-Witten term. Finally, we use dedicated lattice simulations to determine the chiral low-energy effective theory's validity and low-energy constants. This work represents a self-consistent study of this non-Abelian theory. It thereby provides a framework for future phenomenological exploration in connection to the dark matter problem. △ Less

Submitted 15 September, 2022; v1 submitted 10 February, 2022; originally announced February 2022.

Comments: 60 pages, 12 figures; v2: added section 5.4 on eta' meson in the non-degenerate case, minor changes, added references

Journal ref: SciPost Phys. 14, 044 (2023)

Neural Algorithmic Reasoners are Implicit Planners

Authors: Andreea Deac, Petar Veličković, Ognjen Milinković, Pierre-Luc Bacon, Jian Tang, Mladen Nikolić

Abstract: Implicit planning has emerged as an elegant technique for combining learned models of the world with end-to-end model-free reinforcement learning. We study the class of implicit planners inspired by value iteration, an algorithm that is guaranteed to yield perfect policies in fully-specified tabular environments. We find that prior approaches either assume that the environment is provided in such… ▽ More Implicit planning has emerged as an elegant technique for combining learned models of the world with end-to-end model-free reinforcement learning. We study the class of implicit planners inspired by value iteration, an algorithm that is guaranteed to yield perfect policies in fully-specified tabular environments. We find that prior approaches either assume that the environment is provided in such a tabular form -- which is highly restrictive -- or infer "local neighbourhoods" of states to run value iteration over -- for which we discover an algorithmic bottleneck effect. This effect is caused by explicitly running the planning algorithm based on scalar predictions in every state, which can be harmful to data efficiency if such scalars are improperly predicted. We propose eXecuted Latent Value Iteration Networks (XLVINs), which alleviate the above limitations. Our method performs all planning computations in a high-dimensional latent space, breaking the algorithmic bottleneck. It maintains alignment with value iteration by carefully leveraging neural graph-algorithmic reasoning and contrastive self-supervised learning. Across eight low-data settings -- including classical control, navigation and Atari -- XLVINs provide significant improvements to data efficiency against value iteration-based implicit planners, as well as relevant model-free baselines. Lastly, we empirically verify that XLVINs can closely align with value iteration. △ Less

Submitted 11 October, 2021; originally announced October 2021.

Comments: To appear at NeurIPS 2021 (Spotlight talk). 20 pages, 10 figures. arXiv admin note: text overlap with arXiv:2010.13146

Machine Learning over Static and Dynamic Relational Data

Authors: Ahmet Kara, Milos Nikolic, Dan Olteanu, Haozhe Zhang

Abstract: This tutorial overviews principles behind recent works on training and maintaining machine learning models over relational data, with an emphasis on the exploitation of the relational data structure to improve the runtime performance of the learning task. The tutorial has the following parts: 1) Database research for data science 2) Three main ideas to achieve performance improvements 2.1)… ▽ More This tutorial overviews principles behind recent works on training and maintaining machine learning models over relational data, with an emphasis on the exploitation of the relational data structure to improve the runtime performance of the learning task. The tutorial has the following parts: 1) Database research for data science 2) Three main ideas to achieve performance improvements 2.1) Turn the ML problem into a DB problem 2.2) Exploit structure of the data and problem 2.3) Exploit engineering tools of a DB researcher 3) Avenues for future research △ Less

Submitted 29 July, 2021; originally announced July 2021.

Comments: arXiv admin note: text overlap with arXiv:2008.07864

Measuring scientific output of researchers by t-index and Data Envelopment Analysis

Authors: Dusan Teodorovic, Milos Nikolic

Abstract: There is a growing need for ranking universities, departments, research groups, and individual scholars. Usually, the scientific community measures the scientific merits of the researchers by using a variety of indicators that take into account both the productivity of scholars and the impact of their publications. We propose the t-index, the new indicator to measure the scientific merits of the i… ▽ More There is a growing need for ranking universities, departments, research groups, and individual scholars. Usually, the scientific community measures the scientific merits of the researchers by using a variety of indicators that take into account both the productivity of scholars and the impact of their publications. We propose the t-index, the new indicator to measure the scientific merits of the individual researchers. The proposed t-index takes into account the number of citations, number of coauthors on every published paper, and career duration. The t-index makes the possible comparison of researchers at various stages of their careers. We also use in this paper the Data Envelopment Analysis (DEA) to measure the scientific merits of the individual researchers within the observed group of researchers. We chose 15 scholars in the scientific area of transportation engineering and measured their t-index values, as well as DEA scores. △ Less

Submitted 12 July, 2021; originally announced July 2021.

Comments: 15 pages, 2 figures

User equilibrium traffic assignment: k paths subtracting-adding algorithm

Authors: Milos Nikolic, Dusan Teodorovic

Abstract: The traffic assignment problem is one of the most important transportation planning problems. The task faced by transportation planners, traffic engineers, and computer scientists is to generate high quality, approximate solutions of users equilibrium, that enable traffic scenario comparisons in a reasonable CPU time. We introduce the k Paths Subtracting-Adding (k-PSA) algorithm to approximate the… ▽ More The traffic assignment problem is one of the most important transportation planning problems. The task faced by transportation planners, traffic engineers, and computer scientists is to generate high quality, approximate solutions of users equilibrium, that enable traffic scenario comparisons in a reasonable CPU time. We introduce the k Paths Subtracting-Adding (k-PSA) algorithm to approximate the user equilibrium of the traffic assignment problem. The k-PSA algorithm consists of two alternating phases: (1) enlargement of the set of attractive paths; (2) subtracting-adding trips between generated attractive paths for each origin-destination pair of nodes. The proposed algorithm performs the two phases iteratively until the number of paths for each origin-destination pair is k. We tested the proposed algorithm on four benchmark transportation networks from the literature. The performed numerical tests show that the proposed approach generates, in short, computation times, solutions that are, on average, very close to the user equilibrium. △ Less

Submitted 7 July, 2021; originally announced July 2021.

Comments: 15 pages, 8 figures

Probing sub-eV Dark Matter decays with PTOLEMY

Authors: Kyrylo Bondarenko, Alexey Boyarsky, Marco Nikolic, Josef Pradler, Anastasia Sokolenko

Abstract: When the Dark Matter mass is below the eV-scale, its cosmological occupation number exceeds the ones of photons from the cosmic microwave background as well as of relic neutrinos. If such Dark Matter decays to pairs of neutrinos, it implies that experiments that seek the detection of the cosmic neutrino background may as well be sensitive to this additional form of "dark radiation". Here we study… ▽ More When the Dark Matter mass is below the eV-scale, its cosmological occupation number exceeds the ones of photons from the cosmic microwave background as well as of relic neutrinos. If such Dark Matter decays to pairs of neutrinos, it implies that experiments that seek the detection of the cosmic neutrino background may as well be sensitive to this additional form of "dark radiation". Here we study the prospects for detection taking into account various options for the forecasted performance of the future PTOLEMY experiment. From a detailed profile likelihood analysis we find that Dark Matter decays with lifetime as large as $10^4$ Gyr or a sub-% Dark Matter fraction decaying today can be discovered. The prospects are facilitated by the distinct spectral event shape that is introduced from galactic and cosmological neutrino dark radiation fluxes. In the process we also clarify the importance of Pauli-blocking in the Dark Matter decay. The scenarios presented in this work can be considered early physics targets in the development of these instruments with relaxed demands on performance and energy resolution. △ Less

Submitted 17 December, 2020; originally announced December 2020.

FAIR: Fair Adversarial Instance Re-weighting

Authors: Andrija Petrović, Mladen Nikolić, Sandro Radovanović, Boris Delibašić, Miloš Jovanović

Abstract: With growing awareness of societal impact of artificial intelligence, fairness has become an important aspect of machine learning algorithms. The issue is that human biases towards certain groups of population, defined by sensitive features like race and gender, are introduced to the training data through data collection and labeling. Two important directions of fairness ensuring research have foc… ▽ More With growing awareness of societal impact of artificial intelligence, fairness has become an important aspect of machine learning algorithms. The issue is that human biases towards certain groups of population, defined by sensitive features like race and gender, are introduced to the training data through data collection and labeling. Two important directions of fairness ensuring research have focused on (i) instance weighting in order to decrease the impact of more biased instances and (ii) adversarial training in order to construct data representations informative of the target variable, but uninformative of the sensitive attributes. In this paper we propose a Fair Adversarial Instance Re-weighting (FAIR) method, which uses adversarial training to learn instance weighting function that ensures fair predictions. Merging the two paradigms, it inherits desirable properties from both -- interpretability of reweighting and end-to-end trainability of adversarial training. We propose four different variants of the method and, among other things, demonstrate how the method can be cast in a fully probabilistic framework. Additionally, theoretical analysis of FAIR models' properties have been studied extensively. We compare FAIR models to 7 other related and state-of-the-art models and demonstrate that FAIR is able to achieve a better trade-off between accuracy and unfairness. To the best of our knowledge, this is the first model that merges reweighting and adversarial approaches by means of a weighting function that can provide interpretable information about fairness of individual instances. △ Less

Submitted 15 November, 2020; originally announced November 2020.

Scalable Querying of Nested Data

Authors: Jaclyn Smith, Michael Benedikt, Milos Nikolic, Amir Shaikhha

Abstract: While large-scale distributed data processing platforms have become an attractive target for query processing, these systems are problematic for applications that deal with nested collections. Programmers are forced either to perform non-trivial translations of collection programs or to employ automated flattening procedures, both of which lead to performance problems. These challenges only worsen… ▽ More While large-scale distributed data processing platforms have become an attractive target for query processing, these systems are problematic for applications that deal with nested collections. Programmers are forced either to perform non-trivial translations of collection programs or to employ automated flattening procedures, both of which lead to performance problems. These challenges only worsen for nested collections with skewed cardinalities, where both handcrafted rewriting and automated flattening are unable to enforce load balancing across partitions. In this work, we propose a framework that translates a program manipulating nested collections into a set of semantically equivalent shredded queries that can be efficiently evaluated. The framework employs a combination of query compilation techniques, an efficient data representation for nested collections, and automated skew-handling. We provide an extensive experimental evaluation, demonstrating significant improvements provided by the framework in diverse scenarios for nested collection programs. △ Less

Submitted 12 November, 2020; originally announced November 2020.

ACM Class: H.2.3; H.2.4

XLVIN: eXecuted Latent Value Iteration Nets

Authors: Andreea Deac, Petar Veličković, Ognjen Milinković, Pierre-Luc Bacon, Jian Tang, Mladen Nikolić

Abstract: Value Iteration Networks (VINs) have emerged as a popular method to incorporate planning algorithms within deep reinforcement learning, enabling performance improvements on tasks requiring long-range reasoning and understanding of environment dynamics. This came with several limitations, however: the model is not incentivised in any way to perform meaningful planning computations, the underlying s… ▽ More Value Iteration Networks (VINs) have emerged as a popular method to incorporate planning algorithms within deep reinforcement learning, enabling performance improvements on tasks requiring long-range reasoning and understanding of environment dynamics. This came with several limitations, however: the model is not incentivised in any way to perform meaningful planning computations, the underlying state space is assumed to be discrete, and the Markov decision process (MDP) is assumed fixed and known. We propose eXecuted Latent Value Iteration Networks (XLVINs), which combine recent developments across contrastive self-supervised learning, graph representation learning and neural algorithmic reasoning to alleviate all of the above limitations, successfully deploying VIN-style models on generic environments. XLVINs match the performance of VIN-like models when the underlying MDP is discrete, fixed and known, and provides significant improvements to model-free baselines across three general MDP setups. △ Less

Submitted 6 December, 2020; v1 submitted 25 October, 2020; originally announced October 2020.

Comments: NeurIPS 2020 Deep Reinforcement Learning Workshop

The neutrino-floor in the presence of dark radation

Authors: Marco Nikolic, Suchita Kulkarni, Josef Pradler

Abstract: In this work we analyse the ultimate sensitivity of dark matter direct detection experiments, the "neutrino-floor", in the presence of anomalous sources of dark radiation in form of SM or semi-sterile neutrinos. This flux-component is assumed to be produced from dark matter decay. Since dark radiation may mimic dark matter signals, we perform our analysis based on likelihood statistics that allows… ▽ More In this work we analyse the ultimate sensitivity of dark matter direct detection experiments, the "neutrino-floor", in the presence of anomalous sources of dark radiation in form of SM or semi-sterile neutrinos. This flux-component is assumed to be produced from dark matter decay. Since dark radiation may mimic dark matter signals, we perform our analysis based on likelihood statistics that allows to test the distinguishability between signals and backgrounds. We show that the neutrino floor for xenon-based experiments may be lifted in the presence of extra dark radiation. In addition, we explore the testability of neutrino dark radiation from dark matter decay in direct detection experiments. Given the previous bounds from neutrino experiments, we find that xenon-based dark matter searches will not be able to probe new regions of the dark matter progenitor mass and lifetime parameter space when the decay products are SM neutrinos. In turn, if the decay instead happens to a fourth neutrino species with enhanced interactions to baryons, DR can either constitute the dominant background or a discoverable signal in direct detection experiments. △ Less

Submitted 31 August, 2020; originally announced August 2020.

Comments: 15 pages, 9 figures

Hierarchical Protein Function Prediction with Tail-GNNs

Authors: Stefan Spalević, Petar Veličković, Jovana Kovačević, Mladen Nikolić

Abstract: Protein function prediction may be framed as predicting subgraphs (with certain closure properties) of a directed acyclic graph describing the hierarchy of protein functions. Graph neural networks (GNNs), with their built-in inductive bias for relational data, are hence naturally suited for this task. However, in contrast with most GNN applications, the graph is not related to the input, but to th… ▽ More Protein function prediction may be framed as predicting subgraphs (with certain closure properties) of a directed acyclic graph describing the hierarchy of protein functions. Graph neural networks (GNNs), with their built-in inductive bias for relational data, are hence naturally suited for this task. However, in contrast with most GNN applications, the graph is not related to the input, but to the label space. Accordingly, we propose Tail-GNNs, neural networks which naturally compose with the output space of any neural network for multi-task prediction, to provide relationally-reinforced labels. For protein function prediction, we combine a Tail-GNN with a dilated convolutional network which learns representations of the protein sequence, making significant improvement in F_1 score and demonstrating the ability of Tail-GNNs to learn useful representations of labels and exploit them in real-world problem solving. △ Less

Submitted 24 July, 2020; originally announced July 2020.

Journal ref: ICML 2020 - Workshops - Graph RepresGraph Representation Learning and Beyond (GRL+)

F-IVM: Learning over Fast-Evolving Relational Data

Authors: Milos Nikolic, Haozhe Zhang, Ahmet Kara, Dan Olteanu

Abstract: F-IVM is a system for real-time analytics such as machine learning applications over training datasets defined by queries over fast-evolving relational databases. We will demonstrate F-IVM for three such applications: model selection, Chow-Liu trees, and ridge linear regression. F-IVM is a system for real-time analytics such as machine learning applications over training datasets defined by queries over fast-evolving relational databases. We will demonstrate F-IVM for three such applications: model selection, Chow-Liu trees, and ridge linear regression. △ Less

Submitted 31 May, 2020; originally announced June 2020.

Comments: SIGMOD DEMO 2020, 5 pages

Maintaining Triangle Queries under Updates

Authors: Ahmet Kara, Milos Nikolic, Hung Q. Ngo, Dan Olteanu, Haozhe Zhang

Abstract: We consider the problem of incrementally maintaining the triangle queries with arbitrary free variables under single-tuple updates to the input relations. We introduce an approach called IVM$^ε$ that exhibits a trade-off between the update time, the space, and the delay for the enumeration of the query result, such that the update time ranges from the square root to linear in the database size whi… ▽ More We consider the problem of incrementally maintaining the triangle queries with arbitrary free variables under single-tuple updates to the input relations. We introduce an approach called IVM$^ε$ that exhibits a trade-off between the update time, the space, and the delay for the enumeration of the query result, such that the update time ranges from the square root to linear in the database size while the delay ranges from constant to linear time. IVM$^ε$ achieves Pareto worst-case optimality in the update-delay space conditioned on the Online Matrix-Vector Multiplication conjecture. It is strongly Pareto optimal for the triangle queries with zero or three free variables and weakly Pareto optimal for the triangle queries with one or two free variables. △ Less

Submitted 7 April, 2020; originally announced April 2020.

Comments: 47 pages, 18 figures

ACM Class: H.2.4

BitPruning: Learning Bitlengths for Aggressive and Accurate Quantization

Authors: Miloš Nikolić, Ghouthi Boukli Hacene, Ciaran Bannon, Alberto Delmas Lascorz, Matthieu Courbariaux, Yoshua Bengio, Vincent Gripon, Andreas Moshovos

Abstract: Neural networks have demonstrably achieved state-of-the art accuracy using low-bitlength integer quantization, yielding both execution time and energy benefits on existing hardware designs that support short bitlengths. However, the question of finding the minimum bitlength for a desired accuracy remains open. We introduce a training method for minimizing inference bitlength at any granularity whi… ▽ More Neural networks have demonstrably achieved state-of-the art accuracy using low-bitlength integer quantization, yielding both execution time and energy benefits on existing hardware designs that support short bitlengths. However, the question of finding the minimum bitlength for a desired accuracy remains open. We introduce a training method for minimizing inference bitlength at any granularity while maintaining accuracy. Namely, we propose a regularizer that penalizes large bitlength representations throughout the architecture and show how it can be modified to minimize other quantifiable criteria, such as number of operations or memory footprint. We demonstrate that our method learns thrifty representations while maintaining accuracy. With ImageNet, the method produces an average per layer bitlength of 4.13, 3.76 and 4.36 bits on AlexNet, ResNet18 and MobileNet V2 respectively, remaining within 2.0%, 0.5% and 0.5% of the base TOP-1 accuracy. △ Less

Submitted 11 August, 2020; v1 submitted 7 February, 2020; originally announced February 2020.

Trade-offs in Static and Dynamic Evaluation of Hierarchical Queries

Authors: Ahmet Kara, Milos Nikolic, Dan Olteanu, Haozhe Zhang

Abstract: We investigate trade-offs in static and dynamic evaluation of hierarchical queries with arbitrary free variables. In the static setting, the trade-off is between the time to partially compute the query result and the delay needed to enumerate its tuples. In the dynamic setting, we additionally consider the time needed to update the query result under single-tuple inserts or deletes to the database… ▽ More We investigate trade-offs in static and dynamic evaluation of hierarchical queries with arbitrary free variables. In the static setting, the trade-off is between the time to partially compute the query result and the delay needed to enumerate its tuples. In the dynamic setting, we additionally consider the time needed to update the query result under single-tuple inserts or deletes to the database. Our approach observes the degree of values in the database and uses different computation and maintenance strategies for high-degree (heavy) and low-degree (light) values. For the latter it partially computes the result, while for the former it computes enough information to allow for on-the-fly enumeration. We define the preprocessing time, the update time, and the enumeration delay as functions of the light/heavy threshold. By appropriately choosing this threshold, our approach recovers a number of prior results when restricted to hierarchical queries. We show that for a restricted class of hierarchical queries, our approach achieves worst-case optimal update time and enumeration delay conditioned on the Online Matrix-Vector Multiplication Conjecture. △ Less

Submitted 8 August, 2023; v1 submitted 3 July, 2019; originally announced July 2019.

Journal ref: Logical Methods in Computer Science, Volume 19, Issue 3 (August 9, 2023) lmcs:10035

MoËT: Mixture of Expert Trees and its Application to Verifiable Reinforcement Learning

Authors: Marko Vasic, Andrija Petrovic, Kaiyuan Wang, Mladen Nikolic, Rishabh Singh, Sarfraz Khurshid

Abstract: Rapid advancements in deep learning have led to many recent breakthroughs. While deep learning models achieve superior performance, often statistically better than humans, their adoption into safety-critical settings, such as healthcare or self-driving cars is hindered by their inability to provide safety guarantees or to expose the inner workings of the model in a human understandable form. We pr… ▽ More Rapid advancements in deep learning have led to many recent breakthroughs. While deep learning models achieve superior performance, often statistically better than humans, their adoption into safety-critical settings, such as healthcare or self-driving cars is hindered by their inability to provide safety guarantees or to expose the inner workings of the model in a human understandable form. We present MoËT, a novel model based on Mixture of Experts, consisting of decision tree experts and a generalized linear model gating function. Thanks to such gating function the model is more expressive than the standard decision tree. To support non-differentiable decision trees as experts, we formulate a novel training procedure. In addition, we introduce a hard thresholding version, MoËTH, in which predictions are made solely by a single expert chosen via the gating function. Thanks to that property, MoËTH allows each prediction to be easily decomposed into a set of logical rules in a form which can be easily verified. While MoËT is a general use model, we illustrate its power in the reinforcement learning setting. By training MoËT models using an imitation learning procedure on deep RL agents we outperform the previous state-of-the-art technique based on decision trees while preserving the verifiability of the models. Moreover, we show that MoËT can also be used in real-world supervised problems on which it outperforms other verifiable machine learning models. △ Less

Submitted 6 April, 2022; v1 submitted 16 June, 2019; originally announced June 2019.

Journal ref: Neural Networks, Volume 151, 2022, Pages 34-47

Gaussian Conditional Random Fields for Classification

Authors: Andrija Petrović, Mladen Nikolić, Miloš Jovanović, Boris Delibašić

Abstract: Gaussian conditional random fields (GCRF) are a well-known used structured model for continuous outputs that uses multiple unstructured predictors to form its features and at the same time exploits dependence structure among outputs, which is provided by a similarity measure. In this paper, a Gaussian conditional random fields model for structured binary classification (GCRFBC) is proposed. The mo… ▽ More Gaussian conditional random fields (GCRF) are a well-known used structured model for continuous outputs that uses multiple unstructured predictors to form its features and at the same time exploits dependence structure among outputs, which is provided by a similarity measure. In this paper, a Gaussian conditional random fields model for structured binary classification (GCRFBC) is proposed. The model is applicable to classification problems with undirected graphs, intractable for standard classification CRFs. The model representation of GCRFBC is extended by latent variables which yield some appealing properties. Thanks to the GCRF latent structure, the model becomes tractable, efficient and open to improvements previously applied to GCRF regression models. In addition, the model allows for reduction of noise, that might appear if structures were defined directly between discrete outputs. Additionally, two different forms of the algorithm are presented: GCRFBCb (GCRGBC - Bayesian) and GCRFBCnb (GCRFBC - non Bayesian). The extended method of local variational approximation of sigmoid function is used for solving empirical Bayes in Bayesian GCRFBCb variant, whereas MAP value of latent variables is the basis for learning and inference in the GCRFBCnb variant. The inference in GCRFBCb is solved by Newton-Cotes formulas for one-dimensional integration. Both models are evaluated on synthetic data and real-world data. It was shown that both models achieve better prediction performance than unstructured predictors. Furthermore, computational and memory complexity is evaluated. Advantages and disadvantages of the proposed GCRFBCb and GCRFBCnb are discussed in detail. △ Less

Submitted 31 January, 2019; originally announced February 2019.

Comments: Draft paper without experimental evaluation

Laconic Deep Learning Computing

Authors: Sayeh Sharify, Mostafa Mahmoud, Alberto Delmas Lascorz, Milos Nikolic, Andreas Moshovos

Abstract: We motivate a method for transparently identifying ineffectual computations in unmodified Deep Learning models and without affecting accuracy. Specifically, we show that if we decompose multiplications down to the bit level the amount of work performed during inference for image classification models can be consistently reduced by two orders of magnitude. In the best case studied of a sparse varia… ▽ More We motivate a method for transparently identifying ineffectual computations in unmodified Deep Learning models and without affecting accuracy. Specifically, we show that if we decompose multiplications down to the bit level the amount of work performed during inference for image classification models can be consistently reduced by two orders of magnitude. In the best case studied of a sparse variant of AlexNet, this approach can ideally reduce computation work by more than 500x. We present Laconic a hardware accelerator that implements this approach to improve execution time, and energy efficiency for inference with Deep Learning Networks. Laconic judiciously gives up some of the work reduction potential to yield a low-cost, simple, and energy efficient design that outperforms other state-of-the-art accelerators. For example, a Laconic configuration that uses a weight memory interface with just 128 wires outperforms a conventional accelerator with a 2K-wire weight memory interface by 2.3x on average while being 2.13x more energy efficient on average. A Laconic configuration that uses a 1K-wire weight memory interface, outperforms the 2K-wire conventional accelerator by 15.4x and is 1.95x more energy efficient. Laconic does not require but rewards advances in model design such as a reduction in precision, the use of alternate numeric representations that reduce the number of bits that are "1", or an increase in weight or activation sparsity. △ Less

Submitted 10 May, 2018; originally announced May 2018.

DPRed: Making Typical Activation and Weight Values Matter In Deep Learning Computing

Authors: Alberto Delmas, Sayeh Sharify, Patrick Judd, Kevin Siu, Milos Nikolic, Andreas Moshovos

Abstract: We show that selecting a single data type (precision) for all values in Deep Neural Networks, even if that data type is different per layer, amounts to worst case design. Much shorter data types can be used if we target the common case by adjusting the precision at a much finer granularity. We propose Dynamic Precision Reduction (DPRed), where we group weights and activations and encode them using… ▽ More We show that selecting a single data type (precision) for all values in Deep Neural Networks, even if that data type is different per layer, amounts to worst case design. Much shorter data types can be used if we target the common case by adjusting the precision at a much finer granularity. We propose Dynamic Precision Reduction (DPRed), where we group weights and activations and encode them using a precision specific to each group. The per group precisions are selected statically for the weights and dynamically by hardware for the activations. We exploit these precisions to reduce: 1) off-chip storage and off- and on-chip communication, and 2) execution time. DPRed compression reduces off-chip traffic to nearly 35% and 33% on average compared to no compression respectively for 16b and 8b models. This makes it possible to sustain higher performance for a given off-chip memory interface while also boosting energy efficiency. We also demonstrate designs where the time required to process each group of activations and/or weights scales proportionally to the precision they use for convolutional and fully-connected layers. This improves execution time and energy efficiency for both dense and sparse networks. We show the techniques work with 8-bit networks, where 1.82x and 2.81x speedups are achieved for two different hardware variants that take advantage of dynamic precision variability. △ Less

Submitted 17 December, 2018; v1 submitted 16 April, 2018; originally announced April 2018.

Counting Triangles under Updates in Worst-Case Optimal Time

Authors: Ahmet Kara, Hung Q. Ngo, Milos Nikolic, Dan Olteanu, Haozhe Zhang

Abstract: We consider the problem of incrementally maintaining the triangle count query under single-tuple updates to the input relations. We introduce an approach that exhibits a space-time tradeoff such that the space-time product is quadratic in the size of the input database and the update time can be as low as the square root of this size. This lowest update time is worst-case optimal conditioned on th… ▽ More We consider the problem of incrementally maintaining the triangle count query under single-tuple updates to the input relations. We introduce an approach that exhibits a space-time tradeoff such that the space-time product is quadratic in the size of the input database and the update time can be as low as the square root of this size. This lowest update time is worst-case optimal conditioned on the Online Matrix-Vector Multiplication conjecture. The classical and factorized incremental view maintenance approaches are recovered as special cases of our approach within the space-time tradeoff. In particular, they require linear-time update maintenance, which is suboptimal. Our approach also recovers the worst-case optimal time complexity for computing the triangle count in the non-incremental setting. △ Less

Submitted 25 March, 2019; v1 submitted 8 April, 2018; originally announced April 2018.

Comments: simplified notation; incremental maintenance of full triangle query, 4-path count query, count queries with three relations added; improved the space complexity of the dynamic algorithm maintaining the triangle count query

ACM Class: H.2.4

Bit-Tactical: Exploiting Ineffectual Computations in Convolutional Neural Networks: Which, Why, and How

Authors: Alberto Delmas, Patrick Judd, Dylan Malone Stuart, Zissis Poulos, Mostafa Mahmoud, Sayeh Sharify, Milos Nikolic, Andreas Moshovos

Abstract: We show that, during inference with Convolutional Neural Networks (CNNs), more than 2x to $8x ineffectual work can be exposed if instead of targeting those weights and activations that are zero, we target different combinations of value stream properties. We demonstrate a practical application with Bit-Tactical (TCL), a hardware accelerator which exploits weight sparsity, per layer precision varia… ▽ More We show that, during inference with Convolutional Neural Networks (CNNs), more than 2x to $8x ineffectual work can be exposed if instead of targeting those weights and activations that are zero, we target different combinations of value stream properties. We demonstrate a practical application with Bit-Tactical (TCL), a hardware accelerator which exploits weight sparsity, per layer precision variability and dynamic fine-grain precision reduction for activations, and optionally the naturally occurring sparse effectual bit content of activations to improve performance and energy efficiency. TCL benefits both sparse and dense CNNs, natively supports both convolutional and fully-connected layers, and exploits properties of all activations to reduce storage, communication, and computation demands. While TCL does not require changes to the CNN to deliver benefits, it does reward any technique that would amplify any of the aforementioned weight and activation value properties. Compared to an equivalent data-parallel accelerator for dense CNNs, TCLp, a variant of TCL improves performance by 5.05x and is 2.98x more energy efficient while requiring 22% more area. △ Less

Submitted 9 March, 2018; originally announced March 2018.

Comments: An earlier version of this work titled "JaZ: Enabling Innovation Towards Chaff-Free Deep Learning Computing" was submitted for blind review

Incremental View Maintenance with Triple Lock Factorization Benefits

Authors: Milos Nikolic, Dan Olteanu

Abstract: We introduce F-IVM, a unified incremental view maintenance (IVM) approach for a variety of tasks, including gradient computation for learning linear regression models over joins, matrix chain multiplication, and factorized evaluation of conjunctive queries. F-IVM is a higher-order IVM algorithm that reduces the maintenance of the given task to the maintenance of a hierarchy of increasingly simpl… ▽ More We introduce F-IVM, a unified incremental view maintenance (IVM) approach for a variety of tasks, including gradient computation for learning linear regression models over joins, matrix chain multiplication, and factorized evaluation of conjunctive queries. F-IVM is a higher-order IVM algorithm that reduces the maintenance of the given task to the maintenance of a hierarchy of increasingly simpler views. The views are functions map** keys, which are tuples of input data values, to payloads, which are elements from a task-specific ring. Whereas the computation over the keys is the same for all tasks, the computation over the payloads depends on the task. F-IVM achieves efficiency by factorizing the computation of the keys, payloads, and updates. We implemented F-IVM as an extension of DBToaster. We show in a range of scenarios that it can outperform classical first-order IVM, DBToaster's fully recursive higher-order IVM, and plain recomputation by orders of magnitude while using less memory. △ Less

Submitted 28 February, 2018; v1 submitted 21 March, 2017; originally announced March 2017.

Comments: 27 pages, 13 figures, a shorter version appeared in SIGMOD 2018

ACM Class: H.2.4

Measurement of non-monotonic Casimir forces between silicon nanostructures

Authors: L. Tang, M. Wang, C. Y. Ng, M. Nikolic, C. T. Chan, A. W. Rodriguez, H. B. Chan

Abstract: Casimir forces are of fundamental interest because they originate from quantum fluctuations of the electromagnetic field. Apart from controlling the Casimir force via the optical properties of the materials, a number of novel geometries have been proposed to generate repulsive and/or non-monotonic Casimir forces between bodies separated by vacuum gaps. Experimental realization of these geometries,… ▽ More Casimir forces are of fundamental interest because they originate from quantum fluctuations of the electromagnetic field. Apart from controlling the Casimir force via the optical properties of the materials, a number of novel geometries have been proposed to generate repulsive and/or non-monotonic Casimir forces between bodies separated by vacuum gaps. Experimental realization of these geometries, however, is hindered by the difficulties in alignment when the bodies are brought into close proximity. Here, using an on-chip platform with integrated force sensors and actuators, we circumvent the alignment problem and measure the Casimir force between two surfaces with nanoscale protrusions. We demonstrate that the Casimir force depends non-monotonically on the displacement. At some displacements, the Casimir force leads to an effective stiffening of the nanomechanical spring. Our findings pave the way for exploiting the Casimir force in nanomechanical systems using structures of complex and non-conventional shapes. △ Less

Submitted 9 January, 2017; originally announced January 2017.

Comments: 26 pages, 13 figures

Journal ref: Nature Photonics (2017)

Increased Mobility in Presence of Multiple Contacts - Identifying Contact Configurations that Enable Arbitrary Acceleration of CoM

Authors: Milutin Nikolić, Branislav Borovac, Mirko Raković, Milica Žigić

Abstract: Planning of any motion starts by planning the trajectory of the CoM. It is of the highest importance to ensure that the robot will be able to perform planned trajectory. With increasing capabilities of the humanoid robots, the case when contacts are spatially distributed should be considered. In this paper, it is shown that there are some contact configurations in which any acceleration of the cen… ▽ More Planning of any motion starts by planning the trajectory of the CoM. It is of the highest importance to ensure that the robot will be able to perform planned trajectory. With increasing capabilities of the humanoid robots, the case when contacts are spatially distributed should be considered. In this paper, it is shown that there are some contact configurations in which any acceleration of the center of mass (CoM) is feasible. The procedure for identifying such a configurations is presented, as well as its physical meaning. On the other hand, for the configurations in which the constraint on CoM movement exists, it will be shown how to find that linear constraint, which defines the space of feasible motion. The proposed algorithm has a low complexity and to speed up the procedure even further, it will be shown that the whole procedure needs to be run only once when contact configuration changes. As the CoM moves, the new constraints can be calculated from the initial one, thus yielding significant computation speedup. The methods are illustrated in two simulated scenarios. △ Less

Submitted 8 August, 2016; v1 submitted 5 August, 2016; originally announced August 2016.

Comments: 6 pages, 5 figures, 1 table

A simple example of pressure excitation:the 3D square well potential

Authors: V. Celebonovic, M. G. Nikolic

Abstract: High pressure experiments in diamond anvil cells as well as in geophysics show that at certain discrete values of pressure abrupt changes of the mass density and changes of the material structure occur. The aim of this paper is to discuss the influence of high external pressure on atoms and molecules using the $3D$ square well potential as a simple solvable quantum mechanical example. Values of ex… ▽ More High pressure experiments in diamond anvil cells as well as in geophysics show that at certain discrete values of pressure abrupt changes of the mass density and changes of the material structure occur. The aim of this paper is to discuss the influence of high external pressure on atoms and molecules using the $3D$ square well potential as a simple solvable quantum mechanical example. Values of external pressure needed for a transition between energy levels with different values of the main quantum number $n$, as well as the pressure needed to expel the particle from this system are calculated. In both cases discrete values of pressure are obtained. △ Less

Submitted 27 May, 2016; originally announced June 2016.

Comments: 7 pages, plain LaTeX

Short Portfolio Training for CSP Solving

Authors: Mirko Stojadinović, Mladen Nikolić, Filip Marić

Abstract: Many different approaches for solving Constraint Satisfaction Problems (CSPs) and related Constraint Optimization Problems (COPs) exist. However, there is no single solver (nor approach) that performs well on all classes of problems and many portfolio approaches for selecting a suitable solver based on simple syntactic features of the input CSP instance have been developed. In this paper we first… ▽ More Many different approaches for solving Constraint Satisfaction Problems (CSPs) and related Constraint Optimization Problems (COPs) exist. However, there is no single solver (nor approach) that performs well on all classes of problems and many portfolio approaches for selecting a suitable solver based on simple syntactic features of the input CSP instance have been developed. In this paper we first present a simple portfolio method for CSP based on k-nearest neighbors method. Then, we propose a new way of using portfolio systems --- training them shortly in the exploitation time, specifically for the set of instances to be solved and using them on that set. Thorough evaluation has been performed and has shown that the approach yields good results. We evaluated several machine learning techniques for our portfolio. Due to its simplicity and efficiency, the selected k-nearest neighbors method is especially suited for our short training approach and it also yields the best results among the tested methods. We also confirm that our approach yields good results on SAT domain. △ Less

Submitted 8 May, 2015; originally announced May 2015.

Comments: 21 pages

LINVIEW: Incremental View Maintenance for Complex Analytical Queries

Authors: Milos Nikolic, Mohammed ElSeidy, Christoph Koch

Abstract: Many analytics tasks and machine learning problems can be naturally expressed by iterative linear algebra programs. In this paper, we study the incremental view maintenance problem for such complex analytical queries. We develop a framework, called LINVIEW, for capturing deltas of linear algebra programs and understanding their computational cost. Linear algebra operations tend to cause an avalanc… ▽ More Many analytics tasks and machine learning problems can be naturally expressed by iterative linear algebra programs. In this paper, we study the incremental view maintenance problem for such complex analytical queries. We develop a framework, called LINVIEW, for capturing deltas of linear algebra programs and understanding their computational cost. Linear algebra operations tend to cause an avalanche effect where even very local changes to the input matrices spread out and infect all of the intermediate results and the final view, causing incremental view maintenance to lose its performance benefit over re-evaluation. We develop techniques based on matrix factorizations to contain such epidemics of change. As a consequence, our techniques make incremental view maintenance of linear algebra practical and usually substantially cheaper than re-evaluation. We show, both analytically and experimentally, the usefulness of these techniques when applied to standard analytics tasks. Our evaluation demonstrates the efficiency of LINVIEW in generating parallel incremental programs that outperform re-evaluation techniques by more than an order of magnitude. △ Less

Submitted 9 May, 2014; v1 submitted 27 March, 2014; originally announced March 2014.

Comments: 14 pages, SIGMOD

ACM Class: H.2.4; G.1.3

DBToaster: Higher-order Delta Processing for Dynamic, Frequently Fresh Views

Authors: Yanif Ahmad, Oliver Kennedy, Christoph Koch, Milos Nikolic

Abstract: Applications ranging from algorithmic trading to scientific data analysis require realtime analytics based on views over databases that change at very high rates. Such views have to be kept fresh at low maintenance cost and latencies. At the same time, these views have to support classical SQL, rather than window semantics, to enable applications that combine current with aged or historical data.… ▽ More Applications ranging from algorithmic trading to scientific data analysis require realtime analytics based on views over databases that change at very high rates. Such views have to be kept fresh at low maintenance cost and latencies. At the same time, these views have to support classical SQL, rather than window semantics, to enable applications that combine current with aged or historical data. In this paper, we present viewlet transforms, a recursive finite differencing technique applied to queries. The viewlet transform materializes a query and a set of its higher-order deltas as views. These views support each other's incremental maintenance, leading to a reduced overall view maintenance cost. The viewlet transform of a query admits efficient evaluation, the elimination of certain expensive query operations, and aggressive parallelization. We develop viewlet transforms into a workable query execution technique, present a heuristic and cost-based optimization framework, and report on experiments with a prototype dynamic data management system that combines viewlet transforms with an optimizing compilation technique. The system supports tens of thousands of complete view refreshes a second for a wide range of queries. △ Less

Submitted 30 June, 2012; originally announced July 2012.

Comments: VLDB2012

Journal ref: Proceedings of the VLDB Endowment (PVLDB), Vol. 5, No. 10, pp. 968-979 (2012)

Software Verification and Graph Similarity for Automated Evaluation of Students' Assignments

Authors: Milena Vujosevic-Janicic, Mladen Nikolic, Dusan Tosic, Viktor Kuncak

Abstract: In this paper we promote introducing software verification and control flow graph similarity measurement in automated evaluation of students' programs. We present a new grading framework that merges results obtained by combination of these two approaches with results obtained by automated testing, leading to improved quality and precision of automated grading. These two approaches are also useful… ▽ More In this paper we promote introducing software verification and control flow graph similarity measurement in automated evaluation of students' programs. We present a new grading framework that merges results obtained by combination of these two approaches with results obtained by automated testing, leading to improved quality and precision of automated grading. These two approaches are also useful in providing a comprehensible feedback that can help students to improve the quality of their programs We also present our corresponding tools that are publicly available and open source. The tools are based on LLVM low-level intermediate code representation, so they could be applied to a number of programming languages. Experimental evaluation of the proposed grading framework is performed on a corpus of university students' programs written in programming language C. Results of the experiments show that automatically generated grades are highly correlated with manually determined grades suggesting that the presented tools can find real-world applications in studying and grading. △ Less

Submitted 29 June, 2012; originally announced June 2012.

Automatic Methods for Handling Nearly Singular Covariance Structures Using the Cholesky Decomposition of an Indefinite Matrix

Authors: John R. Smith, Milan Nikolic, Stephen P. Smith

Abstract: Linear models have found widespread use in statistical investigations. For every linear model there exists a matrix representation for which the ReML (Restricted Maximum Likelihood) can be constructed from the elements of the corresponding matrix. This method works in the standard manner when the covariance structure is non-singular. It can also be used in the case where the covariance structure i… ▽ More Linear models have found widespread use in statistical investigations. For every linear model there exists a matrix representation for which the ReML (Restricted Maximum Likelihood) can be constructed from the elements of the corresponding matrix. This method works in the standard manner when the covariance structure is non-singular. It can also be used in the case where the covariance structure is singular, because the method identifies particular non-stochastic linear combinations of the observations which must be constrained to zero. In order to use this method, the Cholesky decomposition has to be generalized to symmetric and indefinite matrices using complex arithmetic methods. This method is applied to the problem of determining the spatial size (vertex) for the Higgs Boson decay in the Higgs -> 4 lepton channel. A comparison based on the Chi-Square variable from the vertex fit for Higgs signal and t-tbar background is presented and shows that the background can be greatly suppressed using the Chi-Square variable. One of the major advantages of this novel method over the currently adopted technique of b-tagging is that it is not affected by multiple interactions (pile up). △ Less

Submitted 29 July, 2013; v1 submitted 9 January, 2012; originally announced January 2012.

Comments: 30 pages, 3 figures & 3 tables

MSC Class: 65Zxx ACM Class: G.3; G.4

Simple Algorithm Portfolio for SAT

Authors: Mladen Nikolic, Filip Maric, Predrag Janicic

Abstract: The importance of algorithm portfolio techniques for SAT has long been noted, and a number of very successful systems have been devised, including the most successful one --- SATzilla. However, all these systems are quite complex (to understand, reimplement, or modify). In this paper we propose a new algorithm portfolio for SAT that is extremely simple, but in the same time so efficient that it ou… ▽ More The importance of algorithm portfolio techniques for SAT has long been noted, and a number of very successful systems have been devised, including the most successful one --- SATzilla. However, all these systems are quite complex (to understand, reimplement, or modify). In this paper we propose a new algorithm portfolio for SAT that is extremely simple, but in the same time so efficient that it outperforms SATzilla. For a new SAT instance to be solved, our portfolio finds its k-nearest neighbors from the training set and invokes a solver that performs the best at those instances. The main distinguishing feature of our algorithm portfolio is the locality of the selection procedure --- the selection of a SAT solver is based only on few instances similar to the input one. △ Less

Submitted 13 December, 2011; v1 submitted 1 July, 2011; originally announced July 2011.