Showing 51–100 of 124 results for author: Shah, B

High-performance symbolic-numerics via multiple dispatch

Authors: Shashi Gowda, Yingbo Ma, Alessandro Cheli, Maja Gwozdz, Viral B. Shah, Alan Edelman, Christopher Rackauckas

Abstract: As mathematical computing becomes more democratized in high-level languages, high-performance symbolic-numeric systems are necessary for domain scientists and engineers to get the best performance out of their machine without deep knowledge of code optimization. Naturally, users need different term types either to have different algebraic properties for them, or to use efficient data structures. T… ▽ More As mathematical computing becomes more democratized in high-level languages, high-performance symbolic-numeric systems are necessary for domain scientists and engineers to get the best performance out of their machine without deep knowledge of code optimization. Naturally, users need different term types either to have different algebraic properties for them, or to use efficient data structures. To this end, we developed Symbolics.jl, an extendable symbolic system which uses dynamic multiple dispatch to change behavior depending on the domain needs. In this work we detail an underlying abstract term interface which allows for speed without sacrificing generality. We show that by formalizing a generic API on actions independent of implementation, we can retroactively add optimized data structures to our system without changing the pre-existing term rewriters. We showcase how this can be used to optimize term construction and give a 113x acceleration on general symbolic transformations. Further, we show that such a generic API allows for complementary term-rewriting implementations. We demonstrate the ability to swap between classical term-rewriting simplifiers and e-graph-based term-rewriting simplifiers. We showcase an e-graph ruleset which minimizes the number of CPU cycles during expression evaluation, and demonstrate how it simplifies a real-world reaction-network simulation to halve the runtime. Additionally, we show a reaction-diffusion partial differential equation solver which is able to be automatically converted into symbolic expressions via multiple dispatch tracing, which is subsequently accelerated and parallelized to give a 157x simulation speedup. Together, this presents Symbolics.jl as a next-generation symbolic-numeric computing environment geared towards modeling and simulation. △ Less

Submitted 5 February, 2022; v1 submitted 9 May, 2021; originally announced May 2021.

ACM Class: D.3.3; I.1.1; I.1.3

Mechanical aspects of Near-Infrared Imager Spectrometer and Polarimeter

Authors: Prashanth Kumar Kasarla, Pitamber Singh Patwal, Hitesh Kumar L. Adalja, Satya Narain Mathur, Deekshya Roy Sarkar, Alka Singh, Archita Rai, Prachi Vinod Prajapati, Sachindra Naik, Amish B. Shah, Shashikiran Ganesh, Kiran S. Baliyan

Abstract: Near-infrared Imager Spectrometer and Polarimeter (NISP) is a camera, an intermediate resolution spectrograph and an imaging polarimeter being developed for upcoming 2.5m telescope of Physical Research Laboratory at Mount Abu, India. NISP is designed to work in the Near-IR (0.8-2.5 micron) using a H2RG detector. Collimator and camera lenses would transfer the image from the focal plane of the tele… ▽ More Near-infrared Imager Spectrometer and Polarimeter (NISP) is a camera, an intermediate resolution spectrograph and an imaging polarimeter being developed for upcoming 2.5m telescope of Physical Research Laboratory at Mount Abu, India. NISP is designed to work in the Near-IR (0.8-2.5 micron) using a H2RG detector. Collimator and camera lenses would transfer the image from the focal plane of the telescope to the detector plane. The entire optics, mechanical support structures, detector-SIDECAR assembly will be cooled to cryo-temperatures using an open cycle Liquid Nitrogen tank inside a vacuum Dewar. GFRP support structures would be used to isolate cryogenic system from the Dewar. Two layer thermal shielding would be used to reduce the radiative heat transfer. Molecular sieve (getter) would be used to enhance the vacuum level inside Dewar. Magnet-reedswitch combination are used for absolute positioning of filterwheels. Here we describe the mechanical aspects in detail. △ Less

Submitted 16 December, 2020; originally announced December 2020.

Comments: 11 pages, 12 figures, Submitted to SPIE Conference Astronomical Telescopes + Instrumentation 2020

Journal ref: Proc. SPIE 11447, Ground-based and Airborne Instrumentation for Astronomy VIII, 114476U, 2020

Electronics design and development of Near-Infrared Imager, Spectrometer and Polarimeter

Authors: Deekshya Roy Sarkar, Amish B. Shah, Alka Singh, Pitamber Singh Patwal, Prashanth Kumar Kasarla, Archita Rai, Prachi Vinod Prajapati, Hitesh Kumar L. Adalja, Satya N. Mathur, Sachindra Naik, Shashikiran Ganesh, Kiran S. Baliyan

Abstract: NISP, a multifaceted near-infrared instrument for the upcoming 2.5m IR telescope at MIRO Gurushikhar, Mount Abu, Rajasthan, India is being developed at PRL, Ahmedabad. NISP will have wide (FOV = 10' x 10') field imaging, moderate (R=3000) spectroscopy and imaging polarimetry operating modes. It is designed based on 0.8 to 2.5 micron sensitive, 2048 X 2048 HgCdTe (MCT) array detector from Teledyne.… ▽ More NISP, a multifaceted near-infrared instrument for the upcoming 2.5m IR telescope at MIRO Gurushikhar, Mount Abu, Rajasthan, India is being developed at PRL, Ahmedabad. NISP will have wide (FOV = 10' x 10') field imaging, moderate (R=3000) spectroscopy and imaging polarimetry operating modes. It is designed based on 0.8 to 2.5 micron sensitive, 2048 X 2048 HgCdTe (MCT) array detector from Teledyne. Optical, Mechanical and Electronics subsystems are being designed and developed in-house at PRL. HAWAII-2RG (H2RG) detector will be mounted along with controlling SIDECAR ASIC inside LN2 filled cryogenic cooled Dewar. FPGA based controller for H2RG and ASIC will be mounted outside the Dewar at room temperature. Smart stepper motors will facilitate motion of filter wheels and optical components to realize different operating modes. Detector and ASIC temperatures are servo controlled using Lakeshore's Temperature Controller (TC) 336. Also, several cryogenic temperatures will be monitored by TC for health checking of the instrument. Detector, Motion and Temperature controllers onboard telescope will be interfaced to USB Hub and fiber-optic trans-receiver. Remote Host computer interface to remote end trans-receiver will be equipped with in-house developed GUI software to control all functionalities of NISP. Design and development aspects of NISP Electronics will be presented in this conference. △ Less

Submitted 16 December, 2020; originally announced December 2020.

Comments: 6 pages, 3 figures, Submitted to SPIE Conference Astronomical Telescopes + Instrumentation 2020

Journal ref: Proc. SPIE 11447, Ground-based and Airborne Instrumentation for Astronomy VIII, 1144778, 2020

EMPOL: an EMCCD based optical imaging polarimeter

Authors: Shashikiran Ganesh, Archita Rai, Aravind K, Alka Singh, Prachi Vinod Prajapati, Ashish Mishra, Prashanth Kasarla, Deekshya Roy Sarkar, Pitamber Singh Patwal, Namita Uppal, Sunil Chandra, Satyanarain Mathur, Amish B Shah, Kiran S Baliyan, U C Joshi

Abstract: An Andor 1K $\times$ 1K EMCCD detector has been used to develop an optical imaging polarimeter for use at the Cassegrain focus of 1.2 m telescope of PRL. The optics is derived from an older single-element detector instrument and consists of a rotating half-wave plate as modulator and a Foster prism as an analyser. The field of view of the instrument is 3 $\times$ 3 sq arcmin. We describe the instr… ▽ More An Andor 1K $\times$ 1K EMCCD detector has been used to develop an optical imaging polarimeter for use at the Cassegrain focus of 1.2 m telescope of PRL. The optics is derived from an older single-element detector instrument and consists of a rotating half-wave plate as modulator and a Foster prism as an analyser. The field of view of the instrument is 3 $\times$ 3 sq arcmin. We describe the instrument and the observational methodology in this document. Extensive observations have been carried out with this instrument covering a large variety of sources e.g. near-Earth asteroids, comets, Lynds dark nebulae, open clusters and AGN such as blazars. In the current communication, we discuss some results from the initial calibration runs while the other results will be presented elsewhere. △ Less

Submitted 16 December, 2020; originally announced December 2020.

Comments: 7 pages, 6 figures, Submitted to SPIE Conference Astronomical Telescopes + Instrumentation 2020

Journal ref: Proc. SPIE 11447, Ground-based and Airborne Instrumentation for Astronomy VIII, 114479E, 2020

Optical aspects of Near-Infrared Imager Spectrometer and Polarimeter instrument (NISP)

Authors: Archita Rai, Shashikiran Ganesh, Sukamal K. Paul, Prashanth K. Kasarla, Prachi V. Prajapati, Deekshya R. Sarkar, Alka Singh, Pitambar S. Patwal, Hitesh K L Adalja, Satya N. Mathur, Sachindra Naik, Amish B. Shah, Kiran S. Baliyan

Abstract: As a Near-IR instrument to PRL's upcoming 2.5 m telescope, NISP is designed indigeniously at PRL to serve as a multifaceted instrument. Optical, Mechanical and Electronics subsystems are being designed and developed in-house at PRL. It will consist of imaging, spectroscopy and imaging-polarimetry mode in the wavelength bands Y, J, H, Ks i.e. 0.8 - 2.5 micron. The detector is an 2K x 2K H2RG (MCT)… ▽ More As a Near-IR instrument to PRL's upcoming 2.5 m telescope, NISP is designed indigeniously at PRL to serve as a multifaceted instrument. Optical, Mechanical and Electronics subsystems are being designed and developed in-house at PRL. It will consist of imaging, spectroscopy and imaging-polarimetry mode in the wavelength bands Y, J, H, Ks i.e. 0.8 - 2.5 micron. The detector is an 2K x 2K H2RG (MCT) array detector from Teledyne, which will give a large FOV of 10' x 10' in the imaging mode. Spectroscopic modes with resolving power of R ~ 3000, will be achieved using grisms. Spectroscopy will be available in single order and a cross-dispersed mode shall be planned for simultaneous spectra. The instrument enables multi-wavelength imaging-polarimetry using Wedged-Double Wollaston (WeDoWo) prisms to get single shot Stokes parameters (I, Q, U) for linear polarisation simultaneously, thus increasing the efficiency of polarisation measurements and reducing observation time. △ Less

Submitted 15 December, 2020; originally announced December 2020.

Comments: 8 pages, 10 figures, Submitted to SPIE Conference Astronomical Telescopes + Instrumentation 2020

Journal ref: Proc. SPIE 11447, Ground-based and Airborne Instrumentation for Astronomy VIII, 1144765, 2020

Debiasing Evaluations That are Biased by Evaluations

Authors: **gyan Wang, Ivan Stelmakh, Yuting Wei, Nihar B. Shah

Abstract: It is common to evaluate a set of items by soliciting people to rate them. For example, universities ask students to rate the teaching quality of their instructors, and conference organizers ask authors of submissions to evaluate the quality of the reviews. However, in these applications, students often give a higher rating to a course if they receive higher grades in a course, and authors often g… ▽ More It is common to evaluate a set of items by soliciting people to rate them. For example, universities ask students to rate the teaching quality of their instructors, and conference organizers ask authors of submissions to evaluate the quality of the reviews. However, in these applications, students often give a higher rating to a course if they receive higher grades in a course, and authors often give a higher rating to the reviews if their papers are accepted to the conference. In this work, we call these external factors the "outcome" experienced by people, and consider the problem of mitigating these outcome-induced biases in the given ratings when some information about the outcome is available. We formulate the information about the outcome as a known partial ordering on the bias. We propose a debiasing method by solving a regularized optimization problem under this ordering constraint, and also provide a carefully designed cross-validation method that adaptively chooses the appropriate amount of regularization. We provide theoretical guarantees on the performance of our algorithm, as well as experimental evaluations. △ Less

Submitted 1 December, 2020; originally announced December 2020.

A Large Scale Randomized Controlled Trial on Herding in Peer-Review Discussions

Authors: Ivan Stelmakh, Charvi Rastogi, Nihar B. Shah, Aarti Singh, Hal Daumé III

Abstract: Peer review is the backbone of academia and humans constitute a cornerstone of this process, being responsible for reviewing papers and making the final acceptance/rejection decisions. Given that human decision making is known to be susceptible to various cognitive biases, it is important to understand which (if any) biases are present in the peer-review process and design the pipeline such that t… ▽ More Peer review is the backbone of academia and humans constitute a cornerstone of this process, being responsible for reviewing papers and making the final acceptance/rejection decisions. Given that human decision making is known to be susceptible to various cognitive biases, it is important to understand which (if any) biases are present in the peer-review process and design the pipeline such that the impact of these biases is minimized. In this work, we focus on the dynamics of between-reviewers discussions and investigate the presence of herding behaviour therein. In that, we aim to understand whether reviewers and more senior decision makers get disproportionately influenced by the first argument presented in the discussion when (in case of reviewers) they form an independent opinion about the paper before discussing it with others. Specifically, in conjunction with the review process of ICML 2020 -- a large, top tier machine learning conference -- we design and execute a randomized controlled trial with the goal of testing for the conditional causal effect of the discussion initiator's opinion on the outcome of a paper. △ Less

Submitted 30 November, 2020; originally announced November 2020.

A Novice-Reviewer Experiment to Address Scarcity of Qualified Reviewers in Large Conferences

Authors: Ivan Stelmakh, Nihar B. Shah, Aarti Singh, Hal Daumé III

Abstract: Conference peer review constitutes a human-computation process whose importance cannot be overstated: not only it identifies the best submissions for acceptance, but, ultimately, it impacts the future of the whole research area by promoting some ideas and restraining others. A surge in the number of submissions received by leading AI conferences has challenged the sustainability of the review proc… ▽ More Conference peer review constitutes a human-computation process whose importance cannot be overstated: not only it identifies the best submissions for acceptance, but, ultimately, it impacts the future of the whole research area by promoting some ideas and restraining others. A surge in the number of submissions received by leading AI conferences has challenged the sustainability of the review process by increasing the burden on the pool of qualified reviewers which is growing at a much slower rate. In this work, we consider the problem of reviewer recruiting with a focus on the scarcity of qualified reviewers in large conferences. Specifically, we design a procedure for (i) recruiting reviewers from the population not typically covered by major conferences and (ii) guiding them through the reviewing pipeline. In conjunction with ICML 2020 -- a large, top-tier machine learning conference -- we recruit a small set of reviewers through our procedure and compare their performance with the general population of ICML reviewers. Our experiment reveals that a combination of the recruiting and guiding mechanisms allows for a principled enhancement of the reviewer pool and results in reviews of superior quality compared to the conventional pool of reviews as evaluated by senior members of the program committee (meta-reviewers). △ Less

Submitted 30 November, 2020; originally announced November 2020.

Prior and Prejudice: The Novice Reviewers' Bias against Resubmissions in Conference Peer Review

Authors: Ivan Stelmakh, Nihar B. Shah, Aarti Singh, Hal Daumé III

Abstract: Modern machine learning and computer science conferences are experiencing a surge in the number of submissions that challenges the quality of peer review as the number of competent reviewers is growing at a much slower rate. To curb this trend and reduce the burden on reviewers, several conferences have started encouraging or even requiring authors to declare the previous submission history of the… ▽ More Modern machine learning and computer science conferences are experiencing a surge in the number of submissions that challenges the quality of peer review as the number of competent reviewers is growing at a much slower rate. To curb this trend and reduce the burden on reviewers, several conferences have started encouraging or even requiring authors to declare the previous submission history of their papers. Such initiatives have been met with skepticism among authors, who raise the concern about a potential bias in reviewers' recommendations induced by this information. In this work, we investigate whether reviewers exhibit a bias caused by the knowledge that the submission under review was previously rejected at a similar venue, focusing on a population of novice reviewers who constitute a large fraction of the reviewer pool in leading machine learning and computer science conferences. We design and conduct a randomized controlled trial closely replicating the relevant components of the peer-review pipeline with $133$ reviewers (master's, junior PhD students, and recent graduates of top US universities) writing reviews for $19$ papers. The analysis reveals that reviewers indeed become negatively biased when they receive a signal about paper being a resubmission, giving almost 1 point lower overall score on a 10-point Likert item ($Δ= -0.78, \ 95\% \ \text{CI} = [-1.30, -0.24]$) than reviewers who do not receive such a signal. Looking at specific criteria scores (originality, quality, clarity and significance), we observe that novice reviewers tend to underrate quality the most. △ Less

Submitted 30 November, 2020; originally announced November 2020.

Uncovering Latent Biases in Text: Method and Application to Peer Review

Authors: Emaad Manzoor, Nihar B. Shah

Abstract: Quantifying systematic disparities in numerical quantities such as employment rates and wages between population subgroups provides compelling evidence for the existence of societal biases. However, biases in the text written for members of different subgroups (such as in recommendation letters for male and non-male candidates), though widely reported anecdotally, remain challenging to quantify. I… ▽ More Quantifying systematic disparities in numerical quantities such as employment rates and wages between population subgroups provides compelling evidence for the existence of societal biases. However, biases in the text written for members of different subgroups (such as in recommendation letters for male and non-male candidates), though widely reported anecdotally, remain challenging to quantify. In this work, we introduce a novel framework to quantify bias in text caused by the visibility of subgroup membership indicators. We develop a nonparametric estimation and inference procedure to estimate this bias. We then formalize an identification strategy to causally link the estimated bias to the visibility of subgroup membership indicators, provided observations from time periods both before and after an identity-hiding policy change. We identify an application wherein "ground truth" bias can be inferred to evaluate our framework, instead of relying on synthetic or secondary data. Specifically, we apply our framework to quantify biases in the text of peer reviews from a reputed machine learning conference before and after the conference adopted a double-blind reviewing policy. We show evidence of biases in the review ratings that serves as "ground truth", and show that our proposed framework accurately detects these biases from the review text without having access to the review ratings. △ Less

Submitted 28 October, 2020; originally announced October 2020.

Human-Supervised Semi-Autonomous Mobile Manipulators for Safely and Efficiently Executing Machine Tending Tasks

Authors: Sarah Al-Hussaini, Shantanu Thakar, Hyojeong Kim, Pradeep Rajendran, Brual C. Shah, Jeremy A. Marvel, Satyandra K. Gupta

Abstract: Mobile manipulators can be used for machine tending and material handling tasks in small volume manufacturing applications. These applications usually have semi-structured work environment. The use of a fully autonomous mobile manipulator for such applications can be risky, as an inaccurate model of the workspace may result in damage to expensive equipment. On the other hand, the use of a fully te… ▽ More Mobile manipulators can be used for machine tending and material handling tasks in small volume manufacturing applications. These applications usually have semi-structured work environment. The use of a fully autonomous mobile manipulator for such applications can be risky, as an inaccurate model of the workspace may result in damage to expensive equipment. On the other hand, the use of a fully teleoperated mobile manipulator may require a significant amount of operator time. In this paper, a semi-autonomous mobile manipulator is developed for safely and efficiently carrying out machine tending tasks under human supervision. The robot is capable of generating motion plans from the high-level task description and presenting simulation results to the human for approval. The human operator can authorize the robot to execute the automatically generated plan or provide additional input to the planner to refine the plan. If the level of uncertainty in some parts of the workspace model is high, then the human can decide to perform teleoperation to safely execute the task. Our preliminary user trials show that non-expert operators can quickly learn to use the system and perform machine tending tasks. △ Less

Submitted 16 October, 2020; v1 submitted 10 October, 2020; originally announced October 2020.

Catch Me if I Can: Detecting Strategic Behaviour in Peer Assessment

Authors: Ivan Stelmakh, Nihar B. Shah, Aarti Singh

Abstract: We consider the issue of strategic behaviour in various peer-assessment tasks, including peer grading of exams or homeworks and peer review in hiring or promotions. When a peer-assessment task is competitive (e.g., when students are graded on a curve), agents may be incentivized to misreport evaluations in order to improve their own final standing. Our focus is on designing methods for detection o… ▽ More We consider the issue of strategic behaviour in various peer-assessment tasks, including peer grading of exams or homeworks and peer review in hiring or promotions. When a peer-assessment task is competitive (e.g., when students are graded on a curve), agents may be incentivized to misreport evaluations in order to improve their own final standing. Our focus is on designing methods for detection of such manipulations. Specifically, we consider a setting in which agents evaluate a subset of their peers and output rankings that are later aggregated to form a final ordering. In this paper, we investigate a statistical framework for this problem and design a principled test for detecting strategic behaviour. We prove that our test has strong false alarm guarantees and evaluate its detection ability in practical settings. For this, we design and execute an experiment that elicits strategic behaviour from subjects and release a dataset of patterns of strategic behaviour that may be of independent interest. We then use the collected data to conduct a series of real and semi-synthetic evaluations that demonstrate a strong detection power of our test. △ Less

Submitted 8 October, 2020; originally announced October 2020.

A SUPER* Algorithm to Optimize Paper Bidding in Peer Review

Authors: Tanner Fiez, Nihar B. Shah, Lillian Ratliff

Abstract: A number of applications involve sequential arrival of users, and require showing each user an ordering of items. A prime example (which forms the focus of this paper) is the bidding process in conference peer review where reviewers enter the system sequentially, each reviewer needs to be shown the list of submitted papers, and the reviewer then "bids" to review some papers. The order of the paper… ▽ More A number of applications involve sequential arrival of users, and require showing each user an ordering of items. A prime example (which forms the focus of this paper) is the bidding process in conference peer review where reviewers enter the system sequentially, each reviewer needs to be shown the list of submitted papers, and the reviewer then "bids" to review some papers. The order of the papers shown has a significant impact on the bids due to primacy effects. In deciding on the ordering of papers to show, there are two competing goals: (i) obtaining sufficiently many bids for each paper, and (ii) satisfying reviewers by showing them relevant items. In this paper, we begin by develo** a framework to study this problem in a principled manner. We present an algorithm called SUPER*, inspired by the A* algorithm, for this goal. Theoretically, we show a local optimality guarantee of our algorithm and prove that popular baselines are considerably suboptimal. Moreover, under a community model for the similarities, we prove that SUPER* is near-optimal whereas the popular baselines are considerably suboptimal. In experiments on real data from ICLR 2018 and synthetic data, we find that SUPER* considerably outperforms baselines deployed in existing systems, consistently reducing the number of papers with fewer than requisite bids by 50-75% or more, and is also robust to various real world complexities. △ Less

Submitted 31 July, 2020; v1 submitted 27 June, 2020; originally announced July 2020.

Mitigating Manipulation in Peer Review via Randomized Reviewer Assignments

Authors: Steven Jecmen, Hanrui Zhang, Ryan Liu, Nihar B. Shah, Vincent Conitzer, Fei Fang

Abstract: We consider three important challenges in conference peer review: (i) reviewers maliciously attempting to get assigned to certain papers to provide positive reviews, possibly as part of quid-pro-quo arrangements with the authors; (ii) "torpedo reviewing," where reviewers deliberately attempt to get assigned to certain papers that they dislike in order to reject them; (iii) reviewer de-anonymizatio… ▽ More We consider three important challenges in conference peer review: (i) reviewers maliciously attempting to get assigned to certain papers to provide positive reviews, possibly as part of quid-pro-quo arrangements with the authors; (ii) "torpedo reviewing," where reviewers deliberately attempt to get assigned to certain papers that they dislike in order to reject them; (iii) reviewer de-anonymization on release of the similarities and the reviewer-assignment code. On the conceptual front, we identify connections between these three problems and present a framework that brings all these challenges under a common umbrella. We then present a (randomized) algorithm for reviewer assignment that can optimally solve the reviewer-assignment problem under any given constraints on the probability of assignment for any reviewer-paper pair. We further consider the problem of restricting the joint probability that certain suspect pairs of reviewers are assigned to certain papers, and show that this problem is NP-hard for arbitrary constraints on these joint probabilities but efficiently solvable for a practical special case. Finally, we experimentally evaluate our algorithms on datasets from past conferences, where we observe that they can limit the chance that any malicious reviewer gets assigned to their desired paper to 50% while producing assignments with over 90% of the total optimal similarity. Our algorithms still achieve this similarity while also preventing reviewers with close associations from being assigned to the same paper. △ Less

Submitted 23 October, 2020; v1 submitted 29 June, 2020; originally announced June 2020.

On the Privacy-Utility Tradeoff in Peer-Review Data Analysis

Authors: Wenxin Ding, Nihar B. Shah, Weina Wang

Abstract: A major impediment to research on improving peer review is the unavailability of peer-review data, since any release of such data must grapple with the sensitivity of the peer review data in terms of protecting identities of reviewers from authors. We posit the need to develop techniques to release peer-review data in a privacy-preserving manner. Identifying this problem, in this paper we propose… ▽ More A major impediment to research on improving peer review is the unavailability of peer-review data, since any release of such data must grapple with the sensitivity of the peer review data in terms of protecting identities of reviewers from authors. We posit the need to develop techniques to release peer-review data in a privacy-preserving manner. Identifying this problem, in this paper we propose a framework for privacy-preserving release of certain conference peer-review data -- distributions of ratings, miscalibration, and subjectivity -- with an emphasis on the accuracy (or utility) of the released data. The crux of the framework lies in recognizing that a part of the data pertaining to the reviews is already available in public, and we use this information to post-process the data released by any privacy mechanism in a manner that improves the accuracy (utility) of the data while retaining the privacy guarantees. Our framework works with any privacy-preserving mechanism that operates via releasing perturbed data. We present several positive and negative theoretical results, including a polynomial-time algorithm for improving on the privacy-utility tradeoff. △ Less

Submitted 29 June, 2020; originally announced June 2020.

Two-Sample Testing on Ranked Preference Data and the Role of Modeling Assumptions

Authors: Charvi Rastogi, Sivaraman Balakrishnan, Nihar B. Shah, Aarti Singh

Abstract: A number of applications require two-sample testing on ranked preference data. For instance, in crowdsourcing, there is a long-standing question of whether pairwise comparison data provided by people is distributed similar to ratings-converted-to-comparisons. Other examples include sports data analysis and peer grading. In this paper, we design two-sample tests for pairwise comparison data and ran… ▽ More A number of applications require two-sample testing on ranked preference data. For instance, in crowdsourcing, there is a long-standing question of whether pairwise comparison data provided by people is distributed similar to ratings-converted-to-comparisons. Other examples include sports data analysis and peer grading. In this paper, we design two-sample tests for pairwise comparison data and ranking data. For our two-sample test for pairwise comparison data, we establish an upper bound on the sample complexity required to correctly distinguish between the distributions of the two sets of samples. Our test requires essentially no assumptions on the distributions. We then prove complementary lower bounds showing that our results are tight (in the minimax sense) up to constant factors. We investigate the role of modeling assumptions by proving lower bounds for a range of pairwise comparison models (WST, MST,SST, parameter-based such as BTL and Thurstone). We also provide testing algorithms and associated sample complexity bounds for the problem of two-sample testing with partial (or total) ranking data.Furthermore, we empirically evaluate our results via extensive simulations as well as two real-world datasets consisting of pairwise comparisons. By applying our two-sample test on real-world pairwise comparison data, we conclude that ratings and rankings provided by people are indeed distributed differently. On the other hand, our test recognizes no significant difference in the relative performance of European football teams across two seasons. Finally, we apply our two-sample test on a real-world partial and total ranking dataset and find a statistically significant difference in Sushi preferences across demographic divisions based on gender, age and region of residence. △ Less

Submitted 18 November, 2020; v1 submitted 21 June, 2020; originally announced June 2020.

Comments: 40 pages, 4 figures

Taking Recoveries to Task: Recovery-Driven Development for Recipe-based Robot Tasks

Authors: Siddhartha Banerjee, Angel Daruna, David Kent, Weiyu Liu, Jonathan Balloch, Abhinav Jain, Akshay Krishnan, Muhammad Asif Rana, Harish Ravichandar, Binit Shah, Nithin Shrivatsav, Sonia Chernova

Abstract: Robot task execution when situated in real-world environments is fragile. As such, robot architectures must rely on robust error recovery, adding non-trivial complexity to highly-complex robot systems. To handle this complexity in development, we introduce Recovery-Driven Development (RDD), an iterative task scripting process that facilitates rapid task and recovery development by leveraging hiera… ▽ More Robot task execution when situated in real-world environments is fragile. As such, robot architectures must rely on robust error recovery, adding non-trivial complexity to highly-complex robot systems. To handle this complexity in development, we introduce Recovery-Driven Development (RDD), an iterative task scripting process that facilitates rapid task and recovery development by leveraging hierarchical specification, separation of nominal task and recovery development, and situated testing. We validate our approach with our challenge-winning mobile manipulator software architecture developed using RDD for the FetchIt! Challenge at the IEEE 2019 International Conference on Robotics and Automation. We attribute the success of our system to the level of robustness achieved using RDD, and conclude with lessons learned for develo** such systems. △ Less

Submitted 28 January, 2020; originally announced January 2020.

Comments: Published and presented at International Symposium on Robotics Research (ISRR), 2019 in Hanoi, Vietnam

On Testing for Biases in Peer Review

Authors: Ivan Stelmakh, Nihar B. Shah, Aarti Singh

Abstract: We consider the issue of biases in scholarly research, specifically, in peer review. There is a long standing debate on whether exposing author identities to reviewers induces biases against certain groups, and our focus is on designing tests to detect the presence of such biases. Our starting point is a remarkable recent work by Tomkins, Zhang and Heavlin which conducted a controlled, large-scale… ▽ More We consider the issue of biases in scholarly research, specifically, in peer review. There is a long standing debate on whether exposing author identities to reviewers induces biases against certain groups, and our focus is on designing tests to detect the presence of such biases. Our starting point is a remarkable recent work by Tomkins, Zhang and Heavlin which conducted a controlled, large-scale experiment to investigate existence of biases in the peer reviewing of the WSDM conference. We present two sets of results in this paper. The first set of results is negative, and pertains to the statistical tests and the experimental setup used in the work of Tomkins et al. We show that the test employed therein does not guarantee control over false alarm probability and under correlations between relevant variables coupled with any of the following conditions, with high probability, can declare a presence of bias when it is in fact absent: (a) measurement error, (b) model mismatch, (c) reviewer calibration. Moreover, we show that the setup of their experiment may itself inflate false alarm probability if (d) bidding is performed in non-blind manner or (e) popular reviewer assignment procedure is employed. Our second set of results is positive and is built around a novel approach to testing for biases that we propose. We present a general framework for testing for biases in (single vs. double blind) peer review. We then design hypothesis tests that under minimal assumptions guarantee control over false alarm probability and non-trivial power even under conditions (a)--(c) as well as propose an alternative experimental setup which mitigates issues (d) and (e). Finally, we show that no statistical test can improve over the non-parametric tests we consider in terms of the assumptions required to control for the false alarm probability. △ Less

Submitted 31 December, 2019; originally announced December 2019.

A Deep Learning approach for Hindi Named Entity Recognition

Authors: Bansi Shah, Sunil Kumar Kopparapu

Abstract: Named Entity Recognition is one of the most important text processing requirement in many NLP tasks. In this paper we use a deep architecture to accomplish the task of recognizing named entities in a given Hindi text sentence. Bidirectional Long Short Term Memory (BiLSTM) based techniques have been used for NER task in literature. In this paper, we first tune BiLSTM low-resource scenario to work f… ▽ More Named Entity Recognition is one of the most important text processing requirement in many NLP tasks. In this paper we use a deep architecture to accomplish the task of recognizing named entities in a given Hindi text sentence. Bidirectional Long Short Term Memory (BiLSTM) based techniques have been used for NER task in literature. In this paper, we first tune BiLSTM low-resource scenario to work for Hindi NER and propose two enhancements namely (a) de-noising auto-encoder (DAE) LSTM and (b) conditioning LSTM which show improvement in NER task compared to the BiLSTM approach. We use pre-trained word embedding to represent the words in the corpus, and the NER tags of the words are as defined by the used annotated corpora. Experiments have been performed to analyze the performance of different word embeddings and batch sizes which is essential for training deep models. △ Less

Submitted 5 November, 2019; originally announced November 2019.

Comments: 7 pages; work done during internship at TCS

A Differentiable Programming System to Bridge Machine Learning and Scientific Computing

Authors: Mike Innes, Alan Edelman, Keno Fischer, Chris Rackauckas, Elliot Saba, Viral B Shah, Will Tebbutt

Abstract: Scientific computing is increasingly incorporating the advancements in machine learning and the ability to work with large amounts of data. At the same time, machine learning models are becoming increasingly sophisticated and exhibit many features often seen in scientific computing, stressing the capabilities of machine learning frameworks. Just as the disciplines of scientific computing and machi… ▽ More Scientific computing is increasingly incorporating the advancements in machine learning and the ability to work with large amounts of data. At the same time, machine learning models are becoming increasingly sophisticated and exhibit many features often seen in scientific computing, stressing the capabilities of machine learning frameworks. Just as the disciplines of scientific computing and machine learning have shared common underlying infrastructure in the form of numerical linear algebra, we now have the opportunity to further share new computational infrastructure, and thus ideas, in the form of Differentiable Programming. We describe Zygote, a Differentiable Programming system that is able to take gradients of general program structures. We implement this system in the Julia programming language. Our system supports almost all language constructs (control flow, recursion, mutation, etc.) and compiles high-performance code without requiring any user intervention or refactoring to stage computations. This enables an expressive programming model for deep learning, but more importantly, it enables us to incorporate a large ecosystem of libraries in our models in a straightforward way. We discuss our approach to automatic differentiation, including its support for advanced techniques such as mixed-mode, complex and checkpointed differentiation, and present several examples of differentiating programs. △ Less

Submitted 18 July, 2019; v1 submitted 17 July, 2019; originally announced July 2019.

Comments: Submitted to NeurIPS 2019

Stretching the Effectiveness of MLE from Accuracy to Bias for Pairwise Comparisons

Authors: **gyan Wang, Nihar B. Shah, R. Ravi

Abstract: A number of applications (e.g., AI bot tournaments, sports, peer grading, crowdsourcing) use pairwise comparison data and the Bradley-Terry-Luce (BTL) model to evaluate a given collection of items (e.g., bots, teams, students, search results). Past work has shown that under the BTL model, the widely-used maximum-likelihood estimator (MLE) is minimax-optimal in estimating the item parameters, in te… ▽ More A number of applications (e.g., AI bot tournaments, sports, peer grading, crowdsourcing) use pairwise comparison data and the Bradley-Terry-Luce (BTL) model to evaluate a given collection of items (e.g., bots, teams, students, search results). Past work has shown that under the BTL model, the widely-used maximum-likelihood estimator (MLE) is minimax-optimal in estimating the item parameters, in terms of the mean squared error. However, another important desideratum for designing estimators is fairness. In this work, we consider fairness modeled by the notion of bias in statistics. We show that the MLE incurs a suboptimal rate in terms of bias. We then propose a simple modification to the MLE, which "stretches" the bounding box of the maximum-likelihood optimizer by a small constant factor from the underlying ground truth domain. We show that this simple modification leads to an improved rate in bias, while maintaining minimax-optimality in the mean squared error. In this manner, our proposed class of estimators provably improves fairness represented by bias without loss in accuracy. △ Less

Submitted 10 June, 2019; originally announced June 2019.

Lorentz-violating gaugeon formalism for rank-2 tensor theory

Authors: Sudhaker Upadhyay, Mushtaq B. Shah, Prince A. Ganai

Abstract: We develop a BRST symmetric gaugeon formalism for the Abelian rank-2 antisymmetric tensor field in the Lorentz breaking framework. The Lorentz breaking is achieved here by considering a proper subgroup of Lorentz group together with translation. In this scenario, the gaugeon fields together with the standard fields of the Abelian rank-2 antisymmetric tensor theory get mass. In order to develop the… ▽ More We develop a BRST symmetric gaugeon formalism for the Abelian rank-2 antisymmetric tensor field in the Lorentz breaking framework. The Lorentz breaking is achieved here by considering a proper subgroup of Lorentz group together with translation. In this scenario, the gaugeon fields together with the standard fields of the Abelian rank-2 antisymmetric tensor theory get mass. In order to develop the gaugeon formulation for this theory in VSR, we first introduce a set of dipole vector fields as a quantum gauge freedom to the action. In order to quantize the dipole vector fields, the VSR-modified gauge-fixing and corresponding ghost action are constructed as the classical action is invariant under a VSR-modified gauge transformation. Further, we present a Type I gaugeon formalism for the Abelian rank-2 antisymmetric tensor field theory in VSR. The gauge structures of Fock space constructed with the help of BRST charges are also discussed. △ Less

Submitted 28 May, 2019; originally announced June 2019.

Comments: 13 pages, Accepted for publication in Mod. Phys. Lett. A

Journal ref: Mod. Phys. Lett. A 34 (2019) 1950245

Ramanujan Periodic Subspace Division Multiplexing (RPSDM)

Authors: Goli Srikanth, Vijay Kumar Chakka, Shaik Basheeruddin Shah

Abstract: In this paper, a new modulation method defined as Ramanujan Periodic Subspace Division Multiplexing (RPSDM) is proposed using Ramanujan subspaces. Each subspace contains an integer valued Ramanujan Sum (RS) and its circular downshifts as a basis. The proposed RPSDM decomposes the linear time-invariant wireless channels into a Toeplitz stair block diagonal matrices, whereas Orthogonal Frequency Div… ▽ More In this paper, a new modulation method defined as Ramanujan Periodic Subspace Division Multiplexing (RPSDM) is proposed using Ramanujan subspaces. Each subspace contains an integer valued Ramanujan Sum (RS) and its circular downshifts as a basis. The proposed RPSDM decomposes the linear time-invariant wireless channels into a Toeplitz stair block diagonal matrices, whereas Orthogonal Frequency Division Multiplexing (OFDM) decompose the same into diagonal. Advantages of such structured subspaces representation are studied and compared with an OFDM representation in terms of Peak-Average Power Ratio (PAPR) and Bit-Error-Rate (BER). Zero Forcing (ZF) and Minimum Mean Square Error (MMSE) detectors are applied to evaluate the performance of OFDM and RPSDM techniques. Finally, the simulation results show that the proposed design (with an additional receiver complexity) outperforms OFDM under both detectors. △ Less

Submitted 28 April, 2019; originally announced April 2019.

An Incentive Mechanism for Crowd Sensing with Colluding Agents

Authors: Susu Xu, Weiguang Mao, Yue Cao, Hae Young Noh, Nihar B. Shah

Abstract: Vehicular mobile crowd sensing is a fast-emerging paradigm to collect data about the environment by mounting sensors on vehicles such as taxis. An important problem in vehicular crowd sensing is to design payment mechanisms to incentivize drivers (agents) to collect data, with the overall goal of obtaining the maximum amount of data (across multiple vehicles) for a given budget. Past works on this… ▽ More Vehicular mobile crowd sensing is a fast-emerging paradigm to collect data about the environment by mounting sensors on vehicles such as taxis. An important problem in vehicular crowd sensing is to design payment mechanisms to incentivize drivers (agents) to collect data, with the overall goal of obtaining the maximum amount of data (across multiple vehicles) for a given budget. Past works on this problem consider a setting where each agent operates in isolation---an assumption which is frequently violated in practice. In this paper, we design an incentive mechanism to incentivize agents who can engage in arbitrary collusions. We then show that in a "homogeneous" setting, our mechanism is optimal, and can do as well as any mechanism which knows the agents' preferences a priori. Moreover, if the agents are non-colluding, then our mechanism automatically does as well as any other non-colluding mechanism. We also show that our proposed mechanism has strong (and asymptotically optimal) guarantees for a more general "heterogeneous" setting. Experiments based on synthesized data and real-world data reveal gains of over 30\% attained by our mechanism compared to past literature. △ Less

Submitted 13 September, 2018; originally announced September 2018.

Loss Functions, Axioms, and Peer Review

Authors: Ritesh Noothigattu, Nihar B. Shah, Ariel D. Procaccia

Abstract: It is common to see a handful of reviewers reject a highly novel paper, because they view, say, extensive experiments as far more important than novelty, whereas the community as a whole would have embraced the paper. More generally, the disparate map** of criteria scores to final recommendations by different reviewers is a major source of inconsistency in peer review. In this paper we present a… ▽ More It is common to see a handful of reviewers reject a highly novel paper, because they view, say, extensive experiments as far more important than novelty, whereas the community as a whole would have embraced the paper. More generally, the disparate map** of criteria scores to final recommendations by different reviewers is a major source of inconsistency in peer review. In this paper we present a framework inspired by empirical risk minimization (ERM) for learning the community's aggregate map**. The key challenge that arises is the specification of a loss function for ERM. We consider the class of $L(p,q)$ loss functions, which is a matrix-extension of the standard class of $L_p$ losses on vectors; here the choice of the loss function amounts to choosing the hyperparameters $p, q \in [1,\infty]$. To deal with the absence of ground truth in our problem, we instead draw on computational social choice to identify desirable values of the hyperparameters $p$ and $q$. Specifically, we characterize $p=q=1$ as the only choice of these hyperparameters that satisfies three natural axiomatic properties. Finally, we implement and apply our approach to reviews from IJCAI 2017. △ Less

Submitted 2 March, 2020; v1 submitted 27 August, 2018; originally announced August 2018.

On Strategyproof Conference Peer Review

Authors: Yichong Xu, Han Zhao, Xiaofei Shi, Jeremy Zhang, Nihar B. Shah

Abstract: We consider peer review in a conference setting where there is typically an overlap between the set of reviewers and the set of authors. This overlap can incentivize strategic reviews to influence the final ranking of one's own papers. In this work, we address this problem through the lens of social choice, and present a theoretical framework for strategyproof and efficient peer review. We first p… ▽ More We consider peer review in a conference setting where there is typically an overlap between the set of reviewers and the set of authors. This overlap can incentivize strategic reviews to influence the final ranking of one's own papers. In this work, we address this problem through the lens of social choice, and present a theoretical framework for strategyproof and efficient peer review. We first present and analyze an algorithm for reviewer-assignment and aggregation that guarantees strategyproofness and a natural efficiency property called unanimity, when the authorship graph satisfies a simple property. Our algorithm is based on the so-called partitioning method, and can be thought as a generalization of this method to conference peer review settings. We then empirically show that the requisite property on the authorship graph is indeed satisfied in the submission data from the ICLR conference, and further demonstrate a simple trick to make the partitioning method more practically appealing for conference peer review. Finally, we complement our positive results with negative theoretical results where we prove that under various ways of strengthening the requirements, it is impossible for any algorithm to be strategyproof and efficient. △ Less

Submitted 31 January, 2020; v1 submitted 16 June, 2018; originally announced June 2018.

PeerReview4All: Fair and Accurate Reviewer Assignment in Peer Review

Authors: Ivan Stelmakh, Nihar B. Shah, Aarti Singh

Abstract: We consider the problem of automated assignment of papers to reviewers in conference peer review, with a focus on fairness and statistical accuracy. Our fairness objective is to maximize the review quality of the most disadvantaged paper, in contrast to the commonly used objective of maximizing the total quality over all papers. We design an assignment algorithm based on an incremental max-flow pr… ▽ More We consider the problem of automated assignment of papers to reviewers in conference peer review, with a focus on fairness and statistical accuracy. Our fairness objective is to maximize the review quality of the most disadvantaged paper, in contrast to the commonly used objective of maximizing the total quality over all papers. We design an assignment algorithm based on an incremental max-flow procedure that we prove is near-optimally fair. Our statistical accuracy objective is to ensure correct recovery of the papers that should be accepted. We provide a sharp minimax analysis of the accuracy of the peer-review process for a popular objective-score model as well as for a novel subjective-score model that we propose in the paper. Our analysis proves that our proposed assignment algorithm also leads to a near-optimal statistical accuracy. Finally, we design a novel experiment that allows for an objective comparison of various assignment algorithms, and overcomes the inherent difficulty posed by the absence of a ground truth in experiments on peer-review. The results of this experiment as well as of other experiments on synthetic and real data corroborate the theoretical guarantees of our algorithm. △ Less

Submitted 14 November, 2019; v1 submitted 16 June, 2018; originally announced June 2018.

Your 2 is My 1, Your 3 is My 9: Handling Arbitrary Miscalibrations in Ratings

Authors: **gyan Wang, Nihar B. Shah

Abstract: Cardinal scores (numeric ratings) collected from people are well known to suffer from miscalibrations. A popular approach to address this issue is to assume simplistic models of miscalibration (such as linear biases) to de-bias the scores. This approach, however, often fares poorly because people's miscalibrations are typically far more complex and not well understood. In the absence of simplifyin… ▽ More Cardinal scores (numeric ratings) collected from people are well known to suffer from miscalibrations. A popular approach to address this issue is to assume simplistic models of miscalibration (such as linear biases) to de-bias the scores. This approach, however, often fares poorly because people's miscalibrations are typically far more complex and not well understood. In the absence of simplifying assumptions on the miscalibration, it is widely believed by the crowdsourcing community that the only useful information in the cardinal scores is the induced ranking. In this paper, inspired by the framework of Stein's shrinkage, empirical Bayes, and the classic two-envelope problem, we contest this widespread belief. Specifically, we consider cardinal scores with arbitrary (or even adversarially chosen) miscalibrations which are only required to be consistent with the induced ranking. We design estimators which despite making no assumptions on the miscalibration, strictly and uniformly outperform all possible estimators that rely on only the ranking. Our estimators are flexible in that they can be used as a plug-in for a variety of applications, and we provide a proof-of-concept for A/B testing and ranking. Our results thus provide novel insights in the eternal debate between cardinal and ordinal data. △ Less

Submitted 12 September, 2018; v1 submitted 13 June, 2018; originally announced June 2018.

Non-Local Deformation of a Supersymmetric Field Theory

Authors: Qin Zhao, Mir Faizal, Mushtaq B. Shah, Anha Bhat, Prince A. Ganai, Zaid Zaz, Syed Masood, Jamil Raza, Raja Muhammad Irfan

Abstract: In this paper, we will analyse a supersymmetric field theory deformed by generalized uncertainty principle and Lifshitz scaling. It will be observed that this deformed supersymmetric field theory contains non-local fractional derivative terms. In order to construct such deformed N=1 supersymmetric theory, a harmonic extension of functions will be used. However, the supersymmetry will be only prese… ▽ More In this paper, we will analyse a supersymmetric field theory deformed by generalized uncertainty principle and Lifshitz scaling. It will be observed that this deformed supersymmetric field theory contains non-local fractional derivative terms. In order to construct such deformed N=1 supersymmetric theory, a harmonic extension of functions will be used. However, the supersymmetry will be only preserved for a free theory and will be broken by the inclusion of interaction terms. △ Less

Submitted 4 October, 2017; originally announced October 2017.

Comments: 12 pages, pulished version

Journal ref: Eur.Phys.J.C (2017) 77:612

Low Permutation-rank Matrices: Structural Properties and Noisy Completion

Authors: Nihar B. Shah, Sivaraman Balakrishnan, Martin J. Wainwright

Abstract: We consider the problem of noisy matrix completion, in which the goal is to reconstruct a structured matrix whose entries are partially observed in noise. Standard approaches to this underdetermined inverse problem are based on assuming that the underlying matrix has low rank, or is well-approximated by a low rank matrix. In this paper, we propose a richer model based on what we term the "permutat… ▽ More We consider the problem of noisy matrix completion, in which the goal is to reconstruct a structured matrix whose entries are partially observed in noise. Standard approaches to this underdetermined inverse problem are based on assuming that the underlying matrix has low rank, or is well-approximated by a low rank matrix. In this paper, we propose a richer model based on what we term the "permutation-rank" of a matrix. We first describe how the classical non-negative rank model enforces restrictions that may be undesirable in practice, and how and these restrictions can be avoided by using the richer permutation-rank model. Second, we establish the minimax rates of estimation under the new permutation-based model, and prove that surprisingly, the minimax rates are equivalent up to logarithmic factors to those for estimation under the typical low rank model. Third, we analyze a computationally efficient singular-value-thresholding algorithm, known to be optimal for the low-rank setting, and show that it also simultaneously yields a consistent estimator for the low-permutation rank setting. Finally, we present various structural results characterizing the uniqueness of the permutation-rank decomposition, and characterizing convex approximations of the permutation-rank polytope. △ Less

Submitted 31 August, 2017; originally announced September 2017.

Boundary Effects in Super-Yang-Mills Theory

Authors: Mushtaq B Shah, Mir Faizal, Prince A Ganai, Zaid Zaz, Anha Bhat, Syed Masood

Abstract: In this paper, we shall analyse a three dimensional supersymmetry theory with $\mathcal{N} = 2$. The effective Lagrangian will be given by the sum of the gauge fixing term and the ghost term with the original classical Lagrangian. In presence of a boundary the supersymmetry of this Lagrangian will be broken. However, it will be possible to preserve half the supersymmetry even in presence of a boun… ▽ More In this paper, we shall analyse a three dimensional supersymmetry theory with $\mathcal{N} = 2$. The effective Lagrangian will be given by the sum of the gauge fixing term and the ghost term with the original classical Lagrangian. In presence of a boundary the supersymmetry of this Lagrangian will be broken. However, it will be possible to preserve half the supersymmetry even in presence of a boundary. This will be done by adding a boundary Lagrangian to the effective bulk Lagrangian. The supersymmetric transformation of this new boundary Lagrangian will exactly cancel the boundary term generated from the supersymmetric transformation of the effective bulk Lagrangian. We will obtain the Slavnov-Taylor Identity for this theory. △ Less

Submitted 31 August, 2017; originally announced September 2017.

Comments: 16 pages, Accepted in EPJC

Journal ref: Eur. Phys. J. C 77, 309 (2017)

Design and Analysis of the NIPS 2016 Review Process

Authors: Nihar B. Shah, Behzad Tabibian, Krikamol Muandet, Isabelle Guyon, Ulrike von Luxburg

Abstract: Neural Information Processing Systems (NIPS) is a top-tier annual conference in machine learning. The 2016 edition of the conference comprised more than 2,400 paper submissions, 3,000 reviewers, and 8,000 attendees. This represents a growth of nearly 40% in terms of submissions, 96% in terms of reviewers, and over 100% in terms of attendees as compared to the previous year. The massive scale as we… ▽ More Neural Information Processing Systems (NIPS) is a top-tier annual conference in machine learning. The 2016 edition of the conference comprised more than 2,400 paper submissions, 3,000 reviewers, and 8,000 attendees. This represents a growth of nearly 40% in terms of submissions, 96% in terms of reviewers, and over 100% in terms of attendees as compared to the previous year. The massive scale as well as rapid growth of the conference calls for a thorough quality assessment of the peer-review process and novel means of improvement. In this paper, we analyze several aspects of the data collected during the review process, including an experiment investigating the efficacy of collecting ordinal rankings from reviewers. Our goal is to check the soundness of the review process, and provide insights that may be useful in the design of the review process of subsequent conferences. △ Less

Submitted 23 April, 2018; v1 submitted 31 August, 2017; originally announced August 2017.

Nucleus-acoustic solitary waves and double layers in a magnetized degenerate quantum plasma

Authors: B. Hosen. M. G. Shah, M. R. Hossen, A. A. Mamun

Abstract: The properties of nucleus-acoustic (NA) solitary waves (SWs) and double layers (DLs) in a four-component magnetized degenerate quantum plasma system (containing non-degenerate inertial light nuclei, both non-relativistically and ultra-relativistically degenerate electrons and positrons, and immobile heavy nuclei) are theoretically investigated by the reductive perturbation method. The Korteweg-de… ▽ More The properties of nucleus-acoustic (NA) solitary waves (SWs) and double layers (DLs) in a four-component magnetized degenerate quantum plasma system (containing non-degenerate inertial light nuclei, both non-relativistically and ultra-relativistically degenerate electrons and positrons, and immobile heavy nuclei) are theoretically investigated by the reductive perturbation method. The Korteweg-de Vries (K-dV), the modified K-dV (MK-dV), and the Gardner equations are derived to examine the basic features (viz. amplitude, speed, and width) of NA SWs and DLs. It is found that the effects of the ultra-relativistically degenerate electrons and positrons, stationary heavy nuclei, external magnetic field (obliqueness), etc. significantly modify the basic features of the NA SWs and DLs. The basic features and the underlying physics of NA SWs and DLs, which are relevant to some astrophysical compact objects including white dwarfs and neutron stars, are pinpointed. △ Less

Submitted 18 July, 2017; originally announced July 2017.

Comments: 10 Pages, 16 Figures

Lorentz Violating $p$-form Gauge Theories in Superspace

Authors: Sudhaker Upadhyay, Mushtaq B. Shah, Prince A. Ganai

Abstract: Very special relativity (VSR) keeps the main features of special relativity but breaks rotational invariance due to an intrinsic preferred direction. We study the VSR modified extended BRST and anti-BRST symmetry of the Batalin-Vilkovisky (BV) actions corresponding to the $p=1,2,3$-form gauge theories. Within VSR framework, we discuss the extended BRST invariant and extended BRST and anti-BRST inv… ▽ More Very special relativity (VSR) keeps the main features of special relativity but breaks rotational invariance due to an intrinsic preferred direction. We study the VSR modified extended BRST and anti-BRST symmetry of the Batalin-Vilkovisky (BV) actions corresponding to the $p=1,2,3$-form gauge theories. Within VSR framework, we discuss the extended BRST invariant and extended BRST and anti-BRST invariant superspace formulations for these BV actions. Here we observe that the VSR modified extended BRST invariant BV actions corresponding to the $p=1,2,3$-form gauge theories can be written manifestly covariant manner in a superspace with one Grassmann coordinate. Moreover, two Grassmann coordinates are required to describe the VSR modified extended BRST and extended anti-BRST invariant BV actions in a superspace. These results are consistent with the Lorentz invariant (special relativity) formulation. △ Less

Submitted 3 March, 2017; v1 submitted 19 February, 2017; originally announced February 2017.

Comments: 42 pages, no figure, Accepted for publication in EPJC

Journal ref: Eur. Phys. J. C (2017) 77:157

The Most General Form of Deformation of the Heisenberg Algebra from the Generalized Uncertainty Principle

Authors: Syed Masood, Mir Faizal, Zaid Zaz, Ahmed Farag Ali, Jamil Raza, Mushtaq B Shah

Abstract: In this paper, we will propose the most general form of the deformation of Heisenberg algebra motivated by the generalized uncertainty principle. This deformation of the Heisenberg algebra will deform all quantum mechanical systems. The form of the generalized uncertainty principle used to motivate these results will be motivated by space fractional quantum mechanics and non-locality in quantum me… ▽ More In this paper, we will propose the most general form of the deformation of Heisenberg algebra motivated by the generalized uncertainty principle. This deformation of the Heisenberg algebra will deform all quantum mechanical systems. The form of the generalized uncertainty principle used to motivate these results will be motivated by space fractional quantum mechanics and non-locality in quantum mechanical systems. We also analyse a specific limit of this generalized deformation for one dimensional system, and in that limit, a nonlocal deformation of the momentum operator generates a local deformation of all one dimensional quantum mechanical systems. We analyse the low energy effects of this deformation on a harmonic oscillator, Landau levels, Lamb shift, and potential barrier. We also demonstrate that this deformation leads to a discretization of space. △ Less

Submitted 29 October, 2016; originally announced November 2016.

Comments: 25 pages, no figures. Accepted for publication in Phys. Lett. B

Journal ref: Phys.Lett. B763 (2016) 218-227

Performance of Large Area X-ray Proportional Counters in a Balloon Experiment

Authors: J. Roy, P. C. Agrawal, D. K. Dedhia, R. K. Manchanda, P. B. Shah, V. R. Chitnis, V. M. Gujar, J. V. Parmar, D. M. Pawar, V. B. Kurhade

Abstract: ASTROSAT is India's first satellite fully devoted to astronomical observations covering a wide spectral band from optical to hard X-rays by a complement of 4 co-aligned instruments and a Scanning Sky X-ray Monitor. One of the instruments is Large Area X-ray Proportional Counter with 3 identical detectors. In order to assess the performance of this instrument, a balloon experiment with two prototyp… ▽ More ASTROSAT is India's first satellite fully devoted to astronomical observations covering a wide spectral band from optical to hard X-rays by a complement of 4 co-aligned instruments and a Scanning Sky X-ray Monitor. One of the instruments is Large Area X-ray Proportional Counter with 3 identical detectors. In order to assess the performance of this instrument, a balloon experiment with two prototype Large Area X-ray Proportional Counters (LAXPC) was carried out on 2008 April 14. The design of these LAXPCs was similar to those on the ASTROSAT except that their field of view (FOV) was 3$^{\circ}$ $\times$ 3$^{\circ}$ versus FOV of 1$^{\circ}$ $\times$ 1$^{\circ}$ for the LAXPCs on the ASTROSAT. The LAXPCs are aimed at the timing and spectral studies of X-ray sources in 3-80 keV region. In the balloon experiment, the LAXPC, associated electronics and support systems were mounted on an oriented platform which could be pre-programmed to track any source in the sky. A brief description of the LAXPC design, laboratory tests, calibration and the detector characteristics is presented here. The details of the experiment and background counting rates of the 2 LAXPCs at the float altitude of about 41 km are presented in different energy bands. The bright black hole X-ray binary Cygnus X-1 (Cyg X-1) was observed in the experiment for $\sim$ 3 hours. Details of Cyg X-1 observations, count rates measured from it in different energy intervals and the intensity variations of Cyg X-1 detected during the observations are presented and briefly discussed. △ Less

Submitted 12 September, 2016; originally announced September 2016.

Comments: 23 pages, 14 figures in Experimental Astronomy Astrophysical Instrumentation and Methods August 2016,Vol 42, ISSN 0922-6435

A Permutation-based Model for Crowd Labeling: Optimal Estimation and Robustness

Authors: Nihar B. Shah, Sivaraman Balakrishnan, Martin J. Wainwright

Abstract: The task of aggregating and denoising crowd-labeled data has gained increased significance with the advent of crowdsourcing platforms and massive datasets. We propose a permutation-based model for crowd labeled data that is a significant generalization of the classical Dawid-Skene model, and introduce a new error metric by which to compare different estimators. We derive global minimax rates for t… ▽ More The task of aggregating and denoising crowd-labeled data has gained increased significance with the advent of crowdsourcing platforms and massive datasets. We propose a permutation-based model for crowd labeled data that is a significant generalization of the classical Dawid-Skene model, and introduce a new error metric by which to compare different estimators. We derive global minimax rates for the permutation-based model that are sharp up to logarithmic factors, and match the minimax lower bounds derived under the simpler Dawid-Skene model. We then design two computationally-efficient estimators: the WAN estimator for the setting where the ordering of workers in terms of their abilities is approximately known, and the OBI-WAN estimator where that is not known. For each of these estimators, we provide non-asymptotic bounds on their performance. We conduct synthetic simulations and experiments on real-world crowdsourcing data, and the experimental results corroborate our theoretical findings. △ Less

Submitted 10 January, 2021; v1 submitted 30 June, 2016; originally announced June 2016.

Comments: in IEEE Transactions on Information Theory (online), 2020

Active Ranking from Pairwise Comparisons and when Parametric Assumptions Don't Help

Authors: Reinhard Heckel, Nihar B. Shah, Kannan Ramchandran, Martin J. Wainwright

Abstract: We consider sequential or active ranking of a set of n items based on noisy pairwise comparisons. Items are ranked according to the probability that a given item beats a randomly chosen item, and ranking refers to partitioning the items into sets of pre-specified sizes according to their scores. This notion of ranking includes as special cases the identification of the top-k items and the total or… ▽ More We consider sequential or active ranking of a set of n items based on noisy pairwise comparisons. Items are ranked according to the probability that a given item beats a randomly chosen item, and ranking refers to partitioning the items into sets of pre-specified sizes according to their scores. This notion of ranking includes as special cases the identification of the top-k items and the total ordering of the items. We first analyze a sequential ranking algorithm that counts the number of comparisons won, and uses these counts to decide whether to stop, or to compare another pair of items, chosen based on confidence intervals specified by the data collected up to that point. We prove that this algorithm succeeds in recovering the ranking using a number of comparisons that is optimal up to logarithmic factors. This guarantee does not require any structural properties of the underlying pairwise probability matrix, unlike a significant body of past work on pairwise ranking based on parametric models such as the Thurstone or Bradley-Terry-Luce models. It has been a long-standing open question as to whether or not imposing these parametric assumptions allows for improved ranking algorithms. For stochastic comparison models, in which the pairwise probabilities are bounded away from zero, our second contribution is to resolve this issue by proving a lower bound for parametric models. This shows, perhaps surprisingly, that these popular parametric modeling choices offer at most logarithmic gains for stochastic comparisons. △ Less

Submitted 23 September, 2016; v1 submitted 28 June, 2016; originally announced June 2016.

Comments: improved log factor in main result; added discussion on comparison probabilities close to zero; added numerical results

Feeling the Bern: Adaptive Estimators for Bernoulli Probabilities of Pairwise Comparisons

Authors: Nihar B. Shah, Sivaraman Balakrishnan, Martin J. Wainwright

Abstract: We study methods for aggregating pairwise comparison data in order to estimate outcome probabilities for future comparisons among a collection of n items. Working within a flexible framework that imposes only a form of strong stochastic transitivity (SST), we introduce an adaptivity index defined by the indifference sets of the pairwise comparison probabilities. In addition to measuring the usual… ▽ More We study methods for aggregating pairwise comparison data in order to estimate outcome probabilities for future comparisons among a collection of n items. Working within a flexible framework that imposes only a form of strong stochastic transitivity (SST), we introduce an adaptivity index defined by the indifference sets of the pairwise comparison probabilities. In addition to measuring the usual worst-case risk of an estimator, this adaptivity index also captures the extent to which the estimator adapts to instance-specific difficulty relative to an oracle estimator. We prove three main results that involve this adaptivity index and different algorithms. First, we propose a three-step estimator termed Count-Randomize-Least squares (CRL), and show that it has adaptivity index upper bounded as $\sqrt{n}$ up to logarithmic factors. We then show that that conditional on the hardness of planted clique, no computationally efficient estimator can achieve an adaptivity index smaller than $\sqrt{n}$. Second, we show that a regularized least squares estimator can achieve a poly-logarithmic adaptivity index, thereby demonstrating a $\sqrt{n}$-gap between optimal and computationally achievable adaptivity. Finally, we prove that the standard least squares estimator, which is known to be optimally adaptive in several closely related problems, fails to adapt in the context of estimating pairwise probabilities. △ Less

Submitted 22 March, 2016; originally announced March 2016.

Parametric Prediction from Parametric Agents

Authors: Yuan Luo, Nihar B. Shah, Jianwei Huang, Jean Walrand

Abstract: We consider a problem of prediction based on opinions elicited from heterogeneous rational agents with private information. Making an accurate prediction with a minimal cost requires a joint design of the incentive mechanism and the prediction algorithm. Such a problem lies at the nexus of statistical learning theory and game theory, and arises in many domains such as consumer surveys and mobile c… ▽ More We consider a problem of prediction based on opinions elicited from heterogeneous rational agents with private information. Making an accurate prediction with a minimal cost requires a joint design of the incentive mechanism and the prediction algorithm. Such a problem lies at the nexus of statistical learning theory and game theory, and arises in many domains such as consumer surveys and mobile crowdsourcing. In order to elicit heterogeneous agents' private information and incentivize agents with different capabilities to act in the principal's best interest, we design an optimal joint incentive mechanism and prediction algorithm called COPE (COst and Prediction Elicitation), the analysis of which offers several valuable engineering insights. First, when the costs incurred by the agents are linear in the exerted effort, COPE corresponds to a "crowd contending" mechanism, where the principal only employs the agent with the highest capability. Second, when the costs are quadratic, COPE corresponds to a "crowd-sourcing" mechanism that employs multiple agents with different capabilities at the same time. Numerical simulations show that COPE improves the principal's profit and the network profit significantly (larger than 30% in our simulations), comparing to those mechanisms that assume all agents have equal capabilities. △ Less

Submitted 24 February, 2016; originally announced February 2016.

Limiting the Development of Anti-Cancer Drug Resistance in a Spatial Model of Micrometastases

Authors: Ami B. Shah, Katarzyna A. Rejniak, Jana L. Gevertz

Abstract: While chemoresistance in primary tumors is well-studied, much less is known about the influence of systemic chemotherapy on the development of drug resistance at metastatic sites. In this work, we use a hybrid spatial model of tumor response to a DNA damaging drug to study how the development of chemoresistance in micrometastases depends on the drug dosing schedule. We separately consider cell pop… ▽ More While chemoresistance in primary tumors is well-studied, much less is known about the influence of systemic chemotherapy on the development of drug resistance at metastatic sites. In this work, we use a hybrid spatial model of tumor response to a DNA damaging drug to study how the development of chemoresistance in micrometastases depends on the drug dosing schedule. We separately consider cell populations that harbor pre-existing resistance to the drug, and those that acquire resistance during the course of treatment. For each of these independent scenarios, we consider one hypothetical cell line that is responsive to metronomic chemotherapy, and another that with high probability cannot be eradicated by a metronomic protocol. Motivated by experimental work on ovarian cancer xenografts, we consider all possible combinations of a one week treatment protocol, repeated for three weeks, and constrained by the total weekly drug dose. Simulations reveal a small number of fractionated-dose protocols that are at least as effective as metronomic therapy in eradicating micrometastases with acquired resistance (weak or strong), while also being at least as effective on those that harbor weakly pre-existing resistant cells. Given the responsiveness of very different theoretical cell lines to these few fractionated-dose protocols, these may represent more effective ways to schedule chemotherapy with the goal of limiting metastatic tumor progression. △ Less

Submitted 2 March, 2016; v1 submitted 13 January, 2016; originally announced January 2016.

Comments: 25 pages, 8 figures

MSC Class: 92C50; 37N25

Simple, Robust and Optimal Ranking from Pairwise Comparisons

Authors: Nihar B. Shah, Martin J. Wainwright

Abstract: We consider data in the form of pairwise comparisons of n items, with the goal of precisely identifying the top k items for some value of k < n, or alternatively, recovering a ranking of all the items. We analyze the Copeland counting algorithm that ranks the items in order of the number of pairwise comparisons won, and show it has three attractive features: (a) its computational efficiency leads… ▽ More We consider data in the form of pairwise comparisons of n items, with the goal of precisely identifying the top k items for some value of k < n, or alternatively, recovering a ranking of all the items. We analyze the Copeland counting algorithm that ranks the items in order of the number of pairwise comparisons won, and show it has three attractive features: (a) its computational efficiency leads to speed-ups of several orders of magnitude in computation time as compared to prior work; (b) it is robust in that theoretical guarantees impose no conditions on the underlying matrix of pairwise-comparison probabilities, in contrast to some prior work that applies only to the BTL parametric model; and (c) it is an optimal method up to constant factors, meaning that it achieves the information-theoretic limits for recovering the top k-subset. We extend our results to obtain sharp guarantees for approximate recovery under the Hamming distortion metric, and more generally, to any arbitrary error requirement that satisfies a simple and natural monotonicity condition. △ Less

Submitted 26 April, 2016; v1 submitted 30 December, 2015; originally announced December 2015.

Comments: Changes in version 2: In addition to recovery in the exact and Hamming metrics, v2 analyzes a general, abstract recovery criterion based on a notion of "allowed sets"

Stochastically Transitive Models for Pairwise Comparisons: Statistical and Computational Issues

Authors: Nihar B. Shah, Sivaraman Balakrishnan, Adityanand Guntuboyina, Martin J. Wainwright

Abstract: There are various parametric models for analyzing pairwise comparison data, including the Bradley-Terry-Luce (BTL) and Thurstone models, but their reliance on strong parametric assumptions is limiting. In this work, we study a flexible model for pairwise comparisons, under which the probabilities of outcomes are required only to satisfy a natural form of stochastic transitivity. This class include… ▽ More There are various parametric models for analyzing pairwise comparison data, including the Bradley-Terry-Luce (BTL) and Thurstone models, but their reliance on strong parametric assumptions is limiting. In this work, we study a flexible model for pairwise comparisons, under which the probabilities of outcomes are required only to satisfy a natural form of stochastic transitivity. This class includes parametric models including the BTL and Thurstone models as special cases, but is considerably more general. We provide various examples of models in this broader stochastically transitive class for which classical parametric models provide poor fits. Despite this greater flexibility, we show that the matrix of probabilities can be estimated at the same rate as in standard parametric models. On the other hand, unlike in the BTL and Thurstone models, computing the minimax-optimal estimator in the stochastically transitive model is non-trivial, and we explore various computationally tractable alternatives. We show that a simple singular value thresholding algorithm is statistically consistent but does not achieve the minimax rate. We then propose and study algorithms that achieve the minimax rate over interesting sub-classes of the full stochastically transitive class. We complement our theoretical results with thorough numerical simulations. △ Less

Submitted 27 September, 2016; v1 submitted 19 October, 2015; originally announced October 2015.

Information-theoretically Secure Erasure Codes for Distributed Storage

Authors: Nihar B. Shah, K. V. Rashmi, Kannan Ramchandran, P. Vijay Kumar

Abstract: Repair operations in distributed storage systems potentially expose the data to malicious acts of passive eavesdroppers or active adversaries, which can be detrimental to the security of the system. This paper presents erasure codes and repair algorithms that ensure security of the data in the presence of passive eavesdroppers and active adversaries, while maintaining high availability, reliabilit… ▽ More Repair operations in distributed storage systems potentially expose the data to malicious acts of passive eavesdroppers or active adversaries, which can be detrimental to the security of the system. This paper presents erasure codes and repair algorithms that ensure security of the data in the presence of passive eavesdroppers and active adversaries, while maintaining high availability, reliability and efficiency in the system. Our codes are optimal in that they meet previously proposed lower bounds on the storage, network-bandwidth, and reliability requirements for a wide range of system parameters. Our results thus establish the capacity of such systems. Our codes for security from active adversaries provide an additional appealing feature of `on-demand security' where the desired level of security can be chosen separately for each instance of repair, and our algorithms remain optimal simultaneously for all possible levels. The paper also provides necessary and sufficient conditions governing the transformation of any (non-secure) code into one providing on-demand security. △ Less

Submitted 15 August, 2015; originally announced August 2015.

Estimation from Pairwise Comparisons: Sharp Minimax Bounds with Topology Dependence

Authors: Nihar B. Shah, Sivaraman Balakrishnan, Joseph Bradley, Abhay Parekh, Kannan Ramchandran, Martin J. Wainwright

Abstract: Data in the form of pairwise comparisons arises in many domains, including preference elicitation, sporting competitions, and peer grading among others. We consider parametric ordinal models for such pairwise comparison data involving a latent vector $w^* \in \mathbb{R}^d$ that represents the "qualities" of the $d$ items being compared; this class of models includes the two most widely used parame… ▽ More Data in the form of pairwise comparisons arises in many domains, including preference elicitation, sporting competitions, and peer grading among others. We consider parametric ordinal models for such pairwise comparison data involving a latent vector $w^* \in \mathbb{R}^d$ that represents the "qualities" of the $d$ items being compared; this class of models includes the two most widely used parametric models--the Bradley-Terry-Luce (BTL) and the Thurstone models. Working within a standard minimax framework, we provide tight upper and lower bounds on the optimal error in estimating the quality score vector $w^*$ under this class of models. The bounds depend on the topology of the comparison graph induced by the subset of pairs being compared via its Laplacian spectrum. Thus, in settings where the subset of pairs may be chosen, our results provide principled guidelines for making this choice. Finally, we compare these error rates to those under cardinal measurement models and show that the error rates in the ordinal and cardinal settings have identical scalings apart from constant pre-factors. △ Less

Submitted 6 May, 2015; originally announced May 2015.

Comments: 39 pages, 5 figures. Significant extension of arXiv:1406.6618

Regularized Minimax Conditional Entropy for Crowdsourcing

Authors: Dengyong Zhou, Qiang Liu, John C. Platt, Christopher Meek, Nihar B. Shah

Abstract: There is a rapidly increasing interest in crowdsourcing for data labeling. By crowdsourcing, a large number of labels can be often quickly gathered at low cost. However, the labels provided by the crowdsourcing workers are usually not of high quality. In this paper, we propose a minimax conditional entropy principle to infer ground truth from noisy crowdsourced labels. Under this principle, we der… ▽ More There is a rapidly increasing interest in crowdsourcing for data labeling. By crowdsourcing, a large number of labels can be often quickly gathered at low cost. However, the labels provided by the crowdsourcing workers are usually not of high quality. In this paper, we propose a minimax conditional entropy principle to infer ground truth from noisy crowdsourced labels. Under this principle, we derive a unique probabilistic labeling model jointly parameterized by worker ability and item difficulty. We also propose an objective measurement principle, and show that our method is the only method which satisfies this objective measurement principle. We validate our method through a variety of real crowdsourcing datasets with binary, multiclass or ordinal labels. △ Less

Submitted 24 March, 2015; originally announced March 2015.

Comments: 31 pages

Approval Voting and Incentives in Crowdsourcing

Authors: Nihar B. Shah, Dengyong Zhou, Yuval Peres

Abstract: The growing need for labeled training data has made crowdsourcing an important part of machine learning. The quality of crowdsourced labels is, however, adversely affected by three factors: (1) the workers are not experts; (2) the incentives of the workers are not aligned with those of the requesters; and (3) the interface does not allow workers to convey their knowledge accurately, by forcing the… ▽ More The growing need for labeled training data has made crowdsourcing an important part of machine learning. The quality of crowdsourced labels is, however, adversely affected by three factors: (1) the workers are not experts; (2) the incentives of the workers are not aligned with those of the requesters; and (3) the interface does not allow workers to convey their knowledge accurately, by forcing them to make a single choice among a set of options. In this paper, we address these issues by introducing approval voting to utilize the expertise of workers who have partial knowledge of the true answer, and coupling it with a ("strictly proper") incentive-compatible compensation mechanism. We show rigorous theoretical guarantees of optimality of our mechanism together with a simple axiomatic characterization. We also conduct preliminary empirical studies on Amazon Mechanical Turk which validate our approach. △ Less

Submitted 7 September, 2015; v1 submitted 19 February, 2015; originally announced February 2015.

On the Impossibility of Convex Inference in Human Computation

Authors: Nihar B. Shah, Dengyong Zhou

Abstract: Human computation or crowdsourcing involves joint inference of the ground-truth-answers and the worker-abilities by optimizing an objective function, for instance, by maximizing the data likelihood based on an assumed underlying model. A variety of methods have been proposed in the literature to address this inference problem. As far as we know, none of the objective functions in existing methods… ▽ More Human computation or crowdsourcing involves joint inference of the ground-truth-answers and the worker-abilities by optimizing an objective function, for instance, by maximizing the data likelihood based on an assumed underlying model. A variety of methods have been proposed in the literature to address this inference problem. As far as we know, none of the objective functions in existing methods is convex. In machine learning and applied statistics, a convex function such as the objective function of support vector machines (SVMs) is generally preferred, since it can leverage the high-performance algorithms and rigorous guarantees established in the extensive literature on convex optimization. One may thus wonder if there exists a meaningful convex objective function for the inference problem in human computation. In this paper, we investigate this convexity issue for human computation. We take an axiomatic approach by formulating a set of axioms that impose two mild and natural assumptions on the objective function for the inference. Under these axioms, we show that it is unfortunately impossible to ensure convexity of the inference problem. On the other hand, we show that interestingly, in the absence of a requirement to model "spammers", one can construct reasonable objective functions for crowdsourcing that guarantee convex inference. △ Less

Submitted 21 November, 2014; originally announced November 2014.

Comments: AAAI 2015

Julia: A Fresh Approach to Numerical Computing

Authors: Jeff Bezanson, Alan Edelman, Stefan Karpinski, Viral B. Shah

Abstract: Bridging cultures that have often been distant, Julia combines expertise from the diverse fields of computer science and computational science to create a new approach to numerical computing. Julia is designed to be easy and fast. Julia questions notions generally held as "laws of nature" by practitioners of numerical computing: 1. High-level dynamic programs have to be slow. 2. One must proto… ▽ More Bridging cultures that have often been distant, Julia combines expertise from the diverse fields of computer science and computational science to create a new approach to numerical computing. Julia is designed to be easy and fast. Julia questions notions generally held as "laws of nature" by practitioners of numerical computing: 1. High-level dynamic programs have to be slow. 2. One must prototype in one language and then rewrite in another language for speed or deployment, and 3. There are parts of a system for the programmer, and other parts best left untouched as they are built by the experts. We introduce the Julia programming language and its design --- a dance between specialization and abstraction. Specialization allows for custom treatment. Multiple dispatch, a technique from computer science, picks the right algorithm for the right circumstance. Abstraction, what good computation is really about, recognizes what remains the same after differences are stripped away. Abstractions in mathematics are captured as code through another technique from computer science, generic programming. Julia shows that one can have machine performance without sacrificing human convenience. △ Less

Submitted 19 July, 2015; v1 submitted 6 November, 2014; originally announced November 2014.

Comments: 37 pages

Fundamental Limits on Communication for Oblivious Updates in Storage Networks

Authors: Preetum Nakkiran, Nihar B. Shah, K. V. Rashmi

Abstract: In distributed storage systems, storage nodes intermittently go offline for numerous reasons. On coming back online, nodes need to update their contents to reflect any modifications to the data in the interim. In this paper, we consider a setting where no information regarding modified data needs to be logged in the system. In such a setting, a 'stale' node needs to update its contents by download… ▽ More In distributed storage systems, storage nodes intermittently go offline for numerous reasons. On coming back online, nodes need to update their contents to reflect any modifications to the data in the interim. In this paper, we consider a setting where no information regarding modified data needs to be logged in the system. In such a setting, a 'stale' node needs to update its contents by downloading data from already updated nodes, while neither the stale node nor the updated nodes have any knowledge as to which data symbols are modified and what their value is. We investigate the fundamental limits on the amount of communication necessary for such an "oblivious" update process. We first present a generic lower bound on the amount of communication that is necessary under any storage code with a linear encoding (while allowing non-linear update protocols). This lower bound is derived under a set of extremely weak conditions, giving all updated nodes access to the entire modified data and the stale node access to the entire stale data as side information. We then present codes and update algorithms that are optimal in that they meet this lower bound. Next, we present a lower bound for an important subclass of codes, that of linear Maximum-Distance-Separable (MDS) codes. We then present an MDS code construction and an associated update algorithm that meets this lower bound. These results thus establish the capacity of oblivious updates in terms of the communication requirements under these settings. △ Less

Submitted 5 September, 2014; originally announced September 2014.

Comments: IEEE Global Communications Conference (GLOBECOM) 2014