Showing 1–20 of 20 results for author: McCarthy, J

Dynamic Operational Planning in Warfare: A Stochastic Game Approach to Military Campaigns

Authors: Joseph E. McCarthy, Mathieu Dahan, Chelsea C. White III

Abstract: We study a two-player discounted zero-sum stochastic game model for dynamic operational planning in military campaigns. At each stage, the players manage multiple commanders who order military actions on objectives that have an open line of control. When a battle over the control of an objective occurs, its stochastic outcome depends on the actions and the enabling support provided by the control… ▽ More We study a two-player discounted zero-sum stochastic game model for dynamic operational planning in military campaigns. At each stage, the players manage multiple commanders who order military actions on objectives that have an open line of control. When a battle over the control of an objective occurs, its stochastic outcome depends on the actions and the enabling support provided by the control of other objectives. Each player aims to maximize the cumulative number of objectives they control, weighted by their criticality. To solve this large-scale stochastic game, we derive properties of its Markov perfect equilibria by leveraging the logistics and military operational command and control structure. We show the consequential isotonicity of the optimal value function with respect to the partially ordered state space, which in turn leads to a significant reduction of the state and action spaces. We also accelerate Shapley's value iteration algorithm by eliminating dominated actions and investigating pure equilibria of the matrix game solved at each iteration. We demonstrate the computational value of our equilibrium results on a case study that reflects representative operational-level military campaigns with geopolitical implications. Our analysis reveals a complex interplay between the game's parameters and dynamics in equilibrium, resulting in new military insights for campaign analysts. △ Less

Submitted 1 March, 2024; originally announced March 2024.

Designing for Work with Intelligent Entities: A Review of Perspectives

Authors: James E. McCarthy

Abstract: As the power of Artificial Intelligence (AI) continues to advance, there is increased interest in how best to combine AI-based agents with humans to achieve mission effectiveness. Three perspectives have emerged. The first stems from more conventional human factors traditions and views these entities as highly capable tools that humans can use to accomplish increasingly sophisticated tasks. The se… ▽ More As the power of Artificial Intelligence (AI) continues to advance, there is increased interest in how best to combine AI-based agents with humans to achieve mission effectiveness. Three perspectives have emerged. The first stems from more conventional human factors traditions and views these entities as highly capable tools that humans can use to accomplish increasingly sophisticated tasks. The second "camp" believes that as the sophistication of these entities increases, it becomes increasingly appropriate to talk about them as "teammates" and use the research on human teams as a foundation for further exploration. The third perspective is emerging and finds both the "tools" and "teammate" metaphors flawed and limiting. This perspective emphasizes "joint activity," "joint cognitive activity," or something similar. In this article, we briefly review these three perspectives. △ Less

Submitted 20 February, 2024; v1 submitted 10 February, 2024; originally announced February 2024.

Mixture of Gaussian-distributed Prototypes with Generative Modelling for Interpretable and Trustworthy Image Recognition

Authors: Chong Wang, Yuanhong Chen, Fengbei Liu, Yuyuan Liu, Davis James McCarthy, Helen Frazer, Gustavo Carneiro

Abstract: Prototypical-part methods, e.g., ProtoPNet, enhance interpretability in image recognition by linking predictions to training prototypes, thereby offering intuitive insights into their decision-making. Existing methods, which rely on a point-based learning of prototypes, typically face two critical issues: 1) the learned prototypes have limited representation power and are not suitable to detect Ou… ▽ More Prototypical-part methods, e.g., ProtoPNet, enhance interpretability in image recognition by linking predictions to training prototypes, thereby offering intuitive insights into their decision-making. Existing methods, which rely on a point-based learning of prototypes, typically face two critical issues: 1) the learned prototypes have limited representation power and are not suitable to detect Out-of-Distribution (OoD) inputs, reducing their decision trustworthiness; and 2) the necessary projection of the learned prototypes back into the space of training images causes a drastic degradation in the predictive performance. Furthermore, current prototype learning adopts an aggressive approach that considers only the most active object parts during training, while overlooking sub-salient object regions which still hold crucial classification information. In this paper, we present a new generative paradigm to learn prototype distributions, termed as Mixture of Gaussian-distributed Prototypes (MGProto). The distribution of prototypes from MGProto enables both interpretable image classification and trustworthy recognition of OoD inputs. The optimisation of MGProto naturally projects the learned prototype distributions back into the training image space, thereby addressing the performance degradation caused by prototype projection. Additionally, we develop a novel and effective prototype mining strategy that considers not only the most active but also sub-salient object parts. To promote model compactness, we further propose to prune MGProto by removing prototypes with low importance priors. Experiments on CUB-200-2011, Stanford Cars, Stanford Dogs, and Oxford-IIIT Pets datasets show that MGProto achieves state-of-the-art image recognition and OoD detection performances, while providing encouraging interpretability results. △ Less

Submitted 5 June, 2024; v1 submitted 30 November, 2023; originally announced December 2023.

Improving the State of the Art for Training Human-AI Teams: Technical Report #3 -- Analysis of Testbed Alternatives

Authors: Lillian Asiala, James E. McCarthy, Lixiao Huang

Abstract: Sonalysts is working on an initiative to expand our current expertise in teaming to Human-Artificial Intelligence (AI) teams by develo** original research in this area. To provide a foundation for that research, Sonalysts is investigating the development of a Synthetic Task Environment (STE). In a previous report, we documented the findings of a recent outreach effort in which we asked military… ▽ More Sonalysts is working on an initiative to expand our current expertise in teaming to Human-Artificial Intelligence (AI) teams by develo** original research in this area. To provide a foundation for that research, Sonalysts is investigating the development of a Synthetic Task Environment (STE). In a previous report, we documented the findings of a recent outreach effort in which we asked military Subject Matter Experts (SMEs) and other researchers in the Human-AI teaming domain to identify the qualities that they most valued in a testbed. A surprising finding from that outreach was that several respondents recommended that our team look into existing human-AI teaming testbeds, rather than creating something new. Based on that recommendation, we conducted a systematic investigation of the associated landscape. In this report, we describe the results of that investigation. Building on the survey results, we developed testbed evaluation criteria, identified potential testbeds, and conducted qualitative and quantitative evaluations of candidate testbeds. The evaluation process led to five candidate testbeds for the research team to consider. In the coming months, we will assess the viability of the various alternatives and begin to execute our program of research. △ Less

Submitted 29 August, 2023; originally announced September 2023.

Comments: June 2023, Drs. Asiala and McCarthy are affiliated with Sonalysts, Dr. Huang is affiliated with Arizona State University

Improving the State of the Art for Training Human-AI Teams: Technical Report #2 -- Results of Researcher Knowledge Elicitation Survey

Authors: James E. McCarthy, Lillian Asiala, LeeAnn Maryeski, Dawn Sillars

Abstract: A consensus report produced for the Air Force Research Laboratory (AFRL) by the National Academies of Sciences, Engineering, and Mathematics documented a prevalent and increasing desire to support human-Artificial Intelligence (AI) teaming across military service branches. Sonalysts has begun an internal initiative to explore the training of Human-AI teams. The first step in this effort is to deve… ▽ More A consensus report produced for the Air Force Research Laboratory (AFRL) by the National Academies of Sciences, Engineering, and Mathematics documented a prevalent and increasing desire to support human-Artificial Intelligence (AI) teaming across military service branches. Sonalysts has begun an internal initiative to explore the training of Human-AI teams. The first step in this effort is to develop a Synthetic Task Environment (STE) that is capable of facilitating research on Human-AI teams. Our goal is to create a STE that offers a task environment that could support the breadth of research that stakeholders plan to perform within this domain. As a result, we wanted to sample the priorities of the relevant research community broadly, and the effort documented in this report is our initial attempt to do so. We created a survey that featured two types of questions. The first asked respondents to report their agreement with STE features that we anticipated might be important. The second represented open-ended questions that asked respondents to specify their priorities within several dimensions of the anticipated STE. The research team invited nineteen researchers from academic and Government labs to participate, and 11 were able to complete the survey. The team analyzed their responses to identify themes that emerged and topics that would benefit from further analysis. The most significant finding of the survey was that a number of researchers felt that various open-source STEs that would meet our needs already exist. Researchers also emphasized the need for automated transcription and coding tools to ease the burden of assessing inter-team communications; the importance of robust data capture and export capabilities; and the desirability of extensive flexibility across many aspects of the tool. △ Less

Submitted 29 August, 2023; originally announced September 2023.

Comments: March 2023

Improving the State of the Art for Training Human-AI Teams: Technical Report #1 -- Results of Subject-Matter Expert Knowledge Elicitation Survey

Authors: James E. McCarthy, Lillian Asiala, LeeAnn Maryeski, Nyla Warren

Abstract: A consensus report produced for the Air Force Research Laboratory by the National Academies of Sciences, Engineering, and Mathematics documented a prevalent and increasing desire to support human-Artificial Intelligence (AI) teaming across military service branches. Sonalysts has begun an internal initiative to explore the training of human-AI teams. The first step in this effort is to develop a S… ▽ More A consensus report produced for the Air Force Research Laboratory by the National Academies of Sciences, Engineering, and Mathematics documented a prevalent and increasing desire to support human-Artificial Intelligence (AI) teaming across military service branches. Sonalysts has begun an internal initiative to explore the training of human-AI teams. The first step in this effort is to develop a Synthetic Task Environment (STE) that is capable of facilitating research on human-AI teams. We decided to use Joint All-Domain Command and Control (JADC2) as a focus point for develo** the STE because the volume of sensor inputs and decision options within the JADC2 concept likely requires the use of AI systems to enable timely decisions. Given this focus, we engaged a number of Subject-Matter Experts (SMEs) with Command and Control experience to gain insight into develo** a STE that embodied the teaming challenges associated with JADC2. This report documents our initial engagement with those stakeholders. The research team identified thirteen Sonalysts employees with military backgrounds and Command and Control experience, and invited them to participate. Twelve respondents completed the survey. The team then analyzed the responses to identify themes that emerged and topics that would benefit from further analysis. The results indicated that our SMEs were amenable to research using tasks that were analogous to those encountered in military environments, as long as they required teams to process a great deal of incoming data to arrive at complex decisions. The SMEs felt that the testbed should support 'teams of teams" that represented a matrixed organization, and that it should support a robust array to spoken, text-based, and face-to-face communications. △ Less

Submitted 29 August, 2023; originally announced September 2023.

Comments: March 2023

Iterative Reward Sha** using Human Feedback for Correcting Reward Misspecification

Authors: Jasmina Gajcin, James McCarthy, Rahul Nair, Radu Marinescu, Elizabeth Daly, Ivana Dusparic

Abstract: A well-defined reward function is crucial for successful training of an reinforcement learning (RL) agent. However, defining a suitable reward function is a notoriously challenging task, especially in complex, multi-objective environments. Developers often have to resort to starting with an initial, potentially misspecified reward function, and iteratively adjusting its parameters, based on observ… ▽ More A well-defined reward function is crucial for successful training of an reinforcement learning (RL) agent. However, defining a suitable reward function is a notoriously challenging task, especially in complex, multi-objective environments. Developers often have to resort to starting with an initial, potentially misspecified reward function, and iteratively adjusting its parameters, based on observed learned behavior. In this work, we aim to automate this process by proposing ITERS, an iterative reward sha** approach using human feedback for mitigating the effects of a misspecified reward function. Our approach allows the user to provide trajectory-level feedback on agent's behavior during training, which can be integrated as a reward sha** signal in the following training iteration. We also allow the user to provide explanations of their feedback, which are used to augment the feedback and reduce user effort and feedback frequency. We evaluate ITERS in three environments and show that it can successfully correct misspecified reward functions. △ Less

Submitted 30 August, 2023; originally announced August 2023.

Comments: 7 pages, 2 figures

Sustainable Adaptive Security

Authors: Liliana Pasquale, Kushal Ramkumar, Wanling Cai, John McCarthy, Gavin Doherty, Bashar Nuseibeh

Abstract: With software systems permeating our lives, we are entitled to expect that such systems are secure by design, and that such security endures throughout the use of these systems and their subsequent evolution. Although adaptive security systems have been proposed to continuously protect assets from harm, they can only mitigate threats arising from changes foreseen at design time. In this paper, we… ▽ More With software systems permeating our lives, we are entitled to expect that such systems are secure by design, and that such security endures throughout the use of these systems and their subsequent evolution. Although adaptive security systems have been proposed to continuously protect assets from harm, they can only mitigate threats arising from changes foreseen at design time. In this paper, we propose the notion of Sustainable Adaptive Security (SAS) which reflects such enduring protection by augmenting adaptive security systems with the capability of mitigating newly discovered threats. To achieve this objective, a SAS system should be designed by combining automation (e.g., to discover and mitigate security threats) and human intervention (e.g., to resolve uncertainties during threat discovery and mitigation). In this paper, we use a smart home example to showcase how we can engineer the activities of the MAPE (Monitor, Analysis, Planning, and Execution) loop of systems satisfying sustainable adaptive security. We suggest that using anomaly detection together with abductive reasoning can help discover new threats and guide the evolution of security requirements and controls. We also exemplify situations when humans can be involved in the execution of the activities of the MAPE loop and discuss the requirements to engineer human interventions. △ Less

Submitted 5 June, 2023; originally announced June 2023.

Technical outlier detection via convolutional variational autoencoder for the ADMANI breast mammogram dataset

Authors: Hui Li, Carlos A. Pena Solorzano, Susan Wei, Davis J. McCarthy

Abstract: The ADMANI datasets (annotated digital mammograms and associated non-image datasets) from the Transforming Breast Cancer Screening with AI programme (BRAIx) run by BreastScreen Victoria in Australia are multi-centre, large scale, clinically curated, real-world databases. The datasets are expected to aid in the development of clinically relevant Artificial Intelligence (AI) algorithms for breast ca… ▽ More The ADMANI datasets (annotated digital mammograms and associated non-image datasets) from the Transforming Breast Cancer Screening with AI programme (BRAIx) run by BreastScreen Victoria in Australia are multi-centre, large scale, clinically curated, real-world databases. The datasets are expected to aid in the development of clinically relevant Artificial Intelligence (AI) algorithms for breast cancer detection, early diagnosis, and other applications. To ensure high data quality, technical outliers must be removed before any downstream algorithm development. As a first step, we randomly select 30,000 individual mammograms and use Convolutional Variational Autoencoder (CVAE), a deep generative neural network, to detect outliers. CVAE is expected to detect all sorts of outliers, although its detection performance differs among different types of outliers. Traditional image processing techniques such as erosion and pectoral muscle analysis can compensate for the poor performance of CVAE in certain outlier types. We identify seven types of technical outliers: implant, pacemaker, cardiac loop recorder, improper radiography, atypical lesion/calcification, incorrect exposure parameter and improper placement. The outlier recall rate for the test set is 61% if CVAE, erosion and pectoral muscle analysis each select the top 1% images ranked in ascending or descending order according to image outlier score under each detection method, and 83% if each selects the top 5% images. This study offers an overview of technical outliers in the ADMANI dataset and suggests future directions to improve outlier detection effectiveness. △ Less

Submitted 19 May, 2023; originally announced May 2023.

BRAIxDet: Learning to Detect Malignant Breast Lesion with Incomplete Annotations

Authors: Yuanhong Chen, Yuyuan Liu, Chong Wang, Michael Elliott, Chun Fung Kwok, Carlos Pena-Solorzano, Yu Tian, Fengbei Liu, Helen Frazer, Davis J. McCarthy, Gustavo Carneiro

Abstract: Methods to detect malignant lesions from screening mammograms are usually trained with fully annotated datasets, where images are labelled with the localisation and classification of cancerous lesions. However, real-world screening mammogram datasets commonly have a subset that is fully annotated and another subset that is weakly annotated with just the global classification (i.e., without lesion… ▽ More Methods to detect malignant lesions from screening mammograms are usually trained with fully annotated datasets, where images are labelled with the localisation and classification of cancerous lesions. However, real-world screening mammogram datasets commonly have a subset that is fully annotated and another subset that is weakly annotated with just the global classification (i.e., without lesion localisation). Given the large size of such datasets, researchers usually face a dilemma with the weakly annotated subset: to not use it or to fully annotate it. The first option will reduce detection accuracy because it does not use the whole dataset, and the second option is too expensive given that the annotation needs to be done by expert radiologists. In this paper, we propose a middle-ground solution for the dilemma, which is to formulate the training as a weakly- and semi-supervised learning problem that we refer to as malignant breast lesion detection with incomplete annotations. To address this problem, our new method comprises two stages, namely: 1) pre-training a multi-view mammogram classifier with weak supervision from the whole dataset, and 2) extending the trained classifier to become a multi-view detector that is trained with semi-supervised student-teacher learning, where the training set contains fully and weakly-annotated mammograms. We provide extensive detection results on two real-world screening mammogram datasets containing incomplete annotations, and show that our proposed approach achieves state-of-the-art results in the detection of malignant breast lesions with incomplete annotations. △ Less

Submitted 2 April, 2024; v1 submitted 31 January, 2023; originally announced January 2023.

Comments: Under Review

Learning Support and Trivial Prototypes for Interpretable Image Classification

Authors: Chong Wang, Yuyuan Liu, Yuanhong Chen, Fengbei Liu, Yu Tian, Davis J. McCarthy, Helen Frazer, Gustavo Carneiro

Abstract: Prototypical part network (ProtoPNet) methods have been designed to achieve interpretable classification by associating predictions with a set of training prototypes, which we refer to as trivial prototypes because they are trained to lie far from the classification boundary in the feature space. Note that it is possible to make an analogy between ProtoPNet and support vector machine (SVM) given t… ▽ More Prototypical part network (ProtoPNet) methods have been designed to achieve interpretable classification by associating predictions with a set of training prototypes, which we refer to as trivial prototypes because they are trained to lie far from the classification boundary in the feature space. Note that it is possible to make an analogy between ProtoPNet and support vector machine (SVM) given that the classification from both methods relies on computing similarity with a set of training points (i.e., trivial prototypes in ProtoPNet, and support vectors in SVM). However, while trivial prototypes are located far from the classification boundary, support vectors are located close to this boundary, and we argue that this discrepancy with the well-established SVM theory can result in ProtoPNet models with inferior classification accuracy. In this paper, we aim to improve the classification of ProtoPNet with a new method to learn support prototypes that lie near the classification boundary in the feature space, as suggested by the SVM theory. In addition, we target the improvement of classification results with a new model, named ST-ProtoPNet, which exploits our support prototypes and the trivial prototypes to provide more effective classification. Experimental results on CUB-200-2011, Stanford Cars, and Stanford Dogs datasets demonstrate that ST-ProtoPNet achieves state-of-the-art classification accuracy and interpretability results. We also show that the proposed support prototypes tend to be better localised in the object of interest rather than in the background region. △ Less

Submitted 22 October, 2023; v1 submitted 8 January, 2023; originally announced January 2023.

Comments: ICCV 2023, Code: https://github.com/cwangrun/ST-ProtoPNet

Knowledge Distillation to Ensemble Global and Interpretable Prototype-Based Mammogram Classification Models

Authors: Chong Wang, Yuanhong Chen, Yuyuan Liu, Yu Tian, Fengbei Liu, Davis J. McCarthy, Michael Elliott, Helen Frazer, Gustavo Carneiro

Abstract: State-of-the-art (SOTA) deep learning mammogram classifiers, trained with weakly-labelled images, often rely on global models that produce predictions with limited interpretability, which is a key barrier to their successful translation into clinical practice. On the other hand, prototype-based models improve interpretability by associating predictions with training image prototypes, but they are… ▽ More State-of-the-art (SOTA) deep learning mammogram classifiers, trained with weakly-labelled images, often rely on global models that produce predictions with limited interpretability, which is a key barrier to their successful translation into clinical practice. On the other hand, prototype-based models improve interpretability by associating predictions with training image prototypes, but they are less accurate than global models and their prototypes tend to have poor diversity. We address these two issues with the proposal of BRAIxProtoPNet++, which adds interpretability to a global model by ensembling it with a prototype-based model. BRAIxProtoPNet++ distills the knowledge of the global model when training the prototype-based model with the goal of increasing the classification accuracy of the ensemble. Moreover, we propose an approach to increase prototype diversity by guaranteeing that all prototypes are associated with different training images. Experiments on weakly-labelled private and public datasets show that BRAIxProtoPNet++ has higher classification accuracy than SOTA global and prototype-based models. Using lesion localisation to assess model interpretability, we show BRAIxProtoPNet++ is more effective than other prototype-based models and post-hoc explanation of global models. Finally, we show that the diversity of the prototypes learned by BRAIxProtoPNet++ is superior to SOTA prototype-based approaches. △ Less

Submitted 8 January, 2023; v1 submitted 26 September, 2022; originally announced September 2022.

Comments: MICCAI 2022

Multi-view Local Co-occurrence and Global Consistency Learning Improve Mammogram Classification Generalisation

Authors: Yuanhong Chen, Hu Wang, Chong Wang, Yu Tian, Fengbei Liu, Michael Elliott, Davis J. McCarthy, Helen Frazer, Gustavo Carneiro

Abstract: When analysing screening mammograms, radiologists can naturally process information across two ipsilateral views of each breast, namely the cranio-caudal (CC) and mediolateral-oblique (MLO) views. These multiple related images provide complementary diagnostic information and can improve the radiologist's classification accuracy. Unfortunately, most existing deep learning systems, trained with glob… ▽ More When analysing screening mammograms, radiologists can naturally process information across two ipsilateral views of each breast, namely the cranio-caudal (CC) and mediolateral-oblique (MLO) views. These multiple related images provide complementary diagnostic information and can improve the radiologist's classification accuracy. Unfortunately, most existing deep learning systems, trained with globally-labelled images, lack the ability to jointly analyse and integrate global and local information from these multiple views. By ignoring the potentially valuable information present in multiple images of a screening episode, one limits the potential accuracy of these systems. Here, we propose a new multi-view global-local analysis method that mimics the radiologist's reading procedure, based on a global consistency learning and local co-occurrence learning of ipsilateral views in mammograms. Extensive experiments show that our model outperforms competing methods, in terms of classification accuracy and generalisation, on a large-scale private dataset and two publicly available datasets, where models are exclusively trained and tested with global labels. △ Less

Submitted 21 September, 2022; originally announced September 2022.

Comments: MICCAI 2022

Boolean Decision Rules for Reinforcement Learning Policy Summarisation

Authors: James McCarthy, Rahul Nair, Elizabeth Daly, Radu Marinescu, Ivana Dusparic

Abstract: Explainability of Reinforcement Learning (RL) policies remains a challenging research problem, particularly when considering RL in a safety context. Understanding the decisions and intentions of an RL policy offer avenues to incorporate safety into the policy by limiting undesirable actions. We propose the use of a Boolean Decision Rules model to create a post-hoc rule-based summary of an agent's… ▽ More Explainability of Reinforcement Learning (RL) policies remains a challenging research problem, particularly when considering RL in a safety context. Understanding the decisions and intentions of an RL policy offer avenues to incorporate safety into the policy by limiting undesirable actions. We propose the use of a Boolean Decision Rules model to create a post-hoc rule-based summary of an agent's policy. We evaluate our proposed approach using a DQN agent trained on an implementation of a lava gridworld and show that, given a hand-crafted feature representation of this gridworld, simple generalised rules can be created, giving a post-hoc explainable summary of the agent's policy. We discuss possible avenues to introduce safety into a RL agent's policy by using rules generated by this rule-based model as constraints imposed on the agent's policy, as well as discuss how creating simple rule summaries of an agent's policy may help in the debugging process of RL agents. △ Less

Submitted 18 July, 2022; originally announced July 2022.

N-PAT: A Nested Model-Checker

Authors: Hadrien Bride, Cheng-Hao Cai, ** Song Dong, Rajeev Gore, Zhé Hóu, Brendan Mahony, Jim McCarthy

Abstract: N-PAT is a new model-checking tool that supports the verification of nested-models, i.e. models whose behaviour depends on the results of verification tasks. In this paper, we describe its operation and discuss mechanisms that are tailored to the efficient verification of nested-models. Further, we motivate the advantages of N-PAT over traditional model-checking tools through a network security ca… ▽ More N-PAT is a new model-checking tool that supports the verification of nested-models, i.e. models whose behaviour depends on the results of verification tasks. In this paper, we describe its operation and discuss mechanisms that are tailored to the efficient verification of nested-models. Further, we motivate the advantages of N-PAT over traditional model-checking tools through a network security case study. △ Less

Submitted 11 May, 2020; originally announced May 2020.

A Simple and Effective Model-Based Variable Importance Measure

Authors: Brandon M. Greenwell, Bradley C. Boehmke, Andrew J. McCarthy

Abstract: In the era of "big data", it is becoming more of a challenge to not only build state-of-the-art predictive models, but also gain an understanding of what's really going on in the data. For example, it is often of interest to know which, if any, of the predictors in a fitted model are relatively influential on the predicted outcome. Some modern algorithms---like random forests and gradient boosted… ▽ More In the era of "big data", it is becoming more of a challenge to not only build state-of-the-art predictive models, but also gain an understanding of what's really going on in the data. For example, it is often of interest to know which, if any, of the predictors in a fitted model are relatively influential on the predicted outcome. Some modern algorithms---like random forests and gradient boosted decision trees---have a natural way of quantifying the importance or relative influence of each feature. Other algorithms---like naive Bayes classifiers and support vector machines---are not capable of doing so and model-free approaches are generally used to measure each predictor's importance. In this paper, we propose a standardized, model-based approach to measuring predictor importance across the growing spectrum of supervised learning algorithms. Our proposed method is illustrated through both simulated and real data examples. The R code to reproduce all of the figures in this paper is available in the supplementary materials. △ Less

Submitted 12 May, 2018; originally announced May 2018.

Proceedings of the 4th and 5th International Workshop on Trends in Functional Programming in Education

Authors: Johan Jeuring, Jay McCarthy

Abstract: This volume contains the proceedings of the Fourth and Fifth International Workshops on Trends in Functional Programming in Education, TFPIE 2015 and TFPIE 2016, which were held on June 2, 2015 in Sophia-Antipolis, France, and on June 7, 2016 at the University of Maryland College Park in the USA, respectively. This volume contains the proceedings of the Fourth and Fifth International Workshops on Trends in Functional Programming in Education, TFPIE 2015 and TFPIE 2016, which were held on June 2, 2015 in Sophia-Antipolis, France, and on June 7, 2016 at the University of Maryland College Park in the USA, respectively. △ Less

Submitted 25 November, 2016; originally announced November 2016.

Journal ref: EPTCS 230, 2016

Rooted Cycle Bases

Authors: David Eppstein, J. Michael McCarthy, Brian E. Parrish

Abstract: A cycle basis in an undirected graph is a minimal set of simple cycles whose symmetric differences include all Eulerian subgraphs of the given graph. We define a rooted cycle basis to be a cycle basis in which all cycles contain a specified root edge, and we investigate the algorithmic problem of constructing rooted cycle bases. We show that a given graph has a rooted cycle basis if and only if th… ▽ More A cycle basis in an undirected graph is a minimal set of simple cycles whose symmetric differences include all Eulerian subgraphs of the given graph. We define a rooted cycle basis to be a cycle basis in which all cycles contain a specified root edge, and we investigate the algorithmic problem of constructing rooted cycle bases. We show that a given graph has a rooted cycle basis if and only if the root edge belongs to its 2-core and the 2-core is 2-vertex-connected, and that constructing such a basis can be performed efficiently. We show that in an unweighted or positively weighted graph, it is possible to find the minimum weight rooted cycle basis in polynomial time. Additionally, we show that it is NP-complete to find a fundamental rooted cycle basis (a rooted cycle basis in which each cycle is formed by combining paths in a fixed spanning tree with a single additional edge) but that the problem can be solved by a fixed-parameter-tractable algorithm when parameterized by clique-width. △ Less

Submitted 20 April, 2015; originally announced April 2015.

Comments: 12 pages with 10 additional pages of appendices and 10 figures. Extended version of a paper to appear at the 14th Algorithms and Data Structures Symposium (WADS), Victoria, BC, August 2015

ACM Class: F.2.2

Running Probabilistic Programs Backwards

Authors: Neil Toronto, Jay McCarthy, David Van Horn

Abstract: Many probabilistic programming languages allow programs to be run under constraints in order to carry out Bayesian inference. Running programs under constraints could enable other uses such as rare event simulation and probabilistic verification---except that all such probabilistic languages are necessarily limited because they are defined or implemented in terms of an impoverished theory of proba… ▽ More Many probabilistic programming languages allow programs to be run under constraints in order to carry out Bayesian inference. Running programs under constraints could enable other uses such as rare event simulation and probabilistic verification---except that all such probabilistic languages are necessarily limited because they are defined or implemented in terms of an impoverished theory of probability. Measure-theoretic probability provides a more general foundation, but its generality makes finding computational content difficult. We develop a measure-theoretic semantics for a first-order probabilistic language with recursion, which interprets programs as functions that compute preimages. Preimage functions are generally uncomputable, so we derive an abstract semantics. We implement the abstract semantics and use the implementation to carry out Bayesian inference, stochastic ray tracing (a rare event simulation), and probabilistic verification of floating-point error bounds. △ Less

Submitted 16 January, 2015; v1 submitted 12 December, 2014; originally announced December 2014.

Comments: 26 pages, ESOP 2015 (to appear)

Using Decision Trees for Coreference Resolution

Authors: Joseph F. McCarthy, Wendy G. Lehnert

Abstract: This paper describes RESOLVE, a system that uses decision trees to learn how to classify coreferent phrases in the domain of business joint ventures. An experiment is presented in which the performance of RESOLVE is compared to the performance of a manually engineered set of rules for the same task. The results show that decision trees achieve higher performance than the rules in two of three ev… ▽ More This paper describes RESOLVE, a system that uses decision trees to learn how to classify coreferent phrases in the domain of business joint ventures. An experiment is presented in which the performance of RESOLVE is compared to the performance of a manually engineered set of rules for the same task. The results show that decision trees achieve higher performance than the rules in two of three evaluation metrics developed for the coreference task. In addition to achieving better performance than the rules, RESOLVE provides a framework that facilitates the exploration of the types of knowledge that are useful for solving the coreference problem. △ Less

Submitted 24 May, 1995; originally announced May 1995.

Comments: 6 pages; LaTeX source; 1 uuencoded compressed EPS file (separate); uses ijcai95.sty, named.bst, epsf.tex; to appear in Proc. IJCAI '95