Amazon SageMaker Clarify: Machine Learning Bias Detection and Explainability in the Cloud
Authors:
Michaela Hardt,
Xiaoguang Chen,
Xiaoyi Cheng,
Michele Donini,
Jason Gelman,
Satish Gollaprolu,
John He,
Pedro Larroy,
Xinyu Liu,
Nick McCarthy,
Ashish Rathi,
Scott Rees,
Ankit Siva,
ErhYuan Tsai,
Keerthan Vasist,
Pinar Yilmaz,
Muhammad Bilal Zafar,
Sanjiv Das,
Kevin Haas,
Tyler Hill,
Krishnaram Kenthapadi
Abstract:
Understanding the predictions made by machine learning (ML) models and their potential biases remains a challenging and labor-intensive task that depends on the application, the dataset, and the specific model. We present Amazon SageMaker Clarify, an explainability feature for Amazon SageMaker that launched in December 2020, providing insights into data and ML models by identifying biases and expl…
▽ More
Understanding the predictions made by machine learning (ML) models and their potential biases remains a challenging and labor-intensive task that depends on the application, the dataset, and the specific model. We present Amazon SageMaker Clarify, an explainability feature for Amazon SageMaker that launched in December 2020, providing insights into data and ML models by identifying biases and explaining predictions. It is deeply integrated into Amazon SageMaker, a fully managed service that enables data scientists and developers to build, train, and deploy ML models at any scale. Clarify supports bias detection and feature importance computation across the ML lifecycle, during data preparation, model evaluation, and post-deployment monitoring. We outline the desiderata derived from customer input, the modular architecture, and the methodology for bias and explanation computations. Further, we describe the technical challenges encountered and the tradeoffs we had to make. For illustration, we discuss two customer use cases. We present our deployment results including qualitative customer feedback and a quantitative evaluation. Finally, we summarize lessons learned, and discuss best practices for the successful adoption of fairness and explanation tools in practice.
△ Less
Submitted 7 September, 2021;
originally announced September 2021.
Dynamic Prioritization for Conflict-Free Path Planning of Multi-Robot Systems
Authors:
Aditya Rathi,
Rohith G,
Madhu Vadali
Abstract:
Planning collision-free paths for multi-robot systems (MRS) is a challenging problem because of the safety and efficiency constraints required for real-world solutions. Even though coupled path planning approaches provide optimal collision-free paths for each agent of the MRS, they search the composite space of all the agents and therefore, suffer from exponential increase in computation with the…
▽ More
Planning collision-free paths for multi-robot systems (MRS) is a challenging problem because of the safety and efficiency constraints required for real-world solutions. Even though coupled path planning approaches provide optimal collision-free paths for each agent of the MRS, they search the composite space of all the agents and therefore, suffer from exponential increase in computation with the number of robots. On the other hand, prioritized approaches provide a practical solution to applications with large number of robots, especially when path computation time and collision avoidance take precedence over guaranteed globally optimal solution. While most centrally-planned algorithms use static prioritization, a dynamic prioritization algorithm, PD*, is proposed that employs a novel metric, called freedom index, to decide the priority order of the robots at each time step. This allows the PD* algorithm to simultaneously plan the next step for all robots while ensuring collision-free operation in obstacle ridden environments. Extensive simulations were performed to test and compare the performance of the proposed PD* scheme with other state-of-the-art algorithms. It was found that PD* improves upon the computational time by 25% while providing solutions of similar path lengths. Increase in efficiency was particularly prominent in scenarios with large number of robots and/or higher obstacle densities, where the probability of collisions is higher, suggesting the suitability of PD* in solving such problems.
△ Less
Submitted 6 January, 2021;
originally announced January 2021.
