-
Towards a Responsible AI Metrics Catalogue: A Collection of Metrics for AI Accountability
Authors:
Boming Xia,
Qinghua Lu,
Liming Zhu,
Sung Une Lee,
Yue Liu,
Zhenchang Xing
Abstract:
Artificial Intelligence (AI), particularly through the advent of large-scale generative AI (GenAI) models such as Large Language Models (LLMs), has become a transformative element in contemporary technology. While these models have unlocked new possibilities, they simultaneously present significant challenges, such as concerns over data privacy and the propensity to generate misleading or fabricat…
▽ More
Artificial Intelligence (AI), particularly through the advent of large-scale generative AI (GenAI) models such as Large Language Models (LLMs), has become a transformative element in contemporary technology. While these models have unlocked new possibilities, they simultaneously present significant challenges, such as concerns over data privacy and the propensity to generate misleading or fabricated content. Current frameworks for Responsible AI (RAI) often fall short in providing the granular guidance necessary for tangible application, especially for Accountability-a principle that is pivotal for ensuring transparent and auditable decision-making, bolstering public trust, and meeting increasing regulatory expectations. This study bridges the accountability gap by introducing our effort towards a comprehensive metrics catalogue, formulated through a systematic multivocal literature review (MLR) that integrates findings from both academic and grey literature. Our catalogue delineates process metrics that underpin procedural integrity, resource metrics that provide necessary tools and frameworks, and product metrics that reflect the outputs of AI systems. This tripartite framework is designed to operationalize Accountability in AI, with a special emphasis on addressing the intricacies of GenAI.
△ Less
Submitted 17 January, 2024; v1 submitted 21 November, 2023;
originally announced November 2023.
-
QB4AIRA: A Question Bank for AI Risk Assessment
Authors:
Sung Une Lee,
Harsha Perera,
Boming Xia,
Yue Liu,
Qinghua Lu,
Liming Zhu,
Olivier Salvado,
Jon Whittle
Abstract:
The rapid advancement of Artificial Intelligence (AI), represented by ChatGPT, has raised concerns about responsible AI development and utilization. Existing frameworks lack a comprehensive synthesis of AI risk assessment questions. To address this, we introduce QB4AIRA, a novel question bank developed by refining questions from five globally recognized AI risk frameworks, categorized according to…
▽ More
The rapid advancement of Artificial Intelligence (AI), represented by ChatGPT, has raised concerns about responsible AI development and utilization. Existing frameworks lack a comprehensive synthesis of AI risk assessment questions. To address this, we introduce QB4AIRA, a novel question bank developed by refining questions from five globally recognized AI risk frameworks, categorized according to Australia's AI ethics principles. QB4AIRA comprises 293 prioritized questions covering a wide range of AI risk areas, facilitating effective risk assessment. It serves as a valuable resource for stakeholders in assessing and managing AI risks, while paving the way for new risk frameworks and guidelines. By promoting responsible AI practices, QB4AIRA contributes to responsible AI deployment, mitigating potential risks and harms.
△ Less
Submitted 10 July, 2023; v1 submitted 16 May, 2023;
originally announced May 2023.
-
Driving in Real Life with Inverse Reinforcement Learning
Authors:
Tung Phan-Minh,
Forbes Howington,
Ting-Sheng Chu,
Sang Uk Lee,
Momchil S. Tomov,
Nanxiang Li,
Caglayan Dicle,
Samuel Findler,
Francisco Suarez-Ruiz,
Robert Beaudoin,
Bo Yang,
Sammy Omari,
Eric M. Wolff
Abstract:
In this paper, we introduce the first learning-based planner to drive a car in dense, urban traffic using Inverse Reinforcement Learning (IRL). Our planner, DriveIRL, generates a diverse set of trajectory proposals, filters these trajectories with a lightweight and interpretable safety filter, and then uses a learned model to score each remaining trajectory. The best trajectory is then tracked by…
▽ More
In this paper, we introduce the first learning-based planner to drive a car in dense, urban traffic using Inverse Reinforcement Learning (IRL). Our planner, DriveIRL, generates a diverse set of trajectory proposals, filters these trajectories with a lightweight and interpretable safety filter, and then uses a learned model to score each remaining trajectory. The best trajectory is then tracked by the low-level controller of our self-driving vehicle. We train our trajectory scoring model on a 500+ hour real-world dataset of expert driving demonstrations in Las Vegas within the maximum entropy IRL framework. DriveIRL's benefits include: a simple design due to only learning the trajectory scoring function, relatively interpretable features, and strong real-world performance. We validated DriveIRL on the Las Vegas Strip and demonstrated fully autonomous driving in heavy traffic, including scenarios involving cut-ins, abrupt braking by the lead vehicle, and hotel pickup/dropoff zones. Our dataset will be made public to help further research in this area.
△ Less
Submitted 7 June, 2022;
originally announced June 2022.
-
Design Choices for Data Governance in Platform Ecosystems: A Contingency Model
Authors:
Sung Une Lee,
Liming Zhu,
Ross Jeffery
Abstract:
As platform ecosystems are growing by platform users' data, the importance of data governance has been highlighted. In particular, how to share control and decision rights with platform users are regarded as significant design issues since the role of them is increasing. Platform context should be considered when designing data governance in platform ecosystems (i.e. centralized/decentralized gove…
▽ More
As platform ecosystems are growing by platform users' data, the importance of data governance has been highlighted. In particular, how to share control and decision rights with platform users are regarded as significant design issues since the role of them is increasing. Platform context should be considered when designing data governance in platform ecosystems (i.e. centralized/decentralized governance). However, there is limited research on this issue. Existing models focus on characteristics for enterprises. This results in limited support for platform ecosystems where there are different types of business context such as open strategies or platform maturity. This paper develops a contingency model for platform ecosystems including distinctive contingency factors. The study then discusses which data governance factors should be carefully considered and strengthened for each contingency in order to succeed in governance and to win market. A case study is performed to validate our model and to show its practical implications.
△ Less
Submitted 23 June, 2017;
originally announced June 2017.
-
Data Governance for Platform Ecosystems: Critical Factors and the State of Practice
Authors:
Sung Une Lee,
Liming Zhu,
Ross Jeffery
Abstract:
Recently, platform ecosystem has received attention as a key business concept. Sustainable growth of platform ecosystems is enabled by platform users supplying and/or demanding content from each other: e.g. Facebook, YouTube or Twitter. The importance and value of user data in platform ecosystems is accentuated since platform owners use and sell the data for their business. Serious concern is incr…
▽ More
Recently, platform ecosystem has received attention as a key business concept. Sustainable growth of platform ecosystems is enabled by platform users supplying and/or demanding content from each other: e.g. Facebook, YouTube or Twitter. The importance and value of user data in platform ecosystems is accentuated since platform owners use and sell the data for their business. Serious concern is increasing about data misuse or abuse, privacy issues and revenue sharing between the different stakeholders. Traditional data governance focuses on generic goals and a universal approach to manage the data of an enterprise. It entails limited support for the complicated situation and relationship of a platform ecosystem where multiple participating parties contribute, use data and share profits. This article identifies data governance factors for platform ecosystems through literature review. The study then surveys the data governance state of practice of four platform ecosystems: Facebook, YouTube, EBay and Uber. Finally, 19 governance models in industry and academia are compared against our identified data governance factors for platform ecosystems to reveal the gaps and limitations.
△ Less
Submitted 5 May, 2017;
originally announced May 2017.
