-
Minimax Exploiter: A Data Efficient Approach for Competitive Self-Play
Authors:
Daniel Bairamian,
Philippe Marcotte,
Joshua Romoff,
Gabriel Robert,
Derek Nowrouzezahrai
Abstract:
Recent advances in Competitive Self-Play (CSP) have achieved, or even surpassed, human level performance in complex game environments such as Dota 2 and StarCraft II using Distributed Multi-Agent Reinforcement Learning (MARL). One core component of these methods relies on creating a pool of learning agents -- consisting of the Main Agent, past versions of this agent, and Exploiter Agents -- where…
▽ More
Recent advances in Competitive Self-Play (CSP) have achieved, or even surpassed, human level performance in complex game environments such as Dota 2 and StarCraft II using Distributed Multi-Agent Reinforcement Learning (MARL). One core component of these methods relies on creating a pool of learning agents -- consisting of the Main Agent, past versions of this agent, and Exploiter Agents -- where Exploiter Agents learn counter-strategies to the Main Agents. A key drawback of these approaches is the large computational cost and physical time that is required to train the system, making them impractical to deploy in highly iterative real-life settings such as video game productions. In this paper, we propose the Minimax Exploiter, a game theoretic approach to exploiting Main Agents that leverages knowledge of its opponents, leading to significant increases in data efficiency. We validate our approach in a diversity of settings, including simple turn based games, the arcade learning environment, and For Honor, a modern video game. The Minimax Exploiter consistently outperforms strong baselines, demonstrating improved stability and data efficiency, leading to a robust CSP-MARL method that is both flexible and easy to deploy.
△ Less
Submitted 28 November, 2023;
originally announced November 2023.
-
Deep morphological recognition of kidney stones using intra-operative endoscopic digital videos
Authors:
Vincent Estrade,
Michel Daudon,
Emmanuel Richard,
Jean-Christophe Bernhard,
Franck Bladou,
Gregoire Robert,
Laurent Facq,
Baudouin Denis de Senneville
Abstract:
The collection and the analysis of kidney stone morphological criteria are essential for an aetiological diagnosis of stone disease. However, in-situ LASER-based fragmentation of urinary stones, which is now the most established chirurgical intervention, may destroy the morphology of the targeted stone. In the current study, we assess the performance and added value of processing complete digital…
▽ More
The collection and the analysis of kidney stone morphological criteria are essential for an aetiological diagnosis of stone disease. However, in-situ LASER-based fragmentation of urinary stones, which is now the most established chirurgical intervention, may destroy the morphology of the targeted stone. In the current study, we assess the performance and added value of processing complete digital endoscopic video sequences for the automatic recognition of stone morphological features during a standard-of-care intra-operative session. To this end, a computer-aided video classifier was developed to predict in-situ the morphology of stone using an intra-operative digital endoscopic video acquired in a clinical setting.
The proposed technique was evaluated on pure (i.e. include one morphology) and mixed (i.e. include at least two morphologies) stones involving "Ia/Calcium Oxalate Monohydrate (COM)", "IIb/ Calcium Oxalate Dihydrate (COD)" and "IIIb/Uric Acid (UA)" morphologies. 71 digital endoscopic videos (50 exhibited only one morphological type and 21 displayed two) were analyzed using the proposed video classifier (56840 frames processed in total). Using the proposed approach, diagnostic performances (averaged over both pure and mixed stone types) were as follows: balanced accuracy=88%, sensitivity=80%, specificity=95%, precision=78% and F1-score=78%.
The obtained results demonstrate that AI applied on digital endoscopic video sequences is a promising tool for collecting morphological information during the time-course of the stone fragmentation process without resorting to any human intervention for stone delineation or selection of good quality steady frames. To this end, irrelevant image information must be removed from the prediction process at both frame and pixel levels, which is now feasible thanks to the use of AI-dedicated networks.
△ Less
Submitted 12 May, 2022;
originally announced May 2022.
-
Towards Automatic Recognition of Pure & Mixed Stones using Intraoperative Endoscopic Digital Images
Authors:
Vincent Estrade,
Michel Daudon,
Emmanuel Richard,
Jean-Christophe Bernhard,
Franck Bladou,
Gregoire Robert,
Baudouin Denis de Senneville
Abstract:
Objective: To assess automatic computer-aided in-situ recognition of morphological features of pure and mixed urinary stones using intraoperative digital endoscopic images acquired in a clinical setting. Materials and methods: In this single-centre study, an experienced urologist intraoperatively and prospectively examined the surface and section of all kidney stones encountered. Calcium oxalate m…
▽ More
Objective: To assess automatic computer-aided in-situ recognition of morphological features of pure and mixed urinary stones using intraoperative digital endoscopic images acquired in a clinical setting. Materials and methods: In this single-centre study, an experienced urologist intraoperatively and prospectively examined the surface and section of all kidney stones encountered. Calcium oxalate monohydrate (COM/Ia), dihydrate (COD/IIb) and uric acid (UA/IIIb) morphological criteria were collected and classified to generate annotated datasets. A deep convolutional neural network (CNN) was trained to predict the composition of both pure and mixed stones. To explain the predictions of the deep neural network model, coarse localisation heat-maps were plotted to pinpoint key areas identified by the network. Results: This study included 347 and 236 observations of stone surface and stone section, respectively. A highest sensitivity of 98 % was obtained for the type "pure IIIb/UA" using surface images. The most frequently encountered morphology was that of the type "pure Ia/COM"; it was correctly predicted in 91 % and 94 % of cases using surface and section images, respectively. Of the mixed type "Ia/COM+IIb/COD", Ia/COM was predicted in 84 % of cases using surface images, IIb/COD in 70 % of cases, and both in 65 % of cases. Concerning mixed Ia/COM+IIIb/UA stones, Ia/COM was predicted in 91 % of cases using section images, IIIb/UA in 69 % of cases, and both in 74 % of cases. Conclusions: This preliminary study demonstrates that deep convolutional neural networks are promising to identify kidney stone composition from endoscopic images acquired intraoperatively. Both pure and mixed stone composition could be discriminated. Collected in a clinical setting, surface and section images analysed by deep CNN provide valuable information about stone morphology for computer-aided diagnosis.
△ Less
Submitted 22 May, 2021;
originally announced May 2021.
-
Generalized Gabidulin codes over fields of any characteristic
Authors:
Daniel Augot,
Pierre Loidreau,
Gwezheneg Robert
Abstract:
We generalise Gabidulin codes to the case of infinite fields, eventually with characteristic zero. For this purpose, we consider an abstract field extension and any automorphism in the Galois group. We derive some conditions on the automorphism to be able to have a proper notion of rank metric which is in coherence with linearized polynomials. Under these conditions, we generalize Gabidulin codes…
▽ More
We generalise Gabidulin codes to the case of infinite fields, eventually with characteristic zero. For this purpose, we consider an abstract field extension and any automorphism in the Galois group. We derive some conditions on the automorphism to be able to have a proper notion of rank metric which is in coherence with linearized polynomials. Under these conditions, we generalize Gabidulin codes and provide a decoding algorithm which decode both errors and erasures. Then, we focus on codes over integer rings and how to decode them. We are then faced with the problem of the exponential growth of intermediate values, and to circumvent the problem, it is natural to propose to do computations modulo a prime ideal. For this, we study the reduction of generalized Gabidulin codes over number ideals codes modulo a prime ideal, and show they are classical Gabidulin codes. As a consequence, knowing side information on the size of the errors or the message, we can reduce the decoding problem over the integer ring to a decoding problem over a finite field. We also give examples and timings.
△ Less
Submitted 27 March, 2017;
originally announced March 2017.
-
Rank metric and Gabidulin codes in characteristic zero
Authors:
Gwezheneg Robert,
Pierre Loidreau,
Daniel Augot
Abstract:
We transpose the theory of rank metric and Gabidulin codes to the case of fields of characteristic zero. The Frobenius automorphism is then replaced by any element of the Galois group. We derive some conditions on the automorphism to be able to easily transpose the results obtained by Gabidulin as well and a classical polynomial-time decoding algorithm. We also provide various definitions for the…
▽ More
We transpose the theory of rank metric and Gabidulin codes to the case of fields of characteristic zero. The Frobenius automorphism is then replaced by any element of the Galois group. We derive some conditions on the automorphism to be able to easily transpose the results obtained by Gabidulin as well and a classical polynomial-time decoding algorithm. We also provide various definitions for the rank-metric.
△ Less
Submitted 17 May, 2013;
originally announced May 2013.
