Showing 1–11 of 11 results for author: Henry, R

Morphic Sequences: Complexity and Decidability

Authors: Raphael Henry

Abstract: In this work we recall Pansiot's result on the complexity of pure morphic sequences and we use the tools developed by Devyatov for morphic sequences to prove the decidability of the complexity class of pure morphic sequences. In this work we recall Pansiot's result on the complexity of pure morphic sequences and we use the tools developed by Devyatov for morphic sequences to prove the decidability of the complexity class of pure morphic sequences. △ Less

Submitted 24 June, 2024; originally announced June 2024.

Comments: In Proceedings GASCom 2024, arXiv:2406.14588

ACM Class: G.2.1.

Journal ref: EPTCS 403, 2024, pp. 113-117

Fast leave-one-cluster-out cross-validation by clustered Network Information Criteria (NICc)

Authors: Jiaxing Qiu, Douglas E. Lake, Teague R. Henry

Abstract: This paper introduced a clustered estimator of the Network Information Criterion (NICc) to approximate leave-one-cluster-out cross-validated deviance, which can be used as an alternative to cluster-based cross-validation when modeling clustered data. Stone proved that Akaike Information Criterion (AIC) is an asymptotic equivalence to leave-one-observation-out cross-validation if the parametric mod… ▽ More This paper introduced a clustered estimator of the Network Information Criterion (NICc) to approximate leave-one-cluster-out cross-validated deviance, which can be used as an alternative to cluster-based cross-validation when modeling clustered data. Stone proved that Akaike Information Criterion (AIC) is an asymptotic equivalence to leave-one-observation-out cross-validation if the parametric model is true. Ripley pointed out that the Network Information Criterion (NIC) derived in Stone's proof, is a better approximation to leave-one-observation-out cross-validation when the model is not true. For clustered data, we derived a clustered estimator of NIC, referred to as NICc, by substituting the Fisher information matrix in NIC with its estimator that adjusts for clustering. This adjustment imposes a larger penalty in NICc than the unclustered estimator of NIC when modeling clustered data, thereby preventing overfitting more effectively. In a simulation study and an empirical example, we used linear and logistic regression to model clustered data with Gaussian or binomial response, respectively. We showed that NICc is a better approximation to leave-one-cluster-out deviance and prevents overfitting more effectively than AIC and Bayesian Information Criterion (BIC). NICc leads to more accurate model selection, as determined by cluster-based cross-validation, compared to AIC and BIC. △ Less

Submitted 30 May, 2024; originally announced May 2024.

FineQuant: Unlocking Efficiency with Fine-Grained Weight-Only Quantization for LLMs

Authors: Young ** Kim, Rawn Henry, Raffy Fahim, Hany Hassan Awadalla

Abstract: Large Language Models (LLMs) have achieved state-of-the-art performance across various language tasks but pose challenges for practical deployment due to their substantial memory requirements. Furthermore, the latest generative models suffer from high inference costs caused by the memory bandwidth bottleneck in the auto-regressive decoding process. To address these issues, we propose an efficient… ▽ More Large Language Models (LLMs) have achieved state-of-the-art performance across various language tasks but pose challenges for practical deployment due to their substantial memory requirements. Furthermore, the latest generative models suffer from high inference costs caused by the memory bandwidth bottleneck in the auto-regressive decoding process. To address these issues, we propose an efficient weight-only quantization method that reduces memory consumption and accelerates inference for LLMs. To ensure minimal quality degradation, we introduce a simple and effective heuristic approach that utilizes only the model weights of a pre-trained model. This approach is applicable to both Mixture-of-Experts (MoE) and dense models without requiring additional fine-tuning. To demonstrate the effectiveness of our proposed method, we first analyze the challenges and issues associated with LLM quantization. Subsequently, we present our heuristic approach, which adaptively finds the granularity of quantization, effectively addressing these problems. Furthermore, we implement highly efficient GPU GEMMs that perform on-the-fly matrix multiplication and dequantization, supporting the multiplication of fp16 or bf16 activations with int8 or int4 weights. We evaluate our approach on large-scale open source models such as OPT-175B and internal MoE models, showcasing minimal accuracy loss while achieving up to 3.65 times higher throughput on the same number of GPUs. △ Less

Submitted 16 August, 2023; originally announced August 2023.

Who Says Elephants Can't Run: Bringing Large Scale MoE Models into Cloud Scale Production

Authors: Young ** Kim, Rawn Henry, Raffy Fahim, Hany Hassan Awadalla

Abstract: Mixture of Experts (MoE) models with conditional execution of sparsely activated layers have enabled training models with a much larger number of parameters. As a result, these models have achieved significantly better quality on various natural language processing tasks including machine translation. However, it remains challenging to deploy such models in real-life scenarios due to the large mem… ▽ More Mixture of Experts (MoE) models with conditional execution of sparsely activated layers have enabled training models with a much larger number of parameters. As a result, these models have achieved significantly better quality on various natural language processing tasks including machine translation. However, it remains challenging to deploy such models in real-life scenarios due to the large memory requirements and inefficient inference. In this work, we introduce a highly efficient inference framework with several optimization approaches to accelerate the computation of sparse models and cut down the memory consumption significantly. While we achieve up to 26x speed-up in terms of throughput, we also reduce the model size almost to one eighth of the original 32-bit float model by quantizing expert weights into 4-bit integers. As a result, we are able to deploy 136x larger models with 27% less cost and significantly better quality compared to the existing solutions. This enables a paradigm shift in deploying large scale multilingual MoE transformers models replacing the traditional practice of distilling teacher models into dozens of smaller models per language or task. △ Less

Submitted 17 November, 2022; originally announced November 2022.

Comments: Accepted to SustaiNLP 2022 (EMNLP 2022)

More is Merrier: Relax the Non-Collusion Assumption in Multi-Server PIR

Authors: Tiantian Gong, Ryan Henry, Alexandros Psomas, Aniket Kate

Abstract: A long line of research on secure computation has confirmed that anything that can be computed, can be computed securely using a set of non-colluding parties. Indeed, this non-collusion assumption makes a number of problems solvable, as well as reduces overheads and bypasses computational hardness results, and it is pervasive across different privacy-enhancing technologies. However, it remains hig… ▽ More A long line of research on secure computation has confirmed that anything that can be computed, can be computed securely using a set of non-colluding parties. Indeed, this non-collusion assumption makes a number of problems solvable, as well as reduces overheads and bypasses computational hardness results, and it is pervasive across different privacy-enhancing technologies. However, it remains highly susceptible to covert, undetectable collusion among computing parties. This work stems from an observation that if the number of available computing parties is much higher than the number of parties required to perform a secure computation task, collusion attempts in privacy-preserving computations could be deterred. We focus on the prominent privacy-preserving computation task of multi-server $1$-private information retrieval (PIR) that inherently assumes no pair-wise collusion. For PIR application scenarios, such as those for blockchain light clients, where the available servers can be plentiful, a single server's deviating action is not tremendously beneficial to itself. We can make deviations undesired via small amounts of rewards and penalties, thus significantly raising the bar for collusion resistance. We design and implement a collusion mitigation mechanism on a public bulletin board with payment execution functions, considering only rational and malicious parties with no honest non-colluding servers. Privacy protection is offered for an extended period after the query executions. △ Less

Submitted 4 December, 2023; v1 submitted 19 January, 2022; originally announced January 2022.

Comments: 19 pages, 6 figures

Gym-ANM: Open-source software to leverage reinforcement learning for power system management in research and education

Authors: Robin Henry, Damien Ernst

Abstract: Gym-ANM is a Python package that facilitates the design of reinforcement learning (RL) environments that model active network management (ANM) tasks in electricity networks. Here, we describe how to implement new environments and how to write code to interact with pre-existing ones. We also provide an overview of ANM6-Easy, an environment designed to highlight common ANM challenges. Finally, we di… ▽ More Gym-ANM is a Python package that facilitates the design of reinforcement learning (RL) environments that model active network management (ANM) tasks in electricity networks. Here, we describe how to implement new environments and how to write code to interact with pre-existing ones. We also provide an overview of ANM6-Easy, an environment designed to highlight common ANM challenges. Finally, we discuss the potential impact of Gym-ANM on the scientific community, both in terms of research and education. We hope this package will facilitate collaboration between the power system and RL communities in the search for algorithms to control future energy systems. △ Less

Submitted 18 May, 2021; originally announced May 2021.

Comments: 5 pages, 2 figures, 2 code samples

ACM Class: I.2.8

Gym-ANM: Reinforcement Learning Environments for Active Network Management Tasks in Electricity Distribution Systems

Authors: Robin Henry, Damien Ernst

Abstract: Active network management (ANM) of electricity distribution networks include many complex stochastic sequential optimization problems. These problems need to be solved for integrating renewable energies and distributed storage into future electrical grids. In this work, we introduce Gym-ANM, a framework for designing reinforcement learning (RL) environments that model ANM tasks in electricity dist… ▽ More Active network management (ANM) of electricity distribution networks include many complex stochastic sequential optimization problems. These problems need to be solved for integrating renewable energies and distributed storage into future electrical grids. In this work, we introduce Gym-ANM, a framework for designing reinforcement learning (RL) environments that model ANM tasks in electricity distribution networks. These environments provide new playgrounds for RL research in the management of electricity networks that do not require an extensive knowledge of the underlying dynamics of such systems. Along with this work, we are releasing an implementation of an introductory toy-environment, ANM6-Easy, designed to emphasize common challenges in ANM. We also show that state-of-the-art RL algorithms can already achieve good performance on ANM6-Easy when compared against a model predictive control (MPC) approach. Finally, we provide guidelines to create new Gym-ANM environments differing in terms of (a) the distribution network topology and parameters, (b) the observation space, (c) the modelling of the stochastic processes present in the system, and (d) a set of hyperparameters influencing the reward signal. Gym-ANM can be downloaded at https://github.com/robinhenry/gym-anm. △ Less

Submitted 30 June, 2021; v1 submitted 14 March, 2021; originally announced March 2021.

Comments: 15 main pages, 17 pages of appendix, 10 figures, GitHub repository: https://github.com/robinhenry/gym-anm

ACM Class: I.2.11; I.2.8

Direct Classification of Type 2 Diabetes From Retinal Fundus Images in a Population-based Sample From The Maastricht Study

Authors: Friso G. Heslinga, Josien P. W. Pluim, A. J. H. M. Houben, Miranda T. Schram, Ronald M. A. Henry, Coen D. A. Stehouwer, Marleen J. van Greevenbroek, Tos T. J. M. Berendschot, Mitko Veta

Abstract: Type 2 Diabetes (T2D) is a chronic metabolic disorder that can lead to blindness and cardiovascular disease. Information about early stage T2D might be present in retinal fundus images, but to what extent these images can be used for a screening setting is still unknown. In this study, deep neural networks were employed to differentiate between fundus images from individuals with and without T2D.… ▽ More Type 2 Diabetes (T2D) is a chronic metabolic disorder that can lead to blindness and cardiovascular disease. Information about early stage T2D might be present in retinal fundus images, but to what extent these images can be used for a screening setting is still unknown. In this study, deep neural networks were employed to differentiate between fundus images from individuals with and without T2D. We investigated three methods to achieve high classification performance, measured by the area under the receiver operating curve (ROC-AUC). A multi-target learning approach to simultaneously output retinal biomarkers as well as T2D works best (AUC = 0.746 [$\pm$0.001]). Furthermore, the classification performance can be improved when images with high prediction uncertainty are referred to a specialist. We also show that the combination of images of the left and right eye per individual can further improve the classification performance (AUC = 0.758 [$\pm$0.003]), using a simple averaging approach. The results are promising, suggesting the feasibility of screening for T2D from retinal fundus images. △ Less

Submitted 22 November, 2019; originally announced November 2019.

Comments: to be published in the proceeding of SPIE - Medical Imaging 2020, 6 pages, 1 figure

Critical Time Windows for Renewable Resource Complementarity Assessment

Authors: Mathias Berger, David Radu, Raphael Fonteneau, Robin Henry, Mevludin Glavic, Xavier Fettweis, Marc Le Du, Patrick Panciatici, Lucian Balea, Damien Ernst

Abstract: This paper proposes a systematic framework to assess the complementarity of renewable resources over arbitrary geographical scopes and temporal scales which is particularly well-suited to exploit very large data sets of climatological data. The concept of critical time windows is introduced, and a spatio-temporal criticality indicator is proposed, consisting in a parametrised family of scalar indi… ▽ More This paper proposes a systematic framework to assess the complementarity of renewable resources over arbitrary geographical scopes and temporal scales which is particularly well-suited to exploit very large data sets of climatological data. The concept of critical time windows is introduced, and a spatio-temporal criticality indicator is proposed, consisting in a parametrised family of scalar indicators quantifying the complementarity between renewable resources in both space and time. The criticality indicator is leveraged to devise a family of optimisation problems identifying sets of locations with maximum complementarity under arbitrary geographical deployment constraints. The applicability of the framework is shown in a case study investigating the complementarity between the wind regimes in continental western Europe and southern Greenland, and its usefulness in a power system planning context is demonstrated. Besides showing that the occurrence of low wind power production events can be significantly reduced on a regional scale by exploiting diversity in local wind patterns, results highlight the fact that aggregating wind power production sites located on different continents may result in a lower occurrence of system-wide low wind power production events and indicate potential benefits of intercontinental electrical interconnections. △ Less

Submitted 5 December, 2018; originally announced December 2018.

Robust imaging of hippocampal inner structure at 7T: in vivo acquisition protocol and methodological choices

Authors: Linda Marrakchi-Kacem, Alexandre Vignaud, Julien Sein, Johanne Germain, Thomas R Henry, Cyril Poupon, Lucie Hertz-Pannier, Stéphane Lehéricy, Olivier Colliot, Pierre-François Van de Moortele, Marie Chupin

Abstract: OBJECTIVE:Motion-robust multi-slab imaging of hippocampal inner structure in vivo at 7T.MATERIALS AND METHODS:Motion is a crucial issue for ultra-high resolution imaging, such as can be achieved with 7T MRI. An acquisition protocol was designed for imaging hippocampal inner structure at 7T. It relies on a compromise between anatomical details visibility and robustness to motion. In order to reduce… ▽ More OBJECTIVE:Motion-robust multi-slab imaging of hippocampal inner structure in vivo at 7T.MATERIALS AND METHODS:Motion is a crucial issue for ultra-high resolution imaging, such as can be achieved with 7T MRI. An acquisition protocol was designed for imaging hippocampal inner structure at 7T. It relies on a compromise between anatomical details visibility and robustness to motion. In order to reduce acquisition time and motion artifacts, the full slab covering the hippocampus was split into separate slabs with lower acquisition time. A robust registration approach was implemented to combine the acquired slabs within a final 3D-consistent high-resolution slab covering the whole hippocampus. Evaluation was performed on 50 subjects overall, made of three groups of subjects acquired using three acquisition settings; it focused on three issues: visibility of hippocampal inner structure, robustness to motion artifacts and registration procedure performance.RESULTS:Overall, T2-weighted acquisitions with interleaved slabs proved robust. Multi-slab registration yielded high quality datasets in 96 % of the subjects, thus compatible with further analyses of hippocampal inner structure.CONCLUSION:Multi-slab acquisition and registration setting is efficient for reducing acquisition time and consequently motion artifacts for ultra-high resolution imaging of the inner structure of the hippocampus. △ Less

Submitted 9 May, 2016; originally announced May 2016.

Journal ref: Magnetic Resonance Materials in Physics, Biology and Medicine, Springer Verlag, 2016

Multi-Objective Design Optimization of the Leg Mechanism for a Pi** Inspection Robot

Authors: Renaud Henry, Damien Chablat, Mathieu Porez, Frédéric Boyer, Daniel Kanaan

Abstract: This paper addresses the dimensional synthesis of an adaptive mechanism of contact points ie a leg mechanism of a pi** inspection robot operating in an irradiated area as a nuclear power plant. This studied mechanism is the leading part of the robot sub-system responsible of the locomotion. Firstly, three architectures are chosen from the literature and their properties are described. Then, a me… ▽ More This paper addresses the dimensional synthesis of an adaptive mechanism of contact points ie a leg mechanism of a pi** inspection robot operating in an irradiated area as a nuclear power plant. This studied mechanism is the leading part of the robot sub-system responsible of the locomotion. Firstly, three architectures are chosen from the literature and their properties are described. Then, a method using a multi-objective optimization is proposed to determine the best architecture and the optimal geometric parameters of a leg taking into account environmental and design constraints. In this context, the objective functions are the minimization of the mechanism size and the maximization of the transmission force factor. Representations of the Pareto front versus the objective functions and the design parameters are given. Finally, the CAD model of several solutions located on the Pareto front are presented and discussed. △ Less

Submitted 18 June, 2014; originally announced June 2014.

Comments: Proceedings of the ASME 2014 International Design Engineering Technical Conferences \& Computers and Information in Engineering Conference, Buffalo : United States (2014)