Showing 1–31 of 31 results for author: Rădulescu, R

Divide and Conquer: Provably Unveiling the Pareto Front with Multi-Objective Reinforcement Learning

Authors: Willem Röpke, Mathieu Reymond, Patrick Mannion, Diederik M. Roijers, Ann Nowé, Roxana Rădulescu

Abstract: A significant challenge in multi-objective reinforcement learning is obtaining a Pareto front of policies that attain optimal performance under different preferences. We introduce Iterated Pareto Referent Optimisation (IPRO), a principled algorithm that decomposes the task of finding the Pareto front into a sequence of single-objective problems for which various solution methods exist. This enable… ▽ More A significant challenge in multi-objective reinforcement learning is obtaining a Pareto front of policies that attain optimal performance under different preferences. We introduce Iterated Pareto Referent Optimisation (IPRO), a principled algorithm that decomposes the task of finding the Pareto front into a sequence of single-objective problems for which various solution methods exist. This enables us to establish convergence guarantees while providing an upper bound on the distance to undiscovered Pareto optimal solutions at each step. Empirical evaluations demonstrate that IPRO matches or outperforms methods that require additional domain knowledge. By leveraging problem-specific single-objective solvers, our approach also holds promise for applications beyond multi-objective reinforcement learning, such as in pathfinding and optimisation. △ Less

Submitted 11 February, 2024; originally announced February 2024.

Utility-Based Reinforcement Learning: Unifying Single-objective and Multi-objective Reinforcement Learning

Authors: Peter Vamplew, Cameron Foale, Conor F. Hayes, Patrick Mannion, Enda Howley, Richard Dazeley, Scott Johnson, Johan Källström, Gabriel Ramos, Roxana Rădulescu, Willem Röpke, Diederik M. Roijers

Abstract: Research in multi-objective reinforcement learning (MORL) has introduced the utility-based paradigm, which makes use of both environmental rewards and a function that defines the utility derived by the user from those rewards. In this paper we extend this paradigm to the context of single-objective reinforcement learning (RL), and outline multiple potential benefits including the ability to perfor… ▽ More Research in multi-objective reinforcement learning (MORL) has introduced the utility-based paradigm, which makes use of both environmental rewards and a function that defines the utility derived by the user from those rewards. In this paper we extend this paradigm to the context of single-objective reinforcement learning (RL), and outline multiple potential benefits including the ability to perform multi-policy learning across tasks relating to uncertain objectives, risk-aware RL, discounting, and safe RL. We also examine the algorithmic implications of adopting a utility-based approach. △ Less

Submitted 4 February, 2024; originally announced February 2024.

Comments: Accepted for the Blue Sky Track at AAMAS'24

Emergent Cooperation under Uncertain Incentive Alignment

Authors: Nicole Orzan, Erman Acar, Davide Grossi, Roxana Rădulescu

Abstract: Understanding the emergence of cooperation in systems of computational agents is crucial for the development of effective cooperative AI. Interaction among individuals in real-world settings are often sparse and occur within a broad spectrum of incentives, which often are only partially known. In this work, we explore how cooperation can arise among reinforcement learning agents in scenarios chara… ▽ More Understanding the emergence of cooperation in systems of computational agents is crucial for the development of effective cooperative AI. Interaction among individuals in real-world settings are often sparse and occur within a broad spectrum of incentives, which often are only partially known. In this work, we explore how cooperation can arise among reinforcement learning agents in scenarios characterised by infrequent encounters, and where agents face uncertainty about the alignment of their incentives with those of others. To do so, we train the agents under a wide spectrum of environments ranging from fully competitive, to fully cooperative, to mixed-motives. Under this type of uncertainty we study the effects of mechanisms, such as reputation and intrinsic rewards, that have been proposed in the literature to foster cooperation in mixed-motives environments. Our findings show that uncertainty substantially lowers the agents' ability to engage in cooperative behaviour, when that would be the best course of action. In this scenario, the use of effective reputation mechanisms and intrinsic rewards boosts the agents' capability to act nearly-optimally in cooperative environments, while greatly enhancing cooperation in mixed-motive environments as well. △ Less

Submitted 23 January, 2024; originally announced January 2024.

You Can Tell a Cybercriminal by the Company they Keep: A Framework to Infer the Relevance of Underground Communities to the Threat Landscape

Authors: Michele Campobasso, Radu Rădulescu, Sylvan Brons, Luca Allodi

Abstract: The criminal underground is populated with forum marketplaces where, allegedly, cybercriminals share and trade knowledge, skills, and cybercrime products. However, it is still unclear whether all marketplaces matter the same in the overall threat landscape. To effectively support trade and avoid degenerating into scams-for-scammers places, underground markets must address fundamental economic prob… ▽ More The criminal underground is populated with forum marketplaces where, allegedly, cybercriminals share and trade knowledge, skills, and cybercrime products. However, it is still unclear whether all marketplaces matter the same in the overall threat landscape. To effectively support trade and avoid degenerating into scams-for-scammers places, underground markets must address fundamental economic problems (such as moral hazard, adverse selection) that enable the exchange of actual technology and cybercrime products (as opposed to repackaged malware or years-old password databases). From the relevant literature and manual investigation, we identify several mechanisms that marketplaces implement to mitigate these problems, and we condense them into a market evaluation framework based on the Business Model Canvas. We use this framework to evaluate which mechanisms `successful' marketplaces have in place, and whether these differ from those employed by `unsuccessful' marketplaces. We test the framework on 23 underground forum markets by searching 836 aliases of indicted cybercriminals to identify `successful' marketplaces. We find evidence that marketplaces whose administrators are impartial in trade, verify their sellers, and have the right economic incentives to keep the market functional are more likely to be credible sources of threat. △ Less

Submitted 13 June, 2023; v1 submitted 9 June, 2023; originally announced June 2023.

Comments: The 22nd Workshop on the Economics of Information Security (WEIS'23), July 05--08, 2023, Geneva, Switzerland

Bridging the Gap Between Single and Multi Objective Games

Authors: Willem Röpke, Carla Groenland, Roxana Rădulescu, Ann Nowé, Diederik M. Roijers

Abstract: A classic model to study strategic decision making in multi-agent systems is the normal-form game. This model can be generalised to allow for an infinite number of pure strategies leading to continuous games. Multi-objective normal-form games are another generalisation that model settings where players receive separate payoffs in more than one objective. We bridge the gap between the two models by… ▽ More A classic model to study strategic decision making in multi-agent systems is the normal-form game. This model can be generalised to allow for an infinite number of pure strategies leading to continuous games. Multi-objective normal-form games are another generalisation that model settings where players receive separate payoffs in more than one objective. We bridge the gap between the two models by providing a theoretical guarantee that a game from one setting can always be transformed to a game in the other. We extend the theoretical results to include guaranteed equivalence of Nash equilibria. The map** makes it possible to apply algorithms from one field to the other. We demonstrate this by introducing a fictitious play algorithm for multi-objective games and subsequently applying it to two well-known continuous games. We believe the equivalence relation will lend itself to new insights by translating the theoretical guarantees from one formalism to another. Moreover, it may lead to new computational approaches for continuous games when a problem is more naturally solved in the succinct format of multi-objective games. △ Less

Submitted 1 March, 2023; v1 submitted 13 January, 2023; originally announced January 2023.

Comments: Accepted to AAMAS 2023

Exploring the Pareto front of multi-objective COVID-19 mitigation policies using reinforcement learning

Authors: Mathieu Reymond, Conor F. Hayes, Lander Willem, Roxana Rădulescu, Steven Abrams, Diederik M. Roijers, Enda Howley, Patrick Mannion, Niel Hens, Ann Nowé, Pieter Libin

Abstract: Infectious disease outbreaks can have a disruptive impact on public health and societal processes. As decision making in the context of epidemic mitigation is hard, reinforcement learning provides a methodology to automatically learn prevention strategies in combination with complex epidemic models. Current research focuses on optimizing policies w.r.t. a single objective, such as the pathogen's a… ▽ More Infectious disease outbreaks can have a disruptive impact on public health and societal processes. As decision making in the context of epidemic mitigation is hard, reinforcement learning provides a methodology to automatically learn prevention strategies in combination with complex epidemic models. Current research focuses on optimizing policies w.r.t. a single objective, such as the pathogen's attack rate. However, as the mitigation of epidemics involves distinct, and possibly conflicting criteria (i.a., prevalence, mortality, morbidity, cost), a multi-objective approach is warranted to learn balanced policies. To lift this decision-making process to real-world epidemic models, we apply deep multi-objective reinforcement learning and build upon a state-of-the-art algorithm, Pareto Conditioned Networks (PCN), to learn a set of solutions that approximates the Pareto front of the decision problem. We consider the first wave of the Belgian COVID-19 epidemic, which was mitigated by a lockdown, and study different deconfinement strategies, aiming to minimize both COVID-19 cases (i.e., infections and hospitalizations) and the societal burden that is induced by the applied mitigation measures. We contribute a multi-objective Markov decision process that encapsulates the stochastic compartment model that was used to inform policy makers during the COVID-19 epidemic. As these social mitigation measures are implemented in a continuous action space that modulates the contact matrix of the age-structured epidemic model, we extend PCN to this setting. We evaluate the solution returned by PCN, and observe that it correctly learns to reduce the social burden whenever the hospitalization rates are sufficiently low. In this work, we thus show that multi-objective reinforcement learning is attainable in complex epidemiological models and provides essential insights to balance complex mitigation policies. △ Less

Submitted 11 April, 2022; originally announced April 2022.

Scalar reward is not enough: A response to Silver, Singh, Precup and Sutton (2021)

Authors: Peter Vamplew, Benjamin J. Smith, Johan Kallstrom, Gabriel Ramos, Roxana Radulescu, Diederik M. Roijers, Conor F. Hayes, Fredrik Heintz, Patrick Mannion, Pieter J. K. Libin, Richard Dazeley, Cameron Foale

Abstract: The recent paper `"Reward is Enough" by Silver, Singh, Precup and Sutton posits that the concept of reward maximisation is sufficient to underpin all intelligence, both natural and artificial. We contest the underlying assumption of Silver et al. that such reward can be scalar-valued. In this paper we explain why scalar rewards are insufficient to account for some aspects of both biological and co… ▽ More The recent paper `"Reward is Enough" by Silver, Singh, Precup and Sutton posits that the concept of reward maximisation is sufficient to underpin all intelligence, both natural and artificial. We contest the underlying assumption of Silver et al. that such reward can be scalar-valued. In this paper we explain why scalar rewards are insufficient to account for some aspects of both biological and computational intelligence, and argue in favour of explicitly multi-objective models of reward maximisation. Furthermore, we contend that even if scalar reward functions can trigger intelligent behaviour in specific cases, it is still undesirable to use this approach for the development of artificial general intelligence due to unacceptable risks of unsafe or unethical behaviour. △ Less

Submitted 24 November, 2021; originally announced December 2021.

On Nash Equilibria in Normal-Form Games With Vectorial Payoffs

Authors: Willem Röpke, Diederik M. Roijers, Ann Nowé, Roxana Rădulescu

Abstract: We provide an in-depth study of Nash equilibria in multi-objective normal form games (MONFGs), i.e., normal form games with vectorial payoffs. Taking a utility-based approach, we assume that each player's utility can be modelled with a utility function that maps a vector to a scalar utility. In the case of a mixed strategy, it is meaningful to apply such a scalarisation both before calculating the… ▽ More We provide an in-depth study of Nash equilibria in multi-objective normal form games (MONFGs), i.e., normal form games with vectorial payoffs. Taking a utility-based approach, we assume that each player's utility can be modelled with a utility function that maps a vector to a scalar utility. In the case of a mixed strategy, it is meaningful to apply such a scalarisation both before calculating the expectation of the payoff vector as well as after. This distinction leads to two optimisation criteria. With the first criterion, players aim to optimise the expected value of their utility function applied to the payoff vectors obtained in the game. With the second criterion, players aim to optimise the utility of expected payoff vectors given a joint strategy. Under this latter criterion, it was shown that Nash equilibria need not exist. Our first contribution is to provide a sufficient condition under which Nash equilibria are guaranteed to exist. Secondly, we show that when Nash equilibria do exist under both criteria, no equilibrium needs to be shared between the two criteria, and even the number of equilibria can differ. Thirdly, we contribute a study of pure strategy Nash equilibria under both criteria. We show that when assuming quasiconvex utility functions for players, the sets of pure strategy Nash equilibria under both optimisation criteria are equivalent. This result is further extended to games in which players adhere to different optimisation criteria. Finally, given these theoretical results, we construct an algorithm to compute all pure strategy Nash equilibria in MONFGs where players have a quasiconvex utility function. △ Less

Submitted 16 July, 2022; v1 submitted 13 December, 2021; originally announced December 2021.

Preference Communication in Multi-Objective Normal-Form Games

Authors: Willem Röpke, Diederik M. Roijers, Ann Nowé, Roxana Rădulescu

Abstract: We consider preference communication in two-player multi-objective normal-form games. In such games, the payoffs resulting from joint actions are vector-valued. Taking a utility-based approach, we assume there exists a utility function for each player which maps vectors to scalar utilities and consider agents that aim to maximise the utility of expected payoff vectors. As agents typically do not k… ▽ More We consider preference communication in two-player multi-objective normal-form games. In such games, the payoffs resulting from joint actions are vector-valued. Taking a utility-based approach, we assume there exists a utility function for each player which maps vectors to scalar utilities and consider agents that aim to maximise the utility of expected payoff vectors. As agents typically do not know their opponent's utility function or strategy, they must learn policies to interact with each other. Inspired by Stackelberg games, we introduce four novel preference communication protocols to aid agents in arriving at adequate solutions. Each protocol describes a specific approach for one agent to communicate preferences over their actions and how another agent responds. Additionally, to study when communication emerges, we introduce a communication protocol where agents must learn when to communicate. These protocols are subsequently evaluated on a set of five benchmark games against baseline agents that do not communicate. We find that preference communication can alter the learning process and lead to the emergence of cyclic policies which had not been previously observed in this setting. We further observe that the resulting policies can heavily depend on the characteristics of the game that is played. Lastly, we find that communication naturally emerges in both cooperative and self-interested settings. △ Less

Submitted 10 June, 2022; v1 submitted 17 November, 2021; originally announced November 2021.

A Practical Guide to Multi-Objective Reinforcement Learning and Planning

Authors: Conor F. Hayes, Roxana Rădulescu, Eugenio Bargiacchi, Johan Källström, Matthew Macfarlane, Mathieu Reymond, Timothy Verstraeten, Luisa M. Zintgraf, Richard Dazeley, Fredrik Heintz, Enda Howley, Athirai A. Irissappane, Patrick Mannion, Ann Nowé, Gabriel Ramos, Marcello Restelli, Peter Vamplew, Diederik M. Roijers

Abstract: Real-world decision-making tasks are generally complex, requiring trade-offs between multiple, often conflicting, objectives. Despite this, the majority of research in reinforcement learning and decision-theoretic planning either assumes only a single objective, or that multiple objectives can be adequately handled via a simple linear combination. Such approaches may oversimplify the underlying pr… ▽ More Real-world decision-making tasks are generally complex, requiring trade-offs between multiple, often conflicting, objectives. Despite this, the majority of research in reinforcement learning and decision-theoretic planning either assumes only a single objective, or that multiple objectives can be adequately handled via a simple linear combination. Such approaches may oversimplify the underlying problem and hence produce suboptimal results. This paper serves as a guide to the application of multi-objective methods to difficult problems, and is aimed at researchers who are already familiar with single-objective reinforcement learning and planning methods who wish to adopt a multi-objective perspective on their research, as well as practitioners who encounter multi-objective decision problems in practice. It identifies the factors that may influence the nature of the desired solution, and illustrates by example how these influence the design of multi-objective decision-making systems for complex problems. △ Less

Submitted 17 March, 2021; originally announced March 2021.

Journal ref: Auton Agent Multi-Agent Syst 36, 26 (2022)

Opponent Learning Awareness and Modelling in Multi-Objective Normal Form Games

Authors: Roxana Rădulescu, Timothy Verstraeten, Yijie Zhang, Patrick Mannion, Diederik M. Roijers, Ann Nowé

Abstract: Many real-world multi-agent interactions consider multiple distinct criteria, i.e. the payoffs are multi-objective in nature. However, the same multi-objective payoff vector may lead to different utilities for each participant. Therefore, it is essential for an agent to learn about the behaviour of other agents in the system. In this work, we present the first study of the effects of such opponent… ▽ More Many real-world multi-agent interactions consider multiple distinct criteria, i.e. the payoffs are multi-objective in nature. However, the same multi-objective payoff vector may lead to different utilities for each participant. Therefore, it is essential for an agent to learn about the behaviour of other agents in the system. In this work, we present the first study of the effects of such opponent modelling on multi-objective multi-agent interactions with non-linear utilities. Specifically, we consider two-player multi-objective normal form games with non-linear utility functions under the scalarised expected returns optimisation criterion. We contribute novel actor-critic and policy gradient formulations to allow reinforcement learning of mixed strategies in this setting, along with extensions that incorporate opponent policy reconstruction and learning with opponent learning awareness (i.e., learning while considering the impact of one's policy when anticipating the opponent's learning step). Empirical results in five different MONFGs demonstrate that opponent learning awareness and modelling can drastically alter the learning dynamics in this setting. When equilibria are present, opponent modelling can confer significant benefits on agents that implement it. When there are no Nash equilibria, opponent learning awareness and modelling allows agents to still converge to meaningful solutions that approximate equilibria. △ Less

Submitted 14 November, 2020; originally announced November 2020.

Comments: Under review since 14 November 2020

A utility-based analysis of equilibria in multi-objective normal form games

Authors: Roxana Rădulescu, Patrick Mannion, Yijie Zhang, Diederik M. Roijers, Ann Nowé

Abstract: In multi-objective multi-agent systems (MOMAS), agents explicitly consider the possible tradeoffs between conflicting objective functions. We argue that compromises between competing objectives in MOMAS should be analysed on the basis of the utility that these compromises have for the users of a system, where an agent's utility function maps their payoff vectors to scalar utility values. This util… ▽ More In multi-objective multi-agent systems (MOMAS), agents explicitly consider the possible tradeoffs between conflicting objective functions. We argue that compromises between competing objectives in MOMAS should be analysed on the basis of the utility that these compromises have for the users of a system, where an agent's utility function maps their payoff vectors to scalar utility values. This utility-based approach naturally leads to two different optimisation criteria for agents in a MOMAS: expected scalarised returns (ESR) and scalarised expected returns (SER). In this article, we explore the differences between these two criteria using the framework of multi-objective normal form games (MONFGs). We demonstrate that the choice of optimisation criterion (ESR or SER) can radically alter the set of equilibria in a MONFG when non-linear utility functions are used. △ Less

Submitted 17 January, 2020; originally announced January 2020.

Comments: Under review since 16 January 2020

Multi-Objective Multi-Agent Decision Making: A Utility-based Analysis and Survey

Authors: Roxana Rădulescu, Patrick Mannion, Diederik M. Roijers, Ann Nowé

Abstract: The majority of multi-agent system (MAS) implementations aim to optimise agents' policies with respect to a single objective, despite the fact that many real-world problem domains are inherently multi-objective in nature. Multi-objective multi-agent systems (MOMAS) explicitly consider the possible trade-offs between conflicting objective functions. We argue that, in MOMAS, such compromises should… ▽ More The majority of multi-agent system (MAS) implementations aim to optimise agents' policies with respect to a single objective, despite the fact that many real-world problem domains are inherently multi-objective in nature. Multi-objective multi-agent systems (MOMAS) explicitly consider the possible trade-offs between conflicting objective functions. We argue that, in MOMAS, such compromises should be analysed on the basis of the utility that these compromises have for the users of a system. As is standard in multi-objective optimisation, we model the user utility using utility functions that map value or return vectors to scalar values. This approach naturally leads to two different optimisation criteria: expected scalarised returns (ESR) and scalarised expected returns (SER). We develop a new taxonomy which classifies multi-objective multi-agent decision making settings, on the basis of the reward structures, and which and how utility functions are applied. This allows us to offer a structured view of the field, to clearly delineate the current state-of-the-art in multi-objective multi-agent decision making approaches and to identify promising directions for future research. Starting from the execution phase, in which the selected policies are applied and the utility for the users is attained, we analyse which solution concepts apply to the different settings in our taxonomy. Furthermore, we define and discuss these solution concepts under both ESR and SER optimisation criteria. We conclude with a summary of our main findings and a discussion of many promising future research directions in multi-objective multi-agent systems. △ Less

Submitted 6 September, 2019; originally announced September 2019.

Comments: Under review since 15 May 2019

Towards peptide strings pharmacology

Authors: Razvan Tudor Radulescu

Abstract: Peptide strings have been developed as a concept for the past fourteen years. They are proposed to basically entail various quantum states engendered by the physical interactions of proteins containing peptide portions that are similar to one another and, moreover, amino acid sequences that are sterically complementary to each other. In this survey, additional insights are presented that support t… ▽ More Peptide strings have been developed as a concept for the past fourteen years. They are proposed to basically entail various quantum states engendered by the physical interactions of proteins containing peptide portions that are similar to one another and, moreover, amino acid sequences that are sterically complementary to each other. In this survey, additional insights are presented that support the notion that peptide strings are likely a biophysical phenomenon that warrants further investigations. Specifically, these putative peptide strings traits are the capacity for wave-like interferences and electric current-like properties. Therefore, future experimental validation and quantification of these predicted features as well as application of this potential energy that is stored in the distinct shapes of proteins and peptides may prove beneficial in addressing challenges in physics and medicine. Within the latter field, the key discipline of pharmacology should be most fundamentally advanced by the peptide strings approach. △ Less

Submitted 17 February, 2019; originally announced February 2019.

Comments: 9 pages, 3 figures

Analysing Congestion Problems in Multi-agent Reinforcement Learning

Authors: Roxana Rădulescu, Peter Vrancx, Ann Nowé

Abstract: Congestion problems are omnipresent in today's complex networks and represent a challenge in many research domains. In the context of Multi-agent Reinforcement Learning (MARL), approaches like difference rewards and resource abstraction have shown promising results in tackling such problems. Resource abstraction was shown to be an ideal candidate for solving large-scale resource allocation problem… ▽ More Congestion problems are omnipresent in today's complex networks and represent a challenge in many research domains. In the context of Multi-agent Reinforcement Learning (MARL), approaches like difference rewards and resource abstraction have shown promising results in tackling such problems. Resource abstraction was shown to be an ideal candidate for solving large-scale resource allocation problems in a fully decentralized manner. However, its performance and applicability strongly depends on some, until now, undocumented assumptions. Two of the main congestion benchmark problems considered in the literature are: the Beach Problem Domain and the Traffic Lane Domain. In both settings the highest system utility is achieved when overcrowding one resource and kee** the rest at optimum capacity. We analyse how abstract grou** can promote this behaviour and how feasible it is to apply this approach in a real-world domain (i.e., what assumptions need to be satisfied and what knowledge is necessary). We introduce a new test problem, the Road Network Domain (RND), where the resources are no longer independent, but rather part of a network (e.g., road network), thus choosing one path will also impact the load on other paths having common road segments. We demonstrate the application of state-of-the-art MARL methods for this new congestion model and analyse their performance. RND allows us to highlight an important limitation of resource abstraction and show that the difference rewards approach manages to better capture and inform the agents about the dynamics of the environment. △ Less

Submitted 30 March, 2017; v1 submitted 28 February, 2017; originally announced February 2017.

Comments: Adaptive Learning Agents (ALA) Workshop at AAMAS 2017

MSC Class: 68T05 ACM Class: I.2.11

A Comparison Between the Stability Properties in a DDE Model for Leukemia and the Modified Fractional Counterpart

Authors: Ileana Rodica Radulescu, Doina Candea, Eva Kaslik

Abstract: In this paper, a delay differential equations (DDEs) model of leukemia is introduced and its dynamical properties are investigated in comparison with the modified fractional-order system where the Caputo's derivative is used. The model takes into account three types of division that a stem-like cell can undergo and cell competition between healthy and leukemia cell populations. The action of the i… ▽ More In this paper, a delay differential equations (DDEs) model of leukemia is introduced and its dynamical properties are investigated in comparison with the modified fractional-order system where the Caputo's derivative is used. The model takes into account three types of division that a stem-like cell can undergo and cell competition between healthy and leukemia cell populations. The action of the immune system on the leukemic cell populations is also considered. The stability properties of the equilibrium points are established through numerical results and the differences between the two types of approaches are discussed. Medical conclusions are drawn in view of the obtained numerical simulations. △ Less

Submitted 24 November, 2016; originally announced November 2016.

Comments: 11 pages, ICNPAA 2016 Conference, La Rochelle, France

Journal ref: AIP Conference Proceedings 1798, 020127, 2017

Dynamics of complex-valued fractional-order neural networks

Authors: Eva Kaslik, Ileana Rodica Radulescu

Abstract: The dynamics of complex-valued fractional-order neuronal networks are investigated, focusing on stability, instability and Hopf bifurcations. Sufficient conditions for the asymptotic stability and instability of a steady state of the network are derived, based on the complex system parameters and the fractional order of the system, considering simplified neuronal connectivity structures (hub and r… ▽ More The dynamics of complex-valued fractional-order neuronal networks are investigated, focusing on stability, instability and Hopf bifurcations. Sufficient conditions for the asymptotic stability and instability of a steady state of the network are derived, based on the complex system parameters and the fractional order of the system, considering simplified neuronal connectivity structures (hub and ring). In some specific cases, it is possible to identify the critical values of the fractional order for which Hopf bifurcations may occur. Numerical simulations are presented to illustrate the theoretical findings and to investigate the stability of the limit cycles which appear due to Hopf bifurcations. △ Less

Submitted 8 November, 2016; v1 submitted 2 August, 2016; originally announced August 2016.

Comments: 33 pages, 5 figures

Journal ref: Neural Networks, vol 89: 39-49, 2017

Wave and quantum properties of peptide strings: defining a helix in spacetime

Authors: Razvan Tudor Radulescu

Abstract: Previous studies have described the concept of peptide strings in qualitative terms and illustrated it by applying its corrolaries in order to elucidate basic questions in oncology and rheumatology. The present investigation is the first to quantify these potential sub- and transcellular phenomena. Accordingly, the propagation of peptide strings is proposed here to occur by way of waves that in… ▽ More Previous studies have described the concept of peptide strings in qualitative terms and illustrated it by applying its corrolaries in order to elucidate basic questions in oncology and rheumatology. The present investigation is the first to quantify these potential sub- and transcellular phenomena. Accordingly, the propagation of peptide strings is proposed here to occur by way of waves that in turn are subject to the energy equation established by Planck. As a result of these insights, widespread future applications can now be envisaged for peptide strings both in molecular medicine and quantum optics. △ Less

Submitted 25 April, 2009; originally announced April 2009.

Comments: 9 pages, 1 figure

Tumor suppressor and anti-inflammatory protein: an expanded view on insulin-degrading enzyme (IDE)

Authors: Razvan Tudor Radulescu

Abstract: In 1994, I conjectured that insulin-degrading enzyme (IDE) acts as an inhibitor of malignant transformation by degrading insulin and thus preventing this major growth-stimulatory hormone from binding and thereby inactivating the retinoblastoma tumor suppressor protein (RB). Ten years later, I discovered that a carboxyterminal RB amino acid sequence resembles the catalytic center of IDE. This str… ▽ More In 1994, I conjectured that insulin-degrading enzyme (IDE) acts as an inhibitor of malignant transformation by degrading insulin and thus preventing this major growth-stimulatory hormone from binding and thereby inactivating the retinoblastoma tumor suppressor protein (RB). Ten years later, I discovered that a carboxyterminal RB amino acid sequence resembles the catalytic center of IDE. This structural homology raised the possibility that insulin degradation is a basic mechanism for tumor suppression shared by RB and IDE. Subsequently, a first immunohistochemical study on the differential expression of human IDE in normal tissues, primary tumors and their corresponding lymph node metastases further corroborated the initial conjecture on IDE being an antineoplastic molecule. In this report, it is shown that IDE harbors ankyrin repeat-like amino acid sequences through which it might bind and, as a result, antagonize the pro-inflammatory factor NF-kappaB as well as cyclin-dependent kinases (CDKs). As equally revealed here, IDE also contains 2 RXL cyclin-binding motifs which could contribute to its presumed inhibition of CDKs. These new findings suggest that IDE is potentially able to suppress both inflammation and oncogenesis by several mechanisms that ultimately ensure RB function. △ Less

Submitted 30 November, 2008; originally announced December 2008.

Comments: 5 pages, 2 figures

Contagious obesity: from adenovirus 36 to RB dysfunction

Authors: Razvan Tudor Radulescu

Abstract: Significant overweight represents a major health problem in industrialized countries. Besides its known metabolic origins, this condition may also have an infectious cause, as recently postulated. Here, it is surmised that the potentially causative adenovirus 36 contributes to such disorder by inactivating the retinoblastoma tumor suppressor protein (RB) in a manner reminiscent of a mechanism em… ▽ More Significant overweight represents a major health problem in industrialized countries. Besides its known metabolic origins, this condition may also have an infectious cause, as recently postulated. Here, it is surmised that the potentially causative adenovirus 36 contributes to such disorder by inactivating the retinoblastoma tumor suppressor protein (RB) in a manner reminiscent of a mechanism employed by both another pathogenic adenoviral agent and insulin. The present insight additionally suggests novel modes of interfering with obesity-associated pathology. △ Less

Submitted 15 September, 2008; originally announced September 2008.

Comments: 6 pages, 1 figure

Peptide strings clues to the genesis and treatment of rheumatoid arthritis: rebuilding self-protective immunity amid fungal ruins

Authors: Razvan Tudor Radulescu

Abstract: A recent application of the peptide strings concept has yielded novel perceptions on cell growth regulation, for instance that of oncoprotein metastasis. Here, this interdisciplinary approach at the boundary between physics and biology has been applied to gain a more profound insight into rheumatoid arthritis. As a result of the present investigation, this disease could be viewed as due to a met… ▽ More A recent application of the peptide strings concept has yielded novel perceptions on cell growth regulation, for instance that of oncoprotein metastasis. Here, this interdisciplinary approach at the boundary between physics and biology has been applied to gain a more profound insight into rheumatoid arthritis. As a result of the present investigation, this disease could be viewed as due to a metabolic dysregulation/syndrome-associated breakdown in the immunoglobulin A-based surveillance of the potentially pathogenic fungus Candida albicans that subsequently engenders a widespread self-destruction through cross-reactive auto-epitopes, ultimately amounting to the systemic predominance of a pro-inflammatory peptide string. Its therapeutic counterpart equally proposed in this report might serve as a model for future strategies against autoimmunity. △ Less

Submitted 10 August, 2008; originally announced August 2008.

Comments: 15 pages, 3 figures, 3 tables

Oncoprotein metastasis disjoined

Authors: Razvan Tudor Radulescu

Abstract: As the past decade barely dawned, a fundamentally novel view of cancer relating to signal transduction through intracellular hormones/growth factors and their subunits began to unfold. Further along, it gained additional substance with the advent of the interdisciplinary fields of particle biology and peptide strings which explain (onco)protein dynamics in spacetime, for instance insulin-driven… ▽ More As the past decade barely dawned, a fundamentally novel view of cancer relating to signal transduction through intracellular hormones/growth factors and their subunits began to unfold. Further along, it gained additional substance with the advent of the interdisciplinary fields of particle biology and peptide strings which explain (onco)protein dynamics in spacetime, for instance insulin-driven sub- and trans-cellular carcinogenesis, by physical principles. Here, this new understanding is expanded to introduce the concept of "oncoprotein metastasis" preceding cancer cell spread and, thereby, a particular emphasis is placed on its potential role in the emergence of the pre-metastatic niche. Consistent with this perception, yet unlike currently advocated treatments that target cancer cells only, future antineoplastic strategies should aim to mimic natural tumor suppressors as well as involve both (morphologically) normal and malignant cells. If validated in human patients with advanced cancer disease, its otherwise frequently lethal course may be halted and reversed just in time. △ Less

Submitted 18 December, 2007; originally announced December 2007.

Comments: 6 pages

Cell-permeable tumor suppressor peptides for cancer therapy: back to the future

Authors: Razvan Tudor Radulescu

Abstract: Miniaturization is a hallmark of modern technologies. Notably, this feature has not spared molecular biology and its potential applications. Towards develo** more effective therapeutics against cancer, studies began to explore more than a decade ago how natural tumor suppression could be translated into antineoplastic drugs. To this end, investigators focused on major constituents of a central… ▽ More Miniaturization is a hallmark of modern technologies. Notably, this feature has not spared molecular biology and its potential applications. Towards develo** more effective therapeutics against cancer, studies began to explore more than a decade ago how natural tumor suppression could be translated into antineoplastic drugs. To this end, investigators focused on major constituents of a central pathway that protects cells against neoplastic transformation: the nuclear retinoblastoma protein (RB) pathway. As such, peptide mimetics of RB, p16 and p21 were developed. Likewise, the p53 and von Hippel-Lindau gene products which affect indirectly the RB pathway provided additional templates for the development of anti-proliferative peptides. Each of the peptides derived from these distinct tumor suppressors was made cell-permeable by its ligation to an amino acid sequence conferring cellular internalization. Details reviewed here reveal that through the application of such anti-cancer peptide therapeutics alone or in conjunction whenever synergy is to expect, the dark era of chemotherapy will likely be overcome, at last. △ Less

Submitted 29 November, 2007; originally announced November 2007.

Comments: 6 pages

Across and beyond the cell are peptide strings

Authors: Razvan Tudor Radulescu

Abstract: Until presently, most approaches for understanding physiological or pathological phenomena have been based on the assumption that these processes start from individual cells, a concept introduced primarily by Virchow in the 19th century. Yet, it has also been increasingly recognized that this perception is insufficient, at least when it comes to gras** the mechanisms underlying largely incurab… ▽ More Until presently, most approaches for understanding physiological or pathological phenomena have been based on the assumption that these processes start from individual cells, a concept introduced primarily by Virchow in the 19th century. Yet, it has also been increasingly recognized that this perception is insufficient, at least when it comes to gras** the mechanisms underlying largely incurable diseases such as metastatic cancer or rheumatoid arthritis. Despite this insight, even recently founded disciplines such as systems biology are still locked in this century-old mind-set of cellular building blocks and thus predictably of limited usefulness. Other studies conducted over the past years, however, suggest that there is something more fundamental to life and its various conditions than the cell: peptide strings. Here, I review the origin and nature of these sub- and trans-cellular elements as well as their potential to provide the long-sought answers hitherto inaccessible to cell biology. △ Less

Submitted 1 November, 2007; originally announced November 2007.

Comments: 6 pages, 2 figures

The insulin-RB synapse in health and disease: cellular rocket science

Authors: Razvan Tudor Radulescu

Abstract: Time has come for a survey of our knowledge on the physical interaction between the growth-promoting insulin molecule and retinoblastoma tumor suppressor protein (RB). Theoretical and experimental observations over the past 15 years reviewed here indicate that the insulin-RB dimer may represent an essential molecular crossroads involved in major physiological and pathological conditions. Within… ▽ More Time has come for a survey of our knowledge on the physical interaction between the growth-promoting insulin molecule and retinoblastoma tumor suppressor protein (RB). Theoretical and experimental observations over the past 15 years reviewed here indicate that the insulin-RB dimer may represent an essential molecular crossroads involved in major physiological and pathological conditions. Within this system, the putative tumor suppressor insulin-degrading enzyme (IDE) should be an important modulator. Perhaps most remarkably, the abstraction of this encounter between insulin and RB, two growth-regulatory giants acting either in concert or against each other depending on the respective cellular requirements, reveals that Nature may compute in controlling cell fate and we could follow in its footsteps towards develo** more efficient therapeutics as well as novel technical devices. △ Less

Submitted 1 November, 2007; originally announced November 2007.

Comments: 12 pages

Schizophrenia - a parameters' game?

Authors: Anca R. Radulescu

Abstract: Schizophrenia is a severe, currently incurable, relatively common mental condition. Its symptoms are complex and widespread. It structurally and functionally affects cortical and subcortical regions involved in cognitive, emotional and motivational aspects of behavior. Its cause is unknown, its diagnosis is based on statistical behavior and its treatment is elusive. Our paradigm addresses the… ▽ More Schizophrenia is a severe, currently incurable, relatively common mental condition. Its symptoms are complex and widespread. It structurally and functionally affects cortical and subcortical regions involved in cognitive, emotional and motivational aspects of behavior. Its cause is unknown, its diagnosis is based on statistical behavior and its treatment is elusive. Our paradigm addresses the complexity of schizophrenic symptoms. Building upon recent neural vulnerability and limbic dysregulation hypotheses, it offers a mathematical model for the evolution of the limbic system under perturbation. Dependence on parameters and the concept of "bifurcation" could be the key to understanding the threshold between "normality" and "disease". △ Less

Submitted 3 September, 2007; originally announced September 2007.

Comments: The manuscript is 23 pages long (14 pages of text and 9 of references). It contains five figures, two of which in color

The insulin superfamily of growth-promoting proteins

Authors: Razvan Tudor Radulescu

Abstract: Recently, structural analysis of the human transferrin and growth hormone (GH) amino acid sequences has unravelled that they harbor a motif identical to a pattern found in viral oncoproteins known to bind the primarily nuclear tumor suppressor retinoblastoma protein (RB). Since related signatures had previously been identified also in insulin and the two insulin-like growth factors (IGFs), the a… ▽ More Recently, structural analysis of the human transferrin and growth hormone (GH) amino acid sequences has unravelled that they harbor a motif identical to a pattern found in viral oncoproteins known to bind the primarily nuclear tumor suppressor retinoblastoma protein (RB). Since related signatures had previously been identified also in insulin and the two insulin-like growth factors (IGFs), the aim of the current study has been to investigate whether further hints substantiating these reported homologies can be found in silico. Here, additional similarities are presented supporting the notion of an insulin superfamily of growth-promoting proteins with dual localization in the extracellular environment and the intracellular space, particularly in the nucleus, as well as characterized by a tropism for RB. △ Less

Submitted 29 July, 2007; originally announced July 2007.

Comments: 3 pages, 1 figure

Retinoblastoma protein is the likely common effector for distinct anti-aging pathways

Authors: Razvan Tudor Radulescu

Abstract: The multiple worlds of genetically manipulated laboratory organisms such as transgenic mice or worms with certain gene mutations are somewhat reminiscent of parallel worlds in quantum mechanics. So are various models of aging tested in such organisms. In this context, the tumor suppressor p53 has been found to either accelerate or delay aging, the latter, for instance, in conjunction with ARF, a… ▽ More The multiple worlds of genetically manipulated laboratory organisms such as transgenic mice or worms with certain gene mutations are somewhat reminiscent of parallel worlds in quantum mechanics. So are various models of aging tested in such organisms. In this context, the tumor suppressor p53 has been found to either accelerate or delay aging, the latter, for instance, in conjunction with ARF, another tumor suppressor, as shown very recently. To more easily determine which of these artificial settings comes closest to real life, I discuss here their features in the light of my protein structure-based insights that have led me to propose a physiological anti-aging role for the retinoblastoma tumor suppressor protein (RB) over the past four years. △ Less

Submitted 27 July, 2007; originally announced July 2007.

Comments: 4 pages

Planet RB: a personal contribution to a proteomic map of human retinoblastoma protein

Authors: Razvan Tudor Radulescu

Abstract: As I compress on the canvas of a few pages here major results of my research on the retinoblastoma tumor suppressor protein (RB) spreading over the past 15 years, an exciting picture emerges on this unique host molecule which surpasses in its complexity even that of the most capable viral proteins known to date. Accordingly, RB has the potential to bind not only growth-promoting proteins such as… ▽ More As I compress on the canvas of a few pages here major results of my research on the retinoblastoma tumor suppressor protein (RB) spreading over the past 15 years, an exciting picture emerges on this unique host molecule which surpasses in its complexity even that of the most capable viral proteins known to date. Accordingly, RB has the potential to bind not only growth-promoting proteins such as insulin, but also to attach itself to calcium and oxygen, as well as to be secreted into the extracellular environment. Moreover, RB may exert proteolytic, antimicrobial and anti-aging activities. These condensed structure-based insights on RB are the substance of a scientific revolution I have initiated a long time ago, yet likely to gain even further speed in the years to come, thus expanding both our understanding of life at the molecular level and the possibilities for pharmacological modulation of fundamental biological phenomena, particularly in oncology and gerontology. △ Less

Submitted 13 June, 2007; originally announced June 2007.

Comments: 4 pages, 1 table

Antiproliferative MCR peptides block physical interaction of insulin with retinoblastoma protein (RB) in human lung cancer cells

Authors: Razvan Tudor Radulescu, Kai Kehe

Abstract: Fifteen years ago, a structural analysis of the hormone insulin and the retinoblastoma tumor suppressor protein (RB) revealed that they may physically interact with one another. Subsequently, an RB peptide corresponding to the proposed RB binding site for insulin was found to recognize full-length insulin in vitro. As part of efforts aimed at develo** this RB peptide into an anti-cancer drug,… ▽ More Fifteen years ago, a structural analysis of the hormone insulin and the retinoblastoma tumor suppressor protein (RB) revealed that they may physically interact with one another. Subsequently, an RB peptide corresponding to the proposed RB binding site for insulin was found to recognize full-length insulin in vitro. As part of efforts aimed at develo** this RB peptide into an anti-cancer drug, this molecule was chemically coupled to a cellular internalization signal and termed "MCR peptide". Meanwhile, several such MCR peptide variants have been demonstrated to restrain the proliferation of different human cancer cells in vitro and in vivo. Moreover, one of the MCR peptides coined MCR-10 was shown to be capable of interfering with the complex formation between insulin and RB in HepG2 human hepatoma cells, as monitored by immunofluorescence. This latter result indicating an in vivo association between insulin and RB was confirmed by a follow-up study combining the methods of co-immunoprecipitation and immunoblotting. Here, we provide evidence for the existence of the insulin-RB complex in A549 human non-small cell lung cancer cells. Specifically, we demonstrate this heterodimer by means of a magnetic beads-based immunoprecipitation approach and equally show that this dimer can be disrupted by MCR-4 or MCR-10 each of which is known to possess antiproliferative properties, yet to a much lesser extent by a control peptide. Thus, this investigation has yielded another important proof for the occurrence of the insulin-RB dimer and, furthermore, its validity as a target for antineoplastic MCR peptides. △ Less

Submitted 13 June, 2007; originally announced June 2007.

Comments: 12 pages, 3 figures

Immunohistochemical pitfalls in the demonstration of insulin-degrading enzyme in normal and neoplastic human tissues

Authors: Razvan T. Radulescu, Angelika Jahn, Daniela Hellmann, Gregor Weirich

Abstract: Previously, we have identified the cytoplasmic zinc metalloprotease insulin-degrading enzyme(IDE) in human tissues by an immunohistochemical method involving no antigen retrieval (AR) by pressure cooking to avoid artifacts by endogenous biotin exposure and a detection kit based on the labeled streptavidin biotin (LSAB) method. Thereby, we also employed 3% hydrogen peroxide(H2O2) for the inhibiti… ▽ More Previously, we have identified the cytoplasmic zinc metalloprotease insulin-degrading enzyme(IDE) in human tissues by an immunohistochemical method involving no antigen retrieval (AR) by pressure cooking to avoid artifacts by endogenous biotin exposure and a detection kit based on the labeled streptavidin biotin (LSAB) method. Thereby, we also employed 3% hydrogen peroxide(H2O2) for the inhibition of endogenous peroxidase activity and incubated the tissue sections with the biotinylated secondary antibody at room temperature (RT). We now add the immunohistochemical details that had led us to this optimized procedure as they also bear a more general relevance when demonstrating intracellular tissue antigens. Our most important result is that endogenous peroxidase inhibition by 0.3% H2O2 coincided with an apparently positive IDE staining in an investigated breast cancer specimen whereas combining a block by 3% H2O2 with an incubation of the biotinylated secondary antibody at RT, yet not at 37 degrees Celsius, revealed this specimen as almost entirely IDE-negative. Our present data caution against three different immunohistochemical pitfalls that might cause falsely positive results and artifacts when using an LSAB- and peroxidase-based detection method: pressure cooking for AR, insufficient quenching of endogenous peroxidases and heating of tissue sections while incubating with biotinylated secondary antibodies. △ Less

Submitted 2 May, 2007; originally announced May 2007.

Comments: 17 pages, 6 figures