-
Markov Decision Process and Approximate Dynamic Programming for a Patient Assignment Scheduling problem
Authors:
Malgorzata M. O'Reilly,
Sebastian Krasnicki,
James Montgomery,
Mojtaba Heydar,
Richard Turner,
Pieter Van Dam,
Peter Maree
Abstract:
We study the Patient Assignment Scheduling (PAS) problem in a random environment that arises in the management of patient flow in the hospital systems, due to the stochastic nature of the arrivals as well as the Length of Stay distribution. We develop a Markov Decision Process (MDP) which aims to assign the newly arrived patients in an optimal way so as to minimise the total expected long-run cost…
▽ More
We study the Patient Assignment Scheduling (PAS) problem in a random environment that arises in the management of patient flow in the hospital systems, due to the stochastic nature of the arrivals as well as the Length of Stay distribution. We develop a Markov Decision Process (MDP) which aims to assign the newly arrived patients in an optimal way so as to minimise the total expected long-run cost per unit time over an infinite horizon. We assume Poisson arrival rates that depend on patient types, and Length of Stay distributions that depend on whether patients stay in their primary wards or not. Since the instances of realistic size of this problem are not easy to solve, we develop numerical methods based on Approximate Dynamic Programming. We illustrate the theory with numerical examples with parameters obtained by fitting to data from a tertiary referral hospital in Australia, and demonstrate the application potential of our methodology under practical considerations.
△ Less
Submitted 26 June, 2024;
originally announced June 2024.
-
Matrix-analytic methods for the evolution of species trees, gene trees, and their reconciliation
Authors:
Albert C. Soewongsono,
Jiahao Diao,
Tristan Stark,
Amanda E. Wilson,
David A. Liberles,
Barbara R. Holland,
Malgorzata M. O'Reilly
Abstract:
We consider the reconciliation problem, in which the task is to find a map** of a gene tree into a species tree, so as to maximize the likelihood of such fitting, given the available data. We describe a model for the evolution of the species tree, a subfunctionalisation model for the evolution of the gene tree, and provide an algorithm to compute the likelihood of the reconciliation. We derive o…
▽ More
We consider the reconciliation problem, in which the task is to find a map** of a gene tree into a species tree, so as to maximize the likelihood of such fitting, given the available data. We describe a model for the evolution of the species tree, a subfunctionalisation model for the evolution of the gene tree, and provide an algorithm to compute the likelihood of the reconciliation. We derive our results using the theory of matrix-analytic methods and describe efficient algorithms for the computation of a range of useful metrics. We illustrate the theory with examples and provide the physical interpretations of the discussed quantities, with a focus on the practical applications of the theory to incomplete data.
△ Less
Submitted 8 November, 2023; v1 submitted 11 September, 2023;
originally announced September 2023.
-
Modelling gene content across a phylogeny to determine when genes become associated
Authors:
Jiahao Diao,
Malgorzata M. O'Reilly,
Barbara R. Holland
Abstract:
In this work, we develop a stochastic model of gene gain and loss with the aim of inferring when (if at all) in evolutionary history and association between two genes arises. The data we consider is a species tree along with information on the presence or absence of two genes in each of the species. The biological motivation for our model is that if two genes are involved in the same biochemical p…
▽ More
In this work, we develop a stochastic model of gene gain and loss with the aim of inferring when (if at all) in evolutionary history and association between two genes arises. The data we consider is a species tree along with information on the presence or absence of two genes in each of the species. The biological motivation for our model is that if two genes are involved in the same biochemical pathway, i.e. they are both required for some function, then the rate of gain or loss of one gene in the pathway should depend upon the presence or absence of the other gene in the pathway. However, if the two genes are not functionally linked, then the rate of gain or loss of one gene should be independent of the state of another gene.
We simulate data under this model to determine under what conditions a shift from the independent rates class to the dependent rates class can be detected. For example, how large a tree is required and how large a shift in the rates is needed before Akaike information criterion (AIC) supports a model with two rate classes over a simpler model with just one rate class? If a model with two rate classes is preferred, can it correctly detect where on the evolutionary tree the shift occurred?
△ Less
Submitted 6 September, 2023;
originally announced September 2023.
-
Stochastic niche-based models for the evolution of species
Authors:
Albert Ch. Soewongsono,
Barbara R. Holland,
Malgorzata M. O'Reilly
Abstract:
There have been many studies to examine whether one trait is correlated with another trait across a group of present-day species (for example, do species with larger brains tend to have longer gestation times. Since the introduction of the phylogenetic comparative method some authors have argued that it is necessary to have a biologically realistic model to generate evolutionary trees that incorpo…
▽ More
There have been many studies to examine whether one trait is correlated with another trait across a group of present-day species (for example, do species with larger brains tend to have longer gestation times. Since the introduction of the phylogenetic comparative method some authors have argued that it is necessary to have a biologically realistic model to generate evolutionary trees that incorporates information about the ecological niche occupied by species. Price presented a simple model along these lines in 1997. He defined a two-dimensional niche space formed by two continuous-valued traits, in which new niches arise with trait values drawn from a bivariate normal distribution. When a new niche arises, it is occupied by a descendant species of whichever current species is closest in ecological niche space. In sequence, more species are then evolved from already-existing species to which they are ecologically closest.
Here we explore ways of extending Price's adaptive radiation model. One extension is to increase the dimensionality of the niche space by considering more than two continuous traits. A second extension is to allow both extinction of species (which may leave unoccupied niches) and removal of niches (which causes species occupying them to go extinct). To model this problem, we consider a continuous-time stochastic process which implicitly defines a phylogeny. To explore if trees generated under such a model (or under different parametrizations of the model) are realistic we can compute a variety of summary statistics that can be compared to those of empirically observed phylogenies. For example, there are existing statistics that aim to measure: tree balance, the relative rate of diversification, and phylogenetic signal of traits.
△ Less
Submitted 6 September, 2023;
originally announced September 2023.
-
A Stochastic Fluid Model Approach to the Stationary Distribution of the Maximum Priority Process
Authors:
Hiska M. Boelema,
Daan J. J. Dams,
Malgorzata M. O'Reilly,
Werner R. W. Scheinhardt,
Peter G. Taylor
Abstract:
In traditional priority queues, we assume that every customer upon arrival has a fixed, class-dependent priority, and that a customer may not commence service if a customer with a higher priority is present in the queue. However, in situations where a performance target in terms of the tails of the class-dependent waiting time distributions has to be met, such models of priority queueing may not b…
▽ More
In traditional priority queues, we assume that every customer upon arrival has a fixed, class-dependent priority, and that a customer may not commence service if a customer with a higher priority is present in the queue. However, in situations where a performance target in terms of the tails of the class-dependent waiting time distributions has to be met, such models of priority queueing may not be satisfactory. In fact, there could be situations where high priority classes easily meet their performance target for the maximum waiting time, while lower classes do not.
Here, we are interested in the stationary distribution at the times of commencement of service of this maximum priority process. Until now, there has been no explicit expression for this distribution. We construct a map** of the maximum priority process to a tandem fluid queue, which enables us to find expressions for this stationary distribution. We derive the results for the stationary distribution of the maximum priority process at the times of the commencement of service.
△ Less
Submitted 6 September, 2023;
originally announced September 2023.
-
Sensitivity analysis of Quasi-Birth-and-Death processes
Authors:
Anna Aksamit,
Małgorzata M. O'Reilly,
Zbigniew Palmowski
Abstract:
We perform the sensitivity analysis of a level-dependent QBD with a particular focus on applications in modelling healthcare systems.
We perform the sensitivity analysis of a level-dependent QBD with a particular focus on applications in modelling healthcare systems.
△ Less
Submitted 4 February, 2023;
originally announced February 2023.
-
Random walk on a quadrant: map** to a one-dimensional level-dependent Quasi-Birth-and-Death process (LD-QBD)
Authors:
Małgorzata M. O'Reilly,
Zbigniew Palmowski,
Anna Aksamit
Abstract:
We consider a neighbourhood random walk on a quadrant, $\{(X_1(t),X_2(t),\varphi(t)):t\geq 0\}$, with state space \begin{eqnarray*}
\mathcal{S}&=&\{(n,m,i):n,m=0,1,2,\ldots;i=1,2,\ldots,k(n,m)\}. \end{eqnarray*} Assuming start in state $(n,m,i)$, the process spends exponentially distributed amount of time in $(n,m,i)$ according to some parameter $λ_i^{(n,m)}$. Upon leaving state $(n,m,i)$ the pr…
▽ More
We consider a neighbourhood random walk on a quadrant, $\{(X_1(t),X_2(t),\varphi(t)):t\geq 0\}$, with state space \begin{eqnarray*}
\mathcal{S}&=&\{(n,m,i):n,m=0,1,2,\ldots;i=1,2,\ldots,k(n,m)\}. \end{eqnarray*} Assuming start in state $(n,m,i)$, the process spends exponentially distributed amount of time in $(n,m,i)$ according to some parameter $λ_i^{(n,m)}$. Upon leaving state $(n,m,i)$ the process moves to some state $(n^{'},m^{'},j)$ with $j\in\{1,\ldots,k(n^{'},m^{'})\}$ and $n^{'}\in\{n-1,n,n+1\}$, $m^{'}\in\{m-1,m,m+1\}$, according to some probabilities $(p_{n;a}^{m;b})_{i,j}$ with $a,b\in\{+,-,0\}$. We transform this process into a one-dimensional LD-QBD $\{(Z(t),χ(t)):t\geq 0\}$ with level variable $Z(t)$ and phase variable $χ(t)$. Using this transform we find its transient and stationary analysis using matrix-analytic methods, as well as the distribution at first hitting times.
△ Less
Submitted 4 February, 2023;
originally announced February 2023.
-
Matrix-Analytic Methods for the analysis of Stochastic Fluid-Fluid Models
Authors:
Nigel G. Bean,
Małgorzata M. O'Reilly,
Zbigniew Palmowski
Abstract:
Stochastic fluid-fluid models (SFFMs) offer powerful modeling ability for a wide range of real-life systems of significance. The existing theoretical framework for this class of models is in terms of operator-analytic methods. For the first time, we establish matrix-analytic methods for the efficient analysis of SFFMs. We illustrate the theory with numerical examples.
Stochastic fluid-fluid models (SFFMs) offer powerful modeling ability for a wide range of real-life systems of significance. The existing theoretical framework for this class of models is in terms of operator-analytic methods. For the first time, we establish matrix-analytic methods for the efficient analysis of SFFMs. We illustrate the theory with numerical examples.
△ Less
Submitted 28 February, 2022; v1 submitted 25 October, 2020;
originally announced October 2020.
-
Yaglom limit for Stochastic Fluid Models
Authors:
Nigel G. Bean,
Małgorzata M. O'Reilly,
Zbigniew Palmowski
Abstract:
In this paper we provide the analysis of the limiting conditional distribution (Yaglom limit) for stochastic fluid models (SFMs), a key class of models in the theory of matrix-analytic methods. So far, transient and stationary analyses of the SFMs have been only considered in the literature. The limiting conditional distribution gives useful insights into what happens when the process has been evo…
▽ More
In this paper we provide the analysis of the limiting conditional distribution (Yaglom limit) for stochastic fluid models (SFMs), a key class of models in the theory of matrix-analytic methods. So far, transient and stationary analyses of the SFMs have been only considered in the literature. The limiting conditional distribution gives useful insights into what happens when the process has been evolving for a long time, given its busy period has not ended yet. We derive expressions for the Yaglom limit in terms of the singularity $s^*$ such that the key matrix of the SFM, ${\bfΨ}(s)$, is finite (exists) for all $s\geq s^*$ and infinite for $s<s^*$. We show the uniqueness of the Yaglom limit and illustrate the application of the theory with simple examples.
△ Less
Submitted 20 June, 2020; v1 submitted 28 August, 2019;
originally announced August 2019.
-
A discontinuous Galerkin method for approximating the stationary distribution of stochastic fluid-fluid processes
Authors:
Nigel Bean,
Giang T. Nguyen,
Malgorzata M. O'Reilly,
Vikram Sunkara
Abstract:
Introduced by Bean and O'Reilly (2014), a stochastic fluid-fluid process is a Markov processes $\{X_t, Y_t, \varphi_t\}_{t \geq 0}$, where the first fluid $X_t$ is driven by the Markov chain $\varphi_t$, and the second fluid $Y_t$ is driven by $\varphi_t$ as well as by $X_t$. That paper derived a closed-form expression for the joint stationary distribution, given in terms of operators acting on me…
▽ More
Introduced by Bean and O'Reilly (2014), a stochastic fluid-fluid process is a Markov processes $\{X_t, Y_t, \varphi_t\}_{t \geq 0}$, where the first fluid $X_t$ is driven by the Markov chain $\varphi_t$, and the second fluid $Y_t$ is driven by $\varphi_t$ as well as by $X_t$. That paper derived a closed-form expression for the joint stationary distribution, given in terms of operators acting on measures, which does not lend itself easily to numerical computations.
Here, we construct a discontinuous Galerkin method for approximating this stationary distribution, and illustrate the methodology using an on-off bandwidth sharing system, which is a special case of a stochastic fluid-fluid process.
△ Less
Submitted 29 January, 2019;
originally announced January 2019.
