A compositional framework for Markov processes (Q2798694)

A continuous-time Markov chain is a way to determine the dynamics of a population which is spread across some finite set of states. Population can flow between the states. Under certain conditions the population of the states tend towards an equilibrium in which at any state the inflow of population is equal to its outflow. In an electrical circuit of linear registors, charge can flow along wires, In equilibrium, without any driving voltage from outside, the current along each wire is zero, and the potential at each node is equal.NEWLINEA continuous-time Markov chains is called \textit{detailed balanced} if in equilibrium, for any two vertices connected by an edge, the flow from one of the two vertices to the other equals the opposite flow. The authors call continuous-time Markov chains \textit{Markov processes}, and the principal objective in this paper is to formalize and exploit the well-known analogy between detailed balanced Markov processes and electrical circuits of linear registors ($\text{potential}\Leftrightarrow\text{population}$, $\text{current}\Leftrightarrow\text{flow}$, $\text{conductance}\Leftrightarrow\text{rate constant}$, $\text{power}\Leftrightarrow\text{dissipation}$) [\textit{F. P. Kelly}, Reversibility and stochastic networks. Chichester etc.: John Wiley \& Sons (1979; Zbl 0422.60001); Reversibility and stochastic networks. With a new preface. Reprint of the 1979 ed. Cambridge: Cambridge University Press (2011; Zbl 1260.60001); \textit{J. F. C. Kingman}, J. Appl. Probab. 6, 1--18 (1969; Zbl 0177.21807); \textit{C. St. J. A. Nash-Williams}, Proc. Camb. Philos. Soc. 55, 181--194 (1959; Zbl 0100.13602)].NEWLINENEWLINEThe authors have studied electrical circuits by introducing a framework for \textit{black boxing} a circuit and extracting the relations it determines between potential-current pairs at the input amd output terminals [\textit{J. C. Baez} and \textit{B. Fong}, Theory Appl. Categ. 33, 1158--1222 (2018; Zbl 1402.18005)]. The relation depicts the external behavior as can be observed by someone who is to perform measurements at the terminals. It is significant that black boxing is \textit{compositional} in the sense that if one builds a circuit from smaller pieces, the external behavior of the entire circuit can be determined from those of the pieces. This paper exploits this framework to detailed balanced Markov processes.NEWLINENEWLINEThe paper consists of 14 sections together with a tutorial appendix on decorated cospans [\textit{B. Fong}, Theory Appl. Categ. 30, 1096--1120 (2015; Zbl 1351.18003)]. \S II is an overview of main ideas. \S III is a review of Markov processes. \S IV defines open Markov processes and the open master equation. \S V introduces the concept of detailed balance for open Markov processes. \S VI recalls the principle of minimum power for open circuits of linear registors and expounds how to black box them. \S VII introduces the principle of minimum dissipation for open detailed balanced\ Markov processes and explicate how to black box them. \S VIII claims the analogy between electrical circuits and detailed balanced\ Markov processes formally. \S IX describes how to compose open Markov processes, while \S X does the same for detailed balanced\ Markov processes. \S XI describes the black box functor sending any open detailed balanced\ Markov process to the linear relation which describes its external behavior. \S XII makes the analogy between open detailed balanced\ Markov processes and open circuits a functor. It is shown in \S XIII that the linear relations in the image of these black box functors are Lagrangian relations between symplectic vector spaces, being shown also that the master equation is to be seen as a gradient flow equation. \S XIV summarizes the main findings in the paper.