Optimal control of molecular dynamics using Markov state models (Q715243)

In the article the authors ``propose an approach to solving the Hamilton-Jacobi-Bellman (HJB) equation for optimal control of stochastic molecular dynamics (MD) in high dimensions. The main idea is to first approximate the dominant modes of the molecular dynamics transfer operator by a low-dimensional, so-called Markov state model (MSM), and then solve the HJB for the MSM rather then the full MD. The type of optimal control problems that [they] consider here, and which appear relevant in molecular dynamics applications, belong to the class of ergodic stochastic control problems'' (from authors' introduction). From the abstract: ``A numerical scheme for solving high dimensional stochastic control problems [as described above] on an infinite time horizon \dots is outlined. The scheme rests on the interpretation of the corresponding HJB equation as a nonlinear eigenvalue problem that, using a logarithmic transformation, can be recast as a linear eigenvalue problem, for which the principal eigenvalue and its eigenfunction are sought.'' The authors give a proof of concept that the linear eigenvalue problem ``can be computed efficiently by approximating the underlying stochastic process with a coarse-grained Markov state model for the dominant metastable sets'' (from the abstract). A good feature of the proposed scheme is that the MSM can be sampled from MD simulation data using the uncontrolled dynamics. In conclusion the method is illustrated with ``two numerical examples, one of which involves the task of maximizing the population of \(\alpha\)-helices in an ensemble of small biomolecules (alanine dipeptide)'' (from the abstract).