PhysiGym : bridging the gap between the Gymnasium reinforcement learning application interface and the PhysiCell agent-based modeling framework

ElmarBucher

Indiana University / Intelligent Systems Engineering

"PhysiGym : bridging the gap between the Gymnasium reinforcement learning application interface and the PhysiCell agent-based modeling framework"

Reinforcement learning (RL) is a powerful machine learning paradigm in which an RL agent learns to discover optimal strategies in uncertain environments. The RL control strategy has achieved remarkable success in complex tasks such as playing Chess, Go, and StarCraft. For RL, the prevailing application interface (API) standard is Gymnasium, a Python library [1]. Agent-based (AB) modeling is a mathematical, dynamical system modeling approach where the parts of the system, the so-called agents, autonomously act according to agent-type specific rules. PhysiCell is an AB modeling framework written in C++ and was implemented to model multicellular systems based on Newtonian physics. Cells are the agents. The cell type specifies the rule set the agents apply. Tissue structure emerges from the cell interactions. Substrates like oxygen can be modeled with the integrated BioFVM diffusive transport solver. Additionally, intracellular models can be integrated into cell agents [2]. The resulting AB models are 2 or 3-dimensional, off-lattice, center-based, and multiscale in space and time. In this talk, we will introduce PhysiGym, a well-documented and on all major operating systems tested open-source framework written in C++ and Python that allows to control PhysiCell models over the Gymnasium API. After a brief introduction to AB models and RL, we will discuss the implementation and obtained results from our tumor microenvironment model and the RL algorithms we applied to the model. In the future, PhysiGym can be used to learn from simulations possible mechanisms that might explain how biology systems react to similar real-world control. Furthermore, if cancer patient digital twins are written as PhysiCell models, PhysiGym could ultimately be used by oncologists to explore RL reward functions to improve treatment efficacy, reduce side effects, and slow or prevent resistance. References: [1] https://gymnasium.farama.org/ , [2] https://PhysiCell.org
Additional authors:

---