For use cases that would benefit from being able to run and control runs of an AnyLogic model locally in a Python script, namely for training a reinforcement learning policy, the Alpyne library can be used.
To prepare an AnyLogic model for usage in this context, one must simply create and define a Reinforcement Learning Experiment and create decision points at which some action is desired to be taken. After installing the Alpyne library in your desired Python environment, you can take the model exported from the Reinforcement Learning Experiment and create new runs with some desired configuration, query the model state, take actions, and retrieve outputs—all from your Python environment.
Note that due to the range of libraries and techniques usable for training policies, Alpyne’s focus is on the usage of the model outside of AnyLogic. To query trained policies inside your AnyLogic model requires an implementation appropriate to how it was trained.
Usage is available for any edition of AnyLogic (Personal Learning Edition, University, or Professional), though any limitations still apply, and is compatible with Python 3.10 or above.
The AnyLogic Company has developed Alpyne as a third-party library that is a free and optional connectivity tool, without obligation to provide official support or a promise of compatibility of any kind. Users of the library may ask questions or provide comments on the Issue or Discussion tabs of the GitHub page.
The examples below demonstrate a range of different topics and implementation options. The source model/scripts and further information about each are available to download from the GitHub repository.
This is a toy model useful for getting an initial understanding of Alpyne. It depicts an agent in an 8x8 grid where it’s trying to reach a single “goal” space while avoiding a number of “hole” spaces.
The provided Python scripts allow you to take control over the agent’s movements or to train a simple policy using Q-learning (i.e., no neural networks). The resulting policy is saved as a simple JSON file, which the model is set up to be able to parse and execute on.
This is a modified version of the model in the built-in repository with the same base name. It depicts a single warehouse in a supply chain with a stock that is continuously consumed (exogenous and varying); the rate it’s resupplied at is controllable, with changes taking effect after some lag.
The provided Python script trains a reinforcement learning algorithm to vary the re-supply rate such that the stock is kept at a balanced level, avoiding running out and overstocking. To load the trained policy back into the original model, the Pypeline library is used.
This is a modified version of the AnyLogic-provided example “Border Checkpoint.” It represents the process of passport control and vehicle inspection at the crossing of an international border. The model is set up to allow overriding the schedule for the number of car and bus inspectors (two resource pools) for the duration of the model.
The default model has an optimization experiment intended to find, for a fixed arrival rate, the optimal set of parameters to minimize the mean cost per product. Alpyne is used to train a reinforcement learning policy to optimally set the parameters for any arrival rate.
This is a modified version of the AnyLogic-provided example “Border Checkpoint.” It represents the process of passport control and vehicle inspection at the crossing of an international border. The model is set up to allow overriding the schedule for the number of car and bus inspectors (two resource pools) for the duration of the model.
The Python scripts seek to find the optimal number of each inspector type for each of the three shifts in the schedule, defined in a local Excel file. The strategy uses Bayesian Optimization for its sample efficiency.
AnyLogic simulation is the training and testing platform for AI in business. With AnyLogic general-purpose simulation, you can construct detailed and robust virtual environments for training and testing your AI models. The unique multi-method simulation capabilities provide a comprehensive tool for use in machine learning. Established in use at leading companies across industries, this fully cloud enabled platform with open API is enhancing and accelerating AI development today. Find out more about this powerful machine learning tool in our AI email pack and white paper!
