We and our partners use cookies to give you the best online experience, including to personalize content, advertising, and web analytics. You can reject cookies by changing your browser settings. To learn more about the cookies we use see our Cookie Policy.

Accept & Continue
AnyLogic
Back to blog

What links Machine Learning, Deep Learning, and Simulation?

|

Lyle Wallis in front of some grid worlds

* Take a deep dive with our AI and simulation materials and downloads.

Reinforcement Learning, that’s where all the action is these days says Lyle Wallis, Advisory Analytics Director at PwC. It is also what connects Machine Learning, Deep Learning, and Simulation. How so? And why is this important?

In a presentation at the AnyLogic Conference 2018, Lyle explained the various uses of simulation in artificial intelligence as well as its importance. It was a follow-up to his incredibly popular simulation and AI integration insights from the previous year.

To begin with, the three main uses for combining simulation and AI are given:

  • Simulations for deep reinforcement learning and training neural networks.
  • Modeling the real world – if you are modeling a system with AI in it, then the model should have the AI in it as well.
  • Using AI to calibrate and debug models.

Primarily, the focus of the presentation is on the first use case, where simulations are used to help train reinforcement learning systems, but Lyle explains that the second use case is rapidly being applied in the operations and supply chain worlds. These are industries where machine learning is already in use for tasks such as routing optimization. As a result, for true representation, it is necessary for digital twins and models of these real-world systems to capture their AI elements.


A closer look at reinforcement learning

With the use cases covered, a quick primer on the workings of deep reinforcement learning shows a grid world model at work in AnyLogic. The reason for this, Lyle explains, is that most current examples of deep reinforcement learning make use of grid world type models. [If you are interested, you can setup your own AI grid world in AnyLogic. Benjamin Schumann guides you through the steps in his four-part instructional blog — including videos and downloads.] The grid world model provides insight into the operation of neural networks.

Despite the simple appearance of the grid world model, the power of deep reinforcement learning may be better understood from its successes in the game of Go, and more recently StarCraft. In these cases, world masters lost to DeepMind’s machine learning – specifically deep reinforcement learning.



Applying deep learning

Further into the presentation is a demonstration of how the PwC team moved beyond grid world with an experiment to explore the application of AI in AnyLogic. They took the ‘Candy game’ example model, where companies compete for market share, and, using DL4J, replaced one of the competing companies with an AI agent. The experiment yielded some important observations.

Lyle likens the AI agent to an “optimizer on steroids” that, over the course of its training, would find and exploit every weakness in a model. The implication of this is that if a model is not robust, extra work is needed to properly define the area under investigation. On the flip-side, however, it is possible that previously unknown patterns and novel strategies can be revealed. In the worlds of Go and StarCraft, it is these discoveries that led to machine success and also the welcome interest of expert practitioners. To a company, it can mean competitive advantage, such as when Netflix reengineered its recommendation engine. There is a caveat though.

Complexity quickly increases when expanding a problem domain and can become difficult to manage, as discovered with the ‘Candy game’. Compute time rises rapidly with the number of variables, and there can be problems defining what the end points are and what constitutes success. It is because of these issues that games, with their set actions and environments, provide clarity that the real world cannot.

Of course, cloud computing and specialized hardware can help with computing times, but much still relies on the quality of a model. And it is on this topic of simulation modeling that the presentation concludes. (Although, you should watch on for the Q&A session)

Simulation is a key AI technology, with its three use cases outlined above, and it is especially useful for helping create reinforcement learning models where the training data is too dangerous, expensive, or otherwise impossible to obtain from the real world. Watch the video and hear from Lyle himself, why simulation is a key AI technology.


Video also available on YouKu

⚡ Find out more about the benefits of simulation in AI with our collection of materials and downloads.

The presentation slides are available in PDF.

Related posts