Overview
Engineering Ingegneria Informatica is an international specialist in the field of digital system integration. The company has more than 11,000 employees in more than 50 offices around the world. One of its flagship projects is an ecosystem of platforms that enable other technologies to interact with each other – exchanging value, digitizing processes, developing digital services, and creating value for users, especially through the leverage of eight.
Lagor is an Italian company producing ferromagnetic cores for power transformers. Lagor was the first in Europe to offer an electrical steel cutting service, and it remains the leader in the transformer core market. However, while ramping up production and expanding the business, the company faced issues scaling the manufacturing processes. Lagor contracted Engineering to resolve the bottlenecks and blockages in the production system for manufacturing optimization.
Problem
Power transformer cores are made of many layers of coils, can weigh up to eight tons, and require different production cycles depending on their size and client-specific requirements.
Power transformer core production starts with the layering of cores on top of each other to reach a desired thickness. The materials remain on a steel pallet during the entire production process. These pallets move between the different workstations using roller or shuttle conveyors.
Manufacturing simulation showcase
Manufacturing simulation limits
All cores go through processing, and some continue through a painting station and a curing station, depending on the production cycle. At the end, all cores undergo testing. Steel pallets are never removed from the line, even if they are empty.
Initially, manufacturing line scheduling was conducted manually and for short periods. This method resulted in recurrent issues where objects on the production line created bottlenecks. The most practical way to resolve major blockages was to unload the cores by crane and reset the whole line. With the aim of making the process more efficient, Lagor approached Engineering company. Together, they would streamline production line operations and deliver better management of shop floor movements.
Solution
With the help of AnyLogic manufacturing simulation capabilities, the consultants created a model to act as a digital twin of the production system. With the digital twin technology, they would be able to feed real-time data, direct from the field of operation, into a manufacturing simulation model to better understand the problems and predict the future performance of the production facility. They used an agent-based approach to model the project’s unique features, including the diversity of core types, related production cycles, and a variable production plan.
Production process sequencing simulation
optimized with reinforcement learning
The digital twin technology helped reproduce various production process elements such as:
- Steel pallets – critical resources, where the cores are housed on the production line. Regulations state that pallets must never be unloaded from the line.
- Power transformer cores, which move between workstations depending on the specifics of their production cycle.
- Conveyors – pallet-carrying components, which can be static (rolling) or moving (shuttle).
Learnt policy applied for the optimized production process
After designing the digital twin model, the consultants connected data from the production facility to the supervisory control and data acquisition system (SCADA) to get an updated line status. They also created a line manager – a virtual agent, which seeks the optimal route for each situation in order to avoid unnecessary movements, anticipate possible criticalities, resolve conflicts, and at the same time, respect delivery dates. To govern the line and achieve better results, the line manager used embedded heuristics-based algorithms.
The digital twin of the shop floor, working with the real data, helped reproduce the production and decision-making processes, investigate the production plan, and verify that the selected plan was achievable while respecting delivery dates. With the new digital twin simulation tool, Lagor engineers could successfully rearrange production sequences in a risk-free environment using a “what-if” approach.
Simulating the entire manufacturing production cycle
optimized with reinforcement learning
The system, as such, could already be implemented in production and help avoid issues and reduce costs. Nevertheless, the system had limits. Despite careful manufacturing capacity planning and sequence scheduling featuring highly engineered heuristics, there were still occasional bottlenecks. The consultants wanted to get rid of the complicated heuristics and develop a new method of determining the best movement sequence.
The flexibility and customizability of AnyLogic process simulation software enabled the engineers to create a reinforcement learning solution. By using Pathmind deep reinforcement learning and an AnyLogic manufacturing simulation model, they were able to train agents that could determine the movements of cores on the production line and direct the cores to their destination. Simulation models are perfect gyms for AI algorithms, as they represent a realistic environment where algorithm-linked agents can be trained.
In this case, at the beginning, the learning agent does not know the link between a core’s position and the available actions, so it makes random decisions, which are sometimes physically invalid (red arrows in the video below) and do not result in a state change. During the process of learning, the agent stores all the interactions in its memory and, by exploring new actions, discovers better moves. Every time it reaches the target, the layout is randomized, and the simulation is restarted. A rich experience interacting with the environment enables the agent to eventually infer the best decisions for any given situation. After training, the agent can perform its task in an efficient and effective manner.
Result
Successfully applied deep reinforcement learning resulted in a policy that could effectively manage production line movements and efficiently avoid bottlenecks. The consultants were able to reproduce the entire production process and train the algorithms in a way to avoid the least possible bottlenecks, leading to better production planning optimization and financial savings. Key to the success of the project was AnyLogic digital twin simulation technology, for seamless data integration between the real system and its digital replica, and the possibilities that the software provided for connecting with machine learning technologies.
Find out more about integrating AnyLogic models and AI.
Watch the video of Luigi Manca, presenting this case study at The AnyLogic Conference, or download the presentation.
