Abstract:
This study addresses the critical demand for efficient and intelligent multi-robot collaboration in modern industrial environments by proposing, designing, and implementing a novel industrial digital twin platform for multi-robot cooperative control, which is fundamentally powered by reinforcement learning methodologies. The core objective of this study is to construct a seamless closed-loop framework that bridges high-fidelity virtual simulations with real-world physical execution. This integrated system was specifically engineered to overcome the well-documented limitations prevalent in conventional multi-robot setups, which typically suffer from suboptimal collaborative efficiency, prohibitively high costs associated with physical trial and error, and a pronounced lack of adaptive capabilities in dynamic or unstructured task scenarios. To achieve this, the study is systematically structured around four interconnected pillars of development: (1) holistic design of the platform architecture, (2) construction and integration of the physical robotic system, (3) development of a realistic and interactive simulation environment, and (4) experimental implementation and validation of advanced collaborative control algorithms. The proposed platform architecture establishes a coherent multilayered structure encompassing a Physical Layer, the Digital Twin Layer, and a robust Data Transmission Layer. The Physical Layer comprises a hardware ensemble of multiple robotic manipulators (exemplified by Dofbot arms in this study), integrated vision sensors, such as Intel RealSense for environmental perception, a hierarchical control system utilizing Jetson Nano for high-level computation, and STM32 microcontrollers for low-level actuation. The Digital Twin Layer constructed within the ROS and MoveIt! ecosystem creates a dynamic and physics-accurate virtual replica of the entire physical workspace. This includes detailed models of robots, workpieces, and operational environments. A key innovation is the deep integration of a Multi-agent Reinforcement Learning (MARL) training framework directly into this digital twin. This enables the autonomous learning and optimization of complex cooperative control policies, such as coordinated assembly, pick-and-place, or synchronized motion, in a safe, cost-effective, and infinitely repeatable virtual sandbox before physical deployment. Within this integrated digital twin, sophisticated MARL algorithms, including an enhanced variant of the multi-agent deep deterministic policy gradient algorithm, are employed. Each robotic arm is treated as an independent agent that learns its policy through continuous interaction with the simulated environment, guided by a carefully crafted reward function. A comprehensive experimental validation confirms the efficacy and practical utility of the proposed platform. The results demonstrate the platform's capability for real-time bidirectional synchronization and mapping between virtual and physical spaces. The digital twin accurately mirrors the state of the physical robots, and the control policies optimized in the simulation can be successfully transferred to command the physical arms, achieving designated collaborative tasks with high precision. Crucially, this sim-to-real transfer paradigm drastically reduces the time, financial costs, and operational risks inherent in the direct training of complex policies for physical systems. Furthermore, additional experiments that vary task parameters, such as altered object starting positions or the introduction of obstacles, demonstrate the platform's significant potential for task generalization. The learned policies exhibit notable adaptability to these new conditions with minimal retuning, highlighting the robustness of the approach. In conclusion, this study provides a substantive contribution by delivering a fully operational, open, and reusable digital twin platform that tightly couples MARL with industrial robotics. This provides a validated experimental benchmark and flexible framework for future research and development in intelligent multi-robot systems. The platform not only offers a practical solution to current industrial collaborative challenges, but also paves the way for more advanced investigations into scalable multi-agent coordination, robust sim-to-real transfer learning, adaptive control in the evolving landscape of Industry 4.0, and smart manufacturing.