July 8, 2025
AR bridges gap in industrial robotics
Augmented Reality: Bridging the gap for planning robotics in complex industrial environments
Joseph, who commenced as a Research Fellow at the 精东传媒 of 精东传媒 in 2018, joined the Steel Research Hub later in 2022 where he, together with Phil Commins and Stephen Van Duin, has been working collaboratively with on an Augmented Reality (AR) tool project. This project has delivered a step forward in the way robotic automation is assessed and planned in complex industrial environments, bridging the gap between concept and implementation in robotic system design.
Developing an immersive AR tool using the Microsoft HoloLens 2 enables engineers to rapidly visualise, position and evaluate full-scale robotic systems within live operational spaces and brown-field sites. The platform enables users to scan and capture a geometrical model of the environment in real time, then virtually position and simulate industrial robots directly within those spaces. Users can rapidly test different configurations, program robot motions, trial different tooling, and visualise how robotic systems would operate in a real-world setting—all without needing physical robots or disrupting the site. This approach provides a powerful early-stage design and planning tool for integrating automation into industrial workflows, that accelerates decision-making, reduces integration risk, and lowers the technical barriers to adopting automation technologies in complex industrial environments.
Early in the project – developing and designing the appropriate architecture for the software was a major challenge given the unique robotic application and the limitation in computing power of the HoloLens. This required developing a deep knowledge of the development environment (Unity), the HoloLens hardware, and networking systems to get the application off the ground and push the limits of the hardware. Joseph stated, “The HoloLens is powerful but constrained in terms of processing and rendering, so we’ve had to design clever workarounds to leverage the strengths of the hardware and achieve the performance and fidelity our application demands”.
One of the most rewarding aspects of the research has been developing multi-user capabilities. This allows a trained user to control the simulation while other users wearing their own headsets, such as engineers or managers, can view the same hologram and interact with it passively from their own display. This feature has made the tool far more accessible to people without robotics training and allows for a collaborative and real-time approach to designing and assessing robotic applications. In reflection “Working on this project reinforced the value of building strong collaborations with industry. Being embedded for several months within the BlueScope team was a defining moment. It helped shape the tool around real-world needs rather than hypothetical ones.” Joseph explains. Another key lesson for Joseph was the importance of adaptability. From learning and adopting new technologies to adjusting for hardware limitations, success came from staying flexible and solution focused.
In summary, this project showed how emerging technologies like AR can be made useful in practical, applied ways—not just as flashy demos, but as tools delivering real engineering value.