Shipbuilding With VR, AR and Digital Twinning

The Mexican Navy’s Sierra-class offshore patrol vessel (OPV) ARM Benito Juarez is pictured at sea. Damen Naval designs, builds, and supports ships for a range of international customers, including the German, Indonesian, Netherlands, and Mexican navies.
The Mexican Navy’s Sierra-class offshore patrol vessel (OPV) ARM Benito Juarez is pictured at sea. Damen Naval designs, builds, and supports ships for a range of international customers, including the German, Indonesian, Netherlands, and Mexican navies.

Damen develops simulation training products to add value in ship concept, design, build, and support.

Navies are seeking to introduce new ships more quickly and to generate forces at sea more efficiently to increase frontline presence. The naval industrial base is responding to this requirement in several ways, including by developing synthetic tools that can be used across the spectrum of platform development, including concept, design, build, and in-service support like maintenance and operator training.

One company with a prominent profile in this area is Damen Naval (DN), part of Damen Shipyards Group (DSG). DSG builds maritime platforms from pontoons to tugs to ferries and to commercial ships. DN, which is responsible for building naval ships, is currently engaged in designing, building, maintaining, and supporting a range of ship types for a range of navies, including the Netherlands, Germany, Mexico, and Indonesia.

Alongside designing, building, and maintaining naval ships, DN is actively engaged in examining how the wider use of technology can improve these processes for both DN and its naval customers.

“We have a big emphasis on research and technology, and technology development, as we are part of the naval ‘Triple Helix’ [government, industry, and the research/knowledge community], with the Netherlands Ministry of Defence as launching customer for the Royal Netherlands Navy (RNLN) fleet. As a specialised Western navy, the RNLN has a high level of technical content in its ships and systems,” Björn Mes, a research engineer and xR specialist in DN’s Research and Technology Support team, told Armada International, in an interview on 10 January 2022.

An RNLN Maritime Simulation Centre staff member demonstrates use of an operator training VR set up, which is designed to show operators how to use certain systems.
An RNLN Maritime Simulation Centre staff member demonstrates use of an operator training VR set up, which is designed to show operators how to use certain systems.

The DN research and technology team assess how simulation technologies like Virtual Reality (VR), Augmented Reality (AR), Digital Twinning, and broader use of modelling and simulation add value across the four stages of platform development: concept; design; build; and in-service maintenance and operator training. “We try to find out how to use new technologies like VR, AR, Digital Twinning, and modelling and simulation in the complete value chain of shipbuilding,” said Mes.

“We use such technology for the design process – the concept stages, the early design stages, the detail design of the ships, the design reviews – and after that for the building of ships, where it is really interesting to see what these technologies can do, and then of course during ship operation.”

DN develops its synthetic technologies through proof of concept and field trials work, and then offers them as simulation-based products to support the design, build, and operational support of the shipbuilding programmes it is delivering, Mes explained.

DN’s development of a range of synthetic concepts demonstrates not only the various ways in which such technology can be used in shipbuilding and ship operation, but how it can also add value not only in building and operating a ship effectively but in enabling navies to manage introduction and operation of new platforms at a time of reduced ship numbers in fleets and reduced sailor numbers onboard those ships.

An RNLN Maritime Simulation Centre staff member demonstrates use of an operator training VR set up, which is designed to show operators how to use certain systems.
An RNLN Maritime Simulation Centre staff member demonstrates use of an operator training VR set up, which is designed to show operators how to use certain systems.

Defining Technology

VR, AR, Digital Twinning, and other modelling and simulation technologies are based around different concepts, and are used in shipbuilding programmes in different ways.

“VR is a way to visualise a system that is not available – if it is not yet built, or it is somewhere else,” said Mes. In the latter context, this relates for example to all ships of a certain type being unavailable for training purposes.

“AR is the visualisation of live data in context,” Mes continued. “Live data means sensor information or aggregated data from external sources such as weather satellites or AIS. Together, this data can be everything from sea states to revolutions-per-minute on a propulsion shaft, and can even include data from a maintenance manual – but it always has to be in context.” “If I look at a piece of equipment, an AR application is able to recognise it as what it is and view relevant data – so, dynamic data in context.”

An added element of AR applications is they leave the user’s hands free. “Especially from a maintainer’s perspective, it’s really important that you have your hands free to do actual work,” Mes explained.

Mes noted there are different views on what Digital Twinning is. “In my humble opinion, it is a ‘data lake’, with geometry, behaviour, and simulation in it that grows during the lifetime of a ship.” This ‘data lake’ builds from when the first sketch of a new ship design is drawn up, all the way up to when a fully-fledged computer-aided design (CAD) system is in use, including with updated weapon and sensor information, live information data-base support, and people or tools situated ashore able to access, add to, and use such information.

Mes argued that Digital Twinning enables concepts like condition-based maintenance, which encompass developing a Digital Twin of a system or component onboard a ship and then modelling potential wear and tear to estimate when the real system might require repair and to understand what repair work might need to be done.

An artist’s impression of the F 126 frigate Damen has designed for the German Navy. Damen uses Virtual Reality (VR) simulation technologies to demonstrate design concepts to a customer, such as showing how a multi-purpose frigate can switch operational configurations.
An artist’s impression of the F 126 frigate Damen has designed for the German Navy. Damen uses Virtual Reality (VR) simulation technologies to demonstrate design concepts to a customer, such as showing how a multi-purpose frigate can switch operational configurations.

“I see Digital Twinning as a means to an end, not a goal,” said Mes. “As an example, if you have a Digital Twin and you have a condition-based maintenance system, a technology like AR can be the view of that information. So, the source of the data is the Digital Twin, the business case is the application [such as condition-based maintenance], and AR is the way to view the information in that app.”

Part of this idea, Mes continued, is that a Digital Twin should be a platform on which to add apps that use the same data and enhance functionality by bringing in new or more detailed behaviour and simulations. So, for example, in the design process an app can provide data on the ship’s behaviour like sea-keeping, to inform that ship design.

“Condition-based maintenance is one of those apps, because you use the build data, you use the actual live data, to come to a conclusion,” he explained. “That base data is required to demonstrate the live state of a system, as such data might indicate that something was done during the life-cycle of a component, system, or ship that influences one of those condition-based maintenance variables.”

Applied Simulation

Damen applies the different simulation technologies in different ways across the different shipbuilding phases.

In the concept phase, the company uses VR to demonstrate a design concept to a prospective customer, Mes explained. “So, if we try to build a frigate that is multi-purpose, we use VR as a way to show what a multi-purpose frigate is … to give them an idea of how it looks,” he continued. “We use it to explain … how we would use the stern of a ship, how we can we use containers, how we can use gliders, or how we can switch to a configuration like anti-submarine warfare.”

As concepts at this stage generally are based around preliminary designs and general layout arrangement plans, using VR is key. “It’s more of a concept communication perspective, than actually a technical perspective of how it will fit and what the measurements will be,” said Mes. Here, using VR demonstrates the ship’s size, geometry, and how compartments broadly fit in, for example.

In the design phase – a two-stage phase, when the engineering specifications of the ship concept are developed and refined through basic design and then detail design – Damen also uses VR-based approaches. Here, it is looking to apply VR in two ways, with these different ways assisting customer and engineer, respectively.

First, if the customer has an ad hoc question about the detail of the ship’s lay-out, VR can be used to help demonstrate what the lay-out will look like, including for both major systems and for how compartments will appear. In the former instance, Mes explained that VR can demonstrate visually why a large, primary sensor like a radar dome needs to be positioned in a certain spot on the ship’s superstructure to avoid interfering with the operation of the ship’s crane, for example. In the latter instance, if a customer is interested to understand the lay-out of the ship’s ops room, the ship’s prospective commanding officer can be seated in a chair set at the same height as his seat in the ops room and put on a pair of VR goggles: he would then be able to see the lay-out, and be sure he could see all the consoles from his seat.

Second, Damen is looking at how VR applications can augment the use of CAD systems. “We are actively pursuing the idea to see if we can use that as a regular, embedded procedure in design and review processes,” said Mes. Damen is in the ‘proof of concept’ stage with this idea, he explained: while demonstration work has progressed well, the test will be introducing a real ship into the concept, as that will add design objects and data.

In the build phase, Damen is developing some ideas relating to AR. “One of our unique selling points is that we can build ships abroad: so, we design ships in Vlissingen, and then we build them in the yards in Mexico, or Indonesia, or Germany,” said Mes. “In some countries, language can be an issue …. So, can we use AR technologies, linked to CAD, to explain to a welder how to weld the ship together, looking at where does a piece fit or if we can use the quality control system to help the welder keep the welding torch at the right angle or to measure how long the welder is welding on the same spot?” Similarly, ship designs are drawn up in 3-D, but then are sent to the customer in 2-D before the customer then re-establishes them in 3-D: using AR can help skip the 2-D step, Mes explained.

In the ship operations phase – where synthetic technologies can be applied for both ship maintenance and operator/maintainer training – Mes detailed some key products Damen has developed.

“The training part is really important at the moment because that is a low-hanging fruit for most of our customers, and it’s easily accomplished with the applications we have available,” said Mes. “We use VR for training at this point. It’s been through the ‘proof of concept’ phase; now it’s an actual product.”

Damen has developed several versions of this VR product. “We have maintainer training – so, how do I conduct maintenance on a piece of equipment. We have operator training – so, how do I operate a certain system. We have general introduction training modules,” Mes explained. In the latter instance, Damen has a VR product called ‘Know Your Ship’: “it’s training in VR where you learn what the lay-out of the ship is, where are the fire extinguishers, where are the emergency exits, how fast and heavy is the ship, what are the operational capabilities,” said Mes. “We have training that is similar for the engineering department: it gives an introduction about the ship systems and more specifically the functional chains and dependencies of the different systems; so, at the end of the training, you know that if you want to fire your gun you need chilled water – that sort of knowledge.”

Damen can provide these VR products in multi-user format. It also has produced specialised VR apps, such as one focused on training for internal battle damage assessment in which, for example, an engine room could be isolated.

Damen has not yet fully developed any AR products for sale to customers, as the way a ship moves means ‘ruggedised’ AR solutions must first be developed to enable the AR app to recognise where it is. However, Damen has an embryonic Digital Twinning product emerging, which delivers a full-ship model based on the CAD model.

The Indonesian Navy’s SIGMA 10514 Martadinata-class frigate KRI I Gusti Ngurah Rai is pictured at sea. The Damen-designed Martadinata frigates were built in Indonesia. Augmented Reality (AR) simulation technologies can help shipbuilders generate and use 3-D designs.
The Indonesian Navy’s SIGMA 10514 Martadinata-class frigate KRI I Gusti Ngurah Rai is pictured at sea. The Damen-designed Martadinata frigates were built in Indonesia. Augmented Reality (AR) simulation technologies can help shipbuilders generate and use 3-D designs.

Monitoring and presenting technology

Damen’s researchers also monitor how other industrial sectors are using synthetic technologies like AR, for example to measure pipe sizes and angles when fitting them into tight spaces. In a shipyard, a pipe fitter may have to make regular trips from the pipe shop to the ship to look at where and how a pipe must fit in a particular space.

The researchers came across a company that makes domestic stairlifts, and uses AR to calculate pipe lengths and angles based on routing. Such technology potentially offers time and effort savings. While Damen has not yet introduced the technology into its AR thinking, Mes noted this was an example of how AR and Digital Twinning could be integrated together at a practical level.

In its use of synthetic training products, Damen is looking not only at what technologies to use but also at how to present them. For example, its VR-based training system applications are designed in what Mes referred to as a ‘game-ified’ manner – in other words, drawing on gaming technology. This offers particular assistance to the RNLN, for example, in helping to attract sailors from its target audience of new generations, and helping these recruits maximise their use of and learning from the training product. “We use game mechanics as a technical concept for these training simulation systems: we use it to [enhance] motivation,” said Mes. “A simple example could be a leader board. Part of our target audience may be conscious of their position on the board: they want to rise to the top by overtaking their peers,” he explained. “So, if we let the user retry the training to rise up the board, the unintended consequence is that the learning content is repeated multiple times, helping knowledge retention.”

The Damen research team also devise research topics to support the in-service operator training phase. One new research topic is ‘just in time’ training using AR, said Mes. Here, with many navies reducing their number of platforms but still operating several different ship types, and with sailors cross-decking regularly between types as a result, Mes added there is a need for specific initial training solutions. “You don’t always have the right ship available to train and there is the fact that we have more generic-educated mariners. In previous years, you had a dedicated weapons system engineer or a dedicated platform management person: these days … sailors are more generically educated, so you need information about systems ‘on the fly’, ‘just in time’.”

“You want to have all the data that is available on a piece of equipment accessible for when a user needs it,” Mes explained. “So, if there is something broken and the user is trained to use a ‘Type A’ pump, a ‘Type B’ pump would be maintainable by the user with the right data available. Can we use AR to serve that right data to that maintainer?”

“This is about how those systems work onboard a ship. Can we make connections to static data like geometry, building, the sensor or ship type, and the relevant manuals, and can we use dynamic data as well? Can we link it to a condition-based maintenance system or maintenance management system? Can we make those connections to make that ‘just in time’ learning possible?” Mes asked. “It’s about data flow and information.”

Overall, Mes continued, “Can you use AR to help the crew cope with the huge amount of data whilst … there are less and less people onboard? We have an increase in information and less people to handle that information, so can we build systems to make sure the right person has the right information at the right time, when they need it?” “So, can we help manage those data streams? That’s something we are really interested in using AR for,” he added.

by Dr. Lee Willett