Customizing ultrasonic sensors for composites process optimization and control

This blog is a follow-up to my report on the CosiMo (“composites for sustainable mobility”) project. That project was launched in 2018 by Faurecia Clean Mobility (Nanterre, France) to demonstrate “smart manufacturing” of a composite battery box cover. The 1,100-millimeter-long and 530-millimeter-wide challenge part featured nonwoven glass fiber and unidirectional carbon fiber reinforcements as well as metal and foam inserts, complex geometry including various radii and thicknesses varying from 2.5 to 10 millimeters. The part was molded using thermoplastic resin transfer molding (T-RTM), where caprolactam monomer was injected and then in-situ polymerized during the heated press cycle to form a polyamide 6 (PA6) composite. The main goal of the project was to investigate a fully automated process using closed-loop, sensor-based process control.

This blog explores in more detail the network of 74 sensors used in the CosiMo challenge part, and in particular the ultrasonic sensors developed by the University of Augsburg (UNA, Augsburg, Germany) and how data from all of the sensors was used to build a digital model of the process (digital twin).  

Based on the success of CosiMo and the need for a development center for sensor- and AI-based manufacturing development, UNA has formed the Augsburg AI Production Network, in collaboration with CosiMo project partner German Aerospace Center (DLR) Center of Lightweight Production Technologies (ZLP, Augsburg) and the Fraunhofer Institute for Foundry, Composites and Processing Technology (Fraunhofer IGCV). “We’re setting up a new, 5,000-square meter facility that we will operate in collaboration,” says Dr. Markus Sause, UNA professor and director of the Augsburg AI Production Network. “We will have automated production cells, many focused on composites technologies, which will allow us to demonstrate what AI can do.”

 

CosiMo sensor network

For the CosiMo project, 74 sensors were integrated into the steel RTM tooling supplied by Christian Karl Siebenwurst GmbH & Co. KG (Dietfurt, Germany) for molding the T-RTM battery box cover. Used to monitor process parameters during resin injection, infiltration (flow front monitoring) and in-situ polymerization, this network included:

• 1 vacuum sensor
• 4 pressure/temperature sensors by Kistler (Winterthur, Switzerland)
• 4 temperature sensors by Kistler
• 8 dielectric analysis (DEA)/temperature sensors by Netzsch (Selb, Germany)
• 57 ultrasonic sensors developed by the University of Augsburg.

In my initial report on CosiMo, I discussed how this sensor network was for R&D, not standard production:

“For industrial serial production, no one would install so many sensors and that’s not what we would expect,” says CosiMo manufacturing project leader at DLR ZLP, Jan Faber. “But for this research investigation, this large sensor network was very precise and helped us to have full visibility into the process and material behavior. We could see the local effects in the process in response to variations in part thickness or in the integrated materials, like the foam core.”

So, how was this network designed? “Most were the ultrasonic sensors because they have the lowest cost,” Faber explains. “We used them in a grid at a defined distance to each other. At some positions, we took out the ultrasonic sensors and put in Kistler temperature/pressure or Netzsch DEA/temperature sensors. And for these specific sensors, both the DEA and Kistler, we just tried to have one close to the sprue, where the resin came in, and then one far away. Thus, we could monitor the locations where the resin arrives after having seen the mold temperature for some time already.”

Why use different types of sensors? “One research task was to compare the signals and see how the different sensors perform,” says Faber. This is important because the different types of sensors are based on different measurement principles. For example, DEA sensors measure the AC impedance of the resin, which can be split up into different fractions, that in turn show different effects in the polymer. Kistler, meanwhile, uses piezoresistive sensors that rely on a silicon-based Wheatstone Bridge — a type of circuit used to measure dynamic electrical resistance — connected to a membrane which forms the sensing end installed in the wall of the molding cavity. The membrane extends under pressure, which changes the electrical resistance of the silicon Wheatstone Bridge, which in turn outputs an electrical signal. “We also used Kistler’s new 4001a sensor which measures both pressure and temperature,” notes Faber. “It was very useful to have both values for select local points in the tool.”

 

UNA ultrasonic sensors

The majority of the sensors used in the CosiMo part network were ultrasonic sensors developed by UNA. Sause notes that these were favored not only because of their low cost (<10 euros) but also because they can deliver a wide range of information.

Ultrasonic sensors have been used for decades to inspect finished composite parts, but also have a long history in injection molding. Ultrasonic sensors send a high-frequency sound wave into a medium and measure its reflection (pulse-echo) and/or transmission. In the CosiMo project, the nominal frequency was 2,000 kilohertz. Of the 57 ultrasonic sensors installed in the mold — at a distance of 20 millimeters from the cavity surface — 10 operated in transmission mode, and the rest in pulse-echo mode. Ultrasonic sensors measure the wave’s velocity and decrease in amplitude (attenuation); both are sensitive to the properties of the polymer. Further, wave velocity can be calculated as a function of pressure and temperature, but the ultrasound data must be calibrated to the polymer being investigated.

“The ultrasonic sensor we have developed is actually not very different from what is commercially available,” says Sause. “It's based on a piezoelectric material, which is quite sensitive in converting an elastic motion into an electrical signal. But where typical sensors can operate at temperatures up to 150°C, we had to match the high temperatures of thermoplastic molding at more than 200°C, which was a bit challenging for the sensor electronics.”

“We also had to devise the mechanical system for properly integrating the sensor into the mold,” he explains, “which required spring loading with good sealing to provide a good coupling. There was a large portfolio of requirements that the sensor had to fulfill; so, we had to pretty much design it from scratch. We also did a proper acoustic design of the whole system.”

 

Polymerization and flow front monitoring

“The acoustics focus on the elastic properties of the materials in the mold,” Sause continues. “Basically, the stiffness in a liquid is not the same as when the polymerization is complete. We’re effectively monitoring that transition. We look at the acoustic intensity of the wave reflection at the interface from the mold to the polymer material at each sensor location, and thus continuously track curing over time.”

The mechanics of this measurement are explained by Sause and Faber, et. al., in the SAMPE Europe 2021 conference paper, “Sensor-based process monitoring of in-situ polymerization in T-RTM manufacturing with caprolactam”. The ratio of the acoustic signal echo’s amplitude before and after the preform impregnation depends on the reflective index at the mold-to-part interface, R. This approaches a constant value as the polymerization progresses. Ideally, once polymerization nears completion, the change in R will be zero.

It is also possible, from the acoustic transmission signal, to extract the travel time for the ultrasonic waves to pass through the impregnated preform. This is helpful as the speed of sound through the preform in linked to the degree of polymerization. For the experimentation completed at the time of the SAMPE Europe 2021 paper, the time between the thorough wetting of the preform at the sensor’s location until completion of polymerization was 240 seconds.

“We can also monitor the flow during injection,” explains Sause, “because theacoustic signal information changes as the polymer flow front passes the sensor.” Also verified in the SAMPE Europe 2021 paper, the acoustic signal dropped abruptly from an initial normalized value of 1 to between 0.85-0.9 upon arrival of the caprolactam flow front for the selected sensor at 15 seconds after injection began.

 

Developing a digital twin of the process

Sause says developing the ultrasonic sensor was actually a very small part of what UNA did in the CosiMo project, “and not even the most fascinating part.” The real achievement, he notes, was in the data analysis and development of a digital model of the process to enable closed-loop control during part manufacturing.

“What we’re trying to do is provide information about what is actually occurring in the mold,” he adds. “We have had established routes to adapt the process with temperature and pressure, more or less, but what we are doing now is exploiting the acoustic signal information by doing active pulsing at each sensor location. And this allows us to determine various things such as the shape of the flow front, the time for it to reach each part of the preform and the degree of polymerization at each sensor location.”

“Ideally, you can establish settings to enable closed-loop control and to adjust during the process,” says Sause. “These would include parameters like injection pressure and mold pressure and temperature. You can also use this information to optimize your materials.”

The development of the digital twin was a multi-stage and multi-partner effort. It began with one of the work packages assigned to the DLR ZLP. “We set up a simulation based on a 2D plate, where we wanted to simulate the filling and polymerization using the PAM-RTM simulation software from ESI Group (Rungis, France),” says Faber. “We got the preform permeability data from project partners ITA (Institute for Textile Technology, RWTH Aachen, Germany) and SGL Carbon (Wiesbaden, Germany) and also a reactive model of the thermoplastic polymer reaction from the University of Augsburg. We then integrated all into the simulation and tried to see how close we could actually get to the real sensor data.”

“We found, based on the 2D plate model, that we're actually very close, regarding the filling,” Faber continues. “But we had some temperature effects during polymerization that we could not describe in the simulation. For example, this polymerization reaction is exothermal, and you see a slight increase at the temperature sensors in the real data, but the increase in the simulation was much higher. So, this is an area where we continued to do some work.”

One of the things the team wanted to discover from the simulation was the time required for polymerization to be complete. “What we really wanted was the minimum time we needed in the mold,” notes Faber. This information was derived from the simulation model, and relied, as all of the simulation did, on multiple types of sensors and AI algorithms. For example, Netzsch provided the dielectric sensors used for dielectric analysis, but also helped with parts of the digital twin of the process. “They set up predictive AI models to investigate the mechanisms inside the injection and to predict at which time the polymerization should be finished and what would be the polymerization state,” says Faber.

(Note, Netzsch has combined its competencies in material characterization, kinetic simulation and sensor technology to establish new offerings in AI and cloud solutions. This type of integrated solution has been successfully demonstrated in CosiMo to provide real-time evaluation of the degree of polymerization and will be commercialized under the brand sensXPERT.)

From the complete network of sensors and simulation, it was found that 96% was the maximum polymerization achieved and that required 4.5 minutes. “So that is much faster than infusion processes but much slower than high-speed injection using thermosets, for example.”

“The sensor data fits into the digital twin model,” says Sause. “The idea was to have a model running at the same time as the process that is numerically stabilized by the sensor information. The blue dots provide a coarse grid of ultrasonic sensors, and then we were able to interpolate between those points. For example, we were able to provide a video of how the flow front develops in the CosiMo part as a function of time based on the measurement data from our sensors.”

 

How were the polymerization rate predictions validated? “We had several approaches,” says Sause. “First, a PhD student here at UNA was doing a full set of modeling for the reaction kinetics of the caprolactam-to-PA6 material system. So, he was studying simultaneously infrared spectroscopy together with rheometric data and data from the dielectric analysis sensors to provide a very precise material model for the cured kinetics of that system and to validate that model. So, we used the same model to cross validate our sensors.”

 

Architecture of the digital twin

The architecture of how the digital twin operates is described by Stieber, et. al. in the 2020 technical paper, “Towards real-time process monitoring and machine learning for manufacturing composite structures”:

It consists of three parts: the T-RTM machinery, the in-situ sensors in the mold tooling and the digital twin. This structure and the digital twin, in particular, have been developed with the following objectives O1 – O4 in mind:

• O1: Monitor the manufacturing process of every composite part with the in-situ sensors of the tooling.
• O2: Visualize each manufacturing process in real-time to gain further process insights.
• O3: Train ML models on simulated and real data for predicting the outcome of the manufacturing process.
• O4: Use trained ML models to reduce production rejects.

The sensors then feed data to the edge analytics of the digital twin, which can then be used to predict future flow front dynamics, determine the fiber volume content of each textile preform, predict dry spots and detect anomalies which can reduce reject parts.

The T-RTM machinery comprises two off-the-shelf components: a Wickert(Landau in der Pfalz, Germany) 440-kilonewton hot press and a T-RTM injection unit by KrausMaffei (Munich, Germany). Both of these machines as well as the whole sensor network are connected to a PLC (programmed logic controller) which is responsible for controlling the overall manufacturing process.

Because the sensors need real-time pre-processing to transform the raw data into higher-level information about the flow front or the curing of the resin, the preprocessed data is recorded directly at the edge using an ibaPDA data acquisition unit from iba AG (Fürth, Germany). Thus, the sensor data is recorded and merged with a consistent time stamp linked to the process data and part information with the sensor data per objectives O1 and O2.

To automatically optimize the manufacturing process, the stored data is used to train machine learning (ML) models to address objective O3. These models are responsible, for example, for reconstructing the flow front of the resin from the sensor network, for predicting the flow front progress, and – ideally – for adapting the process parameters to optimize the result either during a process instance or for subsequent runs. Because real data is generally not sufficient to train reliable models, the digital twin team used PAM-RTM simulated runs to augment the measured data. After pre-training using simulated data, the digital twin was fine-tuned with real-world data. This strategy is known as Transfer Learning and has been adopted successfully in computer vision and natural language processing applications. The advanced analytics possible with ML enable increased prediction capabilities and process visualization as well as closed-loop control.

 

Sensor integration for industrial composites production

So, how do composites parts manufacturers apply this technology to develop closed-loop control in their own production processes? Sause says it makes sense to use several types of sensors to cross-validate the flow and cure/polymerization monitoring and to provide complimentary information. And then it’s a matter of defining sensor placement, including which types are placed in each location. “For example,” he notes, “you don’t need pressure and temperature sensors everywhere, but just where you need those key data points, such as resin entry, complex geometry points, etc.”

How many sensors? “For CosiMo, we used 74 sensors, but for a half-meter part you would use less, perhaps 35-40 depending on your geometry and key points of interest,” says Sause. “It also depends on the production process. There are processes in aerospace, for example, that have been used for decades and are well established. For these, you may not need very many sensors and it could be possible to integrate them without much development work at the beginning. But other processes are more unknown and will require more work upfront. For CosiMo, we were looking at using T-RTM with in-situ polymerization to PA6, and this was not as well documented as RTM with epoxy, for example.”

How time-intensive is building a digital model? And is that always required to achieve closed-loop control? “Again, it depends on your process and what level you want to reach,” says Sause. “First of all, having the sensor information integrated allows you to visualize what's happening inside your black box of processing and what parameters to use.

Once that first target is reached, it is a few more steps to achieve closed-loop process control. Halfway, perhaps, is to have the visualization and you can hit the stop button by yourself in order to adapt the process and prevent reject parts. So that too is a start. And then you can develop the digital twin, but it needs training data. So, you have to provide that somehow, which means you can produce a couple of parts to use as input data. But then, you also need to produce parts of low quality. So, it's a matter of your willingness to invest in the training of your machine learning approach.”

However, Sause does acknowledge that the transfer learning approach taken in CosiMo, “is transferable to a certain degree to similar geometries or material systems and other variations that don't completely change the situation. So, you can build up your database in that way, with the possibility to identify efficiency trends using the already acquired data. You can then use that for other questions that you might have alongside.”

So, the way to start seems to be: start using sensors and building digital models to go with the sensors so that you can understand your process? “Absolutely,” says Sause. “And I can assure you that most of the machinery used in production today is still very stupid. Many machines don't have any sensors that can give you the databases for providing feedback as to what's going on. So, yes, integration of sensor systems would be the very first step. But you need to do a systematic data acquisition, and also provide a data storage architecture, to be able to do something with the data. You actually see many companies that have sensors integrated, but then they don't do anything with the data.”

 

AI Production Network

In answering the questions above, Sause kept coming back to various groups within UNA as well as spin-off companies that can provide sensor integration and digital twin services. To that end, the Augsburg AI Production Network was launched in January 2021 by UNA, DLR ZLP and Fraunhofer IGCV. “The collaboration framework comes from projects we have had with DLR and Fraunhofer in the past,” says Sause. “We are taking the blueprint in AI-based production development from CosiMo and now extending it to a very broad scale. So, for example, the budget for our work in CosiMo was 10 million euros, but for the AI Production Network it is 92 million euros as an initial funding. This will provide us the possibility to start many new research projects like CosiMo in exploiting AI in manufacturing. And we are not working just in composites, but we do have a strong focus on them.”

Using ultrasonic sensors to improve CNC machining is one project already started in 2021. “There is a project just about to start on recycling of ceramic matrix composites (CMC),” says Sause, “and we will have several projects with a tight link between additive manufacturing and the reuse of specifically recycled fiber-reinforced materials.”

As for the roles each organization will play, Sause explains there is no competition, as each institution has an allocated budget. “We all have our share, and thus are free to collaborate as we have in the past. Also, we know each other quite well and our competencies, strengths and weaknesses. It’s fun to work together because we know what we can and can’t do. We've been approached by many companies that want to work with us and we're discussing now how to find the ideal match from our side of competencies.”

Regarding the 5,000-square-meter production facility, “we will all run that together,” says Sause. “We’re planning to open by the beginning of 2023 and start installing machines. And we'll provide automated production settings that link together the individual cells. Many of those will have a focus on composite production technologies. And we will do that to also showcase and demonstrate to partners from industry, what AI can do in such a production setting. We also want to have partners from industry coming in and also placing machines in our facility so that we can run projects together to see how to link and integrate new solutions for their sites.”