AI meets hardware: Gray box models for condition monitoring and service life prediction

For mobility applications in the rail or automotive sectors, assessing the condition of complex electronic systems and their remaining service life during operation has always been of crucial importance. It is a basic prerequisite when trying to ensure the functional integrity of the safety-critical electronics throughout their entire working lives. It is therefore not surprising that ensuring reliability and functional safety accounts for approximately 40 percent of the total cost of developing and manufacturing road and rail vehicles.

Under the leadership of Siemens AG – Foundational Technologies – a research consortium has developed a new way of determining the condition of safety-critical electronics in mobility applications in the »SesiM« project.

RealIZM met up with Dr. Johannes Jaeschke, System Reliability Assessment group leader at Fraunhofer IZM, and Tom Dobs, R&D Engineer at Siemens and deputy SesiM project manager, at the EcoReliability working group at Fraunhofer IZM. The two experts provided insights into how new AI-supported methods can be used to assess the condition of safety-critical electronics during both manufacturing and actual operation, and what is important when creating a digital, data-based fingerprint of electronic systems/assemblies.

»The condition of mechanical systems can already be reliably monitored today. By contrast, we still see a considerable need for more research into determining the condition and predicting the service life of complex electronic systems that interact with different components«, Dr. Johannes Jaeschke says about the motivation behind the »SesiM« research project. As an expert in system evaluation, he and his »System Reliability Assessment« working group at Fraunhofer IZM are concerned with identifying and modeling weak points and failure mechanisms in electronics and mapping them in digital twins.

»In order to enable precise real-time condition monitoring throughout the entire service life of electronic systems, physical degradation models had to be combined with data-based artificial intelligence models«, Dr. Johannes Jaeschke explains the overall goal of the »SesiM« research project.

Examples of applications for electronic systems in railway technology and automotive engineering

Three applications from the fields of railway technology and automotive engineering were chosen by the research consortium. For these, they designed and manufactured functional prototype assemblies with reduced complexity and tested them for their reliability. To this end, a range of manufacturing and operating data was collated and evaluated using artificial intelligence (AI) and physics-based models.

A transponder system for automated train control was investigated for the railway sector. The system comprises a balise (data information point) in the track bed, a balise antenna on the underside of the train, and the monitoring hardware inside the train. In order to meet safety requirements, the evaluation electronics are designed as a redundant system on a plug-in card. Such a measure demonstrates the potential for improvement in terms of resource utilization and reparability.

As a second example, a DC/DC converter for the automotive sector was analyzed. All components of the DC/DC converter—power section, logic section, and control—are located on a printed circuit board. Its reliability and service life are determined by one specific part of the system: the power components, solder joint, and high-current printed circuit board.

An integrated bridge rectifier and a partially integrated low-power DC/DC converter in a DFN8 package with external circuitry served as the assembly for general applications. The assemblies generate supply voltages in the low-voltage range and are used in a similar form in many fields of application.

Test circuit board with functional structures for generating the digital fingerprint | © Fraunhofer IZM

»Data mining« in assembly production: potential for predicting service life

»With our new method, we are able to carry out more comprehensive reliability assessments than ever before«, Dr. Jaeschke explains. »It is possible to describe or predict the condition of an electronic system both during manufacture and as a finished product, even in operation.« Until now, data collected during the manufacturing of electronic assemblies has been used for process evaluation and optimization. What is new is the targeted use of extensive and previously unused data from the production process to evaluate a system’s reliability and predict its service life.

Tom Dobs from Siemens sees potential for reliability assessments in data mining in assembly production. »As part of the SesiM project, we tapped into new data points on our production line with newly developed in-line inspection methods.« As example, Dobs cites the stand-off measurement of individual components (developed by project partner GÖPEL electronic) and the AI-supported evaluation of raw data images to understand solder paste application and solder formation. »This data enables us to record the initial state of each individual assembly during production. It gives us a digital fingerprint of the system or assemblies.«

In-line inspection methods include optical processes such as solder paste inspection (SPI), automatic optical inspection (AOI), or automatic X-ray inspection (AXI). These methods were combined with electrical characteristic curve tests and the quantification of process-related fluctuations, such as positioning accuracy, void in the solder, and stand-off. Everything was then transferred into a uniform data model.

»An important prerequisite for data mining in assembly production is the complete monitoring of process steps and the establishment of a technical infrastructure for data acquisition«, Dobs explains. »Then, the collected data must be standardized in order to create a digital fingerprint.«

Four criteria are crucial for processing the collected data in real time: access to data, specifications for the uniform designation of components and pads, a uniform structure for inspection, manufacturing, and field data with standardized interfaces, and the architecture of the database for storing the data.

Project partner Gestalt Robotics set up a cloud data platform and provided automated access interfaces.

Framework for data-based condition monitoring | © SesiM-Konsortium

Gray box models for safety-critical electronic applications

»For safety-critical electronic applications, such as those we are investigating in the SesiM project, it is crucial that we have transparent analyses«, Dr. Jaeschke emphasizes. Machine learning methods such as deep learning are already being used to make progress in anomaly detection or industrial image processing. However, the availability and quality of suitable training data still poses a major challenge. »To increase data transparency and reduce the need for training data, we went with hybrid modeling.«

Various modeling techniques were used to monitor the degradation behavior of the electronic prototype assemblies. As mentioned at the outset, physical models (white box) were combined with data-driven models (black box) to create a comprehensive picture of system behavior. The result is a so-called gray box model. Dr. Jaeschke explains the thinking behind the process: »With this innovative approach, we were able to ensure seamless coupling between the electrical, thermal, and mechanical domains and capture the interactions between these areas.«

Black box models of machine learning are purely statistical models. White box models, on the other hand, refer to analytical and physical descriptions, which are usually very complex to model. Gray box models combine both approaches to combine the best of both worlds.1

Gray box modeling was chosen for the project as part of an overarching data-based modeling approach to condition assessments that links static and dynamic data paths. On the one hand, reliability-relevant production data—static data paths—must be clearly assigned. On the other hand, functional system parameters during operation – dynamic data paths – must also be analyzed. When combined, both paths generate an individual digital fingerprint of the system to be monitored and enable adaptive condition diagnostics that reflect both manufacturing factors and real-world forces.

From test circuit board to rail and automotive demonstrator

Test circuit boards with the sample assemblies were measured and tested with utmost precision during production and in actual operating conditions. Data was also collected under extreme conditions in a laboratory setup. In a next step, the parameters that are actually relevant for representing the system status were identified. Taking physical knowledge into account, a model was created that detects deviations from a predefined ideal state.

The test circuit boards were transferred to a rail and automotive demonstrator containing the DC/DC converters, oscillators, and power MOSFETs, integrated into various test vehicles. These were connected to form a complete assembly in order to illustrate how individual components interact and depend on each other in complex electronic structures.

Rail and automotive demonstrator for AI-supported real-time prediction of a system’s status, based on functional parameters and using gray box model | © Fraunhofer IZM I Volker Mai

To visually represent the models, Gestalt Robotics GmbH, the University of Stuttgart, and Fraunhofer IZM also developed two web applications. These offer a »prediction based on inspection data from manufacturing processes« and can give an estimate of the expected service life. »Predictions based on field data« can then also be used to forecast future indicators.

A view of the graphical user interface with live measurements | © Fraunhofer IZM

Digital twins as the key for predicting remaining service life

»In the research project, we were able to demonstrate new methods for reliability assessments that are more comprehensive than ever before«, Dr. Jaeschke explains. With graybox models, any external manipulation can be detected as quickly as possible, and age-related wear and tear in electronic systems predicted at an early stage, long before any failure occurs.

The researcher is convinced that the initial field of application will be predictive maintenance. »In the long term, I hope that by integrating reliable gray box models into digital twins, we will turn the expected working life into an important criterion in the circular economy when it comes to deciding on matters of reuse, refurbishment, or recycling.«

The project team is currently preparing a follow-up project that will use the insights gained from SesiM to enable informed decisions within sustainability strategies. This will be achieved by collecting unused data and enabling better integration of mobility electronics into circular economy processes.

Tom Dobs sees potential in AI-supported condition assessments for improved maintenance management and thus an important lever for sustainability strategies.

»SesiM« (Self-validation of complex electronic systems in safety-critical mobility applications based on gray box models)

Duration: 07/2021 to 12/2024

Funding body: Federal Ministry for Economic Affairs and Climate Protection (BMWK)

Funding code: 19I21018F

Project partners: Siemens AG, Corporate Technology – consortium leader, Robert Bosch GmbH, Gestalt Robotics GmbH, GÖPEL electronic GmbH, AUCOTEAM GmbH, University of Stuttgart, Fraunhofer IZM

Further information

Source:

¹ Fraunhofer-Gesellschaft (2018): Maschinelles Lernen – eine Analyse zu Kompetenzen, Forschung und Anwendung

Publications: 

Elsotohy, Mariam; Jaeschke, Johannes; Sehr, Frederic; Schneider-Ramelow, Martin: Mission profile-based digital twin framework using functional mockup interfaces for assessing system’s degradation behaviour, Proceedings of Microelectronics Reliability 2023

Dobs, Tom; Elsotohy, Mariam; Jaeschke, Johannes; Sehr, Frederic; Strogies, Joerg, Wilke, Klaus: Multi-domain system level modeling approach for assessment of degradation behaviour under thermal and thermo-mechanical stress, Proceedings of Microelectronics Reliability 2022.