Digital Thread: Deep Learning Applications in Engineering
The challenges in engineering are ever-increasing and the industry players are in search of assistance from new technologies. In recent years, we have heard about the so-called digital thread. Embracing the digital thread has helped engineering departments design better products faster. However, the concept is an empty box without hands-on technologies.
Artificial Intelligence can go beyond pattern recognition for self driving cars and can enter Engineering departments as a full-fledged tool to support critical decisions.
We will show how the introduction of digital threads has opened the door to Deep Learning, an artificial intelligence discipline born from Machine Learning. Deep Learning techniques have become ever more specialized. Since 2018, deep learning algorithms Neural Concept Shape (NCS) have contributed to create a new type of engineering prediction at the service of product design engineers. They are helping the digitalization in the automotive industry and other industries as well.
The Artificial Intelligence laboratory out of which NCS was born is CV Lab in Lausanne (Switzerland). Their Deep Learning application is feature recognition, based on convolutional neural networks; a specialized form of artificial neural networks.
We will first give a practical example of using AI to support innovation; in this case the encasing of a battery that protects the battery from crashes.
Deep Learning Capabilities - a Practical Use Case
Let's take as an example a car battery with an encasing suitable to protect it.
No carmaker can ensure a vehicle will not crash; this will depend on the drivers and other circumstances! However, extensive physical and virtual tests carried out by engineering departments have drastically ameliorated driver, passenger and pedestrian safety in the last few years. Thus, it is now recognized that engineering methodologies can make a difference in the physical integrity of human beings and car components.
We will now summarise a few decades of progress in engineering methodologies in three steps: Physical Testing by specialists, Engineering computation by specialists with CAD and CAE and finally, Engineering real-time prediction by all engineers with Engineering Intelligence.
1st Stage in Engineering: Physical Testing
Crashworthiness (crash resistance) of components or full car bodies can be verified by building a prototype and testing it in a laboratory, with specific and expensive measurements with sensors and cameras placed on the car, its components and the dummies representing drivers, passengers or pedestrians. Prototypes and tests require highly skilled engineers in the construction of prototypes, organization of testing, execution of testing and extraction of the most useful data.
2nd Stage in Engineering: Engineering Computation (CAE) - Software 1.0
The test can be carried out on a computer with a reduction of cost and time with respect to prototypes and laboratory testing. This is possible by using associated virtual versions (digital twins) of the prototypes (CAD) and laboratory test (CAE - 3D crash simulation). Moreover, the collection of data becomes even more systematic in a database than with laboratory testing due to its virtual nature. This stage involves specialized software (CAD, CAE), specialized staff, and computers (in private compute farms or in the Cloud) that yield answers in hours.
Also, data produced by CAE are extremely dense with respect to testing, i.e. virtually on any point of interest. As shown in the figure below, levels of deformation, stress and so on can be captured.
Thus, the second stage created an expectation of increased efficiency. On the other hand, it generated a lot of stored data with deeper insight.
Here comes the question: could data be used to increase efficiency so that a virtual test can be carried out within seconds and without specialized staff?
We will see how this is possible in the third stage of engineering.
3rd Stage in Engineering: Artificial Intelligence Prediction (Deep Learning) - Software 2.0
By putting together virtual descriptions of physical parts (in CAD) and their virtual behaviour during simulation (in CAE) over and over again, companies generate numerous datasets. They are the ideal field of application for Artificial Intelligence; particularly for a new approach attracting interest since 2012: Deep Learning.
As the image shows, deep learning can reproduce classic simulation. However, deep learning requires a fraction of the time that traditional simulation does. Moreover, it can expand by 10 or 100 the audience of potential users.
The task of NCS deep learning algorithm is to link Machine Learning and experienced CAE AI teams with Design engineers to guide designers faster towards radically better concepts.
Beyond the example of crash prediction, whatever can be simulated with CAE can also be predicted by AI when AI is fed with CAE data. Examples are:
- thermal simulation in aerospace and automotive applications
- ground vehicle aerodynamics, both for passenger transport and motorsport racing
- prediction of performance maps for compressors
How To Use Digital Thread in Product Design?
After seeing the tremendous impact of Artificial Intelligence on Engineering with a practical case, let's take a step back to understand the context and how much the above success story could be reproduced, in general, within the industry.
The digital thread refers to integrating data and processes across all stages of product development, which can be achieved through various digital technologies. We will mainly deal with computer-aided design (CAD) and data analytics tools such as CAE (computer-aided engineering).
Companies can create a better product development process by embracing the digital thread. For example, data collected during the design phase can be used to inform and optimize the manufacturing process. Data collected during the operation and maintenance phase can be fed into the design process to improve future products.
The digital thread can also ease collaboration and information sharing, allowing for more informed decision-making and faster turnaround times.
Adopting the digital thread can help engineering departments design better products faster by enabling them to use data and processes throughout the product development lifecycle.
Artificial Intelligence in the Digital Thread
AI, or artificial intelligence, can be a key component of the digital thread in several ways.
First, Artificial Intelligence can analyze and extract insights from the data collected throughout product development. This can help inform and optimize decision-making. Examples are selecting the best materials or manufacturing processes for a given product.
Second, Artificial Intelligence can automate specific tasks and processes within the product development lifecycle. For example, AI-powered CAD software can assist with the design process, and artificial intelligence can suggest alternative designs or detect errors. We will show more of this in conjunction with CAE.
Finally, Artificial Intelligence can enable predictive maintenance, allowing companies to identify and address potential production issues before they occur.
Integrating Artificial Intelligence into the digital thread can help companies design better products faster by enabling them to use data and automation throughout product development.
A branch of artificial intelligence called Deep Learning is a specialized form of Artificial Intelligence and machine learning that can be brought to the service of designers.
After diving into machine learning algorithms and deep learning models, we will show that deep learning capabilities can help engineers with techniques from computer vision and visual recognition.
The Basics of Deep Learning Technology - Machine Learning Algorithms
Machine Learning - How Does It Work?
Artificial intelligence / Machine learning has three main types of learning: supervised, unsupervised, and reinforcement machine learning algorithms.
Machine Learning Approach: Reinforcement
Reinforcement learning is a type of artificial intelligence/machine learning that involves training a model to make a sequence of decisions in an environment to maximize a reward.
Reinforcement machine learning has been successful in various applications, including game-playing, robotic control, and recommendation systems.
Machine Learning Approach: Non-Supervised
Unsupervised machine learning is a type of machine learning where the model is not given any labelled training data and is instead asked to learn from the underlying structure of the input data.
Unsupervised machine learning has succeeded in various applications, for instance, discovering patterns or relationships in data that may not be immediately apparent and can be used to pre-train models for supervised learning tasks.
Machine Learning Approach: Supervised
The supervised learning approach in Deep Learning, such as with NCS, is a type of artificial intelligence/machine learning where a predictive model is trained with input features and corresponding correct output values.
Typical applications include image and speech recognition, signal processing, and natural language recognition.
The algorithm aims to learn a function that can map the input features to the output labels.
During training, the algorithm iteratively makes predictions on the input data and compares its predictions to the correct labels. It then adjusts its internal parameters to reduce errors between its predictions and the correct labels. This process is known as optimization.
Once trained, the model can predict based on new data. Depending on the type of machine learning, the model may be able to continue learning and adapting to new data even after it has been trained.
A typical example is a response surface built to predict according to a reduced-order model, thanks to neural networks. In the following, we will review more sophisticated models for deep neural networks in simulation.
Deep Learning - Introduction
Deep learning is a subfield of machine learning inspired by the brain's structure and function, specifically the neural networks that make up the brain.
It involves training convolutional neural networks composed of multiple layers of interconnected "neurons" on a large dataset.
The network can learn and make intelligent decisions by adapting the weights and biases of the connections between neurons based on the data it is presented with.
Deep learning - What are Deep Neural Networks?
Deep learning applications can provide a supplementary data-based approach to problems for which physical models generate learning data.
The most intuitive deep learning applications are the reproduction of some functions of a real-world biological neural network analogue. Examples can be vision or hearing.
Let's now analyze the structure of a deep learning network (see figure).
The lower convolutional layers can detect basic image features, such as edges.
Convolutional layers are separated by pooling layers adding robustness to feature detection.
Successive layers build from edges to form features with multiple edges or curves. In most network architectures, pooling layers are placed between convolutional layers, and this is done to add robustness to each feature-detecting layer (see figure below).
Benefits of Deep Learning Networks applied to Engineering
The specific branch of artificial intelligence, Deep Learning, is a specialized form of machine learning that can be brought to the service of designers.
Using deep learning in engineering can improve accuracy, automation, prediction capabilities, and efficiency, leading to better and faster decision-making and product development. The success of Neural Concept has been to implement serious computer science, data analysis and advanced machine learning methods to produce user-friendly deep learning applications to simplify engineers' life, without asking them to learn any new software.
Deep learning has the potential to bring several benefits to engineering. We will list four of them.
- Deep Learning benefit: Improved accuracy
- Deep learning can achieve higher accuracy than traditional machine learning, with large and complex datasets.
- Deep Learning benefit: Automation of time-consuming tasks
- Artificial neural networks can automate tasks that are time-consuming or infeasible for humans. Examples are analyzing large data sets or identifying patterns in images.
- Deep Learning benefit: Enhanced prediction capabilities
- Artificial neural networks can be trained to make predictions based on past data. This can be useful for tasks such as predictive maintenance or forecasting demand for a product.
- Deep Learning benefit: Improved efficiency
- Artificial neural networks can often perform tasks better than humans. This allows engineering teams to focus on higher-level tasks and strategies.
Deep Learning: From Theory to Application
We have seen the significant progress introduced by the digital thread concept, how Software 1.0 is feeding Software 2.0, and the theory behind it.
We will now go back to the example seen at the beginning, the battery crash case.
We will recall the example to see the degree of confidence of predictions by the deep neural network and how the deep neural network is implemented in practice.
Deep Learning Application - Battery Crash Case
In the case of the crash prediction, the choice was to empower a wide team of engineers. Thus, rather than keeping the application restricted to a few specialists, it becomes available to all designers.
To this purpose, Neural Concept built an AI-based App working in quasi-real-time as a practical example of deep learning patform for design engineers. The App gives answers to designers while not oversimplifying the approach to simulation (i.e. no "curve-cutting").
In this use case, the critical aspect was predicting if a design configuration produced contact between the battery and its encasing during a crash. A measure of confidence would be to have no false positives. The Covid pandemic has made us sadly familiar with the concept of false positives. For a review: a false positive is the case when we predict an event when, in reality, there is no such an event. A false negative is when we miss capturing the event.
We come back to the event of battery-encasing contact during a crash. We wish to avoid false positives, where an AI-based App will wrongly predict a contact. Also, we wish to have a minimum number of false negatives, i.e. situations when the App misses to identify a contact situation.
Hundreds of designs were fed to the AI-based App to get predictions. Over the hundreds of predictions, the App predicted only one false negative, producing no false positives. This gave designers the confidence to use AI-based Apps to optimise the battery housing design.
A more detailed analysis was carried out to predict displacement and maximum force.
Thus, quite surprisingly, Deep Learning can predict both a static situation and a dynamic situation (a time history).
How does AI compare to "truth"? A good measure of the accuracy of NCS AI-model is the so-called coefficient of determination R2. R2 compares how much the prediction matches available data, in our case, CAE simulation.
The predictive model reached a very high accuracy in all displacement directions, where R2 equal to 1 would mean a perfect match between AI and CAE.
Other Applications Examples
Many more applications are possible for Deep Learning in simulation. Imagination (and a business case) is the only limit. Since deep learning is data-driven, it is not constrained to a specific physics, unlike CAE solvers.
Optimal thermal management is an issue affecting many businesses: for example, heating, ventilating and air conditioning systems are present in automotive, chemical, civil engineering, aerospace and other industries. The optimization of heat exchangers can for instance be fully carried out (both the simulation and the shape optimization stages) with NCS.
Since there is a recurring interest in cold plates and optimized cooling paths, Neural Concept has developed an AI tool verticalized to the specific engineering application.
Moving to a radically different type of industry and physics, the design of ultrasonic piezo-electric actuators is supported by AI predictions giving excellent results in terms of comparison to classic CAE methods.
Remember - only imagination is the limit! We wonder, can electromagnetism be simulated with real-time predictions? Yes, it has been shown that the leading CAE software in the market are excellent "ground truths" to train NCS predictions, ranging from antenna applications to electrical motors.
Conclusion: From Deep Learning to Technological Innovation in Product Design
Now, we should have fewer doubts that deep learning is already in the position to foster technological innovation in the industry.
The key factor is to enable engineers with robust predictive capabilities, without the toils of specialistic tools such as CAE simulation. Deep learning technology for engineers as the core engine will not require extra skills to design engineers since the usage will be driven by Apps.
Thus, by increasing the number of users by orders of magnitude and empowering them with real-time AI-based Apps, it is possible to embrace the concept of product engineering as continuous improvement and adaptation to the surrounding challenges.
With deep learning, design engineers will be supported in conceiving products beyond what had been typically achievable with traditional design tools such as physical cycles of testing, or CAE simulation.
