Neural Concept https://neuralconcept.com 3D Deep Learning Software for Enhanced Engineering Fri, 27 Mar 2020 17:11:20 +0000 en-US hourly 1 https://wordpress.org/?v=5.3.2 In-graph training loop https://neuralconcept.com/blog/in-graph-training-loop/ https://neuralconcept.com/blog/in-graph-training-loop/#respond Fri, 27 Mar 2020 16:17:35 +0000 https://neuralconcept.com/?p=1444 In a previous experiment, we have explored the behaviour and interaction between keras models, tf.functions, saved models and tf.dataset. A summary of this experiment is available here as a notebook. As a follow up to this previous test, we compared the performance of in-graph training loop ( https://www.tensorflow.org/guide/function#advanced_example_an_in-graph_training_loop ) with loop in python and model with annotated __call__ function. […]

The post In-graph training loop appeared first on Neural Concept.

]]>
In a previous experiment, we have explored the behaviour and interaction between keras models, tf.functions, saved models and tf.dataset. A summary of this experiment is available here as a notebook.

As a follow up to this previous test, we compared the performance of in-graph training loop ( https://www.tensorflow.org/guide/function#advanced_example_an_in-graph_training_loop ) with loop in python and model with annotated __call__ function.

We performed training with 100, 200, 400, 800 and 1600 steps with the naval dataset with the following config –

It is seen that the python loop has some upfront initial cost, but over the period of time it runs faster than in-graph training loop.

The initial cost of the python loop can be attributed to the fact that multiple traces are performed for each different shapes, but once all the different shapes are encountered, there are no further delays and it runs faster than the in-graph loop.

The finding also corroborates with the issue https://github.com/tensorflow/tensorflow/issues/35165

In order to avoid retracing for different shapes, we should provide input_signature to the model tf.function. The input_signature is known only after the partial shape information is available, so instead of statically annotating with tf.function, we need to create a tf.function with the right input signature on the fly.

Comparisons of training in python loop, avoiding the retracing –

Follow Up

As a follow up to Tf.function retracing vs specifying an input signature , training on naval dataset is run for 5000 steps, with various combinations of annotating tf functions. Following are the results

ModelOuter functionRetracing happensMean train step (discarding retracing times)
1tf.func with signaturepython funcno0.28
2tf.func without signaturetrain_step is tf.func with signatureno0.22
3tf func without signaturecompute_gradient is tf.func with signatureno0.25
4tf func without signaturecompute_gradient is tf.func without signatureyes0.24
5tf func without signaturetrain_step is tf.func without signatureyes0.14

Refer to attached notebook for details:

We can conclude that retracing is expensive operation, but once the function has retraced all possible shapes, the compiled function is faster.

Making train_step as the tf.function is much faster than making compute_gradient as the tf function .

The post In-graph training loop appeared first on Neural Concept.

]]>
https://neuralconcept.com/blog/in-graph-training-loop/feed/ 0
Deep Neural Network in simulations https://neuralconcept.com/blog/deep-neural-network-in-simulations/ https://neuralconcept.com/blog/deep-neural-network-in-simulations/#respond Wed, 11 Mar 2020 09:23:55 +0000 https://neuralconcept.com/?p=1378 Deep learning and AI in general have taken the entire field of computer science by storm and has now become the dominant approach to solving a wide array of problems, ranging from winning board games to molecular discovery. However, Computer Assisted Design (CAD) and geometry processing are still mostly based on traditional techniques. Indeed, numerical […]

The post Deep Neural Network in simulations appeared first on Neural Concept.

]]>
Deep learning and AI in general have taken the entire field of computer science by storm and has now become the dominant approach to solving a wide array of problems, ranging from winning board games to molecular discovery. However, Computer Assisted Design (CAD) and geometry processing are still mostly based on traditional techniques.

Indeed, numerical simulation techniques have traditionally relied on solving physically derived equations using finite differences and adding heuristic models when those become too complex to solve (Turbulence models in fluid mechanics for example). More recently, Lattice Boltzmann methods have become popular as a means to simulate streaming and collision processes across a limited number of particles. Both classes of techniques remain (computationally) (very) expensive (we are talking about hours to days of simulation), and since the simulation must be re-run each time an engineer wishes to change the shape, this makes the design process slow and costly. 

A typical engineering approach is therefore to test only a few designs without a fine-grained search in the space of potential variations. Hence the company is limited by:

  1. Its financial resources for the given project 
  2. The time constraints of the project
  3. Its engineers’ experience and cognitive biases during the product development phase.

Since this is a severe limitation, there have been many attempts at overcoming it and one of the most famous is reduced-order modeling:

Reduced-Order modeling (ROM) is a class of Machine Learning approaches used to learn a simplified model of a simulator, based on data. This method is a simplification of a high-fidelity dynamical model, built from a large number of numerical simulations. It preserves essential behavior and dominant effects, for the purpose of reducing solution time or storage capacity required for the more complex model. It is applied in a large range of physics and has proven its efficiency for specific applications. It works well when the engineer wants to vary a few, well-defined parameters, with a specific objective in mind. A good overview of the different techniques used is given in this paper: https://www.sciencedirect.com/science/article/abs/pii/S0376042103001131

Figure 2: Classical Response Surface built from a reduced-order model

However, the modeling power of classical ROM methods is limited, and they present several drawbacks:

  1. For some industries, the majority of the simulation data is acquired through experiments, by sensors being placed at various locations, conditions… This data cannot be easily transferred to a reduced-order model. Indeed, it would require that the conditions of experiments are always strictly identical, which is very rarely the case, as you need a total control over the parameters and conditions of simulation
  2. When building the reduced-order model, a parameterization is defined and kept throughout the whole project and the simulations have to be generated using this parametrization. If a company is facing new requirements for a given product, or wants to explore new designs, they may be forced to change their parameterization. Then, they would have to start everything from scratch again and re-generate a bunch of simulations to build a new ROM. It creates silos in the company’s workflow, where ROMs are built for very specific use-cases and are hardly used for later applications.
  3. Some applications require the simulations of very complex phenomena, with discontinuities that may appear (transonic flows…). ROMs tend to « smooth » these discontinuities, giving large errors in these specific regions.

Deep Neural-Networks are an extension of classical Reduced Order Modeling, where the approximating function is not limited to a simple linear model but can be extended through a stack of non-linear operations, called layers. A very recent branch of the Deep-Learning research applies this concept to the processing of geometric information and was able to overcome the limitations of more classical reduced-order models. Based on a neural network architecture, it is able to understand 3D shapes and learns how they interact with the laws of physics. Since it uses raw, 3D, unprocessed geometries as input, it does not suffer from all of the previous drawbacks I mentioned. The engineer is now able to leverage on its historical database (even if the parametrization of a given part has evolved over time!) and integrate experimental data as well. 3D Deep Learning allows to switch from silos workflows to a common base where the information is globally shared and continuously re-used.

It is also orthogonal to the physics, and the same technique can tackle a very large range of physics. Finally, its overall performance is able to improve over time, as it can be fed with new simulations on the fly.

Conclusion: After these few lines, it seems like 3D Deep Learning is the solution to enhance engineering processes! Well it is, but there is also a very important step in order to exploit the full potential of this tool (but also of any machine learning technique): preparing your data so that the model can extract the maximum of information out of it. In a follow-up article, I will come with a few tips and tricks to get the best out of your data when using Machine Learning for numerical simulations. Stay tuned! 

The post Deep Neural Network in simulations appeared first on Neural Concept.

]]>
https://neuralconcept.com/blog/deep-neural-network-in-simulations/feed/ 0
Artificial Intelligence meets Aerodynamics – the Ultimate Drone https://neuralconcept.com/news/artificial-intelligence-meets-aerodynamics-the-ultimate-drone/ Mon, 24 Feb 2020 10:35:49 +0000 https://neuralconcept.com/?p=1348 18 months ago, Neural Concept, EPFL (École polytechnique fédérale de Lausanne), senseFly and AirShaper teamed up for an academic research project to apply deep learning to aerodynamics. Neural Concept Shape was coupled with AirShaper to explore the space of designs, with the ultimate objective to find more efficient designs. Based on those learnings, our software […]

The post Artificial Intelligence meets Aerodynamics – the Ultimate Drone appeared first on Neural Concept.

]]>
18 months ago, Neural Concept, EPFL (École polytechnique fédérale de Lausanne), senseFly and AirShaper teamed up for an academic research project to apply deep learning to aerodynamics.

Neural Concept Shape was coupled with AirShaper to explore the space of designs, with the ultimate objective to find more efficient designs. Based on those learnings, our software started seeing trends and improved its understanding of the application. Soon, it started making predictions on what could be an even better aerodynamic shape! The result? A more efficient drone that will fly further on the same battery charge! This was a first research project and the potential for other applications (cars, transportation, planes, …) is monumental. Are you also working on an application where aerodynamic improvements can make a difference? Just let us know!

We published a report describing the work that has been done and further analyzing what we managed to do, you can read it here.

The post Artificial Intelligence meets Aerodynamics – the Ultimate Drone appeared first on Neural Concept.

]]>
NeuralSampler: Euclidean Point Cloud Auto-Encoder and Sampler https://neuralconcept.com/blog/neuralsampler-euclidean-point-cloud-auto-encoder-and-sampler/ Wed, 22 Jan 2020 11:41:55 +0000 https://neuralconcept.com/?p=1328 We propose an auto-encoder architecture that can both encode and decode clouds of arbitrary size and demonstrate its effectiveness at upsampling sparse point clouds. Interestingly, we can do so using less than half as many parameters as state-of-the-art architectures while still delivering better performance. You can download the full publication here.

The post NeuralSampler: Euclidean Point Cloud Auto-Encoder and Sampler appeared first on Neural Concept.

]]>

We propose an auto-encoder architecture that can both encode and decode clouds of arbitrary size and demonstrate its effectiveness at upsampling sparse point clouds. Interestingly, we can do so using less than half as many parameters as state-of-the-art architectures while still delivering better performance.

You can download the full publication here.

The post NeuralSampler: Euclidean Point Cloud Auto-Encoder and Sampler appeared first on Neural Concept.

]]>
SP80 will be using Neural Concept Shape https://neuralconcept.com/news/sp80-will-be-using-neural-concept-shape/ Thu, 19 Dec 2019 10:15:31 +0000 https://neuralconcept.com/?p=1259 Early September 2019, the ultra aerodynamic bike we helped designing broke two world records of speed. It is with the same objective in mind that we are pleased to announce our collaboration with SP80 : https://sp80.ch/. This team composed of EPFL Alumni and students are working to build the world’s fastest sailboat. They aim at […]

The post SP80 will be using Neural Concept Shape appeared first on Neural Concept.

]]>
Early September 2019, the ultra aerodynamic bike we helped designing broke two world records of speed. It is with the same objective in mind that we are pleased to announce our collaboration with SP80 : https://sp80.ch/.

This team composed of EPFL Alumni and students are working to build the world’s fastest sailboat. They aim at reaching the speed at 80 knots, with the sole power of a kite. To be able to reach such speed, aero and hydrodynamic optimization is mandatory. Moreover, the team needs to model quite complex physical phenomenon such as super cavitation, with numerical simulations that can take a lot of time (hours to days). Using Neural Concept Shape, they will be able to reduce this computation time to milliseconds, allowing them to further optimize the design of their sailboat and ultimately gain these precious knots to break the world record.

Stay tuned for further updates on our collaboration !

The post SP80 will be using Neural Concept Shape appeared first on Neural Concept.

]]>
Neural Concept is in Vancouver for NeurIPS 2019 https://neuralconcept.com/news/neural-concept-is-in-vancouver-for-neurips-2019/ Thu, 12 Dec 2019 14:50:19 +0000 https://neuralconcept.com/?p=1246 Neural Concept is in Vancouver, Canada! Our CEO, Pierre Baqué, will join forces with Airbus engineers to demonstrate applications of Real Time CFD simulations with 3D Mesh Convolutional Networks. If you are there, feel free to reach out to us!

The post Neural Concept is in Vancouver for NeurIPS 2019 appeared first on Neural Concept.

]]>
Neural Concept is in Vancouver, Canada! Our CEO, Pierre Baqué, will join forces with Airbus engineers to demonstrate applications of Real Time CFD simulations with 3D Mesh Convolutional Networks. If you are there, feel free to reach out to us!

The post Neural Concept is in Vancouver for NeurIPS 2019 appeared first on Neural Concept.

]]>
Neural Concept will be at the NAFEMS Conference https://neuralconcept.com/news/neural-concept-3d-deep-learning-for-cae-shape-optimization/ Tue, 08 Oct 2019 09:00:57 +0000 https://staging.arcticonline.com/neural-concept-final/?p=234 NAFEMS is an independent not-for-profit company, with the mission to provide knowledge, international collaboration and educational opportunities for the use and validation of engineering simulation. NAFEMS is widely held to be the leading independent source of information and training for engineering analysts and designers at all levels. On the 15th and 16th of October, will […]

The post Neural Concept will be at the NAFEMS Conference appeared first on Neural Concept.

]]>

NAFEMS is an independent not-for-profit company, with the mission to provide knowledge, international collaboration and educational opportunities for the use and validation of engineering simulation. NAFEMS is widely held to be the leading independent source of information and training for engineering analysts and designers at all levels. On the 15th and 16th of October, will take place the NAFEMS European Conference: Simulation-Based Optimisation in London.

Neural Concept has been invited to hold a presentation there on the 15th of October. Pierre Baqué, our CEO, will speak about 3D Deep-learning Based Surrogate Modeling and Optimisation, its applications and advantages in today’s industrial world. It will be followed by a Q&A session, so do not hesitate to come and discuss with us about the current and upcoming challenges in simulation !

The post Neural Concept will be at the NAFEMS Conference appeared first on Neural Concept.

]]>
Congratulations François! https://neuralconcept.com/news/neural-concept-francois-fleuret-news-machine-learning-3d-simulation/ Mon, 30 Sep 2019 09:00:59 +0000 https://staging.arcticonline.com/neural-concept-final/?p=229 Member of our board, Dr. François Fleuret, currently Senior Scientist at Idiap and head of the machine learning group, has been appointed Adjunct Professor at EPFL (École polytechnique fédérale de Lausanne). Congratulations François for this achievement! https://actu.epfl.ch/news/nominations-of-epfl-professors-132/

The post Congratulations François! appeared first on Neural Concept.

]]>
François Fleuret, neural concept advisors. Machine learning 3D simulation and shape optimization.
Dr. François Fleuret

Member of our board, Dr. François Fleuret, currently Senior Scientist at Idiap and head of the machine learning group, has been appointed Adjunct Professor at EPFL (École polytechnique fédérale de Lausanne). Congratulations François for this achievement!

https://actu.epfl.ch/news/nominations-of-epfl-professors-132/

The post Congratulations François! appeared first on Neural Concept.

]]>
2 world records broken thanks in part to our cutting edge technology https://neuralconcept.com/news/neural-concept-3d-deep-learning-softaware-for-cae-news-shape-optimization-acoustics-simulation/ Thu, 19 Sep 2019 09:00:05 +0000 https://staging.arcticonline.com/neural-concept-final/?p=223 More than one year ago, we started to collaborate with the IUT Annecy to compete for The World Human Powered Speed Challenge. This is a competition involving bicycles designed by teams of university students, and the goal is, of course, to be the fastest. During the Challenge, the pilot will have to ride down a […]

The post 2 world records broken thanks in part to our cutting edge technology appeared first on Neural Concept.

]]>
More than one year ago, we started to collaborate with the IUT Annecy to compete for The World Human Powered Speed Challenge. This is a competition involving bicycles designed by teams of university students, and the goal is, of course, to be the fastest. During the Challenge, the pilot will have to ride down a 200-meter stretch of straight, flat road as fast as possible, after a run-up of 8 km in the Nevada desert.

Start of the race, it is crucial to give the pilots the best launch possible

We used our software, Neural Concept Shape, to optimize the aerodynamic performances of the bike, making sure Fabien Canal and Ilona Peltier, the two pilots, could make the most out of the bike. After a first year where the team beat the french record in the male category , they were back this year. And guess what, the competition is not yet finished and they have already beaten two world records ! Here is the video of their amazing adventure:

We used our software, Neural Concept Shape, to optimize the aerodynamic performances of the bike, making sure Fabien Canal and Ilona Peltier, the two pilots, could make the most out of the bike. After a first year where the team beat the french record in the male category , they were back this year. And guess what, the competition is not yet finished and they have already beaten two world records ! Here is the video of their amazing adventure : https://www.youtube.com/watch?v=PQ67PdsEPgQ&feature=youtu.be

In the men category, Fabien managed to reach a top speed of 136.74 km/h, new French and European record, but also the new world university record !
At 30 years old, Fabien is now the second fastest man in the world, behind the Canadian Todd RICHERT (144.17 km/h).

In the women category, Ilona reached a top speed of 124.05 km/h, new world record across all categories !
At 19 years old, Ilona is now the world’s fastest woman !

Ilona and Fabien with Altaïr 6, their bike for this year’s competition

Neural Concept would like to congratulate the whole team from IUT Annecy for this great achievement, we are very proud to have contributed by designing Altaïr, a bike designed exclusively and entirely by Neural Concept Shape, without any human intervention.

The post 2 world records broken thanks in part to our cutting edge technology appeared first on Neural Concept.

]]>
A new world-record attempt for our favourite aero-bike https://neuralconcept.com/news/neural-concept-news-shape-optimization-and-physics-based-3d-deep-learning-software/ Thu, 05 Sep 2019 09:00:12 +0000 https://staging.arcticonline.com/neural-concept-final/?p=220 We are very excited to announce that the team from the University of Annecy is back to Battle Mountain, Arizona, with the goal of breaking the Human Powered Speed world record! Last year, they broke the french record with a bicycle design that was optimized with the help of Neural Concept Shape. This year, the […]

The post A new world-record attempt for our favourite aero-bike appeared first on Neural Concept.

]]>
We are very excited to announce that the team from the University of Annecy is back to Battle Mountain, Arizona, with the goal of breaking the Human Powered Speed world record! Last year, they broke the french record with a bicycle design that was optimized with the help of Neural Concept Shape. This year, the design remains the same, but major additional features make it a good candidate to go much faster.

  • Very low rolling resistance Michelin Tires
  • Bearing in ceramic in the transmission
  • Camera vision instead of periscope
  • Improvement of pilote’s breath
  • New frame, new fork, therefore new geometry

Their objective is to reach the academic record of 133,78km/h and beat the absolute word record of 144,17km/h!

We have collaborated with this great team in 2018 to optimize the shape of the bike’s shell. We wish them good luck and all the best for their new project!

Follow their adventures on Youtube:

The post A new world-record attempt for our favourite aero-bike appeared first on Neural Concept.

]]>