Researchers in the Mathematical and Scientific Machine Learning Laboratory (MSML) leading by Prof. Ming-Chih Lai at Department of Applied Mathematics in NYCU investigate computational fluid mechanics and explore potential machine learning approaches in solving fluid-structure interaction problems. The study of such incompressible flows with interfaces plays an important role in many natural phenomena and industrial applications, especially for soft matter physics and fluid dynamics of microfluidic systems, and is also of major interest to the applied mathematics community.
Over the past decade, deep learning has gained incredible success in image recognition, natural language processing, computer vision, and many other practical applications in our daily life. But only until recently, it draws much attention to solving partial differential equations using deep neural networks in the scientific computing community. Part of the theoretical reason can be attributed to the various kinds of expressive power for function approximations using DNN. Automatic differentiation in machine learning makes it easier to evaluate derivatives through neural networks, which is probably another reason in practice.
Based on the experience of studying the fast solvers for partial differential equations, Prof. Lai’s group at MSML, and in collaboration with scientists at National Central University, have developed a simple yet efficient way to present discontinuous functions, discontinuity-capturing shallow neural network (DCSNN). There are three major advantages to using this network; namely, (1) jump discontinuity in the function is captured sharply (as shown in Figure); (2) it is completely shallow, comprising only one hidden layer, (3) it is completely mesh-free when it is used to solve PDEs. Compared with traditional grid-based numerical methods, this network can easily solve partial differential equations defined in irregular regions, and even in high dimensions.
The idea behind the design of DCSNN is to introduce an extra input variable (as shown in Figure) that acts as an indicator to distinguish different smooth parts of a function. The network for solving PDEs is based on Physics-Informed Neural Network in literature (Raissi, Perdikaris, and Karniadakis 2019) trained by minimizing the mean squared error loss, which includes the residuals of the governing equations, boundary conditions, and interface jump conditions. With such a simple design and shallow network architecture, the DCSNN model solves the elliptic interface problem with efficiency and accuracy comparable to the well-known numerical methods.
Lai’s group at NYCU also investigate another type of elliptic problem where delta function singularity appears as a sourcing term to the equation. By introducing the energy functional of the problem, the delta function singularity can be replaced by a regular integral along the interface. The network is again designed to be shallow and includes an additional level set function as augmented input feature. The training of the network here is based on deep Ritz method (E and Yu 2018) which trains the network by minimizing the loss function written as a discrete version of the energy functional. Numerical experiments have shown that, with proper design, a shallow network with moderate number of neurons can solve the problem with high accuracy compared to deep neural networks proposed in existing literature. One should notice that, a shallow network is much easier to train than the deep one. This research shows that traditional challenging problems in scientific computing can be solved efficiently with the tool of machine learning. These important advances should bring more applied mathematicians in Taiwan into the field of scientific machine learning.