Loss functions with a large number of saddle points are one of the major obstacles for training modern machine learning (ML) models efficiently. First-order methods such as gradient descent (GD) are usually the methods of choice for training ML models. However, these methods converge to saddle points for certain choices of initial guesses. In this paper, we propose a modification of the recently proposed Laplacian smoothing gradient descent (LSGD) [Osher et al., arXiv:1806.06317], called modified LSGD (mLSGD), and demonstrate its potential to avoid saddle points without sacrificing the convergence rate. Our analysis is based on the attraction region, formed by all starting points for which the considered numerical scheme converges to a saddle point. We investigate the attraction region’s dimension both analytically and numerically. For a canonical class of quadratic functions, we show that the dimension of the attraction region for mLSGD is $\lfloor (n-1)/2\rfloor$, and hence it is significantly smaller than that of GD whose dimension is $n-1$.

In this paper, we study the Cauchy problem for the semilinear heat and Schrödinger equations, with the nonlinear term $f(u)=\unicode[STIX]{x1D706}|u|^{\unicode[STIX]{x1D6FC}}u$. We show that low regularity of $f$ (i.e., $\unicode[STIX]{x1D6FC}>0$ but small) limits the regularity of any possible solution for a certain class of smooth initial data. We employ two different methods, which yield two different types of results. On the one hand, we consider the semilinear equation as a perturbation of the ODE $w_{t}=f(w)$. This yields, in particular, an optimal regularity result for the semilinear heat equation in Hölder spaces. In addition, this approach yields ill-posedness results for the nonlinear Schrödinger equation in certain $H^{s}$-spaces, which depend on the smallness of $\unicode[STIX]{x1D6FC}$ rather than the scaling properties of the equation. Our second method is to consider the semilinear equation as a perturbation of the linear equation via Duhamel’s formula. This yields, in particular, that if $\unicode[STIX]{x1D6FC}$ is sufficiently small and $N$ is sufficiently large, then the nonlinear heat equation is ill-posed in $H^{s}(\mathbb{R}^{N})$ for all $s\geqslant 0$.

The numerical solution of the harmonic heat map flow problems with blowup in finite or infinite time is considered using an adaptive moving mesh method. A properly chosen monitor function is derived so that the moving mesh method can be used to simulate blowup and produce accurate blowup profiles which agree with formal asymptotic analysis. Moreover, the moving mesh method has finite time blowup when the underlying continuous problem does. In situations where the continuous problem has infinite time blowup, the moving mesh method exhibits finite time blowup with a blowup time tending to infinity as the number of mesh points increases. The inadequacy of a uniform mesh solution is clearly demonstrated.