This course establishes a rigorous mathematical foundation for neural networks and explores their modern applications in scientific computing and generative modeling. Students will begin from approximation theory to understand the expressive power of neural networks, then advance through optimization methods, stochastic dynamical systems, and state-of-the-art data-driven and generative modeling techniques.