CS Seminar

SPEAKER: James Lottes, Argonne National Laboratory

TITLE: Algebraic Multigrid for a Petascale Spectral Element Method Navier-Stokes Code

DATE: Thursday, September 13, 2007
TIME: 3:00 P.M.
PLACE: 4124 Siebel Center
201 N. Goodwin Ave., Urbana, IL

ABSTRACT

Nek5000 is spectral element code for solving the time-dependent Navier-Stokes equations that has successfully scaled to 32K processors (the full rack of the IBM BG/W machine). By far the most time-consuming part of the simulation is spent solving the Poisson-like pressure equation at each time step. I will give a brief overview of the current iterative solver, a multilevel additive Schwarz solver. I will show why the fast XX^T direct solver, used at the coarsest level, has scaled very well so far, but will become a barrier for future scaling, motivating its replacement by an approximate AMG solver.

Next I will discuss ongoing work investigating AMG for this application. Because the coarse pressure solver is invoked tens of thousands of times for the same equation, we can afford an almost arbitrarily long time spent in the set-up phase of AMG. I am investigating how this fact can be leveraged to tune the invocation stage to be as fast and efficient as possible.

Past investigators have considered using Sparse Approximate Inverses (SAI) as smoothers in multigrid algorithms. They are attractive because (1) their application is a matrix-vector product and hence inherently parallel, (2) they are inexpensive to compute, and (3) for a given sparsity pattern they are parameter-free. Building on the theory of Falgout, we will show how the SAI idea can be used also to cheaply construct an optimal sparse interpolation operator. We will show that the "residual" of this operator, against the optimal dense interpolation operator, can be used to easily identify those fine degrees of freedom which cannot be adequately interpolated (and hence should be added to the coarse set). This procedure is both simpler and less ad-hoc than coarsening via compatible relaxation. I will present a new compatible relaxation scheme which yields the exact spectrum of the full two-level method while remaining computable, and show that it might be used to improve the selection of an SAI smoother.