We describe NekRS, which is an open source spectral element thermal/fluids code designed for exascale platforms. NekRS supports both incompressible and low-Mach formulations, ALE-based moving meshes, uRANS, and numerous boundary conditions of relevance to engineering flows. Second- and third-order timesteppers are provided, using either standard (CFL-limited) semi-implicit methods or characteristics-based subcycling for advection. A variety of scalable multilevel preconditioners for the pressure Poisson problem have been implemented to ensure optimal performance across a large application space. Several simulation results are presented, including turbulent flow in the full core of a pebble bed reactor that features 352,000 spherical pebbles in an annular domain. The mesh features 98.8 million elements of order N=8, for a total of n=50.5 billion gridpoints. Using 27648 NVIDIA V100s on the OLCF supercomputer, Summit, the wall-clock time for this case is 0.25 seconds per step, corresponding to 6 hours for an entire flow-through time.
*This research is supported by the Department of Energy Exascale Computing Project (17-SC-20-SC). This research used resources of the Oak Ridge Leadership Computing Facility at Oak Ridge National Laboratory, which is supported by the Office of Science of the US Department of Energy under Contract No. DE-AC05-00OR22725.
–
Publication:Paul Fischer, Stefan Kerkemeier, Misun Min, Yu-Hsiang Lan, Malachi Phillips, Thilina Rath- nayake, Elia Merzari, Ananias Tomboulides, Ali Karakus, Noel Chalmers, and Tim Warburton. NekRS, a GPU-accelerated spectral element Navier-Stokes solver (under review).
Paul Fischer, Elia Merzari, Misun Min, Stefan Kerkemeier, Yu-Hsiang Lan, Malachi Phillips, Thilina Rathnayake, April Novak, Derek Gaston, Noel Chalmers, and Tim Warburton. Highly optimized full-core reactor simulations on Summit. (2021).
Yu-Hsiang Lan, Paul Fischer, Elia Merzari, and Misun Min. All-hex meshing strategies for densely packed spheres. In The 29th International Meshing Roundtable, 2021.
