[Paper Review] HPC realization of a controlled turbulent round jet using OpenFOAM

Asim Önder, Johan Meyers|arXiv (Cornell University)|Jun 27, 2014

Fluid Dynamics and Turbulent Flows9 references3 citations

TL;DR

This paper enhances OpenFOAM's high-performance computing (HPC) capabilities for direct numerical simulation (DNS) of a low-Reynolds-number turbulent round jet controlled by zero-net-mass-flux (ZNMF) actuators. By combining parallel statistical averaging with grid partitioning and implementing an incremental projection method, the authors achieve a 30% speedup and an effective parallel efficiency of 87% at 624 processors, significantly improving scalability and reducing wall time for turbulent statistics collection.

ABSTRACT

The present paper investigates high performance computing abilities of OpenFOAM for a low Reynolds number ($Re_D=2000$) axisymmetric jet subject to multiple zero net mass flux (ZNMF) actuators. First, parallel performance of OpenFOAM is tested by performing a scaling study up to $2048$ processors on a supercomputer of Flemish Supercomputer Center(VSC). Then, a method to improve the parallel efficiency is proposed. The method is based on developing a hybrid concept to calculate the statistical moments. This new concept combines ensemble and time averaging in order to allow data sampling in parallel. The motivation is obtaining a reduction in the walltime to collect turbulent statistics which is observed to be the dominating part in the ZNMF controlled jet flow. Employing this parallel statistical averaging approach in combination with regular grid partitioning parallelism, allowed us conducting DNS cases on $P=624$ processors with an overall speed-up of $S_e=540.56$ and a parallel efficiency of $E_e=0.87$. The parallelization using only grid partitioning exhibited inferior performance with $S_e=423.94$ and $E_e=0.68$. In addition to develop a methodology to increase the parallel performance, we reduced the time step cost of the existing unsteady solver as well. To this end, an incremental projection method is implemented into OpenFOAM and a performance gain above $30\%$ is realized.

Motivation & Objective

• To address the limited scalability of OpenFOAM in simulating high-fidelity, ZNMF-controlled turbulent round jets at low Reynolds number (ReD = 2000).
• To reduce the dominant wall time cost associated with collecting turbulent statistics in unsteady DNS simulations.
• To enhance parallel efficiency by combining ensemble averaging with grid partitioning for improved load balancing and data sampling.
• To reduce time step cost in the unsteady solver by replacing the iterative PISO algorithm with a non-iterative incremental projection method.

Proposed method

• Conducted DNS of an axisymmetric turbulent round jet with multiple ZNMF actuators using OpenFOAM on a supercomputer (VSC).
• Implemented a hybrid statistical averaging method combining ensemble and time averaging to enable efficient parallel data sampling.
• Used grid partitioning for domain decomposition and load balancing across up to 2048 processors in scaling studies.
• Introduced an incremental projection method to solve the momentum and continuity equations non-iteratively, preserving second-order accuracy.
• Optimized the solver by replacing the standard iterative PISO algorithm with a non-iterative scheme, reducing time step cost.
• Validated performance improvements via strong scaling tests and comparison of parallel efficiency and speedup across different configurations.

Experimental results

Research questions

• RQ1 Can parallel statistical averaging in combination with grid partitioning significantly improve the scalability of OpenFOAM for DNS of ZNMF-controlled jets?
• RQ2 What is the impact of the incremental projection method on time step performance in OpenFOAM’s unsteady solver?
• RQ3 How does the hybrid statistical averaging approach compare to standard grid-only partitioning in terms of parallel efficiency and speedup?
• RQ4 To what extent can the number of ensemble members be increased to further improve parallel efficiency in statistical data collection?
• RQ5 What is the effect of actuation frequency on coherent structure development and mixing efficiency in controlled turbulent jets?

Key findings

• The hybrid statistical averaging method achieved an effective speedup of 540.56 and parallel efficiency of 87% at 624 processors, significantly outperforming grid-only partitioning (speedup 423.94, efficiency 68%).
• The incremental projection method reduced time step cost by over 30%, demonstrating substantial performance gains in the unsteady solver.
• Controlled cases with actuation frequencies near the natural jet instability (e.g., Con1) showed enhanced coherent structures and prolonged spectral peaks at downstream locations.
• The entrainment rate and scalar decay rate were significantly improved in actuated cases (Con0.5, Con1, Con2), with Con1 showing the best mixing efficiency.
• Spectral analysis revealed distinct actuation-frequency peaks and subharmonics due to vortex pairing, especially in lower-frequency actuation cases.
• The results indicate that increasing the number of ensemble members (R) offers potential for further performance improvement beyond the current R=4 limit.