Hi Bruce, Thanks for the details. Since you already instructed `sbatch` with `-N 2 -n 6` I thought you could simply run with `srun`, i.e. without further `-n`? I believe hydra also reads environment so you could also try `mpiexec` without `-n`. Did you also try `-N 1` and `-N 6`? -- Hui Zhou From: "Palmer, Bruce J" <[email protected]> Date: Thursday, May 7, 2020 at 11:16 AM To: "Zhou, Hui" <[email protected]>, "[email protected]" <[email protected]> Subject: Re: [mpich-discuss] MPICH configure Hi Hui, Sorry for the late reply, I keep getting pulled off on other projects. I’m actually running a test suite most of the time, so the job scripts look like #!/bin/csh #SBATCH -t 02:30:00 #SBATCH -A XGA #SBATCH -p short,slurm,gpu #SBATCH -N 2 #SBATCH -n 6 #SBATCH -o ./test.out #SBATCH -e ./test.err source /etc/profile.d/modules.csh source ~/set_mpich env | grep PATH module list #make check-ga MPIEXEC="mpirun -n 6 " make check-ga MPIEXEC="srun -n 6 " I’ve tried using mpirun, srun and mpiexec in the MPIEXEC variable. If I run a test standalone, then the job submission script is #!/bin/csh #SBATCH -t 02:30:00 #SBATCH -A XGA #SBATCH -p short,slurm,gpu #SBATCH -N 2 #SBATCH -n 6 #SBATCH -o ./test.out #SBATCH -e ./test.err source /etc/profile.d/modules.csh source ~/set_mpich env | grep PATH module list srun -n 6 test.x > test.out Again, I’ve tried running with mpirun, srun, and mpiexec. The environment in the set_mpich file is module purge module load gcc/5.2.0 module load python/2.7.8 module load cmake/3.8.2 module load git module load mkl setenv CC gcc setenv CFLAGS "-pthread" setenv CXX g++ setenv CXXFLAGS "-pthread" setenv FC gfortran setenv FCFLAGS "-pthread" unsetenv F90 unsetenv F90FLAGS setenv PATH /people/d3g293/mpich/mpich-3.3.2/install/bin:${PATH} setenv MANPATH /people/d3g293/mpich/mpich-3.3.2/install/share/man:${MANPATH} setenv LD_LIBRARY_PATH /people/d3g293/mpich/mpich-3.3.2/install/lib:${LD_LIBRARY_PATH} Bruce