My MPICH/Slurm job is failing when the call to MPI_Comm_spawn is made.   The Slurm MPI guide https://slurm.schedmd.com/mpi_guide.html#mpich2 specifically states that MPI_Comm_spawn will work going through Hydra’s PMI 1.1 interface.

How do I ensure that it goes through that interface?

Maybe we’ll have to rebuild Slurm to support PMI 1.1.    This Slurm command  yields the following and PMI 1.1 is not mentioned, although PMI 2 is.

$ srun –mpi=list

srun: MPI types are...

srun: cray_shasta

srun: pmi2

srun: none