Hi Jeff/Giuseppe, I spoke to the computational specialist from research IT and he confirmed that this particular build of MPICH is indeed broken. As Giuseppe pointed out the hydra launcher was running multi threaded jobs on each node because it was unable to read the nodelist. When we passed a nodelist to mpirun and repeated the same test, the results looked more reasonable (~80us latency instead of ~0.05us due to ch3:TCP). Thanks for the help! -- Sajid Ali | PhD Candidate Applied Physics Northwestern University s-sajid-ali.github.io