There is no way for internode latency to be less than ~1 us. Thus, I conclude you measure intranode results with MPICH and internode with everything else.

Check your mpirun commands and try again. 

Jeff

On Tue, Nov 26, 2019 at 10:57 AM Sajid Ali via discuss <discuss@mpich.org> wrote:

Hi MPICH-developers,

I’ve been trying to benchmark mpi software currently available on my univ. cluster with OSU micro benchmarks. I used this script to build and run the benchmark : https://github.com/s-sajid-ali/nu-quest-mpi/blob/master/curr_mpi_bench/submit.sh

And here is a plot of the results I obtained :
curr_mpi.png

Could someone give me a clue as to why the behaviour of mpich bandwidth vs latency is so odd ? Why does the bandwidth increase first and then fall off at high message sizes?

I’m also attaching the config.log for this particular mpich build which seems to indicate that MPICH was built with ch3 and nemesis:tcp. Does this mean that it didn’t pick up the infiniband libraries or that it is somehow using IPoIB? Is there a way to get this information ?

Thanks in advance for the help!


--
Sajid Ali | PhD Candidate
Applied Physics
Northwestern University
s-sajid-ali.github.io

_______________________________________________
discuss mailing list     discuss@mpich.org
To manage subscription options or unsubscribe:
https://lists.mpich.org/mailman/listinfo/discuss
--
Jeff Hammond
jeff.science@gmail.com
http://jeffhammond.github.io/