None

25 Jun 2026

      as less latency than 4KB.

I was looking for explanation of this behavior  but did not get any.

  1.  MPIR_CVAR_CH3_EAGER_MAX_MSG_SIZE is set to 128KB. So none of the abov=
e message size is using Rendezvous protocol. Is there any partition inside =
eager protocol (e.g. 0 - 512 bytes, 1KB - 8KB, 16KB - 64KB)? If yes then wh=
at are the boundaries for them? Can I log them with debug-event-logging?

Setup I am using: 

- two nodes has intel core i7, one with 16gb memory another one 8gb

- mpich 3.2.1, configured and build to use nemesis tcp

- 1gb Ethernet connection

- NFS is using for sharing

- osu_latency: uses MPI_Send and MPI_Recv

- MPIR_CVAR_CH3_EAGER_MAX_MSG_SIZE=3D 131072 (128KB)

Can anyone help me on that? Thanks in advance.

Best Regards,

Abu Naser

_______________________________________________
discuss mailing list     [email protected]<mailto: [email protected]>
To manage subscription options or unsubscribe: 
https: //lists.mpich.org/mailman/listinfo/discuss

--
Jeff Hammond
[email protected]<mailto: [email protected]>
http: //jeffhammond.github.io/

--_000_BLUPR0501MB200383DFBCA0721E71DDAC0E97770BLUPR0501MB2003_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html><head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
...
<style type=3D"text/css" style=3D"display: none;"><!-- P {margin-top:0;margi=
n-bottom: 0;} --></style>
</head>
<body dir=3D"ltr">
<div id=3D"divtagdefaultwrapper" style=3D"font-size: 12pt;color:#000000;font=
-family: Calibri,Helvetica,sans-serif;" dir=3D"ltr">
<p style=3D"margin-top: 0;margin-bottom:0">Hello Jeff,</p>
<p style=3D"margin-top: 0;margin-bottom:0"><br>
</p>
<p style=3D"margin-top: 0;margin-bottom:0">Yes, I am using a switch and othe=
r machines are also connected with that switch.
<br>
</p>
<p style=3D"margin-top: 0;margin-bottom:0">If I remove other machines and ju=
st use my two node with the switch, then will it improve the performance by=
 200 ~ 400 iterations?</p>
<p style=3D"margin-top: 0;margin-bottom:0">Meanwhile I will give a try with =
a single dedicated cable.
<span></span><br>
</p>
<p style=3D"margin-top: 0;margin-bottom:0"><br>
</p>
<p style=3D"margin-top: 0;margin-bottom:0">Thank you.<br>
</p>
<div id=3D"Signature">
<div id=3D"divtagdefaultwrapper" dir=3D"ltr" style=3D"font-size: 12pt; color=
: rgb(0,0,0); font-family:Calibri,Helvetica,sans-serif,"EmojiFont"=
,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,&q=
uot;Segoe UI Symbol","Android Emoji",EmojiSymbols">
<p><br>
</p>
<p align=3D"left"><span style=3D"font-size: 10pt; font-family:Calibri,Helvet=
ica,sans-serif">Best Regards,</span></p>
<span style=3D"font-family: Calibri,Helvetica,sans-serif; font-size:10pt"></=
span>
<div align=3D"left"><span style=3D"font-size: 11pt; font-family:Calibri,Helv=
etica,sans-serif"></span></div>
<span style=3D"font-family: Calibri,Helvetica,sans-serif; font-size:10pt"></=
span>
<p align=3D"left"><span style=3D"font-size: 10pt; font-family:Calibri,Helvet=
ica,sans-serif">Abu Naser</span><br>
</p>
</div>
</div>
</div>
<hr style=3D"display: inline-block;width:98%" tabindex=3D"-1">
<div id=3D"divRplyFwdMsg" dir=3D"ltr"><font face=3D"Calibri, sans-serif" st=
yle=3D"font-size: 11pt" color=3D"#000000"><b>From:</b> Jeff Hammond <jeff=
[email protected]><br>
<b>Sent: </b> Wednesday, June 20, 2018 12:52:06 PM<br>
<b>To: </b> MPICH<br>
<b>Subject: </b> Re: [mpich-discuss] osu_latency test: why 8KB takes less ti=
me than 4KB and 2KB takes less time than 1KB?</font>
<div> </div>
</div>
<meta content=3D"text/html; charset=3Dutf-8">
<div>
<div dir=3D"ltr">Is the ethernet connection a single dedicated cable betwee=
n the two machines or are you running through a switch that handles other t=
raffic?
<div><br>
</div>
<div>My best guess is that this is noise and that you may be able to avoid =
it by running a very long time, e.g. 10000 iterations.</div>
<div><br>
</div>
<div>Jeff</div>
</div>
<div class=3D"x_gmail_extra"><br>
<div class=3D"x_gmail_quote">On Wed, Jun 20, 2018 at 6: 53 AM, Abu Naser <sp=
an dir=3D"ltr">
<<a href=3D"mailto: [email protected]" target=3D"_blank">[email protected]<=
/a>></span> wrote: <br>
<blockquote class=3D"x_gmail_quote" style=3D"margin: 0 0 0 .8ex; border-left=
: 1px #ccc solid; padding-left:1ex">
<div dir=3D"ltr">
<div id=3D"x_m_6077755676379859201divtagdefaultwrapper" dir=3D"ltr" style=
=3D"font-size: 12pt; color:rgb(0,0,0); font-family:Calibri,Helvetica,sans-se=
rif,"EmojiFont","Apple Color Emoji","Segoe UI Emoj=
i",NotoColorEmoji,"Segoe UI Symbol","Android Emoji"=
;,EmojiSymbols">
<p style=3D"margin-top: 0; margin-bottom:0"><br>
</p>
<p style=3D"margin-top: 0; margin-bottom:0">Good day to all,</p>
<p style=3D"margin-top: 0; margin-bottom:0"><br>
</p>
<p style=3D"margin-top: 0; margin-bottom:0">I had run point to point osu_lat=
ency test in two nodes for 200 times.  Followings are the average time=
 in microsecond for various size of the messages -</p>
<p style=3D"margin-top: 0; margin-bottom:0"></p>
<div>1KB    84.8514 us<br>
<span style=3D"color: rgb(255,0,0)">2KB    73.52535</span> us=
<br>
4KB    272.55275 us<br>
<span style=3D"color: rgb(255,0,0)">8KB    234.86385</span> u=
s<br>
16KB    288.88 us<br>
32KB    523.3725 us<br>
64KB    910.4025 us</div>
<p></p>
<p style=3D"margin-top: 0; margin-bottom:0"><br>
</p>
<p style=3D"margin-top: 0; margin-bottom:0">From the above looks like, 2KB m=
essage has less latency than 1 KB and 8KB has less latency than 4KB.
<br>
</p>
<p style=3D"margin-top: 0; margin-bottom:0">I was looking for explanation of=
 this behavior  but did not get any.</p>
<p style=3D"margin-top: 0; margin-bottom:0"><br>
</p>
<ol style=3D"margin-bottom: 0px; margin-top:0px">
<li><span>MPIR_CVAR_CH3_EAGER_MAX_MSG_<wbr>SIZE</span><span> is set to 128K=
B. So none of the above message size is using Rendezvous protocol. Is there=
 any partition inside eager protocol (e.g. 0 - 512 bytes, 1KB - 8KB, 16KB -=
 64KB)? If yes then what are the
 boundaries for them? Can I log them with debug-event-logging? </span><br>
</li></ol>
<p style=3D"margin-top: 0; margin-bottom:0"><br>
</p>
<p style=3D"margin-top: 0; margin-bottom:0">Setup I am using:</p>
<p style=3D"margin-top: 0; margin-bottom:0">- two nodes has intel core i7, o=
ne with 16gb memory another one 8gb</p>
<p style=3D"margin-top: 0; margin-bottom:0">- mpich 3.2.1, configured and bu=
ild to use nemesis tcp</p>
<p style=3D"margin-top: 0; margin-bottom:0">- 1gb Ethernet connection</p>
<p style=3D"margin-top: 0; margin-bottom:0">- NFS is using for sharing<br>
</p>
<p style=3D"margin-top: 0; margin-bottom:0">- osu_latency : uses MPI_Send an=
d MPI_Recv</p>
<p style=3D"margin-top: 0; margin-bottom:0">- <span>MPIR_CVAR_CH3_EAGER_MAX_=
MSG_<wbr>SIZE</span>=3D
<span>131072</span> (128KB)<br>
</p>
<p style=3D"margin-top: 0; margin-bottom:0"><br>
</p>
<p style=3D"margin-top: 0; margin-bottom:0">Can anyone help me on that? Than=
ks in advance.<br>
</p>
<p style=3D"margin-top: 0; margin-bottom:0"><br>
</p>
<p style=3D"margin-top: 0; margin-bottom:0"><br>
</p>
<div id=3D"x_m_6077755676379859201Signature">
<div id=3D"x_m_6077755676379859201divtagdefaultwrapper" dir=3D"ltr" style=
=3D"font-size: 12pt; color:rgb(0,0,0); font-family:Calibri,Helvetica,sans-se=
rif,"EmojiFont","Apple Color Emoji","Segoe UI Emoj=
i",NotoColorEmoji,"Segoe UI Symbol","Android Emoji"=
;,EmojiSymbols">
<p><br>
</p>
<p align=3D"left"><span style=3D"font-size: 10pt; font-family:Calibri,Helvet=
ica,sans-serif">Best Regards,</span></p>
<span style=3D"font-family: Calibri,Helvetica,sans-serif; font-size:10pt"></=
span>
<div align=3D"left"><span style=3D"font-size: 11pt; font-family:Calibri,Helv=
etica,sans-serif"></span></div>
<span style=3D"font-family: Calibri,Helvetica,sans-serif; font-size:10pt"></=
span>
<p align=3D"left"><span style=3D"font-size: 10pt; font-family:Calibri,Helvet=
ica,sans-serif">Abu Naser</span><br>
</p>
</div>
</div>
</div>
</div>
<br>
______________________________<wbr>_________________<br>
discuss mailing list     <a href=3D"mailto: [email protected]=
">[email protected]</a><br>
To manage subscription options or unsubscribe: <br>
<a href=3D"https: //lists.mpich.org/mailman/listinfo/discuss" rel=3D"norefer=
rer" target=3D"_blank">https: //lists.mpich.org/<wbr>mailman/listinfo/discus=
s</a><br>
<br>
</blockquote>
</div>
<br>
<br clear=3D"all">
<div><br>
</div>
-- <br>
<div class=3D"x_gmail_signature">Jeff Hammond<br>
<a href=3D"mailto: [email protected]" target=3D"_blank">jeff.science@gm=
ail.com</a><br>
<a href=3D"http: //jeffhammond.github.io/" target=3D"_blank">http://jeffhamm=
ond.github.io/</a></div>
</div>
</div>
</body>
</html>

--_000_BLUPR0501MB200383DFBCA0721E71DDAC0E97770BLUPR0501MB2003_--

--===============3147691773054401677==
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
discuss mailing list     [email protected]
To manage subscription options or unsubscribe: 
https: //lists.mpich.org/mailman/listinfo/discuss

--===============3147691773054401677==--
Message-ID: <[email protected]>

None

tags

participants (1)