October 2024 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] MPI_Send over ibverbs
by Raffenetti, Ken 08 Oct '24

08 Oct '24

It’s in MPI scope. To my knowledge, that level of acknowledgement is not exposed from the verbs API. Ken From: Niyaz Murshed <Niyaz.Murshed(a)arm.com> Date: Tuesday, October 8, 2024 at 11:08 AM To: Raffenetti, Ken <raffenet(a)anl.gov>, discuss(a)mpich.org <discuss(a)mpich.org> Cc: nd <nd(a)arm.com> Subject: Re: [mpich-discuss] MPI_Send over ibverbs Are you talking about the ack of RC connection? Or is it MPI’s scope? From: Raffenetti, Ken <raffenet@ anl. gov> Date: Tuesday, October 8, 2024 at 11: 05 AM To: discuss@ mpich. org <discuss@ mpich. org> Cc: Niyaz Murshed <Niyaz. Murshed@ arm. com>, ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd Are you talking about the ack of RC connection? Or is it MPI’s scope? From: Raffenetti, Ken <raffenet(a)anl.gov> Date: Tuesday, October 8, 2024 at 11:05 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Niyaz Murshed <Niyaz.Murshed(a)arm.com>, nd <nd(a)arm.com> Subject: Re: [mpich-discuss] MPI_Send over ibverbs An internal acknowledgement message is sent from the receiver to the sender indicating that a particular synchronous send operation was matched. Ken From: Niyaz Murshed via discuss <discuss(a)mpich.org> Date: Tuesday, October 8, 2024 at 10:49 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Niyaz Murshed <Niyaz.Murshed(a)arm.com>, nd <nd(a)arm.com> Subject: Re: [mpich-discuss] MPI_Send over ibverbs Thank you for the reply Joachim. ‘MPI_Ssend has the requirement to return after the message was matched at the receiver’ – From the above statement, how do the sender know that the receiver called a matching receive? I am asking from the point ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd Thank you for the reply Joachim. ‘MPI_Ssend has the requirement to return after the message was matched at the receiver’ – >From the above statement, how do the sender know that the receiver called a matching receive? I am asking from the point of view of MPI_Send over RDMA. In case of RDMA, internally it will call rdma_send which does not know when a matching rdma_recv was called on destination. How do the sender knows when rdma_recv was called? From: Joachim Jenke via discuss <discuss(a)mpich.org> Date: Tuesday, October 8, 2024 at 9:42 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Joachim Jenke <jenke(a)itc.rwth-aachen.de> Subject: Re: [mpich-discuss] MPI_Send over ibverbs Hi, none of the MPI send calls has a requirement to return after the message was received. MPI_Send can return as soon as the caller can write to buffer without impact to the data being transferred. This might be after the data is put to the network, or the MPI library copied the data to an internal buffer. MPI_Ssend has the requirement to return after the message was matched at the receiver, which implies that the matching Receive was called. But it still does not require that the message was received at this point. Best Joachim Am 08.10.24 um 04:55 schrieb Niyaz Murshed via discuss: > Hi, > > Is there a way to check the implementation of MPI_Send? > > As I understand there is an implicit barrier which allows MPI_Send to > return only after the buffer is received by the destination. > > How is this communication done? How does the destination notify the > sender that buffer is received? > > How is this implemented when using rocev2 ? > > Regards, > > Niyaz > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://urldefense.us/v3/__https://lists.mpich.org/mailman/listinfo/discuss… <https://urldefense.us/v3/__https:/lists.mpich.org/mailman/listinfo/discuss_…> -- Dr. rer. nat. Joachim Jenke IT Center Group: High Performance Computing Division: Computational Science and Engineering RWTH Aachen University Seffenter Weg 23 D 52074 Aachen (Germany) Tel: +49 241 80- 24765 Fax: +49 241 80-624765 jenke(a)itc.rwth-aachen.de https://urldefense.us/v3/__http://www.itc.rwth-aachen.de__;!!G_uCfscf7eWS!d… <https://urldefense.us/v3/__http:/www.itc.rwth-aachen.de__;!!G_uCfscf7eWS!aT…>

1 0

Re: [mpich-discuss] MPI_Send over ibverbs
by Niyaz Murshed 08 Oct '24

08 Oct '24

Are you talking about the ack of RC connection? Or is it MPI’s scope? From: Raffenetti, Ken <raffenet(a)anl.gov> Date: Tuesday, October 8, 2024 at 11:05 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Niyaz Murshed <Niyaz.Murshed(a)arm.com>, nd <nd(a)arm.com> Subject: Re: [mpich-discuss] MPI_Send over ibverbs An internal acknowledgement message is sent from the receiver to the sender indicating that a particular synchronous send operation was matched. Ken From: Niyaz Murshed via discuss <discuss(a)mpich.org> Date: Tuesday, October 8, 2024 at 10:49 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Niyaz Murshed <Niyaz.Murshed(a)arm.com>, nd <nd(a)arm.com> Subject: Re: [mpich-discuss] MPI_Send over ibverbs Thank you for the reply Joachim. ‘MPI_Ssend has the requirement to return after the message was matched at the receiver’ – From the above statement, how do the sender know that the receiver called a matching receive? I am asking from the point ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd Thank you for the reply Joachim. ‘MPI_Ssend has the requirement to return after the message was matched at the receiver’ – >From the above statement, how do the sender know that the receiver called a matching receive? I am asking from the point of view of MPI_Send over RDMA. In case of RDMA, internally it will call rdma_send which does not know when a matching rdma_recv was called on destination. How do the sender knows when rdma_recv was called? From: Joachim Jenke via discuss <discuss(a)mpich.org> Date: Tuesday, October 8, 2024 at 9:42 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Joachim Jenke <jenke(a)itc.rwth-aachen.de> Subject: Re: [mpich-discuss] MPI_Send over ibverbs Hi, none of the MPI send calls has a requirement to return after the message was received. MPI_Send can return as soon as the caller can write to buffer without impact to the data being transferred. This might be after the data is put to the network, or the MPI library copied the data to an internal buffer. MPI_Ssend has the requirement to return after the message was matched at the receiver, which implies that the matching Receive was called. But it still does not require that the message was received at this point. Best Joachim Am 08.10.24 um 04:55 schrieb Niyaz Murshed via discuss: > Hi, > > Is there a way to check the implementation of MPI_Send? > > As I understand there is an implicit barrier which allows MPI_Send to > return only after the buffer is received by the destination. > > How is this communication done? How does the destination notify the > sender that buffer is received? > > How is this implemented when using rocev2 ? > > Regards, > > Niyaz > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://urldefense.us/v3/__https://lists.mpich.org/mailman/listinfo/discuss… <https://urldefense.us/v3/__https:/lists.mpich.org/mailman/listinfo/discuss_…> -- Dr. rer. nat. Joachim Jenke IT Center Group: High Performance Computing Division: Computational Science and Engineering RWTH Aachen University Seffenter Weg 23 D 52074 Aachen (Germany) Tel: +49 241 80- 24765 Fax: +49 241 80-624765 jenke(a)itc.rwth-aachen.de https://urldefense.us/v3/__http://www.itc.rwth-aachen.de__;!!G_uCfscf7eWS!c… <https://urldefense.us/v3/__http:/www.itc.rwth-aachen.de__;!!G_uCfscf7eWS!aT…>

1 0

Re: [mpich-discuss] MPI_Send over ibverbs
by Raffenetti, Ken 08 Oct '24

08 Oct '24

An internal acknowledgement message is sent from the receiver to the sender indicating that a particular synchronous send operation was matched. Ken From: Niyaz Murshed via discuss <discuss(a)mpich.org> Date: Tuesday, October 8, 2024 at 10:49 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Niyaz Murshed <Niyaz.Murshed(a)arm.com>, nd <nd(a)arm.com> Subject: Re: [mpich-discuss] MPI_Send over ibverbs Thank you for the reply Joachim. ‘MPI_Ssend has the requirement to return after the message was matched at the receiver’ – From the above statement, how do the sender know that the receiver called a matching receive? I am asking from the point ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd Thank you for the reply Joachim. ‘MPI_Ssend has the requirement to return after the message was matched at the receiver’ – >From the above statement, how do the sender know that the receiver called a matching receive? I am asking from the point of view of MPI_Send over RDMA. In case of RDMA, internally it will call rdma_send which does not know when a matching rdma_recv was called on destination. How do the sender knows when rdma_recv was called? From: Joachim Jenke via discuss <discuss(a)mpich.org> Date: Tuesday, October 8, 2024 at 9:42 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Joachim Jenke <jenke(a)itc.rwth-aachen.de> Subject: Re: [mpich-discuss] MPI_Send over ibverbs Hi, none of the MPI send calls has a requirement to return after the message was received. MPI_Send can return as soon as the caller can write to buffer without impact to the data being transferred. This might be after the data is put to the network, or the MPI library copied the data to an internal buffer. MPI_Ssend has the requirement to return after the message was matched at the receiver, which implies that the matching Receive was called. But it still does not require that the message was received at this point. Best Joachim Am 08.10.24 um 04:55 schrieb Niyaz Murshed via discuss: > Hi, > > Is there a way to check the implementation of MPI_Send? > > As I understand there is an implicit barrier which allows MPI_Send to > return only after the buffer is received by the destination. > > How is this communication done? How does the destination notify the > sender that buffer is received? > > How is this implemented when using rocev2 ? > > Regards, > > Niyaz > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://urldefense.us/v3/__https://lists.mpich.org/mailman/listinfo/discuss… <https://urldefense.us/v3/__https:/lists.mpich.org/mailman/listinfo/discuss_…> -- Dr. rer. nat. Joachim Jenke IT Center Group: High Performance Computing Division: Computational Science and Engineering RWTH Aachen University Seffenter Weg 23 D 52074 Aachen (Germany) Tel: +49 241 80- 24765 Fax: +49 241 80-624765 jenke(a)itc.rwth-aachen.de https://urldefense.us/v3/__http://www.itc.rwth-aachen.de__;!!G_uCfscf7eWS!d… <https://urldefense.us/v3/__http:/www.itc.rwth-aachen.de__;!!G_uCfscf7eWS!aT…>

1 0

Re: [mpich-discuss] MPI_Send over ibverbs
by Niyaz Murshed 08 Oct '24

08 Oct '24

Thank you for the reply Joachim. ‘MPI_Ssend has the requirement to return after the message was matched at the receiver’ – >From the above statement, how do the sender know that the receiver called a matching receive? I am asking from the point of view of MPI_Send over RDMA. In case of RDMA, internally it will call rdma_send which does not know when a matching rdma_recv was called on destination. How do the sender knows when rdma_recv was called? From: Joachim Jenke via discuss <discuss(a)mpich.org> Date: Tuesday, October 8, 2024 at 9:42 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Joachim Jenke <jenke(a)itc.rwth-aachen.de> Subject: Re: [mpich-discuss] MPI_Send over ibverbs Hi, none of the MPI send calls has a requirement to return after the message was received. MPI_Send can return as soon as the caller can write to buffer without impact to the data being transferred. This might be after the data is put to the network, or the MPI library copied the data to an internal buffer. MPI_Ssend has the requirement to return after the message was matched at the receiver, which implies that the matching Receive was called. But it still does not require that the message was received at this point. Best Joachim Am 08.10.24 um 04:55 schrieb Niyaz Murshed via discuss: > Hi, > > Is there a way to check the implementation of MPI_Send? > > As I understand there is an implicit barrier which allows MPI_Send to > return only after the buffer is received by the destination. > > How is this communication done? How does the destination notify the > sender that buffer is received? > > How is this implemented when using rocev2 ? > > Regards, > > Niyaz > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://urldefense.us/v3/__https://lists.mpich.org/mailman/listinfo/discuss… -- Dr. rer. nat. Joachim Jenke IT Center Group: High Performance Computing Division: Computational Science and Engineering RWTH Aachen University Seffenter Weg 23 D 52074 Aachen (Germany) Tel: +49 241 80- 24765 Fax: +49 241 80-624765 jenke(a)itc.rwth-aachen.de https://urldefense.us/v3/__http://www.itc.rwth-aachen.de__;!!G_uCfscf7eWS!a… <https://urldefense.us/v3/__http://www.itc.rwth-aachen.de__;!!G_uCfscf7eWS!a… >

1 0

Re: [mpich-discuss] How to use ppr with mpiexec?
by Zhou, Hui 08 Oct '24

08 Oct '24

If you are using hydra – the default launcher installed with MPICH, the “-hosts” option accepts a comma separated list, so something like “-hosts node1:4,node2:4,node3:4”. The “:4” tells the launcher to assign 4 processes per host, so you don’t need another “-map-by” option. Alternatively, you could use “-hosts node1,node2,node3 -ppn 4” for the same effect. -- Hui Zhou From: Jeffrey Layton via discuss <discuss(a)mpich.org> Date: Tuesday, October 8, 2024 at 6:50 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Jeffrey Layton <laytonjb(a)gmail.com> Subject: [mpich-discuss] How to use ppr with mpiexec? Good morning, I'm trying to "port" an Open MPI script with MPICH and I'm having some trouble. I've got a few questions. What is the form of the "-hosts" option? Is it something like '-hosts node1: 4 node2: 4 ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd Good morning, I'm trying to "port" an Open MPI script with MPICH and I'm having some trouble. I've got a few questions. 1. What is the form of the "-hosts" option? Is it something like '-hosts node1:4 node2:4 node3:4 ...'? 2. The Open MPI script has an option "-map-by ppr:4:node" that tells it to run 4 processes per node and then it runs as many nodes as needed to meet the number of processes. How can I do this in MPICH? Thanks for the help! Jeff

1 0

Re: [mpich-discuss] Building MPICH man pages
by Raffenetti, Ken 08 Oct '24

08 Oct '24

Hi Howard, You’ll need to add --with-doc to your autogen.sh to actually generate the man srcs. From there you might need some additional tools installed to actually generate the output. One is txt2man which should be available via package managers. The other is doctext that you’ll need to grab from Bill’s home page. https://urldefense.us/v3/__https://wgropp.cs.illinois.edu/projects/software… Ken From: Pritchard Jr., Howard via discuss <discuss(a)mpich.org> Date: Tuesday, October 8, 2024 at 8:58 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Pritchard Jr., Howard <howardp(a)lanl.gov> Subject: [mpich-discuss] Building MPICH man pages Hi All, I’m interested in seeing how the MPICH package generates its man pages, and have been trying to build/install main branch (fresh checkout from github). The package builds as far as libs and compiler wrappers are concerned, but there ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd Hi All, I’m interested in seeing how the MPICH package generates its man pages, and have been trying to build/install main branch (fresh checkout from github). The package builds as far as libs and compiler wrappers are concerned, but there do not appear to be any man pages. I’m doing checkout from github git submodule –init –recursive ./autogen.sh ./configure --prefix=a place to call home make -j 4 install do I need to use a special configure or make option? Thanks for any pointers, Howard ------- [signature_61897647] Howard Pritchard Research Scientist HPC-ENV Los Alamos National Laboratory howardp(a)lanl.gov<mailto:[email protected]> [signature_1349344541]<https://urldefense.us/v3/__https:/www.instagram.com/losalamosnatlab/__;!!G_…>[signature_447832744]<https://urldefense.us/v3/__https:/twitter.com/LosAlamosNatLab__;!!G_uCfscf7…>[signature_2725493895]<https://urldefense.us/v3/__https:/www.linkedin.com/company/los-alamos-natio…>[signature_536422773]<https://urldefense.us/v3/__https:/www.facebook.com/LosAlamosNationalLab/__;…>

1 0

Re: [mpich-discuss] MPI_Send over ibverbs
by Joachim Jenke 08 Oct '24

08 Oct '24

Hi, none of the MPI send calls has a requirement to return after the message was received. MPI_Send can return as soon as the caller can write to buffer without impact to the data being transferred. This might be after the data is put to the network, or the MPI library copied the data to an internal buffer. MPI_Ssend has the requirement to return after the message was matched at the receiver, which implies that the matching Receive was called. But it still does not require that the message was received at this point. Best Joachim Am 08.10.24 um 04:55 schrieb Niyaz Murshed via discuss: > Hi, > > Is there a way to check the implementation of MPI_Send? > > As I understand there is an implicit barrier which allows MPI_Send to > return only after the buffer is received by the destination. > > How is this communication done? How does the destination notify the > sender that buffer is received? > > How is this implemented when using rocev2 ? > > Regards, > > Niyaz > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Dr. rer. nat. Joachim Jenke IT Center Group: High Performance Computing Division: Computational Science and Engineering RWTH Aachen University Seffenter Weg 23 D 52074 Aachen (Germany) Tel: +49 241 80- 24765 Fax: +49 241 80-624765 jenke(a)itc.rwth-aachen.de www.itc.rwth-aachen.de

1 0

Building MPICH man pages
by Pritchard Jr., Howard 08 Oct '24

08 Oct '24

Hi All, I’m interested in seeing how the MPICH package generates its man pages, and have been trying to build/install main branch (fresh checkout from github). The package builds as far as libs and compiler wrappers are concerned, but there do not appear to be any man pages. I’m doing checkout from github git submodule –init –recursive ./autogen.sh ./configure --prefix=a place to call home make -j 4 install do I need to use a special configure or make option? Thanks for any pointers, Howard ------- [signature_61897647] Howard Pritchard Research Scientist HPC-ENV Los Alamos National Laboratory howardp(a)lanl.gov<mailto:[email protected]> [signature_1349344541]<https://urldefense.us/v3/__https://www.instagram.com/losalamosnatlab/__;!!G… >[signature_447832744]<https://urldefense.us/v3/__https://twitter.com/LosAlamosNatLab__;!!G_uCfscf… >[signature_2725493895]<https://urldefense.us/v3/__https://www.linkedin.com/company/los-alamos-nati… >[signature_536422773]<https://urldefense.us/v3/__https://www.facebook.com/LosAlamosNationalLab/__… >

1 0

Test - please ignore
by Pritchard Jr., Howard 08 Oct '24

08 Oct '24

------- [signature_61897647] Howard Pritchard Research Scientist HPC-ENV Los Alamos National Laboratory howardp(a)lanl.gov<mailto:[email protected]> [signature_3332789229]<https://urldefense.us/v3/__https://www.instagram.com/losalamosnatlab/__;!!G… >[signature_1163875940]<https://urldefense.us/v3/__https://twitter.com/LosAlamosNatLab__;!!G_uCfscf… >[signature_3060914113]<https://urldefense.us/v3/__https://www.linkedin.com/company/los-alamos-nati… >[signature_3193715674]<https://urldefense.us/v3/__https://www.facebook.com/LosAlamosNationalLab/__… >

1 0

How to use ppr with mpiexec?
by Jeffrey Layton 08 Oct '24

08 Oct '24

Good morning, I'm trying to "port" an Open MPI script with MPICH and I'm having some trouble. I've got a few questions. 1. What is the form of the "-hosts" option? Is it something like '-hosts node1:4 node2:4 node3:4 ...'? 2. The Open MPI script has an option "-map-by ppr:4:node" that tells it to run 4 processes per node and then it runs as many nodes as needed to meet the number of processes. How can I do this in MPICH? Thanks for the help! Jeff

1 0