September 2013 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] Implementation of MPICH collectives
by Jiri Simsa 13 Sep '13

13 Sep '13

Antonio, Thank you for your response. Will setting of this variable result in using the "sock" channel instead of the "nemesis" channel? --Jiri On Fri, Sep 13, 2013 at 8:56 AM, Antonio J. Peña <apenya(a)mcs.anl.gov> wrote: > ** > > > > You can set the MPIR_PARAM_CH3_NO_LOCAL environment variable to 1. Refer > to the README.envvar file. > > > > Antonio > > > > > > On Friday, September 13, 2013 08:53:36 AM Jiri Simsa wrote: > > Pavan, > > > Thank you for your answer. That's precisely what I was looking for. Any > chance there is a way to force the intranode communication to use tcp? > > > --Jiri > > > Within the node, it uses shared memory. Outside the node, it depends on > the netmod you configured with. tcp is the default netmod. > -- Pavan > On Sep 12, 2013, at 2:24 PM, Jiri Simsa wrote: > > The high-order bit of my question is: What OS interface(s) does MPICH > use to transfer data from one MPI process to another? > > > > > > On Thu, Sep 12, 2013 at 1:36 PM, Jiri Simsa <jsimsa(a)cs.cmu.edu> wrote: > > Hello, > > > > I have been trying to understand how MPICH implements collective > operations. To do so, I have been reading the MPICH source code and > stepping through mpiexec executions. > > > > For the sake of this discussion, let's assume that all MPI processes are > executed on the same computer using: mpiexec -n <n> <mpi_program> > > > > This is my current abstract understanding of MPICH: > > > > - mpiexec spawns a hydra_pmi_proxy process, which in turn spawns <n> > instances of <mpi_program> > > - hydra_pmi_proxy process uses socket pairs to communicate with the > instances of <mpi_program> > > > > I am not quite sure though what happens under the hoods when a > collective operation, such as MPI_Allreduce, is executed. I have noticed > that instances of <mpi_program> create and listen on a socket in the course > of executing MPI_Allreduce but I am not sure who connects to these sockets. > Any chance someone could describe the data flow inside of MPICH when a > collective operation, such as MPI_Allreduce, is executed? Thanks! > > > > Best, > > > > --Jiri Simsa > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > > > >

1 0

Re: [mpich-discuss] Implementation of MPICH collectives
by Antonio J. Peña 13 Sep '13

13 Sep '13

You can set the MPIR_PARAM_CH3_NO_LOCAL environment variable to 1. Refer to the README.envvar file. Antonio On Friday, September 13, 2013 08:53:36 AM Jiri Simsa wrote: Pavan, Thank you for your answer. That's precisely what I was looking for. Any chance there is a way to force the intranode communication to use tcp? --Jiri Within the node, it uses shared memory. Outside the node, it depends on the netmod you configured with. tcp is the default netmod. -- Pavan On Sep 12, 2013, at 2:24 PM, Jiri Simsa wrote: > The high-order bit of my question is: What OS interface(s) does MPICH use to transfer data from one MPI process to another?>>> On Thu, Sep 12, 2013 at 1:36 PM, Jiri Simsa <jsimsa(a)cs.cmu.edu[1]> wrote:> Hello,>> I have been trying to understand how MPICH implements collective operations. To do so, I have been reading the MPICH source code and stepping through mpiexec executions.>> For the sake of this discussion, let's assume that all MPI processes are executed on the same computer using: mpiexec -n <n> <mpi_program>>> This is my current abstract understanding of MPICH:>> - mpiexec spawns a hydra_pmi_proxy process, which in turn spawns <n> instances of <mpi_program>> - hydra_pmi_proxy process uses socket pairs to communicate with the instances of <mpi_program>>> I am not quite sure though what happens under the hoods when a collective operation, such as MPI_Allreduce, is executed. I have noticed that instances of <mpi_program> create and listen on a socket in the course of executing MPI_Allreduce but I am not sure who connects to these sockets. Any chance someone could describe the data flow inside of MPICH when a collective operation, such as MPI_Allreduce, is executed? Thanks!>> Best,>> --Jiri Simsa>> _______________________________________________> discuss mailing list discuss(a)mpich.org [2]> To manage subscription options or unsubscribe:> https://lists.mpich.org/mailman/listinfo/discuss[3] --Pavan Balaji http://www.mcs.anl.gov/~balaji[4] -------- [1] mailto:[email protected] [2] mailto:[email protected] [3] https://lists.mpich.org/mailman/listinfo/discuss [4] http://www.mcs.anl.gov/~balaji

1 0

Implementation of MPICH collectives
by Jiri Simsa 13 Sep '13

13 Sep '13

Pavan, Thank you for your answer. That's precisely what I was looking for. Any chance there is a way to force the intranode communication to use tcp? --Jiri Within the node, it uses shared memory. Outside the node, it depends on > the netmod you configured with. tcp is the default netmod. > -- Pavan > On Sep 12, 2013, at 2:24 PM, Jiri Simsa wrote: > > The high-order bit of my question is: What OS interface(s) does MPICH > use to transfer data from one MPI process to another? > > > > > > On Thu, Sep 12, 2013 at 1:36 PM, Jiri Simsa <jsimsa(a)cs.cmu.edu> wrote: > > Hello, > > > > I have been trying to understand how MPICH implements collective > operations. To do so, I have been reading the MPICH source code and > stepping through mpiexec executions. > > > > For the sake of this discussion, let's assume that all MPI processes are > executed on the same computer using: mpiexec -n <n> <mpi_program> > > > > This is my current abstract understanding of MPICH: > > > > - mpiexec spawns a hydra_pmi_proxy process, which in turn spawns <n> > instances of <mpi_program> > > - hydra_pmi_proxy process uses socket pairs to communicate with the > instances of <mpi_program> > > > > I am not quite sure though what happens under the hoods when a > collective operation, such as MPI_Allreduce, is executed. I have noticed > that instances of <mpi_program> create and listen on a socket in the course > of executing MPI_Allreduce but I am not sure who connects to these sockets. > Any chance someone could describe the data flow inside of MPICH when a > collective operation, such as MPI_Allreduce, is executed? Thanks! > > > > Best, > > > > --Jiri Simsa > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] MPI 3 on Win7(64-bit)
by Jeff Hammond 13 Sep '13

13 Sep '13

Microsoft will have to support your usage of their code. Mac OSX is based upon BSD but that doesn't imply you should ask the UCBerkeley CS department for Mac tech support... Jeff Sent from my iPhone On Sep 13, 2013, at 2:41 AM, "Michael.Rachner(a)dlr.de" < Michael.Rachner(a)dlr.de> wrote: Thank you for your information. As a consequence (as recommended by MPICH-FAQ) I downloaded the Microsoft MPI (which is based on MPICH) on my Win7 (64-bit) PC. The installation was successful, but… there is a problem with MPI_REDUCE when using it with MPI_INPLACE, see my next email today. Michael Rachner *Von:* discuss-bounces(a)mpich.org [mailto:[email protected]<discuss-bounces(a)mpich.org>] *Im Auftrag von *Wesley Bland *Gesendet:* Mittwoch, 11. September 2013 16:07 *An:* discuss(a)mpich.org *Betreff:* Re: [mpich-discuss] MPI 3 on Win7(64-bit) Unfortunately, we haven't supported Windows since MPICH version 1.4.1p1. You can find out more at the FAQ page: http://wiki.mpich.org/mpich/index.php/Frequently_Asked_Questions#Q:_Why_can… Thanks, Wesley On Sep 11, 2013, at 8:13 AM, Jeff Hammond <jeff.science(a)gmail.com> wrote: There's a frequently quoted entry in the MPICH FAQ about Windows support that you may wish to consult. Or peruse list archives. Jeff Sent from my iPhone On Sep 11, 2013, at 5:55 AM, "Michael.Rachner(a)dlr.de" < Michael.Rachner(a)dlr.de> wrote: Dear developers of MPICH, Can anyone tell me, when a stable MPI 3 version (binary) of MPICH will be available for Win7(64-bit) operating system? The new feature of MPI 3 of accessing a shared memory data window from the processes on the same node could become very helpful for my Ftn-code. Thank You Michael Rachner _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

source code query
by 陈涛 13 Sep '13

13 Sep '13

To whom it may concern, I am trying to analyzing the source code of MPICH2. Currently I am studying the function MPI_Send(const void *buf, int count, MPI_Datatype datatype, int dest, int tag,MPI_Comm comm) in the file send.c. But it involves a commonly function MPIR_Err_create_code. It is really beyond my ability to understand this one and it drives me crazy. Could you help me with that or recommend me some materials analyzing that? Looking forward to your early reply. Kind regards! Volkan 2013-09-13 你好！我正分析MPICH2的源代码。目前在看send.c文件中的函数MPI_Send(const void *buf, int count, MPI_Datatype datatype, int dest, int tag,MPI_Comm comm) 。其中涉及一个常用的函数MPIR_Err_create_code。我真的是搞不清楚它的作用是什么。请问高手能不能帮我分析一下或推荐相关的资料！不胜感激！祝好！

1 0

MPI_REDUCE with MPI_IN_PLACE does not always work
by Michael.Rachner＠dlr.de 13 Sep '13

13 Sep '13

Dear MPI community, I found a problem when calling MPI_REDUCE with the keyword MPI_IN_PLACE in my Ftn95-code (using the INTEL-12 Ftn95-compiler). Depending on the MPI-implementation and the operating system the problem occurs or not. This is my experience so far: MPICH2 v.1.4.1p1 on Win7(64-bit) PC : It works Microsoft-MPI (v. of 12/11/2012) on WIN7(64-bit) PC : It fails (either assuming the contribution of the root to be zero (i.e. silently a wrong result!), or access violation or floating exception on the root in MPI_REDUCE ) OPENMPI v.1.6.2-2 on WIN7(64-bit) PC : It fails ( in the same manner as with MS-MPI) OPENMPI v.1.4.3 and 1.6.3 on 2 LINUX-Clusters : It works INTEL-MPI v. 4.0.3 and 4.1.0 on 2 LINUX-Clusters : It works My question is: Is the failing possibly caused by an erroneous ('dangerous') Ftn-coding causing some MPI implementations to fail and others not? Or is the problem actually caused by a bug in different MPI implementions? This is my Ftn95-coding: ! subroutine mpiw_reduce_sumfast_real8( rbuffarr, nelem ) ! !=============================================================================== ! ! sbr mpiw_reduce_sumfast_real8 is a wrapper for the MPI-routine MPI_REDUCE ! applied for summing element-wise a real(REAL8) 1d-array rbuffarr(nelem) ! from all processes of communicator commSPRAY ! and store the sums on master in the same array rbuffarr(nelem) , ! i.e. on the master we overwrite the original contribution of the master by: ! ! for i=1..nelem: rbuffarr(i) = SUM_over_iproc (rbuffarr(i) ) , with iproc=1,numprocs ! ! ! mpiw_reduce_sumfast_real8 calls : MPI_REDUCE ! ! last update: 03.09.2013 !=============================================================================== ! use MPIHEADER , only: MPI_SUM, MPI_IN_PLACE use NUMBER_MODEL, only: INT4,REAL8 use MPARAL , only: lmaster, commSPRAY, ierr_mpi, mpiusertype_REAL8 ! implicit none ! integer (INT4) , intent(IN) :: nelem real (REAL8), intent(INOUT), dimension(nelem) :: rbuffarr ! input on master&slaves, result only on master ! real (REAL8) :: rdummyarr(1) if(lmaster) then call MPI_REDUCE( MPI_IN_PLACE, rbuffarr, nelem, mpiusertype_REAL8, MPI_SUM & ,0_INT4, commSPRAY, ierr_mpi ) else ! slaves call MPI_REDUCE( rbuffarr, rdummyarr, nelem, mpiusertype_REAL8, MPI_SUM & ,0_INT4, commSPRAY, ierr_mpi ) endif ! return end subroutine mpiw_reduce_sumfast_real8 Note, that the problem is not dependent on the reduce-operator (here MPI_SUM) chosen. The problem does not occur, when I apply MPI_REDUCE without the MPI_IN_PLACE option. As a reference I cite here the MPI 2.2 standard (of Sept 4, 2009, p. 164): The "in place" option for intracommunicators is specified by passing the value MPI_IN_PLACE to the argument sendbuf at the root. In such a case, the input data is taken at the root from the receive buffer, where it will be replaced by the output data. Does my coding strictly conform to this? Greetings Michael Rachner

1 0

Re: [mpich-discuss] MPI 3 on Win7(64-bit)
by Michael.Rachner＠dlr.de 13 Sep '13

13 Sep '13

Thank you for your information. As a consequence (as recommended by MPICH-FAQ) I downloaded the Microsoft MPI (which is based on MPICH) on my Win7 (64-bit) PC. The installation was successful, but... there is a problem with MPI_REDUCE when using it with MPI_INPLACE, see my next email today. Michael Rachner Von: discuss-bounces(a)mpich.org [mailto:[email protected]] Im Auftrag von Wesley Bland Gesendet: Mittwoch, 11. September 2013 16:07 An: discuss(a)mpich.org Betreff: Re: [mpich-discuss] MPI 3 on Win7(64-bit) Unfortunately, we haven't supported Windows since MPICH version 1.4.1p1. You can find out more at the FAQ page: http://wiki.mpich.org/mpich/index.php/Frequently_Asked_Questions#Q:_Why_can… Thanks, Wesley On Sep 11, 2013, at 8:13 AM, Jeff Hammond <jeff.science(a)gmail.com<mailto:[email protected]>> wrote: There's a frequently quoted entry in the MPICH FAQ about Windows support that you may wish to consult. Or peruse list archives. Jeff Sent from my iPhone On Sep 11, 2013, at 5:55 AM, "Michael.Rachner(a)dlr.de<mailto:[email protected]>" <Michael.Rachner(a)dlr.de<mailto:[email protected]>> wrote: Dear developers of MPICH, Can anyone tell me, when a stable MPI 3 version (binary) of MPICH will be available for Win7(64-bit) operating system? The new feature of MPI 3 of accessing a shared memory data window from the processes on the same node could become very helpful for my Ftn-code. Thank You Michael Rachner _______________________________________________ discuss mailing list discuss(a)mpich.org<mailto:[email protected]> To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org<mailto:[email protected]> To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] discuss Digest, Vol 11, Issue 9
by Amogh Katti 13 Sep '13

13 Sep '13

Thank you so much Wesley. I could install it successfully by disabling c++ and fortran compilation. Amogh. ________________________________________ From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of discuss-request(a)mpich.org [discuss-request(a)mpich.org] Sent: 11 September 2013 18:56 To: discuss(a)mpich.org Subject: discuss Digest, Vol 11, Issue 9 Send discuss mailing list submissions to discuss(a)mpich.org To subscribe or unsubscribe via the World Wide Web, visit https://lists.mpich.org/mailman/listinfo/discuss or, via email, send a message with subject or body 'help' to discuss-request(a)mpich.org You can reach the person managing the list at discuss-owner(a)mpich.org When replying, please edit your Subject line so it is more specific than "Re: Contents of discuss digest..." Today's Topics: 1. Re: MPICH install failure (Wesley Bland) 2. MPI 3 on Win7(64-bit) (Michael.Rachner(a)dlr.de) 3. Re: MPI 3 on Win7(64-bit) (Jeff Hammond) 4. Re: MPI 3 on Win7(64-bit) (Wesley Bland) 5. MPI and bordered block system with LU decomposition (Mauro Crociara) 6. Re: MPI and bordered block system with LU decomposition (Jeff Hammond) 7. Re: MPI and bordered block system with LU decomposition (Mauro Crociara) ---------------------------------------------------------------------- Message: 1 Date: Mon, 9 Sep 2013 07:42:54 -0500 From: Wesley Bland <wbland(a)mcs.anl.gov> To: discuss(a)mpich.org Subject: Re: [mpich-discuss] MPICH install failure Message-ID: <705EDFA1-B2D3-4C0C-9618-8109C48981F5(a)mcs.anl.gov> Content-Type: text/plain; charset="iso-8859-1" It looks like you had some sort of issue in your configure step so the Makefiles were never generated. Can you send the config.log? It has much more useful information than c.txt. Although, on first glance, it appears that you don't have any C++ compiler installed. It's not necessary to do so, but if you don't want it, you need to specify that in your configure by adding `--disable-cxx` to your ./configure line. Thanks, Wesley On Sep 9, 2013, at 7:35 AM, Amogh Katti <A.P.Katti(a)pgr.reading.ac.uk> wrote: > Hello, > > I am a beginner to MPI programming. I tried to install MPICH and the build step failed saying, "No targets specified and no make file found". I have attached the c.txt and m.txt; Please help. > > Many thanks, > Amogh. > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Implementation of MPICH collectives
by Pavan Balaji 12 Sep '13

12 Sep '13

Within the node, it uses shared memory. Outside the node, it depends on the netmod you configured with. tcp is the default netmod. -- Pavan On Sep 12, 2013, at 2:24 PM, Jiri Simsa wrote: > The high-order bit of my question is: What OS interface(s) does MPICH use to transfer data from one MPI process to another? > > > On Thu, Sep 12, 2013 at 1:36 PM, Jiri Simsa <jsimsa(a)cs.cmu.edu> wrote: > Hello, > > I have been trying to understand how MPICH implements collective operations. To do so, I have been reading the MPICH source code and stepping through mpiexec executions. > > For the sake of this discussion, let's assume that all MPI processes are executed on the same computer using: mpiexec -n <n> <mpi_program> > > This is my current abstract understanding of MPICH: > > - mpiexec spawns a hydra_pmi_proxy process, which in turn spawns <n> instances of <mpi_program> > - hydra_pmi_proxy process uses socket pairs to communicate with the instances of <mpi_program> > > I am not quite sure though what happens under the hoods when a collective operation, such as MPI_Allreduce, is executed. I have noticed that instances of <mpi_program> create and listen on a socket in the course of executing MPI_Allreduce but I am not sure who connects to these sockets. Any chance someone could describe the data flow inside of MPICH when a collective operation, such as MPI_Allreduce, is executed? Thanks! > > Best, > > --Jiri Simsa > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] Implementation of MPICH collectives
by Jiri Simsa 12 Sep '13

12 Sep '13

The high-order bit of my question is: What OS interface(s) does MPICH use to transfer data from one MPI process to another? On Thu, Sep 12, 2013 at 1:36 PM, Jiri Simsa <jsimsa(a)cs.cmu.edu> wrote: > Hello, > > I have been trying to understand how MPICH implements collective > operations. To do so, I have been reading the MPICH source code and > stepping through mpiexec executions. > > For the sake of this discussion, let's assume that all MPI processes are > executed on the same computer using: mpiexec -n <n> <mpi_program> > > This is my current abstract understanding of MPICH: > > - mpiexec spawns a hydra_pmi_proxy process, which in turn spawns <n> > instances of <mpi_program> > - hydra_pmi_proxy process uses socket pairs to communicate with the > instances of <mpi_program> > > I am not quite sure though what happens under the hoods when a collective > operation, such as MPI_Allreduce, is executed. I have noticed that > instances of <mpi_program> create and listen on a socket in the course of > executing MPI_Allreduce but I am not sure who connects to these sockets. > Any chance someone could describe the data flow inside of MPICH when a > collective operation, such as MPI_Allreduce, is executed? Thanks! > > Best, > > --Jiri Simsa >

1 0