April 2013 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] Citations for Scatter/Allgather Broadcast and ReduceScatter/Gather Reduce
by Rajeev Thakur 07 Apr '13

07 Apr '13

I see. I didn't know that. Thanks, Rajeev On Apr 7, 2013, at 10:21 PM, Robert van de Geijn wrote: > Rajeev, > > I pointed out to Jack that Fox et al. actually discovered the scatter/allgather broadcast, and that I merely popularized it. So he went and tracked it down. > > Robert > > On Apr 7, 2013, at 10:19 PM, Rajeev Thakur wrote: > >> Jack, >> At the time that code was originally written around 10 years ago, Rabenseifner's algorithm for long-message reduce was not so well known, which is why I think I mentioned it in the code, whereas the long-message broadcast algorithm was relatively well known. A paper describing the collectives in MPICH, published in early 2005, cites two of Robert's papers for the long-message broadcast implemented as scatter-allgather: http://www.mcs.anl.gov/~thakur/papers/ijhpca-coll.pdf. >> >> Regards, >> Rajeev >> >> On Apr 7, 2013, at 9:48 PM, Jack Poulson wrote: >> >>> Dear all, >>> >>> I am putting together a lecture on collective communication and was perusing the MPICH 3.0.3 source code when I noticed that the only references for the long-message implementations of Reduce and Broadcast seem to be "Rabenseifer's algorithm" for Reduce, with a pointer to the following webpage and corresponding 1997 publication: >>> https://fs.hlrs.de/projects/par/mpi//myreduce.html >>> >>> This seems to be a bit strange to me, since this is essentially the Scatter composed with Allgather approach to Broadcast implemented in reverse, which was proposed by Fox, Furmanski, and Walker nearly a decade earlier in "Optimal matrix algorithms on homogeneous hypercube" (see Figs. 1 and 2, where the term "expand" is used instead of the modern "Allgather"): >>> http://dl.acm.org/citation.cfm?id=63125 >>> >>> It would seem to me that, if the source code is to include academic references, it would make sense to also cite this (much earlier) paper. >>> >>> Best Regards, >>> Jack Poulson >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >

1 0

Re: [mpich-discuss] Citations for Scatter/Allgather Broadcast and ReduceScatter/Gather Reduce
by Rajeev Thakur 07 Apr '13

07 Apr '13

Jack, At the time that code was originally written around 10 years ago, Rabenseifner's algorithm for long-message reduce was not so well known, which is why I think I mentioned it in the code, whereas the long-message broadcast algorithm was relatively well known. A paper describing the collectives in MPICH, published in early 2005, cites two of Robert's papers for the long-message broadcast implemented as scatter-allgather: http://www.mcs.anl.gov/~thakur/papers/ijhpca-coll.pdf. Regards, Rajeev On Apr 7, 2013, at 9:48 PM, Jack Poulson wrote: > Dear all, > > I am putting together a lecture on collective communication and was perusing the MPICH 3.0.3 source code when I noticed that the only references for the long-message implementations of Reduce and Broadcast seem to be "Rabenseifer's algorithm" for Reduce, with a pointer to the following webpage and corresponding 1997 publication: > https://fs.hlrs.de/projects/par/mpi//myreduce.html > > This seems to be a bit strange to me, since this is essentially the Scatter composed with Allgather approach to Broadcast implemented in reverse, which was proposed by Fox, Furmanski, and Walker nearly a decade earlier in "Optimal matrix algorithms on homogeneous hypercube" (see Figs. 1 and 2, where the term "expand" is used instead of the modern "Allgather"): > http://dl.acm.org/citation.cfm?id=63125 > > It would seem to me that, if the source code is to include academic references, it would make sense to also cite this (much earlier) paper. > > Best Regards, > Jack Poulson > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Citations for Scatter/Allgather Broadcast and ReduceScatter/Gather Reduce
by Jack Poulson 07 Apr '13

07 Apr '13

Dear all, I am putting together a lecture on collective communication and was perusing the MPICH 3.0.3 source code when I noticed that the only references for the long-message implementations of Reduce and Broadcast seem to be "Rabenseifer's algorithm" for Reduce, with a pointer to the following webpage and corresponding 1997 publication: https://fs.hlrs.de/projects/par/mpi//myreduce.html This seems to be a bit strange to me, since this is essentially the Scatter composed with Allgather approach to Broadcast implemented in reverse, which was proposed by Fox, Furmanski, and Walker nearly a decade earlier in "Optimal matrix algorithms on homogeneous hypercube" (see Figs. 1 and 2, where the term "expand" is used instead of the modern "Allgather"): http://dl.acm.org/citation.cfm?id=63125 It would seem to me that, if the source code is to include academic references, it would make sense to also cite this (much earlier) paper. Best Regards, Jack Poulson

1 0

Re: [mpich-discuss] run more then 1 process
by Pavan Balaji 06 Apr '13

06 Apr '13

Can you provide more information on the exact error you are seeing? -- Pavan On 04/05/2013 04:26 AM US Central Time, Иващук Роман wrote: > Hi all. The problem was that when running fds_mpi there was only one process on PC inspite of the parameter -np 8. The same project of mpi and FDS files on Windows gave 8 processes (amd 8-core PC). The program does not see the file /src/pm/hydra/utils/launch/launch.c and some other files ( running with the option -binding). The file exists, the folder is "shared". > May be some environmental variables are needed? The file "Using the Hydra Process Manager" does not point to the variables for such directory. Please help. Thanks. > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

run more then 1 process
by Иващук Роман 05 Apr '13

05 Apr '13

Hi all. The problem was that when running fds_mpi there was only one process on PC inspite of the parameter -np 8. The same project of mpi and FDS files on Windows gave 8 processes (amd 8-core PC). The program does not see the file /src/pm/hydra/utils/launch/launch.c and some other files ( running with the option -binding). The file exists, the folder is "shared". May be some environmental variables are needed? The file "Using the Hydra Process Manager" does not point to the variables for such directory. Please help. Thanks.

1 0

Re: [mpich-discuss] libmpich.so: undefined reference to ....
by Dave Goodell 04 Apr '13

04 Apr '13

It looks like Delft3D is building its libraries without using "mpicc" or any of the other compiler wrappers. These wrappers add numerous internal MPI libraries to the link line, including "-lmpl", only some of which are captured in the existing Delft3D build system. Please either use the compiler wrapper scripts to build your application or at least ensure that you are linking with all of the flags shown by "mpicc -show". I also strongly recommend moving to MPICH 3.0.3 instead of MPICH2 1.4.1. MPICH2 >=1.5 (and therefore MPICH >=3.0) have a newer build system that handles shared libraries in a substantially more robust manner. -Dave On Apr 4, 2013, at 9:44 AM CDT, "Boyington, Thomas Matthew" <tmboyington(a)tva.gov> wrote: > Dear mpich community, > > I’m getting numerous libmpich.so: undefined reference to XXX errors when building Delft3D (a CFD costal/river model). I’ve read the log files, Googled, and rtm’ed to no avail. I hope that the mpich mailing list can help me out. > > I am running Red Hat Enterprise 6.3 64-bit, and I am building mpich 1.4.1. I have been able to build mpich successfully (I think…the examples described in the README file work) using the commands shown below. I’m using the Intel 2013 Fortran compiler, and I’m using the –enabled-shared, CFLAGS=”-fPIC” and CXXFLAGS=”-fPIC” switches because of a previous Delft3D build error saying to build mpich with the –fPIC switch. > > > configure --prefix=/home/labuser/mpich2-install --enable-shared CFLAGS="-fPIC" CXXFLAGS="-fPIC" F77=ifort FC=ifort |& tee c.txt > > make 2>&1 | tee m.txt > > make install 2>&1 | tee mi.txt > > When building Delft3D (again with the Intel compiler) I get the following errors in the make log. I’ve included my log files as attachments. Any suggestions on how to address this will be greatly appreciated? > > libNefis.a ../../../../../engines_gpl/waq/packages/waq_utils_f/src/interact/.libs/libinteract.a ../../../../../engines_gpl/waq/packages/waq_utils_c/src/.libs/libwaq_utils_c.a ../../../../../engines_gpl/waq/packages/delwaq2_version_number/src/.libs/libdelwaq2_version_number.a -lfmpich -lmpich -lpthread -ldl -lrt -lexpat -lstdc++ -fopenmp -threads > > /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_trid' > > /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_trvalid' > > /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_env2int' > > /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_trrealloc' > > /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_trspace' > > /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_trDebugLevel' > > /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_TrSetMaxMem' > > /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_trlevel' > > /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_trmalloc' > > /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_env2bool' > > /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_env2range' > > /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_trcalloc' > > /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_trfree' > > /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_env2str' > > /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_trstrdup' > > /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_trdump' > > /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_trinit' > > make[5]: *** [delwaq2] Error 1 > > make[5]: Leaving directory `/home/labuser/Delft3D/2367/src/engines_gpl/waq/packages/delwaq2/src' > > make[4]: *** [install-recursive] Error 1 > > make[4]: Leaving directory `/home/labuser/Delft3D/2367/src/engines_gpl/waq/packages/delwaq2' > > make[3]: *** [install-recursive] Error 1 > > make[3]: Leaving directory `/home/labuser/Delft3D/2367/src/engines_gpl/waq/packages' > > make[2]: *** [install-recursive] Error 1 > > make[2]: Leaving directory `/home/labuser/Delft3D/2367/src/engines_gpl/waq' > > make[1]: *** [install-recursive] Error 1 > > make[1]: Leaving directory `/home/labuser/Delft3D/2367/src/engines_gpl' > > make: *** [install-recursive] Error 1 > > > Thanks, > > Matt > > > -- > > T. Matthew Boyington > > Civil Engineer, Water Resources > > Hydrothermal Team > > River Operations & Renewables > > Tennessee Valley Authority > > 400 West Summit Hill Drive, Knoxville, TN 37902 > > Office: (865) 632-8365 > > Cell: (865) 599-8265 > > > <<mi.zip>> <<config_openpa.zip>> <<config.zip>> <<c.zip>> <<m.zip>> > > <mi.zip><config_openpa.zip><config.zip><c.zip><m.zip>_______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

libmpich.so: undefined reference to ....
by Boyington, Thomas Matthew 04 Apr '13

04 Apr '13

Dear mpich community, I'm getting numerous libmpich.so: undefined reference to XXX errors when building Delft3D (a CFD costal/river model). I've read the log files, Googled, and rtm'ed to no avail. I hope that the mpich mailing list can help me out. I am running Red Hat Enterprise 6.3 64-bit, and I am building mpich 1.4.1. I have been able to build mpich successfully (I think...the examples described in the README file work) using the commands shown below. I'm using the Intel 2013 Fortran compiler, and I'm using the -enabled-shared, CFLAGS="-fPIC" and CXXFLAGS="-fPIC" switches because of a previous Delft3D build error saying to build mpich with the -fPIC switch. configure --prefix=/home/labuser/mpich2-install --enable-shared CFLAGS="-fPIC" CXXFLAGS="-fPIC" F77=ifort FC=ifort |& tee c.txt make 2>&1 | tee m.txt make install 2>&1 | tee mi.txt When building Delft3D (again with the Intel compiler) I get the following errors in the make log. I've included my log files as attachments. Any suggestions on how to address this will be greatly appreciated? libNefis.a ../../../../../engines_gpl/waq/packages/waq_utils_f/src/interact/.libs/l ibinteract.a ../../../../../engines_gpl/waq/packages/waq_utils_c/src/.libs/libwaq_uti ls_c.a ../../../../../engines_gpl/waq/packages/delwaq2_version_number/src/.libs /libdelwaq2_version_number.a -lfmpich -lmpich -lpthread -ldl -lrt -lexpat -lstdc++ -fopenmp -threads /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_trid' /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_trvalid' /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_env2int' /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_trrealloc' /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_trspace' /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_trDebugLevel' /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_TrSetMaxMem' /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_trlevel' /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_trmalloc' /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_env2bool' /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_env2range' /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_trcalloc' /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_trfree' /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_env2str' /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_trstrdup' /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_trdump' /home/labuser/mpich2-install/lib/libmpich.so: undefined reference to `MPL_trinit' make[5]: *** [delwaq2] Error 1 make[5]: Leaving directory `/home/labuser/Delft3D/2367/src/engines_gpl/waq/packages/delwaq2/src' make[4]: *** [install-recursive] Error 1 make[4]: Leaving directory `/home/labuser/Delft3D/2367/src/engines_gpl/waq/packages/delwaq2' make[3]: *** [install-recursive] Error 1 make[3]: Leaving directory `/home/labuser/Delft3D/2367/src/engines_gpl/waq/packages' make[2]: *** [install-recursive] Error 1 make[2]: Leaving directory `/home/labuser/Delft3D/2367/src/engines_gpl/waq' make[1]: *** [install-recursive] Error 1 make[1]: Leaving directory `/home/labuser/Delft3D/2367/src/engines_gpl' make: *** [install-recursive] Error 1 Thanks, Matt -- T. Matthew Boyington Civil Engineer, Water Resources Hydrothermal Team River Operations & Renewables Tennessee Valley Authority 400 West Summit Hill Drive, Knoxville, TN 37902 Office: (865) 632-8365 Cell: (865) 599-8265 <<mi.zip>> <<config_openpa.zip>> <<config.zip>> <<c.zip>> <<m.zip>>

1 0

Re: [mpich-discuss] MPI IO, reading MPI Writes without MPI read
by Rob Latham 01 Apr '13

01 Apr '13

On Thu, Mar 28, 2013 at 12:53:42PM -0700, Ryan Crocker wrote: > I just went though the standard, i'm not doing anything to untoward. that i could see. > > I also would really not have to re-run these simulations. There has to be a way to get access to those MPI writes with an MPI read, and like i said, other programs have no trouble reading these in and using them (paravew, and ensight). I would just like to have my own stand alone in fortran. The writes come out of a fortran code compiled with mpich so there should be a way to read them back in with a fortran, mpich compiled code. Shouldn't there? If you have confirmed that the data really does live in that file, with a 244 byte header and three 'ncells_hexa*4'-byte regions of data, then heck yeah we can read it all back into fortran. The quoting's making it hard to read so here's the code again: ncells = parallel_sum(ncells_hexa_,ncells) allocate(buffer3(ncells,3)) openfile=trim(workdir)//'/'//'V/V.000002' call MPI_FILE_OPEN(comm,openfile,MPI_MODE_RDONLY,mpi_info,iunit,ierr) ! Read header bsize = 80 call MPI_FILE_READ(iunit,cbuffer,bsize,MPI_CHARACTER,status,ierr) print*,trim(cbuffer) bsize = 80 call MPI_FILE_READ(iunit,cbuffer,bsize,MPI_CHARACTER,status,ierr) print*,trim(cbuffer) bsize = 1 call MPI_FILE_READ(iunit,ibuffer,bsize,MPI_INTEGER,status,ierr) print*,ibuffer bsize = 80 call MPI_FILE_READ(iunit,cbuffer,bsize,MPI_CHARACTER,status,ierr) print*,trim(cbuffer),ncells ! Read the data disp = 3*80+4+0*ncells*4 call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) call MPI_FILE_READ_ALL(iunit,buffer3(:,1),ncells,MPI_REAL_SP,status,ierr) disp = 3*80+4+1*ncells*4 call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) call MPI_FILE_READ_ALL(iunit,buffer3(:,2),ncells,MPI_REAL_SP,status,ierr) disp = 3*80+4+2*ncells*4 call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) call MPI_FILE_READ_ALL(iunit,buffer3(:,3),ncells,MPI_REAL_SP,status,ierr) ! Close the file call MPI_FILE_CLOSE(iunit,ierr) The symptom is htat 'buffer3' ends up with all the same values, right? I think what Rajeev was suggesting is that you pass not 'buffer3(:,1)' to MPI_File_read_all -- the entire array -- but rather pass in the first element of that (row? column? I have very little fortran experience, sorry). Your use of MPI_FILE_SET_VIEW to move the file pointer to the right place is uncommon but should be correct. One way to rationalize this is that many aspects of the fortran interface are designed to accommodate "speaking to C". ==rob -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA

1 0

Re: [mpich-discuss] MPICH 3.0.3 Ubuntu, Debian and Slackware packages (x86 and amd64)
by Dave Goodell 01 Apr '13

01 Apr '13

Downloads page updated (http://www.mpich.org/downloads/) Thanks! -Dave On Apr 1, 2013, at 2:18 PM CDT, Marcelo Soares Souza <marcelo(a)cebacad.net> wrote: > MPICH 3.0.3 packages > > * Ubuntu 12.04.1 LTS (Precise Pangolin) > > - amd64 > http://www.cebacad.net/files/mpich/ubuntu/mpich-3.0.3/mpich_3.0.3-1ubuntu_a… > > - x86 > http://www.cebacad.net/files/mpich/ubuntu/mpich-3.0.3/mpich_3.0.3-1ubuntu_i… > > * Debian 7.0 (Wheezy) > > - amd64 > http://www.cebacad.net/files/mpich/debian/mpich-3.0.3/mpich_3.0.3-1debian_a… > > - x86 > http://www.cebacad.net/files/mpich/debian/mpich-3.0.3/mpich_3.0.3-1debian_i… > > * Slackware 14.0 > > - amd64 > http://www.cebacad.net/files/mpich/slackware/mpich-3.0.3/mpich-3.0.3-x86_64… > > - x86 > http://www.cebacad.net/files/mpich/slackware/mpich-3.0.3/mpich-3.0.3-i486-g… > > Enjoy! ;) > > -- > Abraços > Marcelo Soares Souza > http://marcelo.cebacad.net > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

MPICH 3.0.3 Ubuntu, Debian and Slackware packages (x86 and amd64)
by Marcelo Soares Souza 01 Apr '13

01 Apr '13

MPICH 3.0.3 packages * Ubuntu 12.04.1 LTS (Precise Pangolin) - amd64 http://www.cebacad.net/files/mpich/ubuntu/mpich-3.0.3/mpich_3.0.3-1ubuntu_a… - x86 http://www.cebacad.net/files/mpich/ubuntu/mpich-3.0.3/mpich_3.0.3-1ubuntu_i… * Debian 7.0 (Wheezy) - amd64 http://www.cebacad.net/files/mpich/debian/mpich-3.0.3/mpich_3.0.3-1debian_a… - x86 http://www.cebacad.net/files/mpich/debian/mpich-3.0.3/mpich_3.0.3-1debian_i… * Slackware 14.0 - amd64 http://www.cebacad.net/files/mpich/slackware/mpich-3.0.3/mpich-3.0.3-x86_64… - x86 http://www.cebacad.net/files/mpich/slackware/mpich-3.0.3/mpich-3.0.3-i486-g… Enjoy! ;) -- Abraços Marcelo Soares Souza http://marcelo.cebacad.net

1 0