September 2017 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] Using MPI_Waitany for MPI_File Requests
by Halim Amer 29 Sep '17

29 Sep '17

yes, the bug is related to progress and generalized requests. Still investigating for a proper patch. Will keep the list posted. Halim www.mcs.anl.gov/~aamer On 9/28/17 10:36 PM, Jeff Hammond wrote: > MPI_Waitsome also works, and I assume you can use that almost anywhere > that MPI_Waitany is used. > > I didn't determine for sure if it is the cause, but MPI_Waitall, > MPI_Wait, and MPI_Waitsome all invoke MPI_Grequest progress, whereas > MPI_Waitany does not, and I recall that ROMIO uses generalized requests. > > $ grep MPIR_Grequest ../src/mpi/pt2pt/wait* > > ../src/mpi/pt2pt/waitall.c:mpi_errno = *MPIR_Grequest*_waitall(count, > request_ptrs); > > ../src/mpi/pt2pt/wait.c:mpi_errno = *MPIR_Grequest*_progress_poke(1, > &request_ptr, status); > > ../src/mpi/pt2pt/waitsome.c:mpi_errno = > *MPIR_Grequest*_progress_poke(incount, > > > Jeff > > On Thu, Sep 28, 2017 at 9:32 AM, Jeff Hammond <jeff.science(a)gmail.com > <mailto:[email protected]>> wrote: > > > There appears to be a bug in Waitany here. It works with 2x Wait > and Waitall. > > Jeff > > jrhammon@klondike:~/Work/INTEL/BUGS$ mpicc mpiio.c && ./a.out > > blocking > > done > > jrhammon@klondike:~/Work/INTEL/BUGS$ mpicc -DNONBLOCKING -DWAITALL > mpiio.c && ./a.out > > nonblocking > > waitall > > done > > jrhammon@klondike:~/Work/INTEL/BUGS$ mpicc -DNONBLOCKING -DWAIT > mpiio.c && ./a.out > > nonblocking > > wait > > done > > wait > > done > > jrhammon@klondike:~/Work/INTEL/BUGS$ mpicc -DNONBLOCKING -DWAITANY > mpiio.c && ./a.out > > nonblocking > > waitany > > ^C > > jrhammon@klondike:~/Work/INTEL/BUGS$ cat mpiio.c > > #include <stdio.h> > > #include <mpi.h> > > > int main(int argc, char** argv) > > { > > MPI_Init(&argc, &argv); > > > int rank; > > MPI_Comm_rank(MPI_COMM_WORLD, &rank); > > > MPI_Request recv_req; > > int int_buf; > > MPI_Irecv(&int_buf, 1, MPI_INT, MPI_ANY_SOURCE, 0, MPI_COMM_WORLD, > > &recv_req); > > > MPI_File fh; > > MPI_File_open(MPI_COMM_WORLD, "output", MPI_MODE_CREATE | > MPI_MODE_RDWR, MPI_INFO_NULL, &fh); > > MPI_File_set_view(fh, 2*sizeof(int)*rank, MPI_INT, MPI_INT, > "native", MPI_INFO_NULL); > > > #ifdef NONBLOCKING > > printf("nonblocking\n"); > > MPI_Request requests[2] = {MPI_REQUEST_NULL, MPI_REQUEST_NULL}; > > MPI_File_iwrite(fh, &rank, 1, MPI_INT, &requests[0]); > > MPI_File_iwrite(fh, &rank, 1, MPI_INT, &requests[1]); > > #if defined(WAITANY) > > int index; > > printf("waitany\n"); > > MPI_Waitany(2, requests, &index, MPI_STATUS_IGNORE); > > MPI_Waitany(2, requests, &index, MPI_STATUS_IGNORE); > > printf("done\n"); > > #elif defined(WAIT) > > printf("wait\n"); > > MPI_Wait(&requests[0], MPI_STATUS_IGNORE); > > printf("done\n"); > > printf("wait\n"); > > MPI_Wait(&requests[1], MPI_STATUS_IGNORE); > > printf("done\n"); > > #elif defined(WAITALL) > > printf("waitall\n"); > > MPI_Waitall(2, requests, MPI_STATUSES_IGNORE); > > printf("done\n"); > > #else > > #error Define WAITANY, WAIT, or WAITALL > > #endif > > #else > > printf("blocking\n"); > > MPI_File_write(fh, &rank, 1, MPI_INT, MPI_STATUS_IGNORE); > > MPI_File_write(fh, &rank, 1, MPI_INT, MPI_STATUS_IGNORE); > > printf("done\n"); > > #endif > > MPI_File_close(&fh); > > > MPI_Finalize(); > > > return 0; > > } > > > > On Thu, Sep 28, 2017 at 6:53 AM, Nils-Arne Dreier > <n.dreier(a)uni-muenster.de <mailto:[email protected]>> wrote: > > Dear MPICH community, > > I'm currently playing around with the File-IO interface of MPI. > For any > reason, the following code does not running through: > > #include <iostream> > #include <mpi.h> > > int main(int argc, char** argv) > { > MPI_Init(&argc, &argv); > > int rank; > MPI_Comm_rank(MPI_COMM_WORLD, &rank); > > MPI_Request recv_req; > int int_buf; > MPI_Irecv(&int_buf, 1, MPI_INT, MPI_ANY_SOURCE, 0, > MPI_COMM_WORLD, > &recv_req); > > MPI_File fh; > MPI_File_open(MPI_COMM_WORLD, "output", MPI_MODE_CREATE | > MPI_MODE_RDWR, > MPI_INFO_NULL, &fh); > MPI_File_set_view(fh, 2*sizeof(int)*rank, MPI_INT, MPI_INT, > "native", MPI_INFO_NULL); > > MPI_Request file_req; > MPI_File_iwrite(fh, &rank, 1, MPI_INT, &file_req); > > MPI_Request file_req2; > MPI_File_iwrite(fh, &rank, 1, MPI_INT, &file_req2); > > MPI_Request requests[2] = {file_req, file_req2}; > int index; > std::cout << rank << ":\twaiting..." << std::endl; > MPI_Waitany(2, requests, &index, MPI_STATUS_IGNORE); > //MPI_Wait(&file_req, MPI_STATUS_IGNORE); > std::cout << rank << ":\tdone" << std::endl; > MPI_File_close(&fh); > > MPI_Finalize(); > > return 0; > } > > Neither if i exchange file_req2 with recv_req, MPI_Waitany > doesn't return. > > Is it possible to use MPI_Waitany with File-IO calls? Did i read > anything over in the MPI-Standard? > > Thank you for your help. > > Thanks, > Nils > > PS: I'm running Ubuntu 16.04 LTS, gcc-4.9 (using -std=c++14) and > MPICH > 3.2. This is the output of mpirun --version: > HYDRA build details: > Version: 3.2 > Release Date: Wed Nov 11 22:06:48 CST 2015 > CC: gcc -Wl,-Bsymbolic-functions > -Wl,-z,relro > CXX: g++ -Wl,-Bsymbolic-functions > -Wl,-z,relro > F77: gfortran -Wl,-Bsymbolic-functions > -Wl,-z,relro > F90: gfortran -Wl,-Bsymbolic-functions > -Wl,-z,relro > Configure options: > '--disable-option-checking' > '--prefix=/usr' '--build=x86_64-linux-gnu' > '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' > '--infodir=${prefix}/share/info' '--sysconfdir=/etc' > '--localstatedir=/var' '--disable-silent-rules' > '--libdir=${prefix}/lib/x86_64-linux-gnu' > '--libexecdir=${prefix}/lib/x86_64-linux-gnu' > '--disable-maintainer-mode' '--disable-dependency-tracking' > '--enable-shared' '--enable-fortran=all' '--disable-rpath' > '--disable-wrapper-rpath' '--sysconfdir=/etc/mpich' > '--libdir=/usr/lib/x86_64-linux-gnu' > '--includedir=/usr/include/mpich' > '--docdir=/usr/share/doc/mpich' '--with-hwloc-prefix=system' > '--enable-checkpointing' '--with-hydra-ckpointlib=blcr' 'CPPFLAGS= > -Wdate-time -D_FORTIFY_SOURCE=2 > -I/build/mpich-jQtQ8p/mpich-3.2/src/mpl/include > -I/build/mpich-jQtQ8p/mpich-3.2/src/mpl/include > -I/build/mpich-jQtQ8p/mpich-3.2/src/openpa/src > -I/build/mpich-jQtQ8p/mpich-3.2/src/openpa/src -D_REENTRANT > -I/build/mpich-jQtQ8p/mpich-3.2/src/mpi/romio/include' 'CFLAGS= > -g -O2 > -fstack-protector-strong -Wformat -Werror=format-security -O2' > 'CXXFLAGS= -g -O2 -fstack-protector-strong -Wformat > -Werror=format-security -O2' 'FFLAGS= -g -O2 > -fstack-protector-strong > -O2' 'FCFLAGS= -g -O2 -fstack-protector-strong -O2' > 'build_alias=x86_64-linux-gnu' 'MPICHLIB_CFLAGS=-g -O2 > -fstack-protector-strong -Wformat -Werror=format-security' > 'MPICHLIB_CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' > 'MPICHLIB_CXXFLAGS=-g -O2 -fstack-protector-strong -Wformat > -Werror=format-security' 'MPICHLIB_FFLAGS=-g -O2 > -fstack-protector-strong' 'MPICHLIB_FCFLAGS=-g -O2 > -fstack-protector-strong' 'LDFLAGS=-Wl,-Bsymbolic-functions > -Wl,-z,relro' 'FC=gfortran' 'F77=gfortran' 'MPILIBNAME=mpich' > '--cache-file=/dev/null' '--srcdir=.' 'CC=gcc' 'LIBS=-lpthread ' > Process Manager: pmi > Launchers available: ssh rsh fork slurm > ll lsf > sge manual persist > Topology libraries available: hwloc > Resource management kernels available: user slurm ll lsf > sge pbs > cobalt > Checkpointing libraries available: blcr > Demux engines available: poll select > > > -- > Nils-Arne Dreier, M.Sc. > Institute for Computational und Applied Mathematics, > University of Münster, Orleans-Ring 10, D-48149 Münster > Tel: +49 251 83-35147 > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > <https://lists.mpich.org/mailman/listinfo/discuss> > > > > > -- > Jeff Hammond > jeff.science(a)gmail.com <mailto:[email protected]> > http://jeffhammond.github.io/ > > > > > -- > Jeff Hammond > jeff.science(a)gmail.com <mailto:[email protected]> > http://jeffhammond.github.io/ > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Fwd: Re: Using MPI_Waitany for MPI_File Requests
by Nils-Arne Dreier 29 Sep '17

29 Sep '17

Hi Jeff, Thanks for your answer! I changed to MPI_Waitsome. During that, I run into another issue: MPI_Waitsome raises a Segmentation fault, depending on the order in which the requests are passed in, if different types of requests are used (file and p2p). Here is the code: #include <iostream> #include <mpi.h> int main(int argc, char** argv) { MPI_Init(&argc, &argv); int rank; MPI_Comm_rank(MPI_COMM_WORLD, &rank); MPI_Request recv_req; int int_buf = 0; MPI_Irecv(&int_buf, 1, MPI_INT, MPI_ANY_SOURCE, 0, MPI_COMM_WORLD, &recv_req); MPI_File fh; MPI_File_open(MPI_COMM_WORLD, "output", MPI_MODE_CREATE | MPI_MODE_RDWR, MPI_INFO_NULL, &fh); MPI_Request file_req; double data = 42; MPI_File_iwrite_at(fh, rank*sizeof(double), &data, 1, MPI_DOUBLE, &file_req); //MPI_Request requests[2] = {recv_req, file_req}; // A MPI_Request requests[2] = {file_req, recv_req}; // B std::cout << rank << ":\twaiting..." << std::endl; int completed = 0; int indices[2]; MPI_Status status[2]; MPI_Waitsome(2, requests, &completed, indices, status); // int index = 0; // MPI_Waitany(2, requests, &index, MPI_STATUS_IGNORE); std::cout << rank << ":\tdone" << std::endl; MPI_File_close(&fh); MPI_Finalize(); return 0; } Exchanging line B with line A does not raises the fault. Thanks, Nils On 29.09.2017 05:36, Jeff Hammond wrote: > MPI_Waitsome also works, and I assume you can use that almost anywhere > that MPI_Waitany is used. > > I didn't determine for sure if it is the cause, but MPI_Waitall, > MPI_Wait, and MPI_Waitsome all invoke MPI_Grequest progress, whereas > MPI_Waitany does not, and I recall that ROMIO uses generalized requests. > > $ grep MPIR_Grequest ../src/mpi/pt2pt/wait* > > ../src/mpi/pt2pt/waitall.c: mpi_errno = > *MPIR_Grequest*_waitall(count, request_ptrs); > > ../src/mpi/pt2pt/wait.c: mpi_errno = > *MPIR_Grequest*_progress_poke(1, &request_ptr, status); > > ../src/mpi/pt2pt/waitsome.c:mpi_errno = > *MPIR_Grequest*_progress_poke(incount, > > > Jeff > > On Thu, Sep 28, 2017 at 9:32 AM, Jeff Hammond <jeff.science(a)gmail.com > <mailto:[email protected]>> wrote: > > > There appears to be a bug in Waitany here. It works with 2x Wait > and Waitall. > > Jeff > > jrhammon@klondike:~/Work/INTEL/BUGS$ mpicc mpiio.c && ./a.out > > blocking > > done > > jrhammon@klondike:~/Work/INTEL/BUGS$ mpicc -DNONBLOCKING -DWAITALL > mpiio.c && ./a.out > > nonblocking > > waitall > > done > > jrhammon@klondike:~/Work/INTEL/BUGS$ mpicc -DNONBLOCKING -DWAIT > mpiio.c && ./a.out > > nonblocking > > wait > > done > > wait > > done > > jrhammon@klondike:~/Work/INTEL/BUGS$ mpicc -DNONBLOCKING -DWAITANY > mpiio.c && ./a.out > > nonblocking > > waitany > > ^C > > jrhammon@klondike:~/Work/INTEL/BUGS$ cat mpiio.c > > #include <stdio.h> > > #include <mpi.h> > > > int main(int argc, char** argv) > > { > > MPI_Init(&argc, &argv); > > > int rank; > > MPI_Comm_rank(MPI_COMM_WORLD, &rank); > > > MPI_Request recv_req; > > int int_buf; > > MPI_Irecv(&int_buf, 1, MPI_INT, MPI_ANY_SOURCE, 0, MPI_COMM_WORLD, > > &recv_req); > > > MPI_File fh; > > MPI_File_open(MPI_COMM_WORLD, "output", MPI_MODE_CREATE | > MPI_MODE_RDWR, MPI_INFO_NULL, &fh); > > MPI_File_set_view(fh, 2*sizeof(int)*rank, MPI_INT, MPI_INT, > "native", MPI_INFO_NULL); > > > #ifdef NONBLOCKING > > printf("nonblocking\n"); > > MPI_Request requests[2] = {MPI_REQUEST_NULL, MPI_REQUEST_NULL}; > > MPI_File_iwrite(fh, &rank, 1, MPI_INT, &requests[0]); > > MPI_File_iwrite(fh, &rank, 1, MPI_INT, &requests[1]); > > #if defined(WAITANY) > > int index; > > printf("waitany\n"); > > MPI_Waitany(2, requests, &index, MPI_STATUS_IGNORE); > > MPI_Waitany(2, requests, &index, MPI_STATUS_IGNORE); > > printf("done\n"); > > #elif defined(WAIT) > > printf("wait\n"); > > MPI_Wait(&requests[0], MPI_STATUS_IGNORE); > > printf("done\n"); > > printf("wait\n"); > > MPI_Wait(&requests[1], MPI_STATUS_IGNORE); > > printf("done\n"); > > #elif defined(WAITALL) > > printf("waitall\n"); > > MPI_Waitall(2, requests, MPI_STATUSES_IGNORE); > > printf("done\n"); > > #else > > #error Define WAITANY, WAIT, or WAITALL > > #endif > > #else > > printf("blocking\n"); > > MPI_File_write(fh, &rank, 1, MPI_INT, MPI_STATUS_IGNORE); > > MPI_File_write(fh, &rank, 1, MPI_INT, MPI_STATUS_IGNORE); > > printf("done\n"); > > #endif > > MPI_File_close(&fh); > > > MPI_Finalize(); > > > return 0; > > } > > > > On Thu, Sep 28, 2017 at 6:53 AM, Nils-Arne Dreier > <n.dreier(a)uni-muenster.de <mailto:[email protected]>> wrote: > > Dear MPICH community, > > I'm currently playing around with the File-IO interface of > MPI. For any > reason, the following code does not running through: > > #include <iostream> > #include <mpi.h> > > int main(int argc, char** argv) > { > MPI_Init(&argc, &argv); > > int rank; > MPI_Comm_rank(MPI_COMM_WORLD, &rank); > > MPI_Request recv_req; > int int_buf; > MPI_Irecv(&int_buf, 1, MPI_INT, MPI_ANY_SOURCE, 0, > MPI_COMM_WORLD, > &recv_req); > > MPI_File fh; > MPI_File_open(MPI_COMM_WORLD, "output", MPI_MODE_CREATE | > MPI_MODE_RDWR, > MPI_INFO_NULL, &fh); > MPI_File_set_view(fh, 2*sizeof(int)*rank, MPI_INT, MPI_INT, > "native", MPI_INFO_NULL); > > MPI_Request file_req; > MPI_File_iwrite(fh, &rank, 1, MPI_INT, &file_req); > > MPI_Request file_req2; > MPI_File_iwrite(fh, &rank, 1, MPI_INT, &file_req2); > > MPI_Request requests[2] = {file_req, file_req2}; > int index; > std::cout << rank << ":\twaiting..." << std::endl; > MPI_Waitany(2, requests, &index, MPI_STATUS_IGNORE); > //MPI_Wait(&file_req, MPI_STATUS_IGNORE); > std::cout << rank << ":\tdone" << std::endl; > MPI_File_close(&fh); > > MPI_Finalize(); > > return 0; > } > > Neither if i exchange file_req2 with recv_req, MPI_Waitany > doesn't return. > > Is it possible to use MPI_Waitany with File-IO calls? Did i read > anything over in the MPI-Standard? > > Thank you for your help. > > Thanks, > Nils > > PS: I'm running Ubuntu 16.04 LTS, gcc-4.9 (using -std=c++14) > and MPICH > 3.2. This is the output of mpirun --version: > HYDRA build details: > Version: 3.2 > Release Date: Wed Nov 11 > 22:06:48 CST 2015 > CC: gcc > -Wl,-Bsymbolic-functions > -Wl,-z,relro > CXX: g++ > -Wl,-Bsymbolic-functions > -Wl,-z,relro > F77: gfortran > -Wl,-Bsymbolic-functions > -Wl,-z,relro > F90: gfortran > -Wl,-Bsymbolic-functions > -Wl,-z,relro > Configure options: > '--disable-option-checking' > '--prefix=/usr' '--build=x86_64-linux-gnu' > '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' > '--infodir=${prefix}/share/info' '--sysconfdir=/etc' > '--localstatedir=/var' '--disable-silent-rules' > '--libdir=${prefix}/lib/x86_64-linux-gnu' > '--libexecdir=${prefix}/lib/x86_64-linux-gnu' > '--disable-maintainer-mode' '--disable-dependency-tracking' > '--enable-shared' '--enable-fortran=all' '--disable-rpath' > '--disable-wrapper-rpath' '--sysconfdir=/etc/mpich' > '--libdir=/usr/lib/x86_64-linux-gnu' > '--includedir=/usr/include/mpich' > '--docdir=/usr/share/doc/mpich' '--with-hwloc-prefix=system' > '--enable-checkpointing' '--with-hydra-ckpointlib=blcr' 'CPPFLAGS= > -Wdate-time -D_FORTIFY_SOURCE=2 > -I/build/mpich-jQtQ8p/mpich-3.2/src/mpl/include > -I/build/mpich-jQtQ8p/mpich-3.2/src/mpl/include > -I/build/mpich-jQtQ8p/mpich-3.2/src/openpa/src > -I/build/mpich-jQtQ8p/mpich-3.2/src/openpa/src -D_REENTRANT > -I/build/mpich-jQtQ8p/mpich-3.2/src/mpi/romio/include' > 'CFLAGS= -g -O2 > -fstack-protector-strong -Wformat -Werror=format-security -O2' > 'CXXFLAGS= -g -O2 -fstack-protector-strong -Wformat > -Werror=format-security -O2' 'FFLAGS= -g -O2 > -fstack-protector-strong > -O2' 'FCFLAGS= -g -O2 -fstack-protector-strong -O2' > 'build_alias=x86_64-linux-gnu' 'MPICHLIB_CFLAGS=-g -O2 > -fstack-protector-strong -Wformat -Werror=format-security' > 'MPICHLIB_CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' > 'MPICHLIB_CXXFLAGS=-g -O2 -fstack-protector-strong -Wformat > -Werror=format-security' 'MPICHLIB_FFLAGS=-g -O2 > -fstack-protector-strong' 'MPICHLIB_FCFLAGS=-g -O2 > -fstack-protector-strong' 'LDFLAGS=-Wl,-Bsymbolic-functions > -Wl,-z,relro' 'FC=gfortran' 'F77=gfortran' 'MPILIBNAME=mpich' > '--cache-file=/dev/null' '--srcdir=.' 'CC=gcc' 'LIBS=-lpthread ' > Process Manager: pmi > Launchers available: ssh rsh fork > slurm ll lsf > sge manual persist > Topology libraries available: hwloc > Resource management kernels available: user slurm ll lsf > sge pbs > cobalt > Checkpointing libraries available: blcr > Demux engines available: poll select > > > -- > Nils-Arne Dreier, M.Sc. > Institute for Computational und Applied Mathematics, > University of Münster, Orleans-Ring 10, D-48149 Münster > Tel: +49 251 83-35147 > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > <mailto:[email protected]> > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > <https://lists.mpich.org/mailman/listinfo/discuss> > > > > > -- > Jeff Hammond > jeff.science(a)gmail.com <mailto:[email protected]> > http://jeffhammond.github.io/ > > > > > -- > Jeff Hammond > jeff.science(a)gmail.com <mailto:[email protected]> > http://jeffhammond.github.io/ > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Nils-Arne Dreier, M.Sc. Institute for Computational und Applied Mathematics, University of Münster, Orleans-Ring 10, D-48149 Münster Tel: +49 251 83-35147 _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Using MPI_Waitany for MPI_File Requests
by Jeff Hammond 28 Sep '17

28 Sep '17

MPI_Waitsome also works, and I assume you can use that almost anywhere that MPI_Waitany is used. I didn't determine for sure if it is the cause, but MPI_Waitall, MPI_Wait, and MPI_Waitsome all invoke MPI_Grequest progress, whereas MPI_Waitany does not, and I recall that ROMIO uses generalized requests. $ grep MPIR_Grequest ../src/mpi/pt2pt/wait* ../src/mpi/pt2pt/waitall.c: mpi_errno = *MPIR_Grequest*_waitall(count, request_ptrs); ../src/mpi/pt2pt/wait.c: mpi_errno = *MPIR_Grequest*_progress_poke(1, &request_ptr, status); ../src/mpi/pt2pt/waitsome.c: mpi_errno = *MPIR_Grequest* _progress_poke(incount, Jeff On Thu, Sep 28, 2017 at 9:32 AM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > > There appears to be a bug in Waitany here. It works with 2x Wait and > Waitall. > > Jeff > > jrhammon@klondike:~/Work/INTEL/BUGS$ mpicc mpiio.c && ./a.out > > blocking > > done > > jrhammon@klondike:~/Work/INTEL/BUGS$ mpicc -DNONBLOCKING -DWAITALL > mpiio.c && ./a.out > > nonblocking > > waitall > > done > > jrhammon@klondike:~/Work/INTEL/BUGS$ mpicc -DNONBLOCKING -DWAIT mpiio.c > && ./a.out > > nonblocking > > wait > > done > > wait > > done > > jrhammon@klondike:~/Work/INTEL/BUGS$ mpicc -DNONBLOCKING -DWAITANY > mpiio.c && ./a.out > > nonblocking > > waitany > > ^C > > jrhammon@klondike:~/Work/INTEL/BUGS$ cat mpiio.c > > #include <stdio.h> > > #include <mpi.h> > > > int main(int argc, char** argv) > > { > > MPI_Init(&argc, &argv); > > > int rank; > > MPI_Comm_rank(MPI_COMM_WORLD, &rank); > > > MPI_Request recv_req; > > int int_buf; > > MPI_Irecv(&int_buf, 1, MPI_INT, MPI_ANY_SOURCE, 0, MPI_COMM_WORLD, > > &recv_req); > > > MPI_File fh; > > MPI_File_open(MPI_COMM_WORLD, "output", MPI_MODE_CREATE | > MPI_MODE_RDWR, MPI_INFO_NULL, &fh); > > MPI_File_set_view(fh, 2*sizeof(int)*rank, MPI_INT, MPI_INT, "native", > MPI_INFO_NULL); > > > #ifdef NONBLOCKING > > printf("nonblocking\n"); > > MPI_Request requests[2] = {MPI_REQUEST_NULL, MPI_REQUEST_NULL}; > > MPI_File_iwrite(fh, &rank, 1, MPI_INT, &requests[0]); > > MPI_File_iwrite(fh, &rank, 1, MPI_INT, &requests[1]); > > #if defined(WAITANY) > > int index; > > printf("waitany\n"); > > MPI_Waitany(2, requests, &index, MPI_STATUS_IGNORE); > > MPI_Waitany(2, requests, &index, MPI_STATUS_IGNORE); > > printf("done\n"); > > #elif defined(WAIT) > > printf("wait\n"); > > MPI_Wait(&requests[0], MPI_STATUS_IGNORE); > > printf("done\n"); > > printf("wait\n"); > > MPI_Wait(&requests[1], MPI_STATUS_IGNORE); > > printf("done\n"); > > #elif defined(WAITALL) > > printf("waitall\n"); > > MPI_Waitall(2, requests, MPI_STATUSES_IGNORE); > > printf("done\n"); > > #else > > #error Define WAITANY, WAIT, or WAITALL > > #endif > > #else > > printf("blocking\n"); > > MPI_File_write(fh, &rank, 1, MPI_INT, MPI_STATUS_IGNORE); > > MPI_File_write(fh, &rank, 1, MPI_INT, MPI_STATUS_IGNORE); > > printf("done\n"); > > #endif > > MPI_File_close(&fh); > > > MPI_Finalize(); > > > return 0; > > } > > > > On Thu, Sep 28, 2017 at 6:53 AM, Nils-Arne Dreier < > n.dreier(a)uni-muenster.de> wrote: > >> Dear MPICH community, >> >> I'm currently playing around with the File-IO interface of MPI. For any >> reason, the following code does not running through: >> >> #include <iostream> >> #include <mpi.h> >> >> int main(int argc, char** argv) >> { >> MPI_Init(&argc, &argv); >> >> int rank; >> MPI_Comm_rank(MPI_COMM_WORLD, &rank); >> >> MPI_Request recv_req; >> int int_buf; >> MPI_Irecv(&int_buf, 1, MPI_INT, MPI_ANY_SOURCE, 0, MPI_COMM_WORLD, >> &recv_req); >> >> MPI_File fh; >> MPI_File_open(MPI_COMM_WORLD, "output", MPI_MODE_CREATE | >> MPI_MODE_RDWR, >> MPI_INFO_NULL, &fh); >> MPI_File_set_view(fh, 2*sizeof(int)*rank, MPI_INT, MPI_INT, >> "native", MPI_INFO_NULL); >> >> MPI_Request file_req; >> MPI_File_iwrite(fh, &rank, 1, MPI_INT, &file_req); >> >> MPI_Request file_req2; >> MPI_File_iwrite(fh, &rank, 1, MPI_INT, &file_req2); >> >> MPI_Request requests[2] = {file_req, file_req2}; >> int index; >> std::cout << rank << ":\twaiting..." << std::endl; >> MPI_Waitany(2, requests, &index, MPI_STATUS_IGNORE); >> //MPI_Wait(&file_req, MPI_STATUS_IGNORE); >> std::cout << rank << ":\tdone" << std::endl; >> MPI_File_close(&fh); >> >> MPI_Finalize(); >> >> return 0; >> } >> >> Neither if i exchange file_req2 with recv_req, MPI_Waitany doesn't return. >> >> Is it possible to use MPI_Waitany with File-IO calls? Did i read >> anything over in the MPI-Standard? >> >> Thank you for your help. >> >> Thanks, >> Nils >> >> PS: I'm running Ubuntu 16.04 LTS, gcc-4.9 (using -std=c++14) and MPICH >> 3.2. This is the output of mpirun --version: >> HYDRA build details: >> Version: 3.2 >> Release Date: Wed Nov 11 22:06:48 CST 2015 >> CC: gcc -Wl,-Bsymbolic-functions >> -Wl,-z,relro >> CXX: g++ -Wl,-Bsymbolic-functions >> -Wl,-z,relro >> F77: gfortran -Wl,-Bsymbolic-functions >> -Wl,-z,relro >> F90: gfortran -Wl,-Bsymbolic-functions >> -Wl,-z,relro >> Configure options: '--disable-option-checking' >> '--prefix=/usr' '--build=x86_64-linux-gnu' >> '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' >> '--infodir=${prefix}/share/info' '--sysconfdir=/etc' >> '--localstatedir=/var' '--disable-silent-rules' >> '--libdir=${prefix}/lib/x86_64-linux-gnu' >> '--libexecdir=${prefix}/lib/x86_64-linux-gnu' >> '--disable-maintainer-mode' '--disable-dependency-tracking' >> '--enable-shared' '--enable-fortran=all' '--disable-rpath' >> '--disable-wrapper-rpath' '--sysconfdir=/etc/mpich' >> '--libdir=/usr/lib/x86_64-linux-gnu' '--includedir=/usr/include/mpich' >> '--docdir=/usr/share/doc/mpich' '--with-hwloc-prefix=system' >> '--enable-checkpointing' '--with-hydra-ckpointlib=blcr' 'CPPFLAGS= >> -Wdate-time -D_FORTIFY_SOURCE=2 >> -I/build/mpich-jQtQ8p/mpich-3.2/src/mpl/include >> -I/build/mpich-jQtQ8p/mpich-3.2/src/mpl/include >> -I/build/mpich-jQtQ8p/mpich-3.2/src/openpa/src >> -I/build/mpich-jQtQ8p/mpich-3.2/src/openpa/src -D_REENTRANT >> -I/build/mpich-jQtQ8p/mpich-3.2/src/mpi/romio/include' 'CFLAGS= -g -O2 >> -fstack-protector-strong -Wformat -Werror=format-security -O2' >> 'CXXFLAGS= -g -O2 -fstack-protector-strong -Wformat >> -Werror=format-security -O2' 'FFLAGS= -g -O2 -fstack-protector-strong >> -O2' 'FCFLAGS= -g -O2 -fstack-protector-strong -O2' >> 'build_alias=x86_64-linux-gnu' 'MPICHLIB_CFLAGS=-g -O2 >> -fstack-protector-strong -Wformat -Werror=format-security' >> 'MPICHLIB_CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' >> 'MPICHLIB_CXXFLAGS=-g -O2 -fstack-protector-strong -Wformat >> -Werror=format-security' 'MPICHLIB_FFLAGS=-g -O2 >> -fstack-protector-strong' 'MPICHLIB_FCFLAGS=-g -O2 >> -fstack-protector-strong' 'LDFLAGS=-Wl,-Bsymbolic-functions >> -Wl,-z,relro' 'FC=gfortran' 'F77=gfortran' 'MPILIBNAME=mpich' >> '--cache-file=/dev/null' '--srcdir=.' 'CC=gcc' 'LIBS=-lpthread ' >> Process Manager: pmi >> Launchers available: ssh rsh fork slurm ll lsf >> sge manual persist >> Topology libraries available: hwloc >> Resource management kernels available: user slurm ll lsf sge pbs >> cobalt >> Checkpointing libraries available: blcr >> Demux engines available: poll select >> >> >> -- >> Nils-Arne Dreier, M.Sc. >> Institute for Computational und Applied Mathematics, >> University of Münster, Orleans-Ring 10, D-48149 Münster >> Tel: +49 251 83-35147 >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> > > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/ _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Using MPI_Waitany for MPI_File Requests
by Jeff Hammond 28 Sep '17

28 Sep '17

There appears to be a bug in Waitany here. It works with 2x Wait and Waitall. Jeff jrhammon@klondike:~/Work/INTEL/BUGS$ mpicc mpiio.c && ./a.out blocking done jrhammon@klondike:~/Work/INTEL/BUGS$ mpicc -DNONBLOCKING -DWAITALL mpiio.c && ./a.out nonblocking waitall done jrhammon@klondike:~/Work/INTEL/BUGS$ mpicc -DNONBLOCKING -DWAIT mpiio.c && ./a.out nonblocking wait done wait done jrhammon@klondike:~/Work/INTEL/BUGS$ mpicc -DNONBLOCKING -DWAITANY mpiio.c && ./a.out nonblocking waitany ^C jrhammon@klondike:~/Work/INTEL/BUGS$ cat mpiio.c #include <stdio.h> #include <mpi.h> int main(int argc, char** argv) { MPI_Init(&argc, &argv); int rank; MPI_Comm_rank(MPI_COMM_WORLD, &rank); MPI_Request recv_req; int int_buf; MPI_Irecv(&int_buf, 1, MPI_INT, MPI_ANY_SOURCE, 0, MPI_COMM_WORLD, &recv_req); MPI_File fh; MPI_File_open(MPI_COMM_WORLD, "output", MPI_MODE_CREATE | MPI_MODE_RDWR, MPI_INFO_NULL, &fh); MPI_File_set_view(fh, 2*sizeof(int)*rank, MPI_INT, MPI_INT, "native", MPI_INFO_NULL); #ifdef NONBLOCKING printf("nonblocking\n"); MPI_Request requests[2] = {MPI_REQUEST_NULL, MPI_REQUEST_NULL}; MPI_File_iwrite(fh, &rank, 1, MPI_INT, &requests[0]); MPI_File_iwrite(fh, &rank, 1, MPI_INT, &requests[1]); #if defined(WAITANY) int index; printf("waitany\n"); MPI_Waitany(2, requests, &index, MPI_STATUS_IGNORE); MPI_Waitany(2, requests, &index, MPI_STATUS_IGNORE); printf("done\n"); #elif defined(WAIT) printf("wait\n"); MPI_Wait(&requests[0], MPI_STATUS_IGNORE); printf("done\n"); printf("wait\n"); MPI_Wait(&requests[1], MPI_STATUS_IGNORE); printf("done\n"); #elif defined(WAITALL) printf("waitall\n"); MPI_Waitall(2, requests, MPI_STATUSES_IGNORE); printf("done\n"); #else #error Define WAITANY, WAIT, or WAITALL #endif #else printf("blocking\n"); MPI_File_write(fh, &rank, 1, MPI_INT, MPI_STATUS_IGNORE); MPI_File_write(fh, &rank, 1, MPI_INT, MPI_STATUS_IGNORE); printf("done\n"); #endif MPI_File_close(&fh); MPI_Finalize(); return 0; } On Thu, Sep 28, 2017 at 6:53 AM, Nils-Arne Dreier <n.dreier(a)uni-muenster.de> wrote: > Dear MPICH community, > > I'm currently playing around with the File-IO interface of MPI. For any > reason, the following code does not running through: > > #include <iostream> > #include <mpi.h> > > int main(int argc, char** argv) > { > MPI_Init(&argc, &argv); > > int rank; > MPI_Comm_rank(MPI_COMM_WORLD, &rank); > > MPI_Request recv_req; > int int_buf; > MPI_Irecv(&int_buf, 1, MPI_INT, MPI_ANY_SOURCE, 0, MPI_COMM_WORLD, > &recv_req); > > MPI_File fh; > MPI_File_open(MPI_COMM_WORLD, "output", MPI_MODE_CREATE | > MPI_MODE_RDWR, > MPI_INFO_NULL, &fh); > MPI_File_set_view(fh, 2*sizeof(int)*rank, MPI_INT, MPI_INT, > "native", MPI_INFO_NULL); > > MPI_Request file_req; > MPI_File_iwrite(fh, &rank, 1, MPI_INT, &file_req); > > MPI_Request file_req2; > MPI_File_iwrite(fh, &rank, 1, MPI_INT, &file_req2); > > MPI_Request requests[2] = {file_req, file_req2}; > int index; > std::cout << rank << ":\twaiting..." << std::endl; > MPI_Waitany(2, requests, &index, MPI_STATUS_IGNORE); > //MPI_Wait(&file_req, MPI_STATUS_IGNORE); > std::cout << rank << ":\tdone" << std::endl; > MPI_File_close(&fh); > > MPI_Finalize(); > > return 0; > } > > Neither if i exchange file_req2 with recv_req, MPI_Waitany doesn't return. > > Is it possible to use MPI_Waitany with File-IO calls? Did i read > anything over in the MPI-Standard? > > Thank you for your help. > > Thanks, > Nils > > PS: I'm running Ubuntu 16.04 LTS, gcc-4.9 (using -std=c++14) and MPICH > 3.2. This is the output of mpirun --version: > HYDRA build details: > Version: 3.2 > Release Date: Wed Nov 11 22:06:48 CST 2015 > CC: gcc -Wl,-Bsymbolic-functions > -Wl,-z,relro > CXX: g++ -Wl,-Bsymbolic-functions > -Wl,-z,relro > F77: gfortran -Wl,-Bsymbolic-functions > -Wl,-z,relro > F90: gfortran -Wl,-Bsymbolic-functions > -Wl,-z,relro > Configure options: '--disable-option-checking' > '--prefix=/usr' '--build=x86_64-linux-gnu' > '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' > '--infodir=${prefix}/share/info' '--sysconfdir=/etc' > '--localstatedir=/var' '--disable-silent-rules' > '--libdir=${prefix}/lib/x86_64-linux-gnu' > '--libexecdir=${prefix}/lib/x86_64-linux-gnu' > '--disable-maintainer-mode' '--disable-dependency-tracking' > '--enable-shared' '--enable-fortran=all' '--disable-rpath' > '--disable-wrapper-rpath' '--sysconfdir=/etc/mpich' > '--libdir=/usr/lib/x86_64-linux-gnu' '--includedir=/usr/include/mpich' > '--docdir=/usr/share/doc/mpich' '--with-hwloc-prefix=system' > '--enable-checkpointing' '--with-hydra-ckpointlib=blcr' 'CPPFLAGS= > -Wdate-time -D_FORTIFY_SOURCE=2 > -I/build/mpich-jQtQ8p/mpich-3.2/src/mpl/include > -I/build/mpich-jQtQ8p/mpich-3.2/src/mpl/include > -I/build/mpich-jQtQ8p/mpich-3.2/src/openpa/src > -I/build/mpich-jQtQ8p/mpich-3.2/src/openpa/src -D_REENTRANT > -I/build/mpich-jQtQ8p/mpich-3.2/src/mpi/romio/include' 'CFLAGS= -g -O2 > -fstack-protector-strong -Wformat -Werror=format-security -O2' > 'CXXFLAGS= -g -O2 -fstack-protector-strong -Wformat > -Werror=format-security -O2' 'FFLAGS= -g -O2 -fstack-protector-strong > -O2' 'FCFLAGS= -g -O2 -fstack-protector-strong -O2' > 'build_alias=x86_64-linux-gnu' 'MPICHLIB_CFLAGS=-g -O2 > -fstack-protector-strong -Wformat -Werror=format-security' > 'MPICHLIB_CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' > 'MPICHLIB_CXXFLAGS=-g -O2 -fstack-protector-strong -Wformat > -Werror=format-security' 'MPICHLIB_FFLAGS=-g -O2 > -fstack-protector-strong' 'MPICHLIB_FCFLAGS=-g -O2 > -fstack-protector-strong' 'LDFLAGS=-Wl,-Bsymbolic-functions > -Wl,-z,relro' 'FC=gfortran' 'F77=gfortran' 'MPILIBNAME=mpich' > '--cache-file=/dev/null' '--srcdir=.' 'CC=gcc' 'LIBS=-lpthread ' > Process Manager: pmi > Launchers available: ssh rsh fork slurm ll lsf > sge manual persist > Topology libraries available: hwloc > Resource management kernels available: user slurm ll lsf sge pbs > cobalt > Checkpointing libraries available: blcr > Demux engines available: poll select > > > -- > Nils-Arne Dreier, M.Sc. > Institute for Computational und Applied Mathematics, > University of Münster, Orleans-Ring 10, D-48149 Münster > Tel: +49 251 83-35147 > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/ _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Using MPI_Waitany for MPI_File Requests
by Nils-Arne Dreier 28 Sep '17

28 Sep '17

Dear MPICH community, I'm currently playing around with the File-IO interface of MPI. For any reason, the following code does not running through: #include <iostream> #include <mpi.h> int main(int argc, char** argv) { MPI_Init(&argc, &argv); int rank; MPI_Comm_rank(MPI_COMM_WORLD, &rank); MPI_Request recv_req; int int_buf; MPI_Irecv(&int_buf, 1, MPI_INT, MPI_ANY_SOURCE, 0, MPI_COMM_WORLD, &recv_req); MPI_File fh; MPI_File_open(MPI_COMM_WORLD, "output", MPI_MODE_CREATE | MPI_MODE_RDWR, MPI_INFO_NULL, &fh); MPI_File_set_view(fh, 2*sizeof(int)*rank, MPI_INT, MPI_INT, "native", MPI_INFO_NULL); MPI_Request file_req; MPI_File_iwrite(fh, &rank, 1, MPI_INT, &file_req); MPI_Request file_req2; MPI_File_iwrite(fh, &rank, 1, MPI_INT, &file_req2); MPI_Request requests[2] = {file_req, file_req2}; int index; std::cout << rank << ":\twaiting..." << std::endl; MPI_Waitany(2, requests, &index, MPI_STATUS_IGNORE); //MPI_Wait(&file_req, MPI_STATUS_IGNORE); std::cout << rank << ":\tdone" << std::endl; MPI_File_close(&fh); MPI_Finalize(); return 0; } Neither if i exchange file_req2 with recv_req, MPI_Waitany doesn't return. Is it possible to use MPI_Waitany with File-IO calls? Did i read anything over in the MPI-Standard? Thank you for your help. Thanks, Nils PS: I'm running Ubuntu 16.04 LTS, gcc-4.9 (using -std=c++14) and MPICH 3.2. This is the output of mpirun --version: HYDRA build details: Version: 3.2 Release Date: Wed Nov 11 22:06:48 CST 2015 CC: gcc -Wl,-Bsymbolic-functions -Wl,-z,relro CXX: g++ -Wl,-Bsymbolic-functions -Wl,-z,relro F77: gfortran -Wl,-Bsymbolic-functions -Wl,-z,relro F90: gfortran -Wl,-Bsymbolic-functions -Wl,-z,relro Configure options: '--disable-option-checking' '--prefix=/usr' '--build=x86_64-linux-gnu' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=${prefix}/lib/x86_64-linux-gnu' '--libexecdir=${prefix}/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--enable-shared' '--enable-fortran=all' '--disable-rpath' '--disable-wrapper-rpath' '--sysconfdir=/etc/mpich' '--libdir=/usr/lib/x86_64-linux-gnu' '--includedir=/usr/include/mpich' '--docdir=/usr/share/doc/mpich' '--with-hwloc-prefix=system' '--enable-checkpointing' '--with-hydra-ckpointlib=blcr' 'CPPFLAGS= -Wdate-time -D_FORTIFY_SOURCE=2 -I/build/mpich-jQtQ8p/mpich-3.2/src/mpl/include -I/build/mpich-jQtQ8p/mpich-3.2/src/mpl/include -I/build/mpich-jQtQ8p/mpich-3.2/src/openpa/src -I/build/mpich-jQtQ8p/mpich-3.2/src/openpa/src -D_REENTRANT -I/build/mpich-jQtQ8p/mpich-3.2/src/mpi/romio/include' 'CFLAGS= -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -O2' 'CXXFLAGS= -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -O2' 'FFLAGS= -g -O2 -fstack-protector-strong -O2' 'FCFLAGS= -g -O2 -fstack-protector-strong -O2' 'build_alias=x86_64-linux-gnu' 'MPICHLIB_CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security' 'MPICHLIB_CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' 'MPICHLIB_CXXFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security' 'MPICHLIB_FFLAGS=-g -O2 -fstack-protector-strong' 'MPICHLIB_FCFLAGS=-g -O2 -fstack-protector-strong' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro' 'FC=gfortran' 'F77=gfortran' 'MPILIBNAME=mpich' '--cache-file=/dev/null' '--srcdir=.' 'CC=gcc' 'LIBS=-lpthread ' Process Manager: pmi Launchers available: ssh rsh fork slurm ll lsf sge manual persist Topology libraries available: hwloc Resource management kernels available: user slurm ll lsf sge pbs cobalt Checkpointing libraries available: blcr Demux engines available: poll select -- Nils-Arne Dreier, M.Sc. Institute for Computational und Applied Mathematics, University of Münster, Orleans-Ring 10, D-48149 Münster Tel: +49 251 83-35147 _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

logging / rlog broken in mpich 3.2 (Missing Makefile.in in src/util/logging/rlog)...
by PERROTON, Laurent 25 Sep '17

25 Sep '17

Hi, I cant configure and compile mpich-3.2 (nor 3.3a2) with logging working, I get the error: config.status: error: cannot find input file: `Makefile.in' configure: error: src/util/logging/rlog configure failed It is apparently a known issue (see just below from the list archive). Is there any progress on this ? Thank you ! Laurent [mpich-discuss] logging Kanaev A.A. kanaev at ibrae.ac.ru Thu Jun 9 12:34:30 CDT 2016 Previous message: [mpich-discuss] logging Next message: [mpich-discuss] Race condition in MPI3 (Fortran, shared memory) Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] Thanks for the response! I'll be glad to have an option of logging mpich 3.1 ________________________________________ From: Oden, Lena [loden at anl.gov] Sent: Thursday, June 09, 2016 3:38 PM To: discuss at mpich.org Subject: Re: [mpich-discuss] logging Hello Anton, Unfortunately, you are right, the rlog support is broken for a while (I think i broke with our change to autogen) I was working on a fix for this, but had to work on other things. Give me a few days and I may will be able to fix it. I would be glad, If you are willing to test my patch. Thanks, Lena > On Jun 9, 2016, at 5:15 AM, Kanaev A.A. <kanaev at ibrae.ac.ru> wrote: > > Hello mpich-dev, > > Since version 2-1.5 there's no Makefile.in in src/util/logging/rlog folder > So when i run configure for mpich with --with-logging=rlog option i receive an error about Makefile.in absence in rlog folder > I tried to run autogen.sh prior to configure but it did nothing as well > > Thanks > Anton > _______________________________________________ > discuss mailing list discuss at mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Missing Nemesis ib
by Halim Amer 21 Sep '17

21 Sep '17

You need to launch two instances of the binary, one as a server and the other as a client. Please read the MXM user guide: http://www.mellanox.com/related-docs/prod_acceleration_software/Mellanox_MX…. As I said before, I suggest you forward your InfiniBand issues to your admin or your Mellanox support. There is only little we can do on the MPICH team, since after all, it's not an MPICH problem here. Get back to us once you have a *working* MXM stack and MPICH is still failing. Halim www.mcs.anl.gov/~aamer On 9/21/17 3:04 AM, Jason Collins wrote: > I tried to run the test in this other way to get more information: > > # mpirun -n 10 ./mxm_perftest > > -------------------------------------------------------------------------- > > Failed to register memory region (MR): > > > Hostname: compute1 > > Address: 1d14000 > > Length: 20480 > > Error: No space left on device > > -------------------------------------------------------------------------- > > -------------------------------------------------------------------------- > > Open MPI has detected that there are UD-capable Verbs devices on your > > system, but none of them were able to be setup properly. This may > > indicate a problem on this system. > > > You job will continue, but Open MPI will ignore the "ud" oob component > > in this run. > > > Hostname: compute1 > > -------------------------------------------------------------------------- > > [1505980772.899531] [compute1:59182:0] sys.c:744 MXM WARN > Conflicting CPU frequencies detected, using: 3599.84 > > Waiting for connection... > > [1505980772.900947] [compute1:59183:0] sys.c:744 MXM WARN > Conflicting CPU frequencies detected, using: 3599.84 > > [1505980772.902329] [compute1:59184:0] sys.c:744 MXM WARN > Conflicting CPU frequencies detected, using: 3599.84 > > [1505980772.903490] [compute1:59185:0] sys.c:744 MXM WARN > Conflicting CPU frequencies detected, using: 3599.84 > > [1505980772.904984] [compute1:59186:0] sys.c:744 MXM WARN > Conflicting CPU frequencies detected, using: 3599.84 > > [1505980772.906288] [compute1:59187:0] sys.c:744 MXM WARN > Conflicting CPU frequencies detected, using: 3599.84 > > [1505980772.907957] [compute1:59188:0] sys.c:744 MXM WARN > Conflicting CPU frequencies detected, using: 3599.84 > > [1505980772.909023] [compute1:59189:0] sys.c:744 MXM WARN > Conflicting CPU frequencies detected, using: 3599.84 > > bind() failed: Address already in use > > bind() failed: Address already in use > > bind() failed: Address already in use > > bind() failed: Address already in use > > bind() failed: Address already in use > > bind() failed: Address already in use > > bind() failed: Address already in use > > ------------------------------------------------------- > > Primary job terminated normally, but 1 process returned > > a non-zero exit code.. Per user-direction, the job has been aborted. > > ------------------------------------------------------- > > bind() failed: Address already in use > > [1505980772.910503] [compute1:59190:0] sys.c:744 MXM WARN > Conflicting CPU frequencies detected, using: 3599.84 > > bind() failed: Address already in use > > [1505980772.911893] [compute1:59191:0] sys.c:744 MXM WARN > Conflicting CPU frequencies detected, using: 3599.84 > > -------------------------------------------------------------------------- > > mpirun detected that one or more processes exited with non-zero status, > thus causing > > the job to be terminated. The first process to do so was: > > > Process name: [[55900,1],1] > > Exit code: 255 > > > El jue., 21 sept. 2017 a las 7:58, Jason Collins > (<jasoncollinsw(a)gmail.com <mailto:[email protected]>>) escribió: > > I ran the test and the result was the following: > > # ./mxm_perftest > > [1505976675.346380] [compute1:55801:0] sys.c:744 MXM WARN > Conflicting CPU frequencies detected, using: 3600.52 > > Waiting for connection... > > > It does nothing else, it remains waiting to establish connection. > <https://audio1.spanishdict.com/audio?lang=en&text=i-ran-the-test-and-the-re…> > > El mié., 20 sept. 2017 a las 17:12, Halim Amer (<aamer(a)anl.gov > <mailto:[email protected]>>) escribió: > > I seems you have mismatch in the OFED stack. Try installing the > Mellanox > OFED stack if you are using the bundled OFED stack right now. > > Make sure MXM works before trying MPICH. Use the > mxm/bin/mxm_perftest > from your MXM installation to test that things work properly. If it > doesn't work, then contact your admin or Mellanox cause it is not an > MPICH problem. > > Halim > www.mcs.anl.gov/~aamer <http://www.mcs.anl.gov/~aamer> > > On 9/19/17 7:14 AM, Jason Collins wrote: > > Thank you very much. > > > > I have compiled with "CH3:nemesis:mxm". The compilation was > successful. > > > > Now I have a new problem. I tested the test "./icp" and get the > > following error. > > > > # mpiexec -f hosts -n 4 ./cpi > > [1505822776.546898] [compute1:16212:0] sys.c:744 MXM WARN > Conflicting > > CPU frequencies detected, using: 3459.84 > > [1505822776.546898] [compute1:16213:0] sys.c:744 MXM WARN > Conflicting > > CPU frequencies detected, using: 3459.84 > > [1505822776.546951] [compute1:16216:0] sys.c:744 MXM WARN > Conflicting > > CPU frequencies detected, using: 3459.84 > > [1505822776.547039] [compute1:16214:0] sys.c:744 MXM WARN > Conflicting > > CPU frequencies detected, using: 3459.84 > > [1505822776.561357] [compute1:16214:0] ib_dev.c:533 MXM WARN > failed call > > to ibv_exp_use_priv_env(): Function not implemented > > [1505822776.561371] [compute1:16214:0] ib_dev.c:544 MXM ERROR > > ibv_query_device() returned 38: Function not implemented > > [1505822776.561386] [compute1:16218:0] ib_dev.c:533 MXM WARN > failed call > > to ibv_exp_use_priv_env(): Function not implemented > > [1505822776.561396] [compute1:16218:0] ib_dev.c:544 MXM ERROR > > ibv_query_device() returned 38: Function not implemented > > [1505822776.561426] [compute1:16225:0] ib_dev.c:533 MXM WARN > failed call > > to ibv_exp_use_priv_env(): Function not implemented > > [1505822776.561442] [compute1:16225:0] ib_dev.c:544 MXM ERROR > > ibv_query_device() returned 38: Function not implemented > > Fatal error in MPI_Init: Other MPI error, error stack: > > MPIR_Init_thread(474).........: > > MPID_Init(190)................: channel initialization failed > > MPIDI_CH3_Init(89)............: > > MPID_nem_init(320)............: > > MPID_nem_mxm_init(158)........: > > MPID_nem_mxm_get_ordering(464): mxm_init failed (Input/output > error) > > > > > > El vie., 15 sept. 2017 a las 16:01, Halim Amer > (<aamer(a)anl.gov <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>>>) escribió: > > > > The "nemesis:ib" netmod does not exist anymore. Try > "ch3:nemesis:mxm" > > with a dependency on Mellanox's MXM library (can be > obtained from the > > HPCX package at www.mellanox.com/products/hpcx > <http://www.mellanox.com/products/hpcx> > > <http://www.mellanox.com/products/hpcx>) or "ch3:nemesis:ofi" > > with a dependency on libfabric (which would be built to > support the IB > > or MXM providers; see https://ofiwg.github.io/libfabric/) > > > > Halim > > www.mcs.anl.gov/~aamer <http://www.mcs.anl.gov/~aamer> > <http://www.mcs.anl.gov/~aamer> > > > > On 9/15/17 4:20 AM, Jason Collins wrote: > > > Hello everyone. > > > > > > Recently, I downloaded Mpich-3.2 > > > > > > I want to configure with support for InfiniBand. I've > put the > > following > > > command: > > > > > > # ./configure --prefix=/my/path > --with-device=ch3:nemesis:ib > > > > > > And I get the following error: > > > > > > configure: error: Network module ib is unknown > > > "./src/mpid/ch3/channels/nemesis/netmod/ib" > > > > > > When I check the path I confirm that in the folder > "netmod" does not > > > exist the folder "ib". How can this be solved? > > > > > > Many thanks. > > > > > > <https://audio1.spanishdict.com/audio?lang=en&text=when-i-check-the-path-i-c…> > > > > > > > > > _______________________________________________ > > > discuss mailing list discuss(a)mpich.org > <mailto:[email protected]> <mailto:[email protected] > <mailto:[email protected]>> > > > To manage subscription options or unsubscribe: > > > https://lists.mpich.org/mailman/listinfo/discuss > > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > <mailto:[email protected]> <mailto:[email protected] > <mailto:[email protected]>> > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > > > > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Missing Nemesis ib
by Jason Collins 21 Sep '17

21 Sep '17

I tried to run the test in this other way to get more information: # mpirun -n 10 ./mxm_perftest -------------------------------------------------------------------------- Failed to register memory region (MR): Hostname: compute1 Address: 1d14000 Length: 20480 Error: No space left on device -------------------------------------------------------------------------- -------------------------------------------------------------------------- Open MPI has detected that there are UD-capable Verbs devices on your system, but none of them were able to be setup properly. This may indicate a problem on this system. You job will continue, but Open MPI will ignore the "ud" oob component in this run. Hostname: compute1 -------------------------------------------------------------------------- [1505980772.899531] [compute1:59182:0] sys.c:744 MXM WARN Conflicting CPU frequencies detected, using: 3599.84 Waiting for connection... [1505980772.900947] [compute1:59183:0] sys.c:744 MXM WARN Conflicting CPU frequencies detected, using: 3599.84 [1505980772.902329] [compute1:59184:0] sys.c:744 MXM WARN Conflicting CPU frequencies detected, using: 3599.84 [1505980772.903490] [compute1:59185:0] sys.c:744 MXM WARN Conflicting CPU frequencies detected, using: 3599.84 [1505980772.904984] [compute1:59186:0] sys.c:744 MXM WARN Conflicting CPU frequencies detected, using: 3599.84 [1505980772.906288] [compute1:59187:0] sys.c:744 MXM WARN Conflicting CPU frequencies detected, using: 3599.84 [1505980772.907957] [compute1:59188:0] sys.c:744 MXM WARN Conflicting CPU frequencies detected, using: 3599.84 [1505980772.909023] [compute1:59189:0] sys.c:744 MXM WARN Conflicting CPU frequencies detected, using: 3599.84 bind() failed: Address already in use bind() failed: Address already in use bind() failed: Address already in use bind() failed: Address already in use bind() failed: Address already in use bind() failed: Address already in use bind() failed: Address already in use ------------------------------------------------------- Primary job terminated normally, but 1 process returned a non-zero exit code.. Per user-direction, the job has been aborted. ------------------------------------------------------- bind() failed: Address already in use [1505980772.910503] [compute1:59190:0] sys.c:744 MXM WARN Conflicting CPU frequencies detected, using: 3599.84 bind() failed: Address already in use [1505980772.911893] [compute1:59191:0] sys.c:744 MXM WARN Conflicting CPU frequencies detected, using: 3599.84 -------------------------------------------------------------------------- mpirun detected that one or more processes exited with non-zero status, thus causing the job to be terminated. The first process to do so was: Process name: [[55900,1],1] Exit code: 255 El jue., 21 sept. 2017 a las 7:58, Jason Collins (<jasoncollinsw(a)gmail.com>) escribió: > I ran the test and the result was the following: > > # ./mxm_perftest > > [1505976675.346380] [compute1:55801:0] sys.c:744 MXM WARN > Conflicting CPU frequencies detected, using: 3600.52 > > Waiting for connection... > > > It does nothing else, it remains waiting to establish connection. > > <https://audio1.spanishdict.com/audio?lang=en&text=i-ran-the-test-and-the-re…> > > El mié., 20 sept. 2017 a las 17:12, Halim Amer (<aamer(a)anl.gov>) escribió: > >> I seems you have mismatch in the OFED stack. Try installing the Mellanox >> OFED stack if you are using the bundled OFED stack right now. >> >> Make sure MXM works before trying MPICH. Use the mxm/bin/mxm_perftest >> from your MXM installation to test that things work properly. If it >> doesn't work, then contact your admin or Mellanox cause it is not an >> MPICH problem. >> >> Halim >> www.mcs.anl.gov/~aamer >> >> On 9/19/17 7:14 AM, Jason Collins wrote: >> > Thank you very much. >> > >> > I have compiled with "CH3:nemesis:mxm". The compilation was successful. >> > >> > Now I have a new problem. I tested the test "./icp" and get the >> > following error. >> > >> > # mpiexec -f hosts -n 4 ./cpi >> > [1505822776.546898] [compute1:16212:0] sys.c:744 MXM WARN Conflicting >> > CPU frequencies detected, using: 3459.84 >> > [1505822776.546898] [compute1:16213:0] sys.c:744 MXM WARN Conflicting >> > CPU frequencies detected, using: 3459.84 >> > [1505822776.546951] [compute1:16216:0] sys.c:744 MXM WARN Conflicting >> > CPU frequencies detected, using: 3459.84 >> > [1505822776.547039] [compute1:16214:0] sys.c:744 MXM WARN Conflicting >> > CPU frequencies detected, using: 3459.84 >> > [1505822776.561357] [compute1:16214:0] ib_dev.c:533 MXM WARN failed call >> > to ibv_exp_use_priv_env(): Function not implemented >> > [1505822776.561371] [compute1:16214:0] ib_dev.c:544 MXM ERROR >> > ibv_query_device() returned 38: Function not implemented >> > [1505822776.561386] [compute1:16218:0] ib_dev.c:533 MXM WARN failed call >> > to ibv_exp_use_priv_env(): Function not implemented >> > [1505822776.561396] [compute1:16218:0] ib_dev.c:544 MXM ERROR >> > ibv_query_device() returned 38: Function not implemented >> > [1505822776.561426] [compute1:16225:0] ib_dev.c:533 MXM WARN failed call >> > to ibv_exp_use_priv_env(): Function not implemented >> > [1505822776.561442] [compute1:16225:0] ib_dev.c:544 MXM ERROR >> > ibv_query_device() returned 38: Function not implemented >> > Fatal error in MPI_Init: Other MPI error, error stack: >> > MPIR_Init_thread(474).........: >> > MPID_Init(190)................: channel initialization failed >> > MPIDI_CH3_Init(89)............: >> > MPID_nem_init(320)............: >> > MPID_nem_mxm_init(158)........: >> > MPID_nem_mxm_get_ordering(464): mxm_init failed (Input/output error) >> > >> > >> > El vie., 15 sept. 2017 a las 16:01, Halim Amer (<aamer(a)anl.gov >> > <mailto:[email protected]>>) escribió: >> > >> > The "nemesis:ib" netmod does not exist anymore. Try >> "ch3:nemesis:mxm" >> > with a dependency on Mellanox's MXM library (can be obtained from >> the >> > HPCX package at www.mellanox.com/products/hpcx >> > <http://www.mellanox.com/products/hpcx>) or "ch3:nemesis:ofi" >> > with a dependency on libfabric (which would be built to support the >> IB >> > or MXM providers; see https://ofiwg.github.io/libfabric/) >> > >> > Halim >> > www.mcs.anl.gov/~aamer <http://www.mcs.anl.gov/~aamer> >> > >> > On 9/15/17 4:20 AM, Jason Collins wrote: >> > > Hello everyone. >> > > >> > > Recently, I downloaded Mpich-3.2 >> > > >> > > I want to configure with support for InfiniBand. I've put the >> > following >> > > command: >> > > >> > > # ./configure --prefix=/my/path --with-device=ch3:nemesis:ib >> > > >> > > And I get the following error: >> > > >> > > configure: error: Network module ib is unknown >> > > "./src/mpid/ch3/channels/nemesis/netmod/ib" >> > > >> > > When I check the path I confirm that in the folder "netmod" does >> not >> > > exist the folder "ib". How can this be solved? >> > > >> > > Many thanks. >> > > >> > < >> https://audio1.spanishdict.com/audio?lang=en&text=when-i-check-the-path-i-c… >> > >> > > >> > > >> > > _______________________________________________ >> > > discuss mailing list discuss(a)mpich.org <mailto:[email protected] >> > >> > > To manage subscription options or unsubscribe: >> > > https://lists.mpich.org/mailman/listinfo/discuss >> > > >> > _______________________________________________ >> > discuss mailing list discuss(a)mpich.org <mailto:[email protected]> >> > To manage subscription options or unsubscribe: >> > https://lists.mpich.org/mailman/listinfo/discuss >> > >> > >> > >> > _______________________________________________ >> > discuss mailing list discuss(a)mpich.org >> > To manage subscription options or unsubscribe: >> > https://lists.mpich.org/mailman/listinfo/discuss >> > >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Missing Nemesis ib
by Jason Collins 21 Sep '17

21 Sep '17

I ran the test and the result was the following: # ./mxm_perftest [1505976675.346380] [compute1:55801:0] sys.c:744 MXM WARN Conflicting CPU frequencies detected, using: 3600.52 Waiting for connection... It does nothing else, it remains waiting to establish connection. <https://audio1.spanishdict.com/audio?lang=en&text=i-ran-the-test-and-the-re…> El mié., 20 sept. 2017 a las 17:12, Halim Amer (<aamer(a)anl.gov>) escribió: > I seems you have mismatch in the OFED stack. Try installing the Mellanox > OFED stack if you are using the bundled OFED stack right now. > > Make sure MXM works before trying MPICH. Use the mxm/bin/mxm_perftest > from your MXM installation to test that things work properly. If it > doesn't work, then contact your admin or Mellanox cause it is not an > MPICH problem. > > Halim > www.mcs.anl.gov/~aamer > > On 9/19/17 7:14 AM, Jason Collins wrote: > > Thank you very much. > > > > I have compiled with "CH3:nemesis:mxm". The compilation was successful. > > > > Now I have a new problem. I tested the test "./icp" and get the > > following error. > > > > # mpiexec -f hosts -n 4 ./cpi > > [1505822776.546898] [compute1:16212:0] sys.c:744 MXM WARN Conflicting > > CPU frequencies detected, using: 3459.84 > > [1505822776.546898] [compute1:16213:0] sys.c:744 MXM WARN Conflicting > > CPU frequencies detected, using: 3459.84 > > [1505822776.546951] [compute1:16216:0] sys.c:744 MXM WARN Conflicting > > CPU frequencies detected, using: 3459.84 > > [1505822776.547039] [compute1:16214:0] sys.c:744 MXM WARN Conflicting > > CPU frequencies detected, using: 3459.84 > > [1505822776.561357] [compute1:16214:0] ib_dev.c:533 MXM WARN failed call > > to ibv_exp_use_priv_env(): Function not implemented > > [1505822776.561371] [compute1:16214:0] ib_dev.c:544 MXM ERROR > > ibv_query_device() returned 38: Function not implemented > > [1505822776.561386] [compute1:16218:0] ib_dev.c:533 MXM WARN failed call > > to ibv_exp_use_priv_env(): Function not implemented > > [1505822776.561396] [compute1:16218:0] ib_dev.c:544 MXM ERROR > > ibv_query_device() returned 38: Function not implemented > > [1505822776.561426] [compute1:16225:0] ib_dev.c:533 MXM WARN failed call > > to ibv_exp_use_priv_env(): Function not implemented > > [1505822776.561442] [compute1:16225:0] ib_dev.c:544 MXM ERROR > > ibv_query_device() returned 38: Function not implemented > > Fatal error in MPI_Init: Other MPI error, error stack: > > MPIR_Init_thread(474).........: > > MPID_Init(190)................: channel initialization failed > > MPIDI_CH3_Init(89)............: > > MPID_nem_init(320)............: > > MPID_nem_mxm_init(158)........: > > MPID_nem_mxm_get_ordering(464): mxm_init failed (Input/output error) > > > > > > El vie., 15 sept. 2017 a las 16:01, Halim Amer (<aamer(a)anl.gov > > <mailto:[email protected]>>) escribió: > > > > The "nemesis:ib" netmod does not exist anymore. Try "ch3:nemesis:mxm" > > with a dependency on Mellanox's MXM library (can be obtained from the > > HPCX package at www.mellanox.com/products/hpcx > > <http://www.mellanox.com/products/hpcx>) or "ch3:nemesis:ofi" > > with a dependency on libfabric (which would be built to support the > IB > > or MXM providers; see https://ofiwg.github.io/libfabric/) > > > > Halim > > www.mcs.anl.gov/~aamer <http://www.mcs.anl.gov/~aamer> > > > > On 9/15/17 4:20 AM, Jason Collins wrote: > > > Hello everyone. > > > > > > Recently, I downloaded Mpich-3.2 > > > > > > I want to configure with support for InfiniBand. I've put the > > following > > > command: > > > > > > # ./configure --prefix=/my/path --with-device=ch3:nemesis:ib > > > > > > And I get the following error: > > > > > > configure: error: Network module ib is unknown > > > "./src/mpid/ch3/channels/nemesis/netmod/ib" > > > > > > When I check the path I confirm that in the folder "netmod" does > not > > > exist the folder "ib". How can this be solved? > > > > > > Many thanks. > > > > > < > https://audio1.spanishdict.com/audio?lang=en&text=when-i-check-the-path-i-c… > > > > > > > > > > > _______________________________________________ > > > discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > > > To manage subscription options or unsubscribe: > > > https://lists.mpich.org/mailman/listinfo/discuss > > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > > > > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Missing Nemesis ib
by Halim Amer 20 Sep '17

20 Sep '17

I seems you have mismatch in the OFED stack. Try installing the Mellanox OFED stack if you are using the bundled OFED stack right now. Make sure MXM works before trying MPICH. Use the mxm/bin/mxm_perftest from your MXM installation to test that things work properly. If it doesn't work, then contact your admin or Mellanox cause it is not an MPICH problem. Halim www.mcs.anl.gov/~aamer On 9/19/17 7:14 AM, Jason Collins wrote: > Thank you very much. > > I have compiled with "CH3:nemesis:mxm". The compilation was successful. > > Now I have a new problem. I tested the test "./icp" and get the > following error. > > # mpiexec -f hosts -n 4 ./cpi > [1505822776.546898] [compute1:16212:0] sys.c:744 MXM WARN Conflicting > CPU frequencies detected, using: 3459.84 > [1505822776.546898] [compute1:16213:0] sys.c:744 MXM WARN Conflicting > CPU frequencies detected, using: 3459.84 > [1505822776.546951] [compute1:16216:0] sys.c:744 MXM WARN Conflicting > CPU frequencies detected, using: 3459.84 > [1505822776.547039] [compute1:16214:0] sys.c:744 MXM WARN Conflicting > CPU frequencies detected, using: 3459.84 > [1505822776.561357] [compute1:16214:0] ib_dev.c:533 MXM WARN failed call > to ibv_exp_use_priv_env(): Function not implemented > [1505822776.561371] [compute1:16214:0] ib_dev.c:544 MXM ERROR > ibv_query_device() returned 38: Function not implemented > [1505822776.561386] [compute1:16218:0] ib_dev.c:533 MXM WARN failed call > to ibv_exp_use_priv_env(): Function not implemented > [1505822776.561396] [compute1:16218:0] ib_dev.c:544 MXM ERROR > ibv_query_device() returned 38: Function not implemented > [1505822776.561426] [compute1:16225:0] ib_dev.c:533 MXM WARN failed call > to ibv_exp_use_priv_env(): Function not implemented > [1505822776.561442] [compute1:16225:0] ib_dev.c:544 MXM ERROR > ibv_query_device() returned 38: Function not implemented > Fatal error in MPI_Init: Other MPI error, error stack: > MPIR_Init_thread(474).........: > MPID_Init(190)................: channel initialization failed > MPIDI_CH3_Init(89)............: > MPID_nem_init(320)............: > MPID_nem_mxm_init(158)........: > MPID_nem_mxm_get_ordering(464): mxm_init failed (Input/output error) > > > El vie., 15 sept. 2017 a las 16:01, Halim Amer (<aamer(a)anl.gov > <mailto:[email protected]>>) escribió: > > The "nemesis:ib" netmod does not exist anymore. Try "ch3:nemesis:mxm" > with a dependency on Mellanox's MXM library (can be obtained from the > HPCX package at www.mellanox.com/products/hpcx > <http://www.mellanox.com/products/hpcx>) or "ch3:nemesis:ofi" > with a dependency on libfabric (which would be built to support the IB > or MXM providers; see https://ofiwg.github.io/libfabric/) > > Halim > www.mcs.anl.gov/~aamer <http://www.mcs.anl.gov/~aamer> > > On 9/15/17 4:20 AM, Jason Collins wrote: > > Hello everyone. > > > > Recently, I downloaded Mpich-3.2 > > > > I want to configure with support for InfiniBand. I've put the > following > > command: > > > > # ./configure --prefix=/my/path --with-device=ch3:nemesis:ib > > > > And I get the following error: > > > > configure: error: Network module ib is unknown > > "./src/mpid/ch3/channels/nemesis/netmod/ib" > > > > When I check the path I confirm that in the folder "netmod" does not > > exist the folder "ib". How can this be solved? > > > > Many thanks. > > > <https://audio1.spanishdict.com/audio?lang=en&text=when-i-check-the-path-i-c…> > > > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0