September 2016 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] compilation error on cygwin 2.6.0
by Guo, Yanfei 23 Sep '16

23 Sep '16

Hi, We no longer support Windows or GNU ports such as Cygwin and MinGW. You may have a better luck with MS-MPI. https://wiki.mpich.org/mpich/index.php/Frequently_Asked_Questions#Q:_Why_ca… Yanfei Guo Postdoctoral Researcher MCS Division, ANL On 9/23/16, 4:35 AM, "Prashanth Nadukandi" <prashanth.nadukandi(a)manchester.ac.uk> wrote: ________________________________________ Good morning, I tried to build mpich-3.2 on cygwin (version 2.6.0; 64-bit) using the configure option --with-pm=gforker. There are some errors (c.txt and m.txt attached; VERBOSE=1) after executing make. The objective is to use the available multiple cores in my desktop (one node) for testing MPI programs. The version of GCC is 5.4.0. I will be grateful to you for any help in this regard. Thank you for your time. -- Prashanth. Dr. Prashanth Nadukandi Marie Skłodowska-Curie Fellow School of Mathematics University of Manchester, UK office: +44(0) 161 306 3640 web: nadukandi.es/ <http://nadukandi.es/> _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

compilation error on cygwin 2.6.0
by Prashanth Nadukandi 23 Sep '16

23 Sep '16

________________________________ Good morning, I tried to build mpich-3.2 on cygwin (version 2.6.0; 64-bit) using the configure option --with-pm=gforker. There are some errors (c.txt and m.txt attached; VERBOSE=1) after executing make. The objective is to use the available multiple cores in my desktop (one node) for testing MPI programs. The version of GCC is 5.4.0. I will be grateful to you for any help in this regard. Thank you for your time. -- Prashanth. Dr. Prashanth Nadukandi Marie Skłodowska-Curie Fellow School of Mathematics University of Manchester, UK office: +44(0) 161 306 3640 web: nadukandi.es/<http://nadukandi.es/> _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPI Window limit on mpich
by Jeff Hammond 19 Sep '16

19 Sep '16

The upper limit has not changed by more than one bit (102X to 204X) in as long as I have been paying attention. There is a need to keep the bits in (comm,tag,rank) small, although I don't know the details in MPICH but the issue was discussed in https://cug.org/5-publications/proceedings_attendee_lists/CUG09CD/S09_Proce… . A better option for RMA programs is to provide a window info key to disable the duplicating of communicators for every window. This would force the implementation to generate them on-the-fly for window collectives (e.g. MPI_Win_fence and MPI_Win_free) but there are many RMA programs that do not use MPI_Win_fence (the most reasonable use of RMA is passive target with flush synchronization, which is non-collective). If MPI_Win_free is on the critical path, the application is poorly written. Jeff On Mon, Sep 19, 2016 at 11:15 AM, Marvin Smith <Marvin.Smith(a)sncorp.com> wrote: > Thanks for the quick reply. Do you see this value increasing in the > future? In the meantime, I have a solution with less windows. > > Thanks, > Marvin > > > > From: "Oden, Lena" <loden(a)anl.gov> > To: "discuss(a)mpich.org" <discuss(a)mpich.org> > Date: 09/19/2016 11:11 AM > Subject: Re: [mpich-discuss] MPI Window limit on mpich > ------------------------------ > > > > Hi Marvin, > > currently, this is a limitation inside MPICH. > For every new window, MPICH internally creates a new communicator - for > every communicator a new (unique) context ID is required - > and the number of different context IDs is limited to 2048. > > This context-id/ communicator is required for internal synchronization > (e.g. barriers) . We have to ensure, that his communication is > not interfering with other communication on other windows or communicators. > > If you use more communicators, you should run into this problem earlier, > because you already create other communicators > (the limit is per process) > > Lena > > On Sep 19, 2016, at 12:49 PM, Marvin Smith <*Marvin.Smith(a)sncorp.com* > <Marvin.Smith(a)sncorp.com>> wrote: > > Good morning, > > I wanted to present an issue I am having with MPICH and validate > whether this is a configuration problem, a limitation with MPICH, or a bug. > > I am writing an application which uses a large number of MPI windows, each > window is given a relatively large amount of memory. This has never been > a problem before, however we discovered if you allocate more than 2045 > windows, you get an exception thrown. > > Notes: > > - I am compiling using g++, version 4.8.5 on Red Hat Enterprise Linux > version 7.2. > - My MPICH version is listed at the bottom of this email. It was > installed via yum and is the RHEL default. > - I have attached sample output, to include the stdout/stderr. Also > included is a Makefile and a simple example. > - The boundary of failure is between 2045 and 2046 windows. > - I have verified on my system this problem repeats even if I > distribute windows between multiple communicators. > - I have not tested yet against ompi or mvapich. > > > > #----------------------------------------------------------- > ------------------------------------# > #- Here is my output > -# > #----------------------------------------------------------- > ------------------------------------# > > mpirun -np 2 -hosts localhost ./mpi-win-test 2046 1 > Initialized Rank: 0, Number Processors: 2, Hostname: test-machine > Initialized Rank: 1, Number Processors: 2, Hostname: test-machine > Fatal error in MPI_Win_create_dynamic: Other MPI error, error stack: > MPI_Win_create_dynamic(154)..........: MPI_Win_create_dynamic(MPI_INFO_NULL, > MPI_COMM_WORLD, win=0x10c1464) failed > MPID_Win_create_dynamic(139).........: > win_init(254)........................: > MPIR_Comm_dup_impl(55)...............: > MPIR_Comm_copy(1552).................: > MPIR_Get_contextid(799)..............: > MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID > because of fragmentation (0/2048 free on this process; ignore_id=0) > Fatal error in MPI_Win_create_dynamic: Other MPI error, error stack: > MPI_Win_create_dynamic(154)..........: MPI_Win_create_dynamic(MPI_INFO_NULL, > MPI_COMM_WORLD, win=0x19ef444) failed > MPID_Win_create_dynamic(139).........: > win_init(254)........................: > MPIR_Comm_dup_impl(55)...............: > MPIR_Comm_copy(1552).................: > MPIR_Get_contextid(799)..............: > MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID > because of fragmentation (0/2048 free on this process; ignore_id=0) > > ============================================================ > ======================= > = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES > = EXIT CODE: 1 > = CLEANING UP REMAINING PROCESSES > = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES > ============================================================ > ======================= > make: *** [run] Error 1 > > > > * > #------------------------------------------------------------------------------------------#* > * #- Here is my Sample Makefile > -#* > * > #------------------------------------------------------------------------------------------#* > > # Path to mpich on RHEL7 > MPI_INCL=-I/usr/include/mpich-x86_64 > MPI_LIBS=-L/usr/lib64/mpich/lib -lmpich > > # C++11 Bindings (Being lazy with string) > CXX_ARGS=-std=c++11 > > # Make th test > all: mpi-win-test > > mpi-win-test: mpi-win-test.cpp > g++ $< -o $@ $(MPI_INCL) $(MPI_LIBS) $(CXX_ARGS) > > > # Args for application > NUM_WINDOWS=2046 > USE_DYNAMIC=1 > > # Sample run usage > # > # Args: > # - Number of Windows > # - Type of windows (1 dynamic, 0 static) > run: > mpirun -np 2 -hosts localhost ./mpi-win-test $(NUM_WINDOWS) > $(USE_DYNAMIC) > > > > * > #------------------------------------------------------------------------------------------#* > * #- Here is my Sample Application > -#* > * > #------------------------------------------------------------------------------------------#* > > > #include <mpi.h> > > #include <iostream> > #include <string> > #include <vector> > > using namespace std; > > int main( int argc, char* argv[] ) > { > // Number of MPI Windows > int num_windows = std::stoi(argv[1]); > > bool use_dynamic = std::stoi(argv[2]); > > // Initialize MPI > MPI_Init( &argc, &argv ); > > // Get the rank and size > int rank, nprocs; > MPI_Comm_size( MPI_COMM_WORLD, &nprocs ); > MPI_Comm_rank( MPI_COMM_WORLD, &rank ); > > // Get the processor name > char hostname[MPI_MAX_PROCESSOR_NAME]; > int hostname_len; > MPI_Get_processor_name( hostname, &hostname_len); > > // Print Message > for( int i=0; i<nprocs; i++ ){ > MPI_Barrier(MPI_COMM_WORLD); > if( i == rank ){ > std::cout << "Initialized Rank: " << rank << ", Number > Processors: " << nprocs << ", Hostname: " << hostname << std::endl; > } > } > > > // MPI Variables > vector<MPI_Aint> sdisp_remotes(num_windows); > vector<MPI_Aint> sdisp_locals(num_windows); > > // Create MPI Windows > vector<MPI_Win> windows(num_windows); > > int64_t buffer_size = 1000; > char* buffer = new char[buffer_size]; > > for( int i=0; i<num_windows; i++ ) > { > if( use_dynamic ) > { > MPI_Win_create_dynamic( MPI_INFO_NULL, MPI_COMM_WORLD, > &windows[i] ); > } > > else > { > MPI_Win_create( &buffer, > buffer_size, > 1, > MPI_INFO_NULL, > MPI_COMM_WORLD, > &windows[i] ); > } > } > > > // Exception always occurs prior to reaching this point. > > > // More Code Here that I am removing for brevity > > // Wait at the barrier > MPI_Barrier( MPI_COMM_WORLD ); > > // Remove all windows > for( int i=0; i<num_windows; i++) > { > // Destroy the MPI Window > MPI_Win_free( &windows[i] ); > } > windows.clear(); > > // Clear buffer > delete [] buffer; > buffer = nullptr; > > // Close MPI > MPI_Finalize(); > > return 0; > } > > * > #--------------------------------------------------------------------------#* > * #- MPICH Version Output -#* > * > #--------------------------------------------------------------------------#* > MPICH Version: 3.0.4 > MPICH Release date: Wed Apr 24 10:08:10 CDT 2013 > MPICH Device: ch3:nemesis > MPICH configure: --build=x86_64-redhat-linux-gnu > --host=x86_64-redhat-linux-gnu --program-prefix= > --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr > --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc > --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 > --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib > --mandir=/usr/share/man --infodir=/usr/share/info --enable-sharedlibs=gcc > --enable-shared --enable-lib-depend --disable-rpath --enable-fc > --with-device=ch3:nemesis --with-pm=hydra:gforker > --sysconfdir=/etc/mpich-x86_64 --includedir=/usr/include/mpich-x86_64 > --bindir=/usr/lib64/mpich/bin --libdir=/usr/lib64/mpich/lib > --datadir=/usr/share/mpich --mandir=/usr/share/man/mpich > --docdir=/usr/share/mpich/doc --htmldir=/usr/share/mpich/doc > --with-hwloc-prefix=system FC=gfortran F77=gfortran CFLAGS=-m64 -O2 -g > -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC > CXXFLAGS=-m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions > -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches > -m64 -mtune=generic -fPIC FCFLAGS=-m64 -O2 -g -pipe -Wall > -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC > FFLAGS=-m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions > -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches > -m64 -mtune=generic -fPIC LDFLAGS=-Wl,-z,noexecstack MPICH2LIB_CFLAGS=-O2 > -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions > -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches > -m64 -mtune=generic MPICH2LIB_CXXFLAGS=-O2 -g -pipe -Wall > -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic > MPICH2LIB_FCFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions > -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches > -m64 -mtune=generic MPICH2LIB_FFLAGS=-O2 -g -pipe -Wall > -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic > MPICH CC: cc -m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 > -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 > -grecord-gcc-switches -m64 -mtune=generic -fPIC -O2 > MPICH CXX: c++ -m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 > -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 > -grecord-gcc-switches -m64 -mtune=generic -fPIC -O2 > MPICH F77: gfortran -m64 -O2 -g -pipe -Wall > -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC > -O2 > MPICH FC: gfortran -m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 > -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 > -grecord-gcc-switches -m64 -mtune=generic -fPIC -O2 > CONFIDENTIALITY NOTICE - SNC EMAIL: This email and any attachments are > confidential, may contain proprietary, protected, or export controlled > information, and are intended for the use of the intended recipients only. > Any review, reliance, distribution, disclosure, or forwarding of this email > and/or attachments outside of Sierra Nevada Corporation (SNC) without > express written approval of the sender, except to the extent required to > further properly approved SNC business purposes, is strictly prohibited. If > you are not the intended recipient of this email, please notify the sender > immediately, and delete all copies without reading, printing, or saving in > any manner. --- Thank You. > _______________________________________________ > discuss mailing list *discuss(a)mpich.org* <discuss(a)mpich.org> > To manage subscription options or unsubscribe: > *https://lists.mpich.org/mailman/listinfo/discuss* > <https://lists.mpich.org/mailman/listinfo/discuss> > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > CONFIDENTIALITY NOTICE - SNC EMAIL: This email and any attachments are > confidential, may contain proprietary, protected, or export controlled > information, and are intended for the use of the intended recipients only. > Any review, reliance, distribution, disclosure, or forwarding of this email > and/or attachments outside of Sierra Nevada Corporation (SNC) without > express written approval of the sender, except to the extent required to > further properly approved SNC business purposes, is strictly prohibited. If > you are not the intended recipient of this email, please notify the sender > immediately, and delete all copies without reading, printing, or saving in > any manner. --- Thank You. > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/ _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPI Window limit on mpich
by Marvin Smith 19 Sep '16

19 Sep '16

Thanks for the quick reply. Do you see this value increasing in the future? In the meantime, I have a solution with less windows. Thanks, Marvin From: "Oden, Lena" <loden(a)anl.gov> To: "discuss(a)mpich.org" <discuss(a)mpich.org> Date: 09/19/2016 11:11 AM Subject: Re: [mpich-discuss] MPI Window limit on mpich Hi Marvin, currently, this is a limitation inside MPICH. For every new window, MPICH internally creates a new communicator - for every communicator a new (unique) context ID is required - and the number of different context IDs is limited to 2048. This context-id/ communicator is required for internal synchronization (e.g. barriers) . We have to ensure, that his communication is not interfering with other communication on other windows or communicators. If you use more communicators, you should run into this problem earlier, because you already create other communicators (the limit is per process) Lena On Sep 19, 2016, at 12:49 PM, Marvin Smith <Marvin.Smith(a)sncorp.com> wrote: Good morning, I wanted to present an issue I am having with MPICH and validate whether this is a configuration problem, a limitation with MPICH, or a bug. I am writing an application which uses a large number of MPI windows, each window is given a relatively large amount of memory. This has never been a problem before, however we discovered if you allocate more than 2045 windows, you get an exception thrown. Notes: I am compiling using g++, version 4.8.5 on Red Hat Enterprise Linux version 7.2. My MPICH version is listed at the bottom of this email. It was installed via yum and is the RHEL default. I have attached sample output, to include the stdout/stderr. Also included is a Makefile and a simple example. The boundary of failure is between 2045 and 2046 windows. I have verified on my system this problem repeats even if I distribute windows between multiple communicators. I have not tested yet against ompi or mvapich. #-----------------------------------------------------------------------------------------------# #- Here is my output -# #-----------------------------------------------------------------------------------------------# mpirun -np 2 -hosts localhost ./mpi-win-test 2046 1 Initialized Rank: 0, Number Processors: 2, Hostname: test-machine Initialized Rank: 1, Number Processors: 2, Hostname: test-machine Fatal error in MPI_Win_create_dynamic: Other MPI error, error stack: MPI_Win_create_dynamic(154)..........: MPI_Win_create_dynamic(MPI_INFO_NULL, MPI_COMM_WORLD, win=0x10c1464) failed MPID_Win_create_dynamic(139).........: win_init(254)........................: MPIR_Comm_dup_impl(55)...............: MPIR_Comm_copy(1552).................: MPIR_Get_contextid(799)..............: MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID because of fragmentation (0/2048 free on this process; ignore_id=0) Fatal error in MPI_Win_create_dynamic: Other MPI error, error stack: MPI_Win_create_dynamic(154)..........: MPI_Win_create_dynamic(MPI_INFO_NULL, MPI_COMM_WORLD, win=0x19ef444) failed MPID_Win_create_dynamic(139).........: win_init(254)........................: MPIR_Comm_dup_impl(55)...............: MPIR_Comm_copy(1552).................: MPIR_Get_contextid(799)..............: MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID because of fragmentation (0/2048 free on this process; ignore_id=0) =================================================================================== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 1 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =================================================================================== make: *** [run] Error 1 #------------------------------------------------------------------------------------------# #- Here is my Sample Makefile -# #------------------------------------------------------------------------------------------# # Path to mpich on RHEL7 MPI_INCL=-I/usr/include/mpich-x86_64 MPI_LIBS=-L/usr/lib64/mpich/lib -lmpich # C++11 Bindings (Being lazy with string) CXX_ARGS=-std=c++11 # Make th test all: mpi-win-test mpi-win-test: mpi-win-test.cpp g++ $< -o $@ $(MPI_INCL) $(MPI_LIBS) $(CXX_ARGS) # Args for application NUM_WINDOWS=2046 USE_DYNAMIC=1 # Sample run usage # # Args: # - Number of Windows # - Type of windows (1 dynamic, 0 static) run: mpirun -np 2 -hosts localhost ./mpi-win-test $(NUM_WINDOWS) $(USE_DYNAMIC) #------------------------------------------------------------------------------------------# #- Here is my Sample Application -# #------------------------------------------------------------------------------------------# #include <mpi.h> #include <iostream> #include <string> #include <vector> using namespace std; int main( int argc, char* argv[] ) { // Number of MPI Windows int num_windows = std::stoi(argv[1]); bool use_dynamic = std::stoi(argv[2]); // Initialize MPI MPI_Init( &argc, &argv ); // Get the rank and size int rank, nprocs; MPI_Comm_size( MPI_COMM_WORLD, &nprocs ); MPI_Comm_rank( MPI_COMM_WORLD, &rank ); // Get the processor name char hostname[MPI_MAX_PROCESSOR_NAME]; int hostname_len; MPI_Get_processor_name( hostname, &hostname_len); // Print Message for( int i=0; i<nprocs; i++ ){ MPI_Barrier(MPI_COMM_WORLD); if( i == rank ){ std::cout << "Initialized Rank: " << rank << ", Number Processors: " << nprocs << ", Hostname: " << hostname << std::endl; } } // MPI Variables vector<MPI_Aint> sdisp_remotes(num_windows); vector<MPI_Aint> sdisp_locals(num_windows); // Create MPI Windows vector<MPI_Win> windows(num_windows); int64_t buffer_size = 1000; char* buffer = new char[buffer_size]; for( int i=0; i<num_windows; i++ ) { if( use_dynamic ) { MPI_Win_create_dynamic( MPI_INFO_NULL, MPI_COMM_WORLD, &windows[i] ); } else { MPI_Win_create( &buffer, buffer_size, 1, MPI_INFO_NULL, MPI_COMM_WORLD, &windows[i] ); } } // Exception always occurs prior to reaching this point. // More Code Here that I am removing for brevity // Wait at the barrier MPI_Barrier( MPI_COMM_WORLD ); // Remove all windows for( int i=0; i<num_windows; i++) { // Destroy the MPI Window MPI_Win_free( &windows[i] ); } windows.clear(); // Clear buffer delete [] buffer; buffer = nullptr; // Close MPI MPI_Finalize(); return 0; } #--------------------------------------------------------------------------# #- MPICH Version Output -# #--------------------------------------------------------------------------# MPICH Version: 3.0.4 MPICH Release date: Wed Apr 24 10:08:10 CDT 2013 MPICH Device: ch3:nemesis MPICH configure: --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --enable-sharedlibs=gcc --enable-shared --enable-lib-depend --disable-rpath --enable-fc --with-device=ch3:nemesis --with-pm=hydra:gforker --sysconfdir=/etc/mpich-x86_64 --includedir=/usr/include/mpich-x86_64 --bindir=/usr/lib64/mpich/bin --libdir=/usr/lib64/mpich/lib --datadir=/usr/share/mpich --mandir=/usr/share/man/mpich --docdir=/usr/share/mpich/doc --htmldir=/usr/share/mpich/doc --with-hwloc-prefix=system FC=gfortran F77=gfortran CFLAGS=-m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC CXXFLAGS=-m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC FCFLAGS=-m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC FFLAGS=-m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC LDFLAGS=-Wl,-z,noexecstack MPICH2LIB_CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic MPICH2LIB_CXXFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic MPICH2LIB_FCFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic MPICH2LIB_FFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic MPICH CC: cc -m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC -O2 MPICH CXX: c++ -m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC -O2 MPICH F77: gfortran -m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC -O2 MPICH FC: gfortran -m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC -O2 CONFIDENTIALITY NOTICE - SNC EMAIL: This email and any attachments are confidential, may contain proprietary, protected, or export controlled information, and are intended for the use of the intended recipients only. Any review, reliance, distribution, disclosure, or forwarding of this email and/or attachments outside of Sierra Nevada Corporation (SNC) without express written approval of the sender, except to the extent required to further properly approved SNC business purposes, is strictly prohibited. If you are not the intended recipient of this email, please notify the sender immediately, and delete all copies without reading, printing, or saving in any manner. --- Thank You. _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss CONFIDENTIALITY NOTICE - SNC EMAIL: This email and any attachments are confidential, may contain proprietary, protected, or export controlled information, and are intended for the use of the intended recipients only. Any review, reliance, distribution, disclosure, or forwarding of this email and/or attachments outside of Sierra Nevada Corporation (SNC) without express written approval of the sender, except to the extent required to further properly approved SNC business purposes, is strictly prohibited. If you are not the intended recipient of this email, please notify the sender immediately, and delete all copies without reading, printing, or saving in any manner. --- Thank You. _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPI Window limit on mpich
by Jeff Hammond 19 Sep '16

19 Sep '16

I believe that creating 2046 communicators will also fail, so this doesn't have anything to do with windows themselves, just their apparent need to create an associated window. The limitation in MPICH to create more than ~2000 communicators is known. Why do you need to create ~2000 dynamic windows? I can't think of a reasonable application that would do this. Jeff On Mon, Sep 19, 2016 at 10:49 AM, Marvin Smith <Marvin.Smith(a)sncorp.com> wrote: > Good morning, > > I wanted to present an issue I am having with MPICH and validate > whether this is a configuration problem, a limitation with MPICH, or a bug. > > I am writing an application which uses a large number of MPI windows, each > window is given a relatively large amount of memory. This has never been > a problem before, however we discovered if you allocate more than 2045 > windows, you get an exception thrown. > > Notes: > > - I am compiling using g++, version 4.8.5 on Red Hat Enterprise Linux > version 7.2. > - My MPICH version is listed at the bottom of this email. It was > installed via yum and is the RHEL default. > - I have attached sample output, to include the stdout/stderr. Also > included is a Makefile and a simple example. > - The boundary of failure is between 2045 and 2046 windows. > - I have verified on my system this problem repeats even if I > distribute windows between multiple communicators. > - I have not tested yet against ompi or mvapich. > > > > #----------------------------------------------------------- > ------------------------------------# > #- Here is my output > -# > #----------------------------------------------------------- > ------------------------------------# > > mpirun -np 2 -hosts localhost ./mpi-win-test 2046 1 > Initialized Rank: 0, Number Processors: 2, Hostname: test-machine > Initialized Rank: 1, Number Processors: 2, Hostname: test-machine > Fatal error in MPI_Win_create_dynamic: Other MPI error, error stack: > MPI_Win_create_dynamic(154)..........: MPI_Win_create_dynamic(MPI_INFO_NULL, > MPI_COMM_WORLD, win=0x10c1464) failed > MPID_Win_create_dynamic(139).........: > win_init(254)........................: > MPIR_Comm_dup_impl(55)...............: > MPIR_Comm_copy(1552).................: > MPIR_Get_contextid(799)..............: > MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID > because of fragmentation (0/2048 free on this process; ignore_id=0) > Fatal error in MPI_Win_create_dynamic: Other MPI error, error stack: > MPI_Win_create_dynamic(154)..........: MPI_Win_create_dynamic(MPI_INFO_NULL, > MPI_COMM_WORLD, win=0x19ef444) failed > MPID_Win_create_dynamic(139).........: > win_init(254)........................: > MPIR_Comm_dup_impl(55)...............: > MPIR_Comm_copy(1552).................: > MPIR_Get_contextid(799)..............: > MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID > because of fragmentation (0/2048 free on this process; ignore_id=0) > > ============================================================ > ======================= > = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES > = EXIT CODE: 1 > = CLEANING UP REMAINING PROCESSES > = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES > ============================================================ > ======================= > make: *** [run] Error 1 > > > > > *#------------------------------------------------------------------------------------------#* > *#- Here is my Sample Makefile > -#* > > *#------------------------------------------------------------------------------------------#* > > # Path to mpich on RHEL7 > MPI_INCL=-I/usr/include/mpich-x86_64 > MPI_LIBS=-L/usr/lib64/mpich/lib -lmpich > > # C++11 Bindings (Being lazy with string) > CXX_ARGS=-std=c++11 > > # Make th test > all: mpi-win-test > > mpi-win-test: mpi-win-test.cpp > g++ $< -o $@ $(MPI_INCL) $(MPI_LIBS) $(CXX_ARGS) > > > # Args for application > NUM_WINDOWS=2046 > USE_DYNAMIC=1 > > # Sample run usage > # > # Args: > # - Number of Windows > # - Type of windows (1 dynamic, 0 static) > run: > mpirun -np 2 -hosts localhost ./mpi-win-test $(NUM_WINDOWS) > $(USE_DYNAMIC) > > > > > *#------------------------------------------------------------------------------------------#* > *#- Here is my Sample Application > -#* > > *#------------------------------------------------------------------------------------------#* > > > #include <mpi.h> > > #include <iostream> > #include <string> > #include <vector> > > using namespace std; > > int main( int argc, char* argv[] ) > { > // Number of MPI Windows > int num_windows = std::stoi(argv[1]); > > bool use_dynamic = std::stoi(argv[2]); > > // Initialize MPI > MPI_Init( &argc, &argv ); > > // Get the rank and size > int rank, nprocs; > MPI_Comm_size( MPI_COMM_WORLD, &nprocs ); > MPI_Comm_rank( MPI_COMM_WORLD, &rank ); > > // Get the processor name > char hostname[MPI_MAX_PROCESSOR_NAME]; > int hostname_len; > MPI_Get_processor_name( hostname, &hostname_len); > > // Print Message > for( int i=0; i<nprocs; i++ ){ > MPI_Barrier(MPI_COMM_WORLD); > if( i == rank ){ > std::cout << "Initialized Rank: " << rank << ", Number > Processors: " << nprocs << ", Hostname: " << hostname << std::endl; > } > } > > > // MPI Variables > vector<MPI_Aint> sdisp_remotes(num_windows); > vector<MPI_Aint> sdisp_locals(num_windows); > > // Create MPI Windows > vector<MPI_Win> windows(num_windows); > > int64_t buffer_size = 1000; > char* buffer = new char[buffer_size]; > > for( int i=0; i<num_windows; i++ ) > { > if( use_dynamic ) > { > MPI_Win_create_dynamic( MPI_INFO_NULL, MPI_COMM_WORLD, > &windows[i] ); > } > > else > { > MPI_Win_create( &buffer, > buffer_size, > 1, > MPI_INFO_NULL, > MPI_COMM_WORLD, > &windows[i] ); > } > } > > > // Exception always occurs prior to reaching this point. > > > // More Code Here that I am removing for brevity > > // Wait at the barrier > MPI_Barrier( MPI_COMM_WORLD ); > > // Remove all windows > for( int i=0; i<num_windows; i++) > { > // Destroy the MPI Window > MPI_Win_free( &windows[i] ); > } > windows.clear(); > > // Clear buffer > delete [] buffer; > buffer = nullptr; > > // Close MPI > MPI_Finalize(); > > return 0; > } > > > *#--------------------------------------------------------------------------#* > *#- MPICH Version Output -#* > > *#--------------------------------------------------------------------------#* > MPICH Version: 3.0.4 > MPICH Release date: Wed Apr 24 10:08:10 CDT 2013 > MPICH Device: ch3:nemesis > MPICH configure: --build=x86_64-redhat-linux-gnu > --host=x86_64-redhat-linux-gnu --program-prefix= > --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr > --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc > --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 > --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib > --mandir=/usr/share/man --infodir=/usr/share/info --enable-sharedlibs=gcc > --enable-shared --enable-lib-depend --disable-rpath --enable-fc > --with-device=ch3:nemesis --with-pm=hydra:gforker > --sysconfdir=/etc/mpich-x86_64 --includedir=/usr/include/mpich-x86_64 > --bindir=/usr/lib64/mpich/bin --libdir=/usr/lib64/mpich/lib > --datadir=/usr/share/mpich --mandir=/usr/share/man/mpich > --docdir=/usr/share/mpich/doc --htmldir=/usr/share/mpich/doc > --with-hwloc-prefix=system FC=gfortran F77=gfortran CFLAGS=-m64 -O2 -g > -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC > CXXFLAGS=-m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions > -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches > -m64 -mtune=generic -fPIC FCFLAGS=-m64 -O2 -g -pipe -Wall > -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC > FFLAGS=-m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions > -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches > -m64 -mtune=generic -fPIC LDFLAGS=-Wl,-z,noexecstack MPICH2LIB_CFLAGS=-O2 > -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions > -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches > -m64 -mtune=generic MPICH2LIB_CXXFLAGS=-O2 -g -pipe -Wall > -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic > MPICH2LIB_FCFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions > -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches > -m64 -mtune=generic MPICH2LIB_FFLAGS=-O2 -g -pipe -Wall > -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic > MPICH CC: cc -m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 > -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 > -grecord-gcc-switches -m64 -mtune=generic -fPIC -O2 > MPICH CXX: c++ -m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 > -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 > -grecord-gcc-switches -m64 -mtune=generic -fPIC -O2 > MPICH F77: gfortran -m64 -O2 -g -pipe -Wall > -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC > -O2 > MPICH FC: gfortran -m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 > -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 > -grecord-gcc-switches -m64 -mtune=generic -fPIC -O2 > CONFIDENTIALITY NOTICE - SNC EMAIL: This email and any attachments are > confidential, may contain proprietary, protected, or export controlled > information, and are intended for the use of the intended recipients only. > Any review, reliance, distribution, disclosure, or forwarding of this email > and/or attachments outside of Sierra Nevada Corporation (SNC) without > express written approval of the sender, except to the extent required to > further properly approved SNC business purposes, is strictly prohibited. If > you are not the intended recipient of this email, please notify the sender > immediately, and delete all copies without reading, printing, or saving in > any manner. --- Thank You. > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/ _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPI Window limit on mpich
by Oden, Lena 19 Sep '16

19 Sep '16

Hi Marvin, currently, this is a limitation inside MPICH. For every new window, MPICH internally creates a new communicator - for every communicator a new (unique) context ID is required - and the number of different context IDs is limited to 2048. This context-id/ communicator is required for internal synchronization (e.g. barriers) . We have to ensure, that his communication is not interfering with other communication on other windows or communicators. If you use more communicators, you should run into this problem earlier, because you already create other communicators (the limit is per process) Lena On Sep 19, 2016, at 12:49 PM, Marvin Smith <Marvin.Smith(a)sncorp.com<mailto:[email protected]>> wrote: Good morning, I wanted to present an issue I am having with MPICH and validate whether this is a configuration problem, a limitation with MPICH, or a bug. I am writing an application which uses a large number of MPI windows, each window is given a relatively large amount of memory. This has never been a problem before, however we discovered if you allocate more than 2045 windows, you get an exception thrown. Notes: * I am compiling using g++, version 4.8.5 on Red Hat Enterprise Linux version 7.2. * My MPICH version is listed at the bottom of this email. It was installed via yum and is the RHEL default. * I have attached sample output, to include the stdout/stderr. Also included is a Makefile and a simple example. * The boundary of failure is between 2045 and 2046 windows. * I have verified on my system this problem repeats even if I distribute windows between multiple communicators. * I have not tested yet against ompi or mvapich. #-----------------------------------------------------------------------------------------------# #- Here is my output -# #-----------------------------------------------------------------------------------------------# mpirun -np 2 -hosts localhost ./mpi-win-test 2046 1 Initialized Rank: 0, Number Processors: 2, Hostname: test-machine Initialized Rank: 1, Number Processors: 2, Hostname: test-machine Fatal error in MPI_Win_create_dynamic: Other MPI error, error stack: MPI_Win_create_dynamic(154)..........: MPI_Win_create_dynamic(MPI_INFO_NULL, MPI_COMM_WORLD, win=0x10c1464) failed MPID_Win_create_dynamic(139).........: win_init(254)........................: MPIR_Comm_dup_impl(55)...............: MPIR_Comm_copy(1552).................: MPIR_Get_contextid(799)..............: MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID because of fragmentation (0/2048 free on this process; ignore_id=0) Fatal error in MPI_Win_create_dynamic: Other MPI error, error stack: MPI_Win_create_dynamic(154)..........: MPI_Win_create_dynamic(MPI_INFO_NULL, MPI_COMM_WORLD, win=0x19ef444) failed MPID_Win_create_dynamic(139).........: win_init(254)........................: MPIR_Comm_dup_impl(55)...............: MPIR_Comm_copy(1552).................: MPIR_Get_contextid(799)..............: MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID because of fragmentation (0/2048 free on this process; ignore_id=0) =================================================================================== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 1 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =================================================================================== make: *** [run] Error 1 #------------------------------------------------------------------------------------------# #- Here is my Sample Makefile -# #------------------------------------------------------------------------------------------# # Path to mpich on RHEL7 MPI_INCL=-I/usr/include/mpich-x86_64 MPI_LIBS=-L/usr/lib64/mpich/lib -lmpich # C++11 Bindings (Being lazy with string) CXX_ARGS=-std=c++11 # Make th test all: mpi-win-test mpi-win-test: mpi-win-test.cpp g++ $< -o $@ $(MPI_INCL) $(MPI_LIBS) $(CXX_ARGS) # Args for application NUM_WINDOWS=2046 USE_DYNAMIC=1 # Sample run usage # # Args: # - Number of Windows # - Type of windows (1 dynamic, 0 static) run: mpirun -np 2 -hosts localhost ./mpi-win-test $(NUM_WINDOWS) $(USE_DYNAMIC) #------------------------------------------------------------------------------------------# #- Here is my Sample Application -# #------------------------------------------------------------------------------------------# #include <mpi.h> #include <iostream> #include <string> #include <vector> using namespace std; int main( int argc, char* argv[] ) { // Number of MPI Windows int num_windows = std::stoi(argv[1]); bool use_dynamic = std::stoi(argv[2]); // Initialize MPI MPI_Init( &argc, &argv ); // Get the rank and size int rank, nprocs; MPI_Comm_size( MPI_COMM_WORLD, &nprocs ); MPI_Comm_rank( MPI_COMM_WORLD, &rank ); // Get the processor name char hostname[MPI_MAX_PROCESSOR_NAME]; int hostname_len; MPI_Get_processor_name( hostname, &hostname_len); // Print Message for( int i=0; i<nprocs; i++ ){ MPI_Barrier(MPI_COMM_WORLD); if( i == rank ){ std::cout << "Initialized Rank: " << rank << ", Number Processors: " << nprocs << ", Hostname: " << hostname << std::endl; } } // MPI Variables vector<MPI_Aint> sdisp_remotes(num_windows); vector<MPI_Aint> sdisp_locals(num_windows); // Create MPI Windows vector<MPI_Win> windows(num_windows); int64_t buffer_size = 1000; char* buffer = new char[buffer_size]; for( int i=0; i<num_windows; i++ ) { if( use_dynamic ) { MPI_Win_create_dynamic( MPI_INFO_NULL, MPI_COMM_WORLD, &windows[i] ); } else { MPI_Win_create( &buffer, buffer_size, 1, MPI_INFO_NULL, MPI_COMM_WORLD, &windows[i] ); } } // Exception always occurs prior to reaching this point. // More Code Here that I am removing for brevity // Wait at the barrier MPI_Barrier( MPI_COMM_WORLD ); // Remove all windows for( int i=0; i<num_windows; i++) { // Destroy the MPI Window MPI_Win_free( &windows[i] ); } windows.clear(); // Clear buffer delete [] buffer; buffer = nullptr; // Close MPI MPI_Finalize(); return 0; } #--------------------------------------------------------------------------# #- MPICH Version Output -# #--------------------------------------------------------------------------# MPICH Version: 3.0.4 MPICH Release date: Wed Apr 24 10:08:10 CDT 2013 MPICH Device: ch3:nemesis MPICH configure: --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --enable-sharedlibs=gcc --enable-shared --enable-lib-depend --disable-rpath --enable-fc --with-device=ch3:nemesis --with-pm=hydra:gforker --sysconfdir=/etc/mpich-x86_64 --includedir=/usr/include/mpich-x86_64 --bindir=/usr/lib64/mpich/bin --libdir=/usr/lib64/mpich/lib --datadir=/usr/share/mpich --mandir=/usr/share/man/mpich --docdir=/usr/share/mpich/doc --htmldir=/usr/share/mpich/doc --with-hwloc-prefix=system FC=gfortran F77=gfortran CFLAGS=-m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC CXXFLAGS=-m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC FCFLAGS=-m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC FFLAGS=-m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC LDFLAGS=-Wl,-z,noexecstack MPICH2LIB_CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic MPICH2LIB_CXXFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic MPICH2LIB_FCFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic MPICH2LIB_FFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic MPICH CC: cc -m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC -O2 MPICH CXX: c++ -m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC -O2 MPICH F77: gfortran -m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC -O2 MPICH FC: gfortran -m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC -O2 CONFIDENTIALITY NOTICE - SNC EMAIL: This email and any attachments are confidential, may contain proprietary, protected, or export controlled information, and are intended for the use of the intended recipients only. Any review, reliance, distribution, disclosure, or forwarding of this email and/or attachments outside of Sierra Nevada Corporation (SNC) without express written approval of the sender, except to the extent required to further properly approved SNC business purposes, is strictly prohibited. If you are not the intended recipient of this email, please notify the sender immediately, and delete all copies without reading, printing, or saving in any manner. --- Thank You. _______________________________________________ discuss mailing list discuss(a)mpich.org<mailto:[email protected]> To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

MPI Window limit on mpich
by Marvin Smith 19 Sep '16

19 Sep '16

Good morning, I wanted to present an issue I am having with MPICH and validate whether this is a configuration problem, a limitation with MPICH, or a bug. I am writing an application which uses a large number of MPI windows, each window is given a relatively large amount of memory. This has never been a problem before, however we discovered if you allocate more than 2045 windows, you get an exception thrown. Notes: I am compiling using g++, version 4.8.5 on Red Hat Enterprise Linux version 7.2. My MPICH version is listed at the bottom of this email. It was installed via yum and is the RHEL default. I have attached sample output, to include the stdout/stderr. Also included is a Makefile and a simple example. The boundary of failure is between 2045 and 2046 windows. I have verified on my system this problem repeats even if I distribute windows between multiple communicators. I have not tested yet against ompi or mvapich. #-----------------------------------------------------------------------------------------------# #- Here is my output -# #-----------------------------------------------------------------------------------------------# mpirun -np 2 -hosts localhost ./mpi-win-test 2046 1 Initialized Rank: 0, Number Processors: 2, Hostname: test-machine Initialized Rank: 1, Number Processors: 2, Hostname: test-machine Fatal error in MPI_Win_create_dynamic: Other MPI error, error stack: MPI_Win_create_dynamic(154)..........: MPI_Win_create_dynamic(MPI_INFO_NULL, MPI_COMM_WORLD, win=0x10c1464) failed MPID_Win_create_dynamic(139).........: win_init(254)........................: MPIR_Comm_dup_impl(55)...............: MPIR_Comm_copy(1552).................: MPIR_Get_contextid(799)..............: MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID because of fragmentation (0/2048 free on this process; ignore_id=0) Fatal error in MPI_Win_create_dynamic: Other MPI error, error stack: MPI_Win_create_dynamic(154)..........: MPI_Win_create_dynamic(MPI_INFO_NULL, MPI_COMM_WORLD, win=0x19ef444) failed MPID_Win_create_dynamic(139).........: win_init(254)........................: MPIR_Comm_dup_impl(55)...............: MPIR_Comm_copy(1552).................: MPIR_Get_contextid(799)..............: MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID because of fragmentation (0/2048 free on this process; ignore_id=0) =================================================================================== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 1 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =================================================================================== make: *** [run] Error 1 #------------------------------------------------------------------------------------------# #- Here is my Sample Makefile -# #------------------------------------------------------------------------------------------# # Path to mpich on RHEL7 MPI_INCL=-I/usr/include/mpich-x86_64 MPI_LIBS=-L/usr/lib64/mpich/lib -lmpich # C++11 Bindings (Being lazy with string) CXX_ARGS=-std=c++11 # Make th test all: mpi-win-test mpi-win-test: mpi-win-test.cpp g++ $< -o $@ $(MPI_INCL) $(MPI_LIBS) $(CXX_ARGS) # Args for application NUM_WINDOWS=2046 USE_DYNAMIC=1 # Sample run usage # # Args: # - Number of Windows # - Type of windows (1 dynamic, 0 static) run: mpirun -np 2 -hosts localhost ./mpi-win-test $(NUM_WINDOWS) $(USE_DYNAMIC) #------------------------------------------------------------------------------------------# #- Here is my Sample Application -# #------------------------------------------------------------------------------------------# #include <mpi.h> #include <iostream> #include <string> #include <vector> using namespace std; int main( int argc, char* argv[] ) { // Number of MPI Windows int num_windows = std::stoi(argv[1]); bool use_dynamic = std::stoi(argv[2]); // Initialize MPI MPI_Init( &argc, &argv ); // Get the rank and size int rank, nprocs; MPI_Comm_size( MPI_COMM_WORLD, &nprocs ); MPI_Comm_rank( MPI_COMM_WORLD, &rank ); // Get the processor name char hostname[MPI_MAX_PROCESSOR_NAME]; int hostname_len; MPI_Get_processor_name( hostname, &hostname_len); // Print Message for( int i=0; i<nprocs; i++ ){ MPI_Barrier(MPI_COMM_WORLD); if( i == rank ){ std::cout << "Initialized Rank: " << rank << ", Number Processors: " << nprocs << ", Hostname: " << hostname << std::endl; } } // MPI Variables vector<MPI_Aint> sdisp_remotes(num_windows); vector<MPI_Aint> sdisp_locals(num_windows); // Create MPI Windows vector<MPI_Win> windows(num_windows); int64_t buffer_size = 1000; char* buffer = new char[buffer_size]; for( int i=0; i<num_windows; i++ ) { if( use_dynamic ) { MPI_Win_create_dynamic( MPI_INFO_NULL, MPI_COMM_WORLD, &windows[i] ); } else { MPI_Win_create( &buffer, buffer_size, 1, MPI_INFO_NULL, MPI_COMM_WORLD, &windows[i] ); } } // Exception always occurs prior to reaching this point. // More Code Here that I am removing for brevity // Wait at the barrier MPI_Barrier( MPI_COMM_WORLD ); // Remove all windows for( int i=0; i<num_windows; i++) { // Destroy the MPI Window MPI_Win_free( &windows[i] ); } windows.clear(); // Clear buffer delete [] buffer; buffer = nullptr; // Close MPI MPI_Finalize(); return 0; } #--------------------------------------------------------------------------# #- MPICH Version Output -# #--------------------------------------------------------------------------# MPICH Version: 3.0.4 MPICH Release date: Wed Apr 24 10:08:10 CDT 2013 MPICH Device: ch3:nemesis MPICH configure: --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --enable-sharedlibs=gcc --enable-shared --enable-lib-depend --disable-rpath --enable-fc --with-device=ch3:nemesis --with-pm=hydra:gforker --sysconfdir=/etc/mpich-x86_64 --includedir=/usr/include/mpich-x86_64 --bindir=/usr/lib64/mpich/bin --libdir=/usr/lib64/mpich/lib --datadir=/usr/share/mpich --mandir=/usr/share/man/mpich --docdir=/usr/share/mpich/doc --htmldir=/usr/share/mpich/doc --with-hwloc-prefix=system FC=gfortran F77=gfortran CFLAGS=-m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC CXXFLAGS=-m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC FCFLAGS=-m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC FFLAGS=-m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC LDFLAGS=-Wl,-z,noexecstack MPICH2LIB_CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic MPICH2LIB_CXXFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic MPICH2LIB_FCFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic MPICH2LIB_FFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic MPICH CC: cc -m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC -O2 MPICH CXX: c++ -m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC -O2 MPICH F77: gfortran -m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC -O2 MPICH FC: gfortran -m64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC -O2 CONFIDENTIALITY NOTICE - SNC EMAIL: This email and any attachments are confidential, may contain proprietary, protected, or export controlled information, and are intended for the use of the intended recipients only. Any review, reliance, distribution, disclosure, or forwarding of this email and/or attachments outside of Sierra Nevada Corporation (SNC) without express written approval of the sender, except to the extent required to further properly approved SNC business purposes, is strictly prohibited. If you are not the intended recipient of this email, please notify the sender immediately, and delete all copies without reading, printing, or saving in any manner. --- Thank You. _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

RES: ENC: Help me
by André Frederico Muller 12 Sep '16

12 Sep '16

Thank you Mr. Kenneth. I will make the right question, and post it later. -----Mensagem original----- De: Kenneth Raffenetti [mailto:[email protected]] Enviada em: segunda-feira, 12 de setembro de 2016 10:37 Para: discuss(a)mpich.org Assunto: Re: [mpich-discuss] ENC: Help me Hi, If you have a specific question about how to use MPICH or problems installing it, we can help. This list is not for general MPI or parallel programming questions. Ken On 09/12/2016 07:36 AM, André Frederico Muller wrote: > Hello MPICH TEAM. Im from Brazil. Could you help me with the MPICH-3.2 > configuration? > > > > I would like to run processes of DNA mapping between two servers with > parallel processing. MPICH could help with this?I need these two > servers working as one. > > > > > > Thank you in advance. > > > > *André Frederico Müller* > > afmuller(a)asav.org.br <mailto:[email protected]> > > > > Gerência de Tecnologia da Informação |*ASAV* Av. Unisinos, 950 | > Bairro Cristo Rei - CEP 93022 000 São Leopoldo | Rio Grande do Sul | > Brasil > > > *Aviso legal*: Esta mensagem eletrônica e seus respectivos anexos, > podem conter informações confidenciais. Se você não é o(a) > destinatário(a) > correto(a) e/ou o conteúdo do /e-mail/ não lhe diz respeito > notifique-nos, respondendo a este /e-mail/ com cópia para > esirc(a)asav.org.br <mailto:[email protected]>, e apague-o imediatamente. > Fica, desde já, notificado que qualquer ação baseada no conteúdo desta > mensagem é estritamente proibida e ilegal. > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss Aviso legal: Esta mensagem eletrônica e seus respectivos anexos, podem conter informações confidenciais. Se você não é o(a) destinatário(a) correto(a) e/ou o conteúdo do e-mail não lhe diz respeito notifique-nos, respondendo a este e-mail com cópia para esirc(a)asav.org.br<mailto:[email protected]>, e apague-o imediatamente. Fica, desde já, notificado que qualquer ação baseada no conteúdo desta mensagem é estritamente proibida e ilegal. _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] ENC: Help me
by Kenneth Raffenetti 12 Sep '16

12 Sep '16

Hi, If you have a specific question about how to use MPICH or problems installing it, we can help. This list is not for general MPI or parallel programming questions. Ken On 09/12/2016 07:36 AM, André Frederico Muller wrote: > Hello MPICH TEAM. Im from Brazil. Could you help me with the MPICH-3.2 > configuration? > > > > I would like to run processes of DNA mapping between two servers with > parallel processing. MPICH could help with this?I need these two servers > working as one. > > > > > > Thank you in advance. > > > > *André Frederico Müller* > > afmuller(a)asav.org.br <mailto:[email protected]> > > > > Gerência de Tecnologia da Informação |*ASAV* > Av. Unisinos, 950 | Bairro Cristo Rei - CEP 93022 000 > São Leopoldo | Rio Grande do Sul | Brasil > > > *Aviso legal*: Esta mensagem eletrônica e seus respectivos anexos, podem > conter informações confidenciais. Se você não é o(a) destinatário(a) > correto(a) e/ou o conteúdo do /e-mail/ não lhe diz respeito > notifique-nos, respondendo a este /e-mail/ com cópia para > esirc(a)asav.org.br <mailto:[email protected]>, e apague-o imediatamente. > Fica, desde já, notificado que qualquer ação baseada no conteúdo desta > mensagem é estritamente proibida e ilegal. > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

ENC: Help me
by André Frederico Muller 12 Sep '16

12 Sep '16

Hello MPICH TEAM. Im from Brazil. Could you help me with the MPICH-3.2 configuration? I would like to run processes of DNA mapping between two servers with parallel processing. MPICH could help with this? I need these two servers working as one. Thank you in advance. André Frederico Müller afmuller(a)asav.org.br<mailto:[email protected]> Gerência de Tecnologia da Informação | ASAV Av. Unisinos, 950 | Bairro Cristo Rei - CEP 93022 000 São Leopoldo | Rio Grande do Sul | Brasil Aviso legal: Esta mensagem eletrônica e seus respectivos anexos, podem conter informações confidenciais. Se você não é o(a) destinatário(a) correto(a) e/ou o conteúdo do e-mail não lhe diz respeito notifique-nos, respondendo a este e-mail com cópia para esirc(a)asav.org.br<mailto:[email protected]>, e apague-o imediatamente. Fica, desde já, notificado que qualquer ação baseada no conteúdo desta mensagem é estritamente proibida e ilegal. _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0