February 2015 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] [mvapich-discuss] Lustre 2.4 bug is back in MV2-2.1a
by Jonathan Perkins 26 Feb '15

26 Feb '15

On Thu, Feb 26, 2015 at 11:43:31AM -0600, Rob Latham wrote: > > > On 10/01/2014 12:25 PM, Jonathan Perkins wrote: > >On Wed, Oct 01, 2014 at 11:25:33AM -0500, Rob Latham wrote: > >> > >> > >>On 10/01/2014 10:33 AM, Jonathan Perkins wrote: > >>>Hello MPICH community. Recently we had a report of a regression on the > >>>MVAPICH discuss list and I've found that the regression appears in > >>>MPICH-3.1.2 itself. > >>> > >>>We plan on just removing the _XOPEN_SOURCE lines but thought that you'd > >>>like to know about this so that it could be fixed in MPICH as well. > >>>Thanks! > >> > >>so, on solaris we needed to put back _XOPEN_SOURCE lines with > >>--enable-strict in order to bring in pread/pwrite declarations, I think. > >>but i had forgotten about the caddr_t time compile issue. > >> > >>thanks for the report. > > > >Ah, thanks for that info. I figured that there was a chance that the > >proper fix might be a little bit more involved. We'll keep our eye on > >how you guys deal with this. Thanks! > > took me long enough, but I've pushed some fixes to romio that should put > this to rest once and for all. no more mucking with XOPEN_SOURCE in ROMIO. > fallback versions of pread/pwrite: > - http://git.mpich.org/mpich.git/commit/4e80e1d2b > - http://git.mpich.org/mpich.git/commit/5a10283bf Thanks for following up with us. These fixes look good and we'll replace our changes with these when we next update with the version of MPICH containing this. -- Jonathan Perkins _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] [mvapich-discuss] Lustre 2.4 bug is back in MV2-2.1a
by Rob Latham 26 Feb '15

26 Feb '15

On 10/01/2014 12:25 PM, Jonathan Perkins wrote: > On Wed, Oct 01, 2014 at 11:25:33AM -0500, Rob Latham wrote: >> >> >> On 10/01/2014 10:33 AM, Jonathan Perkins wrote: >>> Hello MPICH community. Recently we had a report of a regression on the >>> MVAPICH discuss list and I've found that the regression appears in >>> MPICH-3.1.2 itself. >>> >>> We plan on just removing the _XOPEN_SOURCE lines but thought that you'd >>> like to know about this so that it could be fixed in MPICH as well. >>> Thanks! >> >> so, on solaris we needed to put back _XOPEN_SOURCE lines with >> --enable-strict in order to bring in pread/pwrite declarations, I think. >> but i had forgotten about the caddr_t time compile issue. >> >> thanks for the report. > > Ah, thanks for that info. I figured that there was a chance that the > proper fix might be a little bit more involved. We'll keep our eye on > how you guys deal with this. Thanks! took me long enough, but I've pushed some fixes to romio that should put this to rest once and for all. no more mucking with XOPEN_SOURCE in ROMIO. fallback versions of pread/pwrite: - http://git.mpich.org/mpich.git/commit/4e80e1d2b - http://git.mpich.org/mpich.git/commit/5a10283bf ==rob -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] [OMPI devel] ROMIO+Lustre problems in OpenMPI 1.8.3
by Rob Latham 26 Feb '15

26 Feb '15

On 11/07/2014 06:26 AM, Ralph Castain wrote: > Hi Rob > > Following up on this: I cannot find any reference to XOPEN_SOURCE in our included ROMIO source for Lustre. I only found one reference anywhere in ROMIO: > > romio/adio/ad_xfs/ad_xfs.h:11:#define _XOPEN_SOURCE 500 > > Any other suggestions on what could be causing the problem? I've fixed this in ROMIO by not mucking around with XOPEN_SOURCE at all, in either lustre or xfs or anywhere. http://git.mpich.org/mpich.git/commit/4e80e1d2b and http://git.mpich.org/mpich.git/commit/5a10283bf7 ==rob > > Thanks > Ralph > > >> On Oct 28, 2014, at 7:32 AM, Rob Latham <robl(a)mcs.anl.gov> wrote: >> >> >> >> On 10/28/2014 06:00 AM, Paul Kapinos wrote: >>> Dear Open MPI and ROMIO developer, >>> >>> We use Open MPI v.1.6.x and 1.8.x in our cluster. >>> We have Lustre file system; we wish to use MPI_IO. >>> So the OpenMPI's are compiled with this flag: >>>> --with-io-romio-flags='--with-file-system=testfs+ufs+nfs+lustre' >>> >>> In our newest installation openmpi/1.8.3 we found that MPI_IO is *broken*. >>> >>> Short seek for root of the evil bring the following to light: >>> >>> - the ROMIO component 'MCA io: romio' isn't here at all in the affected >>> version, because >>> >>> - configure of ROMIO has *failed* (cf. logs (a,b,c). >>> - because lustre_user.h was found but could not be compiled. >> >> lustre_user.h cannot be compiled because quota defines won't compile. Ugh, what a mess. >> >> A while back I noticed this and fixed it by removing an XOPEN_SOURCE feature test macro: >> >> http://trac.mpich.org/projects/mpich/ticket/1973 >> >> Then, on solaris with --enable-strict we needed to put *back* the XOPEN_SOURCE macro or else pread and pwrite would be undefined. >> >> So what I really need to to is delete XOPEN_SOURCE since it causes such headaches, and on the rare platforms that only have pread/pwrite defined if you take extraordinary measures, if at all, I'll have a ROMIO pread and pwrite that simply do seek + write (or read). >> >> For now, please delete the XOPEN_SOURCE line at the very beginning of src/mpi/romio/adio/ad_lustre/ad_lustre_rwcontig.c >> >> ==rob >> >> >>> >>> >>> In our system, there are two lustre_user.h available: >>> $ locate lustre_user.h >>> /usr/include/linux/lustre_user.h >>> /usr/include/lustre/lustre_user.h >>> As I'm not very convinient with lustre, I just attach both of them. >>> >>> pk224850@cluster:~[509]$ uname -a >>> Linux cluster.rz.RWTH-Aachen.DE 2.6.32-431.29.2.el6.x86_64 #1 SMP Tue >>> Sep 9 13:45:55 CDT 2014 x86_64 x86_64 x86_64 GNU/Linux >>> >>> pk224850@cluster:~[510]$ cat /etc/issue >>> Scientific Linux release 6.5 (Carbon) >>> >>> Note that openmpi/1.8.1 seem to be fully OK (MPI_IO works) in our >>> environment. >>> >>> Best >>> >>> Paul Kapinos >>> >>> P.S. Is there a confugure flag, which will enforce ROMIO? That is when >>> ROMIO not available, configure would fail. This would make such hidden >>> errors publique at installation time.. >>> >>> >>> >>> >>> >>> >>> a) Log in Open MPI's config.log: >>> ------------------------------------------------------------------------------ >>> >>> configure:226781: OMPI configuring in ompi/mca/io/romio/romio >>> configure:226866: running /bin/sh './configure' >>> --with-file-system=testfs+ufs+nfs+lustre FROM_OMPI=yes CC="icc >>> -std=c99" CFLAGS="-DNDEBUG -O3 -ip -axAVX,SSE4.2,SSE4.1 -fp-model fast=2 >>> -m64 -finline-functions -fno-strict-aliasing -restrict -fexceptions >>> -Qoption,cpp,--extended_float_types -pthread" CPPFLAGS=" >>> -I/w0/tmp/pk224850/linuxc2_9713/openmpi-1.8.3_linux64_intel/opal/mca/hwloc/hwloc172/hwloc/include >>> -I/w0/tmp/pk224850/linuxc2_9713/openmpi-1.8.3_linux64_intel/opal/mca/event/libevent2021/libevent >>> -I/w0/tmp/pk224850/linuxc2_9713/openmpi-1.8.3_linux64_intel/opal/mca/event/libevent2021/libevent/include" >>> FFLAGS="-O3 -ip -axAVX,SSE4.2,SSE4.1 -fp-model fast=2 -m64 " >>> LDFLAGS="-O3 -ip -axAVX,SSE4.2,SSE4.1 -fp-model fast=2 -m64 >>> -fexceptions " --enable-shared --disable-static >>> --with-file-system=testfs+ufs+nfs+lustre >>> --prefix=/opt/MPI/openmpi-1.8.3/linux/intel --disable-aio >>> --cache-file=/dev/null --srcdir=. --disable-option-checking >>> configure:226876: /bin/sh './configure' *failed* for >>> ompi/mca/io/romio/romio >>> configure:226911: WARNING: ROMIO distribution did not configure >>> successfully >>> configure:227425: checking if MCA component io:romio can compile >>> configure:227427: result: no >>> ------------------------------------------------------------------------------ >>> >>> >>> >>> >>> b) dump of Open MPI's 'configure' output to the console: >>> ------------------------------------------------------------------------------ >>> >>> checking lustre/lustre_user.h usability... no >>> checking lustre/lustre_user.h presence... yes >>> configure: WARNING: lustre/lustre_user.h: present but cannot be compiled >>> configure: WARNING: lustre/lustre_user.h: check for missing >>> prerequisite headers? >>> configure: WARNING: lustre/lustre_user.h: see the Autoconf documentation >>> configure: WARNING: lustre/lustre_user.h: section "Present But >>> Cannot Be Compiled" >>> configure: WARNING: lustre/lustre_user.h: proceeding with the compiler's >>> result >>> configure: WARNING: ## -------------------------------- ## >>> configure: WARNING: ## Report this to discuss(a)mpich.org ## >>> configure: WARNING: ## -------------------------------- ## >>> checking for lustre/lustre_user.h... no >>> configure: error: LUSTRE support requested but cannot find >>> lustre/lustre_user.h header file >>> configure: /bin/sh './configure' *failed* for ompi/mca/io/romio/romio >>> configure: WARNING: ROMIO distribution did not configure successfully >>> checking if MCA component io:romio can compile... no >>> ------------------------------------------------------------------------------ >>> >>> >>> c) ompi/mca/io/romio/romio's config.log: >>> ------------------------------------------------------------------------------ >>> >>> configure:20962: checking lustre/lustre_user.h usability >>> configure:20962: icc -std=c99 -c -DNDEBUG -O3 -ip -axAVX,SSE4.2,SSE4.1 >>> -fp-model fast=2 -m64 -finline-functions -fno-strict-aliasing -restrict >>> -fexceptions -Qoption,cpp,--extended_float_types -pthread >>> -I/w0/tmp/pk224850/linuxc2_9713/openmpi-1.8.3_linux64_intel/opal/mca/hwloc/hwloc172/hwloc/include >>> -I/w0/tmp/pk224850/linuxc2_9713/openmpi-1.8.3_linux64_intel/opal/mca/event/libevent2021/libevent >>> -I/w0/tmp/pk224850/linuxc2_9713/openmpi-1.8.3_linux64_intel/opal/mca/event/libevent2021/libevent/include >>> conftest.c >&5 >>> /usr/include/sys/quota.h(221): error: identifier "caddr_t" is undefined >>> caddr_t __addr) __THROW; >>> ^ >>> >>> compilation aborted for conftest.c (code 2) >>> configure:20962: $? = 2 >>> configure: failed program was: >>> | /* confdefs.h */ >>> | #define PACKAGE_NAME "ROMIO" >>> | #define PACKAGE_TARNAME "romio" >>> | #define PACKAGE_VERSION "Open MPI" >>> | #define PACKAGE_STRING "ROMIO Open MPI" >>> | #define PACKAGE_BUGREPORT "discuss(a)mpich.org" >>> | #define PACKAGE_URL "http://www.mpich.org/" >>> | #define PACKAGE "romio" >>> | #define VERSION "Open MPI" >>> | #define STDC_HEADERS 1 >>> | #define HAVE_SYS_TYPES_H 1 >>> | #define HAVE_SYS_STAT_H 1 >>> | #define HAVE_STDLIB_H 1 >>> | #define HAVE_STRING_H 1 >>> | #define HAVE_MEMORY_H 1 >>> | #define HAVE_STRINGS_H 1 >>> | #define HAVE_INTTYPES_H 1 >>> | #define HAVE_STDINT_H 1 >>> | #define HAVE_UNISTD_H 1 >>> | #define HAVE_DLFCN_H 1 >>> | #define LT_OBJDIR ".libs/" >>> | #define HAVE_MPI_OFFSET 1 >>> | #define HAVE_MEMALIGN 1 >>> | #define HAVE_UNISTD_H 1 >>> | #define HAVE_FCNTL_H 1 >>> | #define HAVE_MALLOC_H 1 >>> | #define HAVE_STDDEF_H 1 >>> | #define HAVE_SYS_TYPES_H 1 >>> | #define u_char unsigned char >>> | #define u_short unsigned short >>> | #define u_int unsigned int >>> | #define u_long unsigned long >>> | #define SIZEOF_INT 4 >>> | #define SIZEOF_VOID_P 8 >>> | #define INT_LT_POINTER 1 >>> | #define HAVE_INT_LT_POINTER 1 >>> | #define SIZEOF_LONG_LONG 8 >>> | #define HAVE_LONG_LONG_64 1 >>> | #define HAVE_MPI_LONG_LONG_INT 1 >>> | #define HAVE_MPI_INFO 1 >>> | #define ROMIO_NFS 1 >>> | #define ROMIO_UFS 1 >>> | #define ROMIO_TESTFS 1 >>> | /* end confdefs.h. */ >>> | #include <stdio.h> >>> | #ifdef HAVE_SYS_TYPES_H >>> | # include <sys/types.h> >>> | #endif >>> | #ifdef HAVE_SYS_STAT_H >>> | # include <sys/stat.h> >>> | #endif >>> | #ifdef STDC_HEADERS >>> | # include <stdlib.h> >>> | # include <stddef.h> >>> | #else >>> | # ifdef HAVE_STDLIB_H >>> | # include <stdlib.h> >>> | # endif >>> | #endif >>> | #ifdef HAVE_STRING_H >>> | # if !defined STDC_HEADERS && defined HAVE_MEMORY_H >>> | # include <memory.h> >>> | # endif >>> | # include <string.h> >>> | #endif >>> | #ifdef HAVE_STRINGS_H >>> | # include <strings.h> >>> | #endif >>> | #ifdef HAVE_INTTYPES_H >>> | # include <inttypes.h> >>> | #endif >>> | #ifdef HAVE_STDINT_H >>> | # include <stdint.h> >>> | #endif >>> | #ifdef HAVE_UNISTD_H >>> | # include <unistd.h> >>> | #endif >>> | #include <lustre/lustre_user.h> >>> configure:20962: result: no >>> configure:20962: checking lustre/lustre_user.h presence >>> configure:20962: icc -std=c99 -E >>> -I/w0/tmp/pk224850/linuxc2_9713/openmpi-1.8.3_linux64_intel/opal/mca/hwloc/hwloc172/hwloc/include >>> -I/w0/tmp/pk224850/linuxc2_9713/openmpi-1.8.3_linux64_intel/opal/mca/event/libevent2021/libevent >>> -I/w0/tmp/pk224850/linuxc2_9713/openmpi-1.8.3_linux64_intel/opal/mca/event/libevent2021/libevent/include >>> conftest.c >>> configure:20962: $? = 0 >>> configure:20962: result: yes >>> configure:20962: WARNING: lustre/lustre_user.h: present but cannot be >>> compiled >>> configure:20962: WARNING: lustre/lustre_user.h: check for missing >>> prerequisite headers? >>> configure:20962: WARNING: lustre/lustre_user.h: see the Autoconf >>> documentation >>> configure:20962: WARNING: lustre/lustre_user.h: section "Present But >>> Cannot Be Compiled" >>> configure:20962: WARNING: lustre/lustre_user.h: proceeding with the >>> compiler's result >>> configure:20962: checking for lustre/lustre_user.h >>> configure:20962: result: no >>> configure:20971: error: LUSTRE support requested but cannot find >>> lustre/lustre_user.h header file >>> ------------------------------------------------------------------------------ >>> >>> >>> >>> >>> >>> >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >>> >> >> -- >> Rob Latham >> Mathematics and Computer Science Division >> Argonne National Lab, IL USA >> _______________________________________________ >> devel mailing list >> devel(a)open-mpi.org >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel >> Link to this post: http://www.open-mpi.org/community/lists/devel/2014/10/16109.php > -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPI_Finalize hangs in dynamic connection in case of failed process
by Wesley Bland 26 Feb '15

26 Feb '15

First, I believe the sock device is untested with most of the MPICH fault tolerance features, so YMMV here. Is there a reason that you aren’t calling MPI_Disconnect for the failed process? Did you try it an something bad happened? That seems like the most straightforward way of doing things. Otherwise, this sounds like a known issue that we’re seeing from time to time with MPI_Finalize and the FT work. It’s something I’m trying to figure out now. If you can reduce your code down to the minimum and send it to me, I can use it as a test case to try to fix the problem. Thanks, Wesley > On Feb 19, 2015, at 5:15 AM, Roy, Hirak <Hirak_Roy(a)mentor.com> wrote: > > Hi All, > > I am using MPICH with sock connection. > I also setup processes using dynamic connection method (MPI_Comm_connect/MPI_Comm_accept). It’s a master-slave architecture where master accepts the connections from slaves. > > Now if one of the process dies (or get killed), I can still recover from this (without using checkpoint/restore method). > For the particular process in master, I do not call MPI_disconnect (it hangs and does not complete). > As a result, my MPI_Finalize in master hangs and does not complete. > Do you have a workaround to forcefully complete MPI_Finalize or MPI_disconnect? > I tried MPI_Comm_free on the failed connection. However, it does not solve the hang in finalize. > > Thanks, > Hirak > _______________________________________________ > discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss <https://lists.mpich.org/mailman/listinfo/discuss> _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Is there any optimization of collective calls (MPI_Allreduce) for 2^n ranks?
by Aiman Fang 26 Feb '15

26 Feb '15

Thank you all for your help. We were doing experiments and somehow needed to compare the performance of power-of-two and nonpower-of-two processes in collective communication. After adding MPI_Barrier before MPI_Allreduce, we did observe improvement in performance of nonpower-of-two processes. Best, Aiman On Feb 25, 2015, at 8:58 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > See papers by Sameer Kumar and coworkers on Google Scholar for > details. You need to look at the pamid device code in MPICH to see > the collectives that are actually used on BGQ, but that won't actually > help since the all the BGQ collective magic is usually inside of PAMI. > PAMI is open-source but not exactly easy to understand. > > Unlike previous generation of Blue Gene, BGQ collective optimizations > are much more general and often appear for communicator sizes other > than 2^n. > > From my experiments when I was at ALCF, you may find that calling > MPI_Barrier before MPI_Allreduce can improve performance > significantly. > > This is hardly the full story. It would be useful to know more about > what you are trying to accomplish. > > Best, > > Jeff > > On Wed, Feb 25, 2015 at 3:07 PM, Junchao Zhang <jczhang(a)mcs.anl.gov> wrote: >> Yes. Many collectives have optimizations for power-of-two processes. In >> MPICH's source code allreduce.c, you can find the following comments. >> >> /* This is the default implementation of allreduce. The algorithm is: >> >> Algorithm: MPI_Allreduce >> >> For the heterogeneous case, we call MPI_Reduce followed by MPI_Bcast >> in order to meet the requirement that all processes must have the >> same result. For the homogeneous case, we use the following algorithms. >> >> >> For long messages and for builtin ops and if count >= pof2 (where >> pof2 is the nearest power-of-two less than or equal to the number >> of processes), we use Rabenseifner's algorithm (see >> http://www.hlrs.de/mpi/myreduce.html) >> This algorithm implements the allreduce in two steps: first a >> reduce-scatter, followed by an allgather. A recursive-halving >> algorithm (beginning with processes that are distance 1 apart) is >> used for the reduce-scatter, and a recursive doubling >> algorithm is used for the allgather. The non-power-of-two case is >> handled by dropping to the nearest lower power-of-two: the first >> few even-numbered processes send their data to their right neighbors >> (rank+1), and the reduce-scatter and allgather happen among the remaining >> power-of-two processes. At the end, the first few even-numbered >> processes get the result from their right neighbors. >> >> For the power-of-two case, the cost for the reduce-scatter is >> lgp.alpha + n.((p-1)/p).beta + n.((p-1)/p).gamma. The cost for the >> allgather lgp.alpha + n.((p-1)/p).beta. Therefore, the >> total cost is: >> Cost = 2.lgp.alpha + 2.n.((p-1)/p).beta + n.((p-1)/p).gamma >> >> For the non-power-of-two case, >> Cost = (2.floor(lgp)+2).alpha + (2.((p-1)/p) + 2).n.beta + >> n.(1+(p-1)/p).gamma >> >> >> For short messages, for user-defined ops, and for count < pof2 >> we use a recursive doubling algorithm (similar to the one in >> MPI_Allgather). We use this algorithm in the case of user-defined ops >> because in this case derived datatypes are allowed, and the user >> could pass basic datatypes on one process and derived on another as >> long as the type maps are the same. Breaking up derived datatypes >> to do the reduce-scatter is tricky. >> >> Cost = lgp.alpha + n.lgp.beta + n.lgp.gamma >> >> Possible improvements: >> >> End Algorithm: MPI_Allreduce >> */ >> >> --Junchao Zhang >> >> On Wed, Feb 25, 2015 at 2:59 PM, Aiman Fang <aimanf(a)cs.uchicago.edu> wrote: >>> >>> Hi, >>> >>> I came across a problem in experiments that makes me wondering if there is >>> any optimization of collective calls, such as MPI_Allreduce, for 2^n number >>> of ranks? >>> >>> We did some experiments on Argonne Vesta system to measure the time of >>> MPI_Allreduce calls using 511, 512 and 513 processes. (one process per >>> node). In each run, the synthetic benchmark first did some computation and >>> then called MPI_Allreduce 30 times, for total 100 loops. We measured the >>> total time spent on communication. >>> >>> We found that 512-process run gives the best performance. The time for >>> 511, 512 and 513 processes are 0.1492, 0.1449 and 0.1547 seconds >>> respectively. 512-proc outperforms 511-proc by 3.7%, and 513-proc by 6.7%. >>> >>> The mpich version we used is as follows. >>> >>> $ mpichversion >>> MPICH Version: 3.1.2 >>> MPICH Release date: Mon Jul 21 16:00:21 CDT 2014 >>> MPICH Device: pamid >>> MPICH configure: --prefix=/home/fujita/soft/mpich-3.1.2 >>> --host=powerpc64-bgq-linux --with-device=pamid --with-file-system=gpfs:BGQ >>> --disable-wrapper-rpath >>> MPICH CC: powerpc64-bgq-linux-gcc -O2 >>> MPICH CXX: powerpc64-bgq-linux-g++ -O2 >>> MPICH F77: powerpc64-bgq-linux-gfortran -O2 >>> MPICH FC: powerpc64-bgq-linux-gfortran -O2 >>> >>> Thanks! >>> >>> Best, >>> Aiman >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] mpich-3.1.4 error in mx_init (error no mx device entry in /dev
by "Antonio J. Peña" 26 Feb '15

26 Feb '15

No problem, I'm glad it worked. Antonio On 02/26/2015 06:31 AM, Khalid Hasanov wrote: > Hi Antonio, > > Please ignore my request. It was my bad. I've found my mistake, I > compiled with Open MPI but run with MPICH. It works fine now. > > Sorry for any inconveniences. > > Regards, > Khalid > > On Thu, Feb 26, 2015 at 2:44 AM, Khalid Hasanov <xalid.h(a)gmail.com > <mailto:[email protected]>> wrote: > > Hi Antonio, > > I have attached the log file. > > Thanks. > > Khalid > > > > On Thu, Feb 26, 2015 at 2:16 AM, Antonio J. Peña <apenya(a)anl.gov > <mailto:[email protected]>> wrote: > > Hi Khalid, > > Can you send us your config.log file? > > Antonio > > -------------------------------------------------------------- > > Antonio J. Peña > > Postdoctoral Appointee > > Mathematics and Computer Science Division (MCS) > > Argonne National Laboratory > > 9700 S. Cass Avenue > > Argonne, IL 60439 > > apenya(a)anl.gov <mailto:[email protected]> > > *From:*Khalid Hasanov [mailto:[email protected] > <mailto:[email protected]>] > *Sent:* Wednesday, February 25, 2015 8:15 PM > *To:* discuss(a)mpich.org <mailto:[email protected]> > *Subject:* [mpich-discuss] mpich-3.1.4 error in mx_init (error > no mx device entry in /dev > > Hello, > > I have an issue with mpich-3.1.4 when I run mpirun on a > cluster which has Infiniband-20G and Gigabit Ethernet networks: > error in mx_init (error no mx device entry in /dev > > As far as I know in Open MPI exactly the same error message is > considered as a warning which may come from collisions between > IB and MYRINET technologies and these warnings can be disabled > by MCA parameters. > > Is there any similar option in mpich3 to disable/solve the error? > > Best regards, > > Khalid > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Antonio J. Peña Postdoctoral Appointee Mathematics and Computer Science Division Argonne National Laboratory 9700 South Cass Avenue, Bldg. 240, Of. 3148 Argonne, IL 60439-4847 apenya(a)mcs.anl.gov www.mcs.anl.gov/~apenya _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] mpich-3.1.4 error in mx_init (error no mx device entry in /dev
by Khalid Hasanov 26 Feb '15

26 Feb '15

Hi Antonio, Please ignore my request. It was my bad. I've found my mistake, I compiled with Open MPI but run with MPICH. It works fine now. Sorry for any inconveniences. Regards, Khalid On Thu, Feb 26, 2015 at 2:44 AM, Khalid Hasanov <xalid.h(a)gmail.com> wrote: > Hi Antonio, > > I have attached the log file. > > Thanks. > > Khalid > > > > On Thu, Feb 26, 2015 at 2:16 AM, Antonio J. Peña <apenya(a)anl.gov> wrote: > >> >> >> Hi Khalid, >> >> >> >> Can you send us your config.log file? >> >> >> >> Antonio >> >> >> >> -------------------------------------------------------------- >> >> Antonio J. Peña >> >> Postdoctoral Appointee >> >> Mathematics and Computer Science Division (MCS) >> >> Argonne National Laboratory >> >> 9700 S. Cass Avenue >> >> Argonne, IL 60439 >> >> apenya(a)anl.gov >> >> >> >> >> >> *From:* Khalid Hasanov [mailto:[email protected]] >> *Sent:* Wednesday, February 25, 2015 8:15 PM >> *To:* discuss(a)mpich.org >> *Subject:* [mpich-discuss] mpich-3.1.4 error in mx_init (error no mx >> device entry in /dev >> >> >> >> Hello, >> >> I have an issue with mpich-3.1.4 when I run mpirun on a cluster which has >> Infiniband-20G and Gigabit Ethernet networks: >> error in mx_init (error no mx device entry in /dev >> >> As far as I know in Open MPI exactly the same error message is considered >> as a warning which may come from collisions between IB and MYRINET >> technologies and these warnings can be disabled by MCA parameters. >> >> Is there any similar option in mpich3 to disable/solve the error? >> >> >> >> Best regards, >> >> Khalid >> >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> > > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Is there any optimization of collective calls (MPI_Allreduce) for 2^n ranks?
by Jeff Hammond 25 Feb '15

25 Feb '15

See papers by Sameer Kumar and coworkers on Google Scholar for details. You need to look at the pamid device code in MPICH to see the collectives that are actually used on BGQ, but that won't actually help since the all the BGQ collective magic is usually inside of PAMI. PAMI is open-source but not exactly easy to understand. Unlike previous generation of Blue Gene, BGQ collective optimizations are much more general and often appear for communicator sizes other than 2^n.

1 0

Re: [mpich-discuss] mpich-3.1.4 error in mx_init (error no mx device entry in /dev
by Khalid Hasanov 25 Feb '15

25 Feb '15

Hi Antonio, I have attached the log file. Thanks. Khalid On Thu, Feb 26, 2015 at 2:16 AM, Antonio J. Peña <apenya(a)anl.gov> wrote: > > > Hi Khalid, > > > > Can you send us your config.log file? > > > > Antonio > > > > -------------------------------------------------------------- > > Antonio J. Peña > > Postdoctoral Appointee > > Mathematics and Computer Science Division (MCS) > > Argonne National Laboratory > > 9700 S. Cass Avenue > > Argonne, IL 60439 > > apenya(a)anl.gov > > > > > > *From:* Khalid Hasanov [mailto:[email protected]] > *Sent:* Wednesday, February 25, 2015 8:15 PM > *To:* discuss(a)mpich.org > *Subject:* [mpich-discuss] mpich-3.1.4 error in mx_init (error no mx > device entry in /dev > > > > Hello, > > I have an issue with mpich-3.1.4 when I run mpirun on a cluster which has > Infiniband-20G and Gigabit Ethernet networks: > error in mx_init (error no mx device entry in /dev > > As far as I know in Open MPI exactly the same error message is considered > as a warning which may come from collisions between IB and MYRINET > technologies and these warnings can be disabled by MCA parameters. > > Is there any similar option in mpich3 to disable/solve the error? > > > > Best regards, > > Khalid > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] mpich-3.1.4 error in mx_init (error no mx device entry in /dev
by Antonio J. Peña 25 Feb '15

25 Feb '15

Hi Khalid, Can you send us your config.log file? Antonio -------------------------------------------------------------- Antonio J. Peña Postdoctoral Appointee Mathematics and Computer Science Division (MCS) Argonne National Laboratory 9700 S. Cass Avenue Argonne, IL 60439 apenya(a)anl.gov From: Khalid Hasanov [mailto:[email protected]] Sent: Wednesday, February 25, 2015 8:15 PM To: discuss(a)mpich.org Subject: [mpich-discuss] mpich-3.1.4 error in mx_init (error no mx device entry in /dev Hello, I have an issue with mpich-3.1.4 when I run mpirun on a cluster which has Infiniband-20G and Gigabit Ethernet networks: error in mx_init (error no mx device entry in /dev As far as I know in Open MPI exactly the same error message is considered as a warning which may come from collisions between IB and MYRINET technologies and these warnings can be disabled by MCA parameters. Is there any similar option in mpich3 to disable/solve the error? Best regards, Khalid _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0