July 2013 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] mpich on Mac os x
by Jeff Hammond 08 Jul '13

08 Jul '13

Is this the FLASH astrophysics code developed at UC? That code is run with MPICH all the time without issues so one has to assume that your program usage is at least partly causing the problem. Jeff On Mon, Jul 8, 2013 at 9:17 AM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: > > Also make sure you are freeing unused communicators. > > -- Pavan > > On 07/08/2013 08:14 AM, Wesley Bland wrote: >> >> It seems that you're creating more communicators than MPICH can handle. >> It's possible that you might be able to get around this by setting the >> environment variable MPIR_PARAM_CTXID_EAGER_SIZE to something smaller than >> its default (which is 2). That frees up a few more communicators, but there >> is a pathological case where even with fewer communicators than the max, >> MPICH won't be able to agree on a new communicator id when needed. Try >> changing that environment variable and see if that fixes things. >> >> Wesley >> >> On Jul 8, 2013, at 5:33 AM, Reem Alraddadi <raba500(a)york.ac.uk> wrote: >> >>> Hi all, >>> I am using mpich-3.0.4 on Mac os x version 10.7.5 to run FLASH code. It >>> works fine in the beginning of the run and then I got the following error: >>> >>> Fatal error in MPI_Comm_create: Other MPI error, error stack: >>> MPI_Comm_create(600).................: MPI_Comm_create(comm=0x84000002, >>> group=0xc8001349, new_comm=0x7fff606a8614) failed >>> MPI_Comm_create(577).................: >>> MPIR_Comm_create_intra(241)..........: >>> MPIR_Get_contextid(799)..............: >>> MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID >>> because of fragmentation (169/2048 free on this process; ignore_id=0) >>> Fatal error in MPI_Comm_create: Other MPI error, error stack: >>> MPI_Comm_create(600).................: MPI_Comm_create(comm=0x84000002, >>> group=0xc80012b6, new_comm=0x7fff670cc614) failed >>> MPI_Comm_create(577).................: >>> MPIR_Comm_create_intra(241)..........: >>> MPIR_Get_contextid(799)..............: >>> MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID >>> because of fragmentation (316/2048 free on this process; ignore_id=0) >>> Fatal error in MPI_Comm_create: Other MPI error, error stack: >>> MPI_Comm_create(600).................: MPI_Comm_create(comm=0x84000004, >>> group=0xc800000e, new_comm=0x7fff629d5614) failed >>> MPI_Comm_create(577).................: >>> MPIR_Comm_create_intra(241)..........: >>> MPIR_Get_contextid(799)..............: >>> MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID >>> because of fragmentation (2020/2048 free on this process; ignore_id=0) >>> Fatal error in MPI_Comm_create: Other MPI error, error stack: >>> MPI_Comm_create(600).................: MPI_Comm_create(comm=0x84000002, >>> group=0xc8000020, new_comm=0x7fff639ae614) failed >>> MPI_Comm_create(577).................: >>> MPIR_Comm_create_intra(241)..........: >>> MPIR_Get_contextid(799)..............: >>> MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID >>> because of fragmentation (2002/2048 free on this process; ignore_id=0 >>> >>> Is there a way to fix that ? >>> >>> Thanks, >>> Reem >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Jeff Hammond jeff.science(a)gmail.com

1 0

Re: [mpich-discuss] "undefined references"
by Jeff Hammond 08 Jul '13

08 Jul '13

I built Siesta against MPICH a week or two ago, albeit on BGQ (https://wiki.alcf.anl.gov/parts/index.php/SIESTA) and had no issues. What is the result of "mpif90 -show"? I wonder if you are mixing MPI builds or otherwise have a problem with your F90 configuration. Jeff On Mon, Jul 8, 2013 at 9:33 AM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: > > This doesn't look like an MPI error. Did you contact the authors of > whatever package you are trying to compile? > > -- Pavan > > On 07/08/2013 08:25 AM, Hana Milani wrote: >> >> Hello, >> >> I have compiled a code (siesta) by mpich2, blacs, lapack, blas and >> scalapack. It works fine under grid computation. But when I want to >> "make" a utility, I receive the following errors: >> >> [Home@localhost Gen-basis]$ make >> /usr/local/bin/mpif90 -o gen-basis \ >> alloc.o basis_types.o precision.o recipes.o parallel.o >> parsing.o xcmod.o basis_io.o chemical.o atm_transfer.o atm_types.o >> atmparams.o old_atmfuncs.o radial.o io.o paste.o flib_spline.o >> bessph.o m_filter.o basis_specs.o atom.o memoryinfo.o memory.o >> periodic_table.o pseudopotential.o pxf.o dot.o xc.o atom_options.o >> arw.o sys.o timer.o xml.o m_walltime.o gen-basis.o nag.o libfdf.a >> /home/Home/Packages/scalapack-2.0.2/libscalapack.a >> /home/Home/Packages/BLACS/LIB/blacsCinit_MPI-LINUX-0.a >> /home/Home/Packages/BLACS/LIB/blacsF77init_MPI-LINUX-0.a >> /home/Home/Packages/BLACS/LIB/blacs_MPI-LINUX-0.a >> /home/Home/Packages/lapack-3.4.2/liblapack.a >> /home/Home/Packages/BLAS/blas_LINUX.a >> alloc.o: In function `print_report': >> >> /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/alloc.F90:1705: >> undefined reference to `__mpi__r8_s_MOD_mpi_reduce_t' >> >> /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/alloc.F90:1707: >> undefined reference to `__mpi__r8_v_MOD_mpi_reduce_t' >> >> /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/alloc.F90:1711: >> undefined reference to `__mpi__integer_s_MOD_mpi_bcast_t' >> >> /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/alloc.F90:1723: >> undefined reference to `__mpi__r8_s_MOD_mpi_recv_t' >> >> /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/alloc.F90:1718: >> undefined reference to `__mpi__r8_s_MOD_mpi_send_t' >> alloc.o: In function `__alloc_MOD_alloc_report': >> >> /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/alloc.F90:318: >> undefined reference to `__mpi__integer_s_MOD_mpi_bcast_t' >> >> /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/alloc.F90:319: >> undefined reference to `__mpi__integer_s_MOD_mpi_bcast_t' >> >> /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/alloc.F90:320: >> undefined reference to `__mpi__character_s_MOD_mpi_bcast_t' >> xcmod.o: In function `__xcmod_MOD_setxc': >> /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/xcmod.F:136: >> undefined >> reference to `__mpi__integer_s_MOD_mpi_bcast_t' >> /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/xcmod.F:138: >> undefined >> reference to `__mpi__character_v_MOD_mpi_bcast_t' >> /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/xcmod.F:140: >> undefined >> reference to `__mpi__character_v_MOD_mpi_bcast_t' >> /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/xcmod.F:142: >> undefined >> reference to `__mpi__r8_v_MOD_mpi_bcast_t' >> /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/xcmod.F:144: >> undefined >> reference to `__mpi__r8_v_MOD_mpi_bcast_t' >> memory.o: In function `printmemory_': >> /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/memory.F:144: >> undefined reference to `__mpi__integer_s_MOD_mpi_allreduce_t' >> /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/memory.F:167: >> undefined reference to `__mpi__integer_sv_MOD_mpi_gather_t' >> timer.o: In function `timer1_': >> /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/timer.F:117: >> undefined >> reference to `__mpi__r8_s_MOD_mpi_allreduce_t' >> /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/timer.F:164: >> undefined >> reference to `__mpi__r8_v_MOD_mpi_reduce_t' >> /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/timer.F:139: >> undefined >> reference to `__mpi__r8_v_MOD_mpi_reduce_t' >> collect2: error: ld returned 1 exit status >> make: *** [gen-basis] Error 1 >> >> I will be really grateful if you kindly help me to solve this problem. >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Jeff Hammond jeff.science(a)gmail.com

1 0

Re: [mpich-discuss] discuss Digest, Vol 9, Issue 13
by Reem Alraddadi 08 Jul '13

08 Jul '13

Hi Wesley. I wrote the following: mpirun --np 4 --env MPIR_PARAM_CTXID_EAGER_SIZE 1 ./flash4 but the error still the same. Did I do it in the wrong way?? Thanks, Reem > > Message: 5 > Date: Mon, 8 Jul 2013 08:14:48 -0500 > From: Wesley Bland <wbland(a)mcs.anl.gov> > To: discuss(a)mpich.org > Subject: Re: [mpich-discuss] mpich on Mac os x > Message-ID: <8DC984B2-4E4B-4BFE-806E-203463A7A4E4(a)mcs.anl.gov> > Content-Type: text/plain; charset=iso-8859-1 > > It seems that you're creating more communicators than MPICH can handle. > It's possible that you might be able to get around this by setting the > environment variable MPIR_PARAM_CTXID_EAGER_SIZE to something smaller than > its default (which is 2). That frees up a few more communicators, but there > is a pathological case where even with fewer communicators than the max, > MPICH won't be able to agree on a new communicator id when needed. Try > changing that environment variable and see if that fixes things. > > Wesley > > On Jul 8, 2013, at 5:33 AM, Reem Alraddadi <raba500(a)york.ac.uk> wrote: > > > Hi all, > > I am using mpich-3.0.4 on Mac os x version 10.7.5 to run FLASH code. It > works fine in the beginning of the run and then I got the following error: > > > > Fatal error in MPI_Comm_create: Other MPI error, error stack: > > MPI_Comm_create(600).................: MPI_Comm_create(comm=0x84000002, > group=0xc8001349, new_comm=0x7fff606a8614) failed > > MPI_Comm_create(577).................: > > MPIR_Comm_create_intra(241)..........: > > MPIR_Get_contextid(799)..............: > > MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID > because of fragmentation (169/2048 free on this process; ignore_id=0) > > Fatal error in MPI_Comm_create: Other MPI error, error stack: > > MPI_Comm_create(600).................: MPI_Comm_create(comm=0x84000002, > group=0xc80012b6, new_comm=0x7fff670cc614) failed > > MPI_Comm_create(577).................: > > MPIR_Comm_create_intra(241)..........: > > MPIR_Get_contextid(799)..............: > > MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID > because of fragmentation (316/2048 free on this process; ignore_id=0) > > Fatal error in MPI_Comm_create: Other MPI error, error stack: > > MPI_Comm_create(600).................: MPI_Comm_create(comm=0x84000004, > group=0xc800000e, new_comm=0x7fff629d5614) failed > > MPI_Comm_create(577).................: > > MPIR_Comm_create_intra(241)..........: > > MPIR_Get_contextid(799)..............: > > MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID > because of fragmentation (2020/2048 free on this process; ignore_id=0) > > Fatal error in MPI_Comm_create: Other MPI error, error stack: > > MPI_Comm_create(600).................: MPI_Comm_create(comm=0x84000002, > group=0xc8000020, new_comm=0x7fff639ae614) failed > > MPI_Comm_create(577).................: > > MPIR_Comm_create_intra(241)..........: > > MPIR_Get_contextid(799)..............: > > MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID > because of fragmentation (2002/2048 free on this process; ignore_id=0 > > > > Is there a way to fix that ? > > > > Thanks, > > Reem > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > >

1 0

Re: [mpich-discuss] "undefined references"
by Pavan Balaji 08 Jul '13

08 Jul '13

This doesn't look like an MPI error. Did you contact the authors of whatever package you are trying to compile? -- Pavan On 07/08/2013 08:25 AM, Hana Milani wrote: > Hello, > > I have compiled a code (siesta) by mpich2, blacs, lapack, blas and > scalapack. It works fine under grid computation. But when I want to > "make" a utility, I receive the following errors: > > [Home@localhost Gen-basis]$ make > /usr/local/bin/mpif90 -o gen-basis \ > alloc.o basis_types.o precision.o recipes.o parallel.o > parsing.o xcmod.o basis_io.o chemical.o atm_transfer.o atm_types.o > atmparams.o old_atmfuncs.o radial.o io.o paste.o flib_spline.o > bessph.o m_filter.o basis_specs.o atom.o memoryinfo.o memory.o > periodic_table.o pseudopotential.o pxf.o dot.o xc.o atom_options.o > arw.o sys.o timer.o xml.o m_walltime.o gen-basis.o nag.o libfdf.a > /home/Home/Packages/scalapack-2.0.2/libscalapack.a > /home/Home/Packages/BLACS/LIB/blacsCinit_MPI-LINUX-0.a > /home/Home/Packages/BLACS/LIB/blacsF77init_MPI-LINUX-0.a > /home/Home/Packages/BLACS/LIB/blacs_MPI-LINUX-0.a > /home/Home/Packages/lapack-3.4.2/liblapack.a > /home/Home/Packages/BLAS/blas_LINUX.a > alloc.o: In function `print_report': > /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/alloc.F90:1705: > undefined reference to `__mpi__r8_s_MOD_mpi_reduce_t' > /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/alloc.F90:1707: > undefined reference to `__mpi__r8_v_MOD_mpi_reduce_t' > /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/alloc.F90:1711: > undefined reference to `__mpi__integer_s_MOD_mpi_bcast_t' > /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/alloc.F90:1723: > undefined reference to `__mpi__r8_s_MOD_mpi_recv_t' > /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/alloc.F90:1718: > undefined reference to `__mpi__r8_s_MOD_mpi_send_t' > alloc.o: In function `__alloc_MOD_alloc_report': > /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/alloc.F90:318: > undefined reference to `__mpi__integer_s_MOD_mpi_bcast_t' > /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/alloc.F90:319: > undefined reference to `__mpi__integer_s_MOD_mpi_bcast_t' > /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/alloc.F90:320: > undefined reference to `__mpi__character_s_MOD_mpi_bcast_t' > xcmod.o: In function `__xcmod_MOD_setxc': > /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/xcmod.F:136: undefined > reference to `__mpi__integer_s_MOD_mpi_bcast_t' > /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/xcmod.F:138: undefined > reference to `__mpi__character_v_MOD_mpi_bcast_t' > /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/xcmod.F:140: undefined > reference to `__mpi__character_v_MOD_mpi_bcast_t' > /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/xcmod.F:142: undefined > reference to `__mpi__r8_v_MOD_mpi_bcast_t' > /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/xcmod.F:144: undefined > reference to `__mpi__r8_v_MOD_mpi_bcast_t' > memory.o: In function `printmemory_': > /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/memory.F:144: > undefined reference to `__mpi__integer_s_MOD_mpi_allreduce_t' > /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/memory.F:167: > undefined reference to `__mpi__integer_sv_MOD_mpi_gather_t' > timer.o: In function `timer1_': > /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/timer.F:117: undefined > reference to `__mpi__r8_s_MOD_mpi_allreduce_t' > /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/timer.F:164: undefined > reference to `__mpi__r8_v_MOD_mpi_reduce_t' > /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/timer.F:139: undefined > reference to `__mpi__r8_v_MOD_mpi_reduce_t' > collect2: error: ld returned 1 exit status > make: *** [gen-basis] Error 1 > > I will be really grateful if you kindly help me to solve this problem. > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] Support for MIC in MPICH 3.0.4
by Pavan Balaji 08 Jul '13

08 Jul '13

Hi Maciej, Intel is aware of this problem that occurs when a large number of MIC processes are used. They are looking into it. Right now, the only workaround we can offer is to use fewer MIC processes and use threads on the MIC instead. -- Pavan On 07/08/2013 01:34 AM, Maciej.Golebiewski(a)csiro.au wrote: > Hi, > > I have managed to build MPICH for MIC device and for host system with support for SCIF and using Intel compilers. I am able to start MPI processes on MIC device using mpirun from host or mpiexec on the device itself. > > I can also run application with some ranks on the host and some on the device. > > I can't however start more than 4 ranks on any device, if I try to run my application across more than 1 node (be it host or MIC cards): > > env I_MPI_MIC=enable mpirun -n 1 -host mike.it.csiro.au ./mpptest -bcast : -n 30 -host mic0 /tmp/mpptest.scif -bcast > 0: 1: 00000011: 00000170: readv err 0 > Fatal error in MPI_Allreduce: Other MPI error, error stack: > MPI_Allreduce(861)...............: MPI_Allreduce(sbuf=0x7fff62945a08, rbuf=0x7c5de0, count=1, MPI_DOUBLE, MPI_MAX, MPI_COMM_WORLD) failed > MPIR_Allreduce_impl(719).........: > MPIR_Allreduce_intra(201)........: > allreduce_intra_or_coll_fn(110)..: > MPIR_Allreduce_intra(362)........: > MPIC_Sendrecv(213)...............: > MPIC_Wait(569)...................: > MPIDI_CH3I_Progress(367).........: > MPID_nem_mpich_blocking_recv(894): > state_commrdy_handler(175).......: > state_commrdy_handler(138).......: > MPID_nem_scif_recv_handler(115)..: Communication error with rank 1 > [...bunch of more error messages follows...] > > This problem does not occur of all ranks run on the same node: > > env I_MPI_MIC=enable mpirun -n 30 -host mic0 /tmp/mpptest.scif -bcast > set default > set font variable > set curve window y 0.15 0.90 > set order d d d x y d > title left 'time (us)', bottom 'Size (bytes)', > top 'Comm Perf for MPI (pkmic-mic0)', > 'type = blocking' > > #p0 p1 dist len ave time (us) rate > 0 29 29 0 11.448860 0.00 > 0 29 29 32 12.009144 2.665e+6 > [...more expected output follows...] > > Last November there was a short thread on this list (archive: http://lists.mpich.org/pipermail/discuss/2012-November/000109.html) about seemingly similiar problem, but seemed to fizzle out before any solution was offered. > > I wonder whether there was any progress on it, or perhaps someone else has encountered this problem, too and managed to find a workaround/solution or at least has an explanation as to what is happening. > > In case someone would like to try and reproduce it, my configure invocation for MPICH on MIC was: > > ./configure CC="icc" CXX=icpc F77="ifort -warn noalignments" FC="ifort -warn noalignments" \ > CXXFLAGS=-mmic CFLAGS=-mmic FFLAGS=-mmic FCFLAGS=-mmic \ > MPICHLIB_CFLAGS="-g" MPICHLIB_CXXFLAGS="-g" MPICHLIB_FFLAGS="-g" MPICHLIB_FCFLAGS="-g" \ > MPICHLIB_LDFLAGS="-g" LIBS=-lscif LDFLAGS=-L/opt/intel/mic/filesystem/base/lib64 \ > --with-device=ch3:nemesis:scif,tcp --disable-romio --disable-mpe --enable-smpcoll \ > --enable-fast=defopt --prefix=/apps/mpich-scif/3.0.4/mic --enable-error-checking=runtime \ > --enable-timing=runtime --host=i686-linux-gnu \ > --with-cross=maint/fcrosscompile/cross_values.txt.mike > > The settings in cross_values.txt.mike were: > > # The Fortran related cross compilation values. > CROSS_F77_SIZEOF_INTEGER="4" > CROSS_F77_SIZEOF_REAL="4" > CROSS_F77_SIZEOF_DOUBLE_PRECISION="8" > CROSS_F77_TRUE_VALUE="-1" > CROSS_F77_FALSE_VALUE="0" > CROSS_F90_ADDRESS_KIND="8" > CROSS_F90_OFFSET_KIND="8" > CROSS_F90_INTEGER_KIND="4" > CROSS_F90_REAL_MODEL=" 6 , 37" > CROSS_F90_DOUBLE_MODEL=" 15 , 307" > CROSS_F90_INTEGER_MODEL=" 9" > CROSS_F90_ALL_INTEGER_MODELS=" 2 , 1, 4 , 2, 9 , 4, 18 , 8," > CROSS_F90_INTEGER_MODEL_MAP=" { 2 , 1 , 1 }, { 4 , 2 , 2 }, { 9 , 4 , 4 }, { 18 , 8 , 8 }," > > And finally, the configure invocation for MPICH on the host was: > > ./configure CC="icc" CXX=icpc F77="ifort -warn noalignments" FC="ifort -warn noalignments" \ > MPICHLIB_CFLAGS="-g" MPICHLIB_CXXFLAGS="-g" MPICHLIB_FFLAGS="-g" MPICHLIB_FCFLAGS="-g" \ > MPICHLIB_LDFLAGS="-g" LIBS=-lscif LDFLAGS=-L/usr/lib64 --with-device=ch3:nemesis:scif,tcp \ > --disable-romio --disable-mpe --enable-smpcoll --enable-fast=defopt \ > --prefix=/apps/mpich-scif/3.0.4/intel64 --enable-error-checking=runtime \ > --enable-timing=runtime > > Thanks, > > Maciej > > Maciej Golebiewski > Applications Support, Advanced Scientific Computing CSIRO IM&T > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] mpich on Mac os x
by Pavan Balaji 08 Jul '13

08 Jul '13

Also make sure you are freeing unused communicators. -- Pavan On 07/08/2013 08:14 AM, Wesley Bland wrote: > It seems that you're creating more communicators than MPICH can handle. It's possible that you might be able to get around this by setting the environment variable MPIR_PARAM_CTXID_EAGER_SIZE to something smaller than its default (which is 2). That frees up a few more communicators, but there is a pathological case where even with fewer communicators than the max, MPICH won't be able to agree on a new communicator id when needed. Try changing that environment variable and see if that fixes things. > > Wesley > > On Jul 8, 2013, at 5:33 AM, Reem Alraddadi <raba500(a)york.ac.uk> wrote: > >> Hi all, >> I am using mpich-3.0.4 on Mac os x version 10.7.5 to run FLASH code. It works fine in the beginning of the run and then I got the following error: >> >> Fatal error in MPI_Comm_create: Other MPI error, error stack: >> MPI_Comm_create(600).................: MPI_Comm_create(comm=0x84000002, group=0xc8001349, new_comm=0x7fff606a8614) failed >> MPI_Comm_create(577).................: >> MPIR_Comm_create_intra(241)..........: >> MPIR_Get_contextid(799)..............: >> MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID because of fragmentation (169/2048 free on this process; ignore_id=0) >> Fatal error in MPI_Comm_create: Other MPI error, error stack: >> MPI_Comm_create(600).................: MPI_Comm_create(comm=0x84000002, group=0xc80012b6, new_comm=0x7fff670cc614) failed >> MPI_Comm_create(577).................: >> MPIR_Comm_create_intra(241)..........: >> MPIR_Get_contextid(799)..............: >> MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID because of fragmentation (316/2048 free on this process; ignore_id=0) >> Fatal error in MPI_Comm_create: Other MPI error, error stack: >> MPI_Comm_create(600).................: MPI_Comm_create(comm=0x84000004, group=0xc800000e, new_comm=0x7fff629d5614) failed >> MPI_Comm_create(577).................: >> MPIR_Comm_create_intra(241)..........: >> MPIR_Get_contextid(799)..............: >> MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID because of fragmentation (2020/2048 free on this process; ignore_id=0) >> Fatal error in MPI_Comm_create: Other MPI error, error stack: >> MPI_Comm_create(600).................: MPI_Comm_create(comm=0x84000002, group=0xc8000020, new_comm=0x7fff639ae614) failed >> MPI_Comm_create(577).................: >> MPIR_Comm_create_intra(241)..........: >> MPIR_Get_contextid(799)..............: >> MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID because of fragmentation (2002/2048 free on this process; ignore_id=0 >> >> Is there a way to fix that ? >> >> Thanks, >> Reem >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

"undefined references"
by Hana Milani 08 Jul '13

08 Jul '13

Hello, I have compiled a code (siesta) by mpich2, blacs, lapack, blas and scalapack. It works fine under grid computation. But when I want to "make" a utility, I receive the following errors: [Home@localhost Gen-basis]$ make /usr/local/bin/mpif90 -o gen-basis \ alloc.o basis_types.o precision.o recipes.o parallel.o parsing.o xcmod.o basis_io.o chemical.o atm_transfer.o atm_types.o atmparams.o old_atmfuncs.o radial.o io.o paste.o flib_spline.o bessph.o m_filter.o basis_specs.o atom.o memoryinfo.o memory.o periodic_table.o pseudopotential.o pxf.o dot.o xc.o atom_options.o arw.o sys.o timer.o xml.o m_walltime.o gen-basis.o nag.o libfdf.a /home/Home/Packages/scalapack-2.0.2/libscalapack.a /home/Home/Packages/BLACS/LIB/blacsCinit_MPI-LINUX-0.a /home/Home/Packages/BLACS/LIB/blacsF77init_MPI-LINUX-0.a /home/Home/Packages/BLACS/LIB/blacs_MPI-LINUX-0.a /home/Home/Packages/lapack-3.4.2/liblapack.a /home/Home/Packages/BLAS/blas_LINUX.a alloc.o: In function `print_report': /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/alloc.F90:1705: undefined reference to `__mpi__r8_s_MOD_mpi_reduce_t' /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/alloc.F90:1707: undefined reference to `__mpi__r8_v_MOD_mpi_reduce_t' /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/alloc.F90:1711: undefined reference to `__mpi__integer_s_MOD_mpi_bcast_t' /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/alloc.F90:1723: undefined reference to `__mpi__r8_s_MOD_mpi_recv_t' /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/alloc.F90:1718: undefined reference to `__mpi__r8_s_MOD_mpi_send_t' alloc.o: In function `__alloc_MOD_alloc_report': /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/alloc.F90:318: undefined reference to `__mpi__integer_s_MOD_mpi_bcast_t' /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/alloc.F90:319: undefined reference to `__mpi__integer_s_MOD_mpi_bcast_t' /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/alloc.F90:320: undefined reference to `__mpi__character_s_MOD_mpi_bcast_t' xcmod.o: In function `__xcmod_MOD_setxc': /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/xcmod.F:136: undefined reference to `__mpi__integer_s_MOD_mpi_bcast_t' /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/xcmod.F:138: undefined reference to `__mpi__character_v_MOD_mpi_bcast_t' /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/xcmod.F:140: undefined reference to `__mpi__character_v_MOD_mpi_bcast_t' /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/xcmod.F:142: undefined reference to `__mpi__r8_v_MOD_mpi_bcast_t' /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/xcmod.F:144: undefined reference to `__mpi__r8_v_MOD_mpi_bcast_t' memory.o: In function `printmemory_': /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/memory.F:144: undefined reference to `__mpi__integer_s_MOD_mpi_allreduce_t' /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/memory.F:167: undefined reference to `__mpi__integer_sv_MOD_mpi_gather_t' timer.o: In function `timer1_': /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/timer.F:117: undefined reference to `__mpi__r8_s_MOD_mpi_allreduce_t' /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/timer.F:164: undefined reference to `__mpi__r8_v_MOD_mpi_reduce_t' /home/Home/Packages/siesta-TEST-3.2/Util/Gen-basis/../../Src/timer.F:139: undefined reference to `__mpi__r8_v_MOD_mpi_reduce_t' collect2: error: ld returned 1 exit status make: *** [gen-basis] Error 1 I will be really grateful if you kindly help me to solve this problem.

1 0

Re: [mpich-discuss] mpich on Mac os x
by Wesley Bland 08 Jul '13

08 Jul '13

It seems that you're creating more communicators than MPICH can handle. It's possible that you might be able to get around this by setting the environment variable MPIR_PARAM_CTXID_EAGER_SIZE to something smaller than its default (which is 2). That frees up a few more communicators, but there is a pathological case where even with fewer communicators than the max, MPICH won't be able to agree on a new communicator id when needed. Try changing that environment variable and see if that fixes things. Wesley On Jul 8, 2013, at 5:33 AM, Reem Alraddadi <raba500(a)york.ac.uk> wrote: > Hi all, > I am using mpich-3.0.4 on Mac os x version 10.7.5 to run FLASH code. It works fine in the beginning of the run and then I got the following error: > > Fatal error in MPI_Comm_create: Other MPI error, error stack: > MPI_Comm_create(600).................: MPI_Comm_create(comm=0x84000002, group=0xc8001349, new_comm=0x7fff606a8614) failed > MPI_Comm_create(577).................: > MPIR_Comm_create_intra(241)..........: > MPIR_Get_contextid(799)..............: > MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID because of fragmentation (169/2048 free on this process; ignore_id=0) > Fatal error in MPI_Comm_create: Other MPI error, error stack: > MPI_Comm_create(600).................: MPI_Comm_create(comm=0x84000002, group=0xc80012b6, new_comm=0x7fff670cc614) failed > MPI_Comm_create(577).................: > MPIR_Comm_create_intra(241)..........: > MPIR_Get_contextid(799)..............: > MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID because of fragmentation (316/2048 free on this process; ignore_id=0) > Fatal error in MPI_Comm_create: Other MPI error, error stack: > MPI_Comm_create(600).................: MPI_Comm_create(comm=0x84000004, group=0xc800000e, new_comm=0x7fff629d5614) failed > MPI_Comm_create(577).................: > MPIR_Comm_create_intra(241)..........: > MPIR_Get_contextid(799)..............: > MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID because of fragmentation (2020/2048 free on this process; ignore_id=0) > Fatal error in MPI_Comm_create: Other MPI error, error stack: > MPI_Comm_create(600).................: MPI_Comm_create(comm=0x84000002, group=0xc8000020, new_comm=0x7fff639ae614) failed > MPI_Comm_create(577).................: > MPIR_Comm_create_intra(241)..........: > MPIR_Get_contextid(799)..............: > MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID because of fragmentation (2002/2048 free on this process; ignore_id=0 > > Is there a way to fix that ? > > Thanks, > Reem > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

mpich on Mac os x
by Reem Alraddadi 08 Jul '13

08 Jul '13

Hi all, I am using mpich-3.0.4 on Mac os x version 10.7.5 to run FLASH code. It works fine in the beginning of the run and then I got the following error: Fatal error in MPI_Comm_create: Other MPI error, error stack: MPI_Comm_create(600).................: MPI_Comm_create(comm=0x84000002, group=0xc8001349, new_comm=0x7fff606a8614) failed MPI_Comm_create(577).................: MPIR_Comm_create_intra(241)..........: MPIR_Get_contextid(799)..............: MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID because of fragmentation (169/2048 free on this process; ignore_id=0) Fatal error in MPI_Comm_create: Other MPI error, error stack: MPI_Comm_create(600).................: MPI_Comm_create(comm=0x84000002, group=0xc80012b6, new_comm=0x7fff670cc614) failed MPI_Comm_create(577).................: MPIR_Comm_create_intra(241)..........: MPIR_Get_contextid(799)..............: MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID because of fragmentation (316/2048 free on this process; ignore_id=0) Fatal error in MPI_Comm_create: Other MPI error, error stack: MPI_Comm_create(600).................: MPI_Comm_create(comm=0x84000004, group=0xc800000e, new_comm=0x7fff629d5614) failed MPI_Comm_create(577).................: MPIR_Comm_create_intra(241)..........: MPIR_Get_contextid(799)..............: MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID because of fragmentation (2020/2048 free on this process; ignore_id=0) Fatal error in MPI_Comm_create: Other MPI error, error stack: MPI_Comm_create(600).................: MPI_Comm_create(comm=0x84000002, group=0xc8000020, new_comm=0x7fff639ae614) failed MPI_Comm_create(577).................: MPIR_Comm_create_intra(241)..........: MPIR_Get_contextid(799)..............: MPIR_Get_contextid_sparse_group(1146): Cannot allocate context ID because of fragmentation (2002/2048 free on this process; ignore_id=0 Is there a way to fix that ? Thanks, Reem

1 0

Support for MIC in MPICH 3.0.4
by Maciej.Golebiewski＠csiro.au 08 Jul '13

08 Jul '13

Hi, I have managed to build MPICH for MIC device and for host system with support for SCIF and using Intel compilers. I am able to start MPI processes on MIC device using mpirun from host or mpiexec on the device itself. I can also run application with some ranks on the host and some on the device. I can't however start more than 4 ranks on any device, if I try to run my application across more than 1 node (be it host or MIC cards): env I_MPI_MIC=enable mpirun -n 1 -host mike.it.csiro.au ./mpptest -bcast : -n 30 -host mic0 /tmp/mpptest.scif -bcast 0: 1: 00000011: 00000170: readv err 0 Fatal error in MPI_Allreduce: Other MPI error, error stack: MPI_Allreduce(861)...............: MPI_Allreduce(sbuf=0x7fff62945a08, rbuf=0x7c5de0, count=1, MPI_DOUBLE, MPI_MAX, MPI_COMM_WORLD) failed MPIR_Allreduce_impl(719).........: MPIR_Allreduce_intra(201)........: allreduce_intra_or_coll_fn(110)..: MPIR_Allreduce_intra(362)........: MPIC_Sendrecv(213)...............: MPIC_Wait(569)...................: MPIDI_CH3I_Progress(367).........: MPID_nem_mpich_blocking_recv(894): state_commrdy_handler(175).......: state_commrdy_handler(138).......: MPID_nem_scif_recv_handler(115)..: Communication error with rank 1 [...bunch of more error messages follows...] This problem does not occur of all ranks run on the same node: env I_MPI_MIC=enable mpirun -n 30 -host mic0 /tmp/mpptest.scif -bcast set default set font variable set curve window y 0.15 0.90 set order d d d x y d title left 'time (us)', bottom 'Size (bytes)', top 'Comm Perf for MPI (pkmic-mic0)', 'type = blocking' #p0 p1 dist len ave time (us) rate 0 29 29 0 11.448860 0.00 0 29 29 32 12.009144 2.665e+6 [...more expected output follows...] Last November there was a short thread on this list (archive: http://lists.mpich.org/pipermail/discuss/2012-November/000109.html) about seemingly similiar problem, but seemed to fizzle out before any solution was offered. I wonder whether there was any progress on it, or perhaps someone else has encountered this problem, too and managed to find a workaround/solution or at least has an explanation as to what is happening. In case someone would like to try and reproduce it, my configure invocation for MPICH on MIC was: ./configure CC="icc" CXX=icpc F77="ifort -warn noalignments" FC="ifort -warn noalignments" \ CXXFLAGS=-mmic CFLAGS=-mmic FFLAGS=-mmic FCFLAGS=-mmic \ MPICHLIB_CFLAGS="-g" MPICHLIB_CXXFLAGS="-g" MPICHLIB_FFLAGS="-g" MPICHLIB_FCFLAGS="-g" \ MPICHLIB_LDFLAGS="-g" LIBS=-lscif LDFLAGS=-L/opt/intel/mic/filesystem/base/lib64 \ --with-device=ch3:nemesis:scif,tcp --disable-romio --disable-mpe --enable-smpcoll \ --enable-fast=defopt --prefix=/apps/mpich-scif/3.0.4/mic --enable-error-checking=runtime \ --enable-timing=runtime --host=i686-linux-gnu \ --with-cross=maint/fcrosscompile/cross_values.txt.mike The settings in cross_values.txt.mike were: # The Fortran related cross compilation values. CROSS_F77_SIZEOF_INTEGER="4" CROSS_F77_SIZEOF_REAL="4" CROSS_F77_SIZEOF_DOUBLE_PRECISION="8" CROSS_F77_TRUE_VALUE="-1" CROSS_F77_FALSE_VALUE="0" CROSS_F90_ADDRESS_KIND="8" CROSS_F90_OFFSET_KIND="8" CROSS_F90_INTEGER_KIND="4" CROSS_F90_REAL_MODEL=" 6 , 37" CROSS_F90_DOUBLE_MODEL=" 15 , 307" CROSS_F90_INTEGER_MODEL=" 9" CROSS_F90_ALL_INTEGER_MODELS=" 2 , 1, 4 , 2, 9 , 4, 18 , 8," CROSS_F90_INTEGER_MODEL_MAP=" { 2 , 1 , 1 }, { 4 , 2 , 2 }, { 9 , 4 , 4 }, { 18 , 8 , 8 }," And finally, the configure invocation for MPICH on the host was: ./configure CC="icc" CXX=icpc F77="ifort -warn noalignments" FC="ifort -warn noalignments" \ MPICHLIB_CFLAGS="-g" MPICHLIB_CXXFLAGS="-g" MPICHLIB_FFLAGS="-g" MPICHLIB_FCFLAGS="-g" \ MPICHLIB_LDFLAGS="-g" LIBS=-lscif LDFLAGS=-L/usr/lib64 --with-device=ch3:nemesis:scif,tcp \ --disable-romio --disable-mpe --enable-smpcoll --enable-fast=defopt \ --prefix=/apps/mpich-scif/3.0.4/intel64 --enable-error-checking=runtime \ --enable-timing=runtime Thanks, Maciej Maciej Golebiewski Applications Support, Advanced Scientific Computing CSIRO IM&T

1 0