November 2018 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] _get_addr error while running application using MPICH
by Zhou, Hui 13 Nov '18

13 Nov '18

I thought mpirun always assumes default behavior when -np <number of processes> is not given. With hydra, it default to 1. Openmpi would default to number of hardware threads I believe. Zhifeng, could you duplicate the error with a simpler FORTRAN program and post your source code so I can try reproduce? — Hui Zhou T: 630-252-3430 On Nov 13, 2018, at 10:27 AM, Si, Min via discuss <discuss(a)mpich.org<mailto:[email protected]>> wrote: Hi Zhifeng, You have to always execute the MPI program by using mpirun or mpiexec with -np <number of processes>. At MPI_Init, the process tries to read information (e.g., MPI rank, number of processes) from the PMI server which is another process launched by the mpiexec command. This information is necessary for MPI library to initialize communication (e.g., connecting two processes). Best regards, Min On 2018/11/04 10:27, Zhifeng Yang via discuss wrote: Hi After I installed mpich and used it in a FORTRAN code. There is an error while running this FORTRAN code called real.exe by using the following command $./real.exe or $mpirun ./real.exe real.exe: error: _get_addr: No such file or directory real.exe: error: _get_addr: No such file or directory real.exe: error: _get_addr: No such file or directory Fatal error in MPI_Init: Other MPI error, error stack: MPIR_Init_thread(784).....: MPID_Init(1323)...........: channel initialization failed MPIDI_CH3_Init(120).......: MPID_nem_init_ckpt(852)...: MPIDI_CH3I_Seg_commit(364): PMI_Barrier returned -1 But while using mpirun with specifying number of processors. as follows $mpirun -np 1 ./real.exe There is no error any more. I am not sure why? do you have any explanation? Thank you very much Best regards Zhifeng _______________________________________________ discuss mailing list discuss(a)mpich.org<mailto:[email protected]> To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org<mailto:[email protected]> To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] [BUG REPORT] /usr/local/lib/libmpicxx.so.12: undefined reference to `MPIR_Keyval_set_proxy'
by Si, Min 13 Nov '18

13 Nov '18

Hi Yuri, If I read your configure correctly, is the user program linked to OpenMPI instead of MPICH ? Best regards, Min On 2018/10/31 1:31, Yuri via discuss wrote: > The mpich-3.2.1 built in the FreeBSD port has this problem of missing > function when other project tries to link with it: > > > /usr/bin/c++ -O2 -pipe -fno-omit-frame-pointer -fstack-protector > -fno-strict-aliasing -fno-omit-frame-pointer -ffast-math -pthread > -std=c++11 -O2 -pipe -fno-omit-frame-pointer -fstack-protector > -fno-strict-aliasing -fno-omit-frame-pointer /usr/local/lib/libmpir.so > -Wl,-rpath=/usr/local/lib/gcc7 -L/usr/local/lib/gcc7 -B/usr/local/bin > -fstack-protector CMakeFiles/ssages.dir/src/main.cpp.o > CMakeFiles/ssages.dir/hooks/lammps/Driver.cpp.o -o ssages > -L/usr/local/mpi/openmpi/lib/libmpi_cxx.so > -L/usr/local/mpi/openmpi/lib/libmpi.so > -L/usr/ports/science/ssages/work/SSAGES-public-0.8.3-alpha/_lammps_/src > -Wl,-rpath,/usr/local/mpi/openmpi/lib/libmpi_cxx.so:/usr/local/mpi/openmpi/lib/libmpi.so:/usr/ports/science/ssages/work/SSAGES-public-0.8.3-alpha/_lammps_/src:/usr/local/mpi/openmpi/lib > libssages.a -llammps_mpi /usr/local/mpi/openmpi/lib/libmpi_cxx.so > /usr/local/mpi/openmpi/lib/libmpi.so > CMakeFiles/ssages.dir/hooks/lammps/Driver.cpp.o: In function > `SSAGES::Driver::Run()': > Driver.cpp:(.text+0x3f): undefined reference to > `LAMMPS_NS::LAMMPS::LAMMPS(int, char**, ompi_communicator_t*)' > //usr/local/lib/libmpicxx.so.12: undefined reference to > `MPIR_Keyval_set_proxy' > //usr/local/lib/libmpicxx.so.12: undefined reference to > `MPIR_Errhandler_set_cxx' > //usr/local/lib/libmpicxx.so.12: undefined reference to `MPIR_Op_set_cxx' > > > These functions are either missing from this library, or it isn't > linked to some ther library. > > > Yuri > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] _get_addr error while running application using MPICH
by Si, Min 13 Nov '18

13 Nov '18

Hi Zhifeng, You have to always execute the MPI program by using mpirun or mpiexec with -np <number of processes>. At MPI_Init, the process tries to read information (e.g., MPI rank, number of processes) from the PMI server which is another process launched by the mpiexec command. This information is necessary for MPI library to initialize communication (e.g., connecting two processes). Best regards, Min On 2018/11/04 10:27, Zhifeng Yang via discuss wrote: Hi After I installed mpich and used it in a FORTRAN code. There is an error while running this FORTRAN code called real.exe by using the following command $./real.exe or $mpirun ./real.exe real.exe: error: _get_addr: No such file or directory real.exe: error: _get_addr: No such file or directory real.exe: error: _get_addr: No such file or directory Fatal error in MPI_Init: Other MPI error, error stack: MPIR_Init_thread(784).....: MPID_Init(1323)...........: channel initialization failed MPIDI_CH3_Init(120).......: MPID_nem_init_ckpt(852)...: MPIDI_CH3I_Seg_commit(364): PMI_Barrier returned -1 But while using mpirun with specifying number of processors. as follows $mpirun -np 1 ./real.exe There is no error any more. I am not sure why? do you have any explanation? Thank you very much Best regards Zhifeng _______________________________________________ discuss mailing list discuss(a)mpich.org<mailto:[email protected]> To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Communication issue with simple test?
by Si, Min 13 Nov '18

13 Nov '18

Hi Thibaut, In order to help us isolate this problem, could you please try the following options: - Use MPICH 3.3rc instead of MPICH 3.3b1 - Use ch3:tcp instead of ch3:sock (tcp is used by default, so please just delete `--with-device=ch3:sock` when you configure) - Try `mpiexec -f path_to_hostfile -n 8 hostname` Best regards, Min On 2018/11/13 7:30, Appel, Thibaut via discuss wrote: Dear MPICH users, I'm having an issue with, apparently, communication between nodes of the local cluster we're using. To fix ideas, the cluster is made of 8 different nodes and it was set-up with now outdated versions of the Intel compilers and MPI libraries. I did an mpbboot/mpdringtest and it seems to work fine. Now, I would like to use my application code with MPICH 3.3 installed from PETSc and with gcc-8 installed from linuxbrew on the different nodes. I tested a simple Fortran program: program test use mpi use ISO_fortran_env, only: output_unit implicit none integer :: irank, nproc, ierr character(len=80) :: hostname call MPI_INIT(ierr) call MPI_COMM_SIZE(MPI_COMM_WORLD,nproc,ierr) call MPI_COMM_RANK(MPI_COMM_WORLD,irank,ierr) if (irank == 0) THEN WRITE(output_unit,'(1X,A,I3,A)') 'Started test program with', nproc, ' MPI processes' end if call MPI_BARRIER(MPI_COMM_WORLD,ierr) call HOSTNM(hostname,ierr) WRITE(output_unit,'(1X,A,I3,A)') 'I am processor #', irank, ' running on '//hostname call MPI_FINALIZE(ierr) end program test It works fine on the local host. But when I try to launch it on all the nodes with "mpiexec -f path_to_hostfile -n 8 path_to_my_program" I get: Started test program with 8 MPI processes Fatal error in PMPI_Barrier: Unknown error class, error stack: PMPI_Barrier(287)...........................: MPI_Barrier(MPI_COMM_WORLD) failed PMPI_Barrier(273)...........................: MPIR_Barrier_impl(173)......................: MPIR_Barrier_intra_auto(108)................: MPIR_Barrier_intra_recursive_doubling(47)...: MPIC_Sendrecv(347)..........................: MPIC_Wait(73)...............................: MPIDI_CH3i_Progress_wait(242)...............: an error occurred while handling an event returned by MPIDI_CH3I_Sock_Wait() MPIDI_CH3I_Progress_handle_sock_event(698)..: MPIDI_CH3_Sockconn_handle_connect_event(597): [ch3:sock] failed to connnect to remote process MPIDI_CH3I_Sock_post_connect_ifaddr(1774)...: unexpected operating system error (set=0,sock=5,errno=101:Network is unreachable) Note that when I comment the call to MPI_BARRIER, it works fine as well. Therefore, communication between nodes seems to be an issue. All the nodes see and have access to the same mpiexec/mpifort executables: I checked $PATH and 'which mpiexec'/'which mpifort'. Could you give me ways for an efficient diagnostic of what's possibly wrong? mpiexec --version gives HYDRA build details: Version: 3.3b1 Release Date: Mon Feb 5 10:16:15 CST 2018 CC: gcc-8 -fPIC -fstack-protector -O3 -march=native CXX: g++-8 -fstack-protector -O3 -march=native -fPIC F77: gfortran-8 -fPIC -ffree-line-length-0 -O3 -march=native F90: gfortran-8 -fPIC -ffree-line-length-0 -O3 -march=native Configure options: '--disable-option-checking' '--prefix=/home/petsc/icm_cplx' 'MAKE=/usr/bin/make' '--libdir=/home/petsc/icm_cplx/lib' 'CC=gcc-8' 'CFLAGS=-fPIC -fstack-protector -O3 -march=native -O2' 'AR=/usr/bin/ar' 'ARFLAGS=cr' 'CXX=g++-8' 'CXXFLAGS=-fstack-protector -O3 -march=native -fPIC -O2' 'F77=gfortran-8' 'FFLAGS=-fPIC -ffree-line-length-0 -O3 -march=native -O2' 'FC=gfortran-8' 'FCFLAGS=-fPIC -ffree-line-length-0 -O3 -march=native -O2' '--enable-shared' '--with-device=ch3:sock' '--with-pm=hydra' '--enable-g=meminit' '--cache-file=/dev/null' '--srcdir=.' 'LDFLAGS=' 'LIBS=' 'CPPFLAGS= -I/home/petsc/icm_cplx/externalpackages/mpich-3.3b1/src/mpl/include -I/home/petsc/icm_cplx/externalpackages/mpich-3.3b1/src/mpl/include -I/home/petsc/icm_cplx/externalpackages/mpich-3.3b1/src/openpa/src -I/home/petsc/icm_cplx/externalpackages/mpich-3.3b1/src/openpa/src -D_REENTRANT -I/home/petsc/icm_cplx/externalpackages/mpich-3.3b1/src/mpi/romio/include' 'MPLLIBNAME=mpl' Process Manager: pmi Launchers available: ssh rsh fork slurm ll lsf sge manual persist Topology libraries available: hwloc Resource management kernels available: user slurm ll lsf sge pbs cobalt Checkpointing libraries available: Demux engines available: poll select Thank you, Thibaut _______________________________________________ discuss mailing list discuss(a)mpich.org<mailto:[email protected]> To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Communication issue with simple test?
by Appel, Thibaut 13 Nov '18

13 Nov '18

Dear MPICH users, I'm having an issue with, apparently, communication between nodes of the local cluster we're using. To fix ideas, the cluster is made of 8 different nodes and it was set-up with now outdated versions of the Intel compilers and MPI libraries. I did an mpbboot/mpdringtest and it seems to work fine. Now, I would like to use my application code with MPICH 3.3 installed from PETSc and with gcc-8 installed from linuxbrew on the different nodes. I tested a simple Fortran program: program test use mpi use ISO_fortran_env, only: output_unit implicit none integer :: irank, nproc, ierr character(len=80) :: hostname call MPI_INIT(ierr) call MPI_COMM_SIZE(MPI_COMM_WORLD,nproc,ierr) call MPI_COMM_RANK(MPI_COMM_WORLD,irank,ierr) if (irank == 0) THEN WRITE(output_unit,'(1X,A,I3,A)') 'Started test program with', nproc, ' MPI processes' end if call MPI_BARRIER(MPI_COMM_WORLD,ierr) call HOSTNM(hostname,ierr) WRITE(output_unit,'(1X,A,I3,A)') 'I am processor #', irank, ' running on '//hostname call MPI_FINALIZE(ierr) end program test It works fine on the local host. But when I try to launch it on all the nodes with "mpiexec -f path_to_hostfile -n 8 path_to_my_program" I get: Started test program with 8 MPI processes Fatal error in PMPI_Barrier: Unknown error class, error stack: PMPI_Barrier(287)...........................: MPI_Barrier(MPI_COMM_WORLD) failed PMPI_Barrier(273)...........................: MPIR_Barrier_impl(173)......................: MPIR_Barrier_intra_auto(108)................: MPIR_Barrier_intra_recursive_doubling(47)...: MPIC_Sendrecv(347)..........................: MPIC_Wait(73)...............................: MPIDI_CH3i_Progress_wait(242)...............: an error occurred while handling an event returned by MPIDI_CH3I_Sock_Wait() MPIDI_CH3I_Progress_handle_sock_event(698)..: MPIDI_CH3_Sockconn_handle_connect_event(597): [ch3:sock] failed to connnect to remote process MPIDI_CH3I_Sock_post_connect_ifaddr(1774)...: unexpected operating system error (set=0,sock=5,errno=101:Network is unreachable) Note that when I comment the call to MPI_BARRIER, it works fine as well. Therefore, communication between nodes seems to be an issue. All the nodes see and have access to the same mpiexec/mpifort executables: I checked $PATH and 'which mpiexec'/'which mpifort'. Could you give me ways for an efficient diagnostic of what's possibly wrong? mpiexec --version gives HYDRA build details: Version: 3.3b1 Release Date: Mon Feb 5 10:16:15 CST 2018 CC: gcc-8 -fPIC -fstack-protector -O3 -march=native CXX: g++-8 -fstack-protector -O3 -march=native -fPIC F77: gfortran-8 -fPIC -ffree-line-length-0 -O3 -march=native F90: gfortran-8 -fPIC -ffree-line-length-0 -O3 -march=native Configure options: '--disable-option-checking' '--prefix=/home/petsc/icm_cplx' 'MAKE=/usr/bin/make' '--libdir=/home/petsc/icm_cplx/lib' 'CC=gcc-8' 'CFLAGS=-fPIC -fstack-protector -O3 -march=native -O2' 'AR=/usr/bin/ar' 'ARFLAGS=cr' 'CXX=g++-8' 'CXXFLAGS=-fstack-protector -O3 -march=native -fPIC -O2' 'F77=gfortran-8' 'FFLAGS=-fPIC -ffree-line-length-0 -O3 -march=native -O2' 'FC=gfortran-8' 'FCFLAGS=-fPIC -ffree-line-length-0 -O3 -march=native -O2' '--enable-shared' '--with-device=ch3:sock' '--with-pm=hydra' '--enable-g=meminit' '--cache-file=/dev/null' '--srcdir=.' 'LDFLAGS=' 'LIBS=' 'CPPFLAGS= -I/home/petsc/icm_cplx/externalpackages/mpich-3.3b1/src/mpl/include -I/home/petsc/icm_cplx/externalpackages/mpich-3.3b1/src/mpl/include -I/home/petsc/icm_cplx/externalpackages/mpich-3.3b1/src/openpa/src -I/home/petsc/icm_cplx/externalpackages/mpich-3.3b1/src/openpa/src -D_REENTRANT -I/home/petsc/icm_cplx/externalpackages/mpich-3.3b1/src/mpi/romio/include' 'MPLLIBNAME=mpl' Process Manager: pmi Launchers available: ssh rsh fork slurm ll lsf sge manual persist Topology libraries available: hwloc Resource management kernels available: user slurm ll lsf sge pbs cobalt Checkpointing libraries available: Demux engines available: poll select Thank you, Thibaut

1 0

Re: [mpich-discuss] how to dynamically schedule process in mpich
by Shuwei Zhao 13 Nov '18

13 Nov '18

Hi, Hui, Thanks for your response, I have already read through the docs that you said. It's very useful. although I did some research still have some questions: IMHO, there are 2 possible ways to do dynamic scheduling of cpu cores: 1. using mpi_comm_spawn to spawn a new hydra_pmi_proxy on a different port or using mpi_comm_spawn_multiple to spawn new processes on the same port but my question is that how to start the new processes on a remote machine, using farm resource manager(qsub,bsub,etc)? *Is there a reserved keyword in the MPI_info that can qualify for this function?* 2. using mpi_comm_connect to connect an already started mpi application to connect to another mpi application, but I didn't find some useful testcases yet. Do you know if this way is possible? Thanks, Shuwei On Thu, Nov 8, 2018 at 4:52 PM Zhou, Hui <zhouh(a)anl.gov> wrote: > I am by far not an expert on MPI dynamic process at this point. I will > point you to the MPI standards https://www.mpi-forum.org/docs/, in > particular, chapter 10. MPICH implements most of the standards. Try follow > the standards to see what you can do. If you hit specific problem, post > back here and some one will help you. > > Cheers, > — > Hui Zhou > > > > > > > > > > On Nov 6, 2018, at 5:20 PM, Shuwei Zhao via discuss <discuss(a)mpich.org> > wrote: > > Hi, Experts, > > I have a question about if mpich can do dynamically allocate/schedule/kill > process on the runtime? If that’s doable, how can we do that? > > Let’s see an example: > qsub -P my_queue -pe mpi 8 mpiexec my_exe > Distribute mpich application on 8 processes each on a single machine. > During the application runtime, can we dynamically (allocate & > start)/(remove & kill) one process and (add to)/(delete from) the current > communication group ? > > Appreciate your help ahead! > > Thanks, > Shuwei > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > >

1 0

Re: [mpich-discuss] how to dynamically schedule process in mpich
by Zhou, Hui 08 Nov '18

08 Nov '18

I am by far not an expert on MPI dynamic process at this point. I will point you to the MPI standards https://www.mpi-forum.org/docs/, in particular, chapter 10. MPICH implements most of the standards. Try follow the standards to see what you can do. If you hit specific problem, post back here and some one will help you. Cheers, — Hui Zhou On Nov 6, 2018, at 5:20 PM, Shuwei Zhao via discuss <discuss(a)mpich.org<mailto:[email protected]>> wrote: Hi, Experts, I have a question about if mpich can do dynamically allocate/schedule/kill process on the runtime? If that’s doable, how can we do that? Let’s see an example: qsub -P my_queue -pe mpi 8 mpiexec my_exe Distribute mpich application on 8 processes each on a single machine. During the application runtime, can we dynamically (allocate & start)/(remove & kill) one process and (add to)/(delete from) the current communication group ? Appreciate your help ahead! Thanks, Shuwei _______________________________________________ discuss mailing list discuss(a)mpich.org<mailto:[email protected]> To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] how to dynamically schedule process in mpich
by Zhou, Hui 08 Nov '18

08 Nov '18

I am by far not an expert on MPI dynamic process at this point. I will point you to the MPI standards https://www.mpi-forum.org/docs/, in particular, chapter 10. MPICH implements most of the standards. Try follow the standards to see what you can do. If you hit specific problem, post back here and some one will help you. Cheers, — Hui Zhou On Nov 6, 2018, at 5:20 PM, Shuwei Zhao via discuss <discuss(a)mpich.org<mailto:[email protected]>> wrote: Hi, Experts, I have a question about if mpich can do dynamically allocate/schedule/kill process on the runtime? If that’s doable, how can we do that? Let’s see an example: qsub -P my_queue -pe mpi 8 mpiexec my_exe Distribute mpich application on 8 processes each on a single machine. During the application runtime, can we dynamically (allocate & start)/(remove & kill) one process and (add to)/(delete from) the current communication group ? Appreciate your help ahead! Thanks, Shuwei _______________________________________________ discuss mailing list discuss(a)mpich.org<mailto:[email protected]> To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] [petsc-dev] Is mpich/master:a8a2b30fd21 tested with Petsc?
by Jed Brown 08 Nov '18

08 Nov '18

This bug is present in Ubuntu 18.10, which distributes unpatched mpich-3.3b2. I just submitted a bug report: https://bugs.launchpad.net/ubuntu/+source/mpich/+bug/1802372 Eric Chamberland <Eric.Chamberland(a)giref.ulaval.ca> writes: > Hi, > > mainly for PETSc users: please do no waste your time using MPI released > with Intel Parallel Studio 2019 since it is the buggy MPICH 3.3b2 for > which this initial thread has been created... > > I just wrote a remind about this also on Intel forum: > > https://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/t… > > Eric > > On 19/04/18 09:01 AM, Eric Chamberland wrote: >> Hi, >> >> this morning, mpich/master with PETSc is 100% working again for us. >> >> Thanks to both commits: >> >> https://github.com/pmodels/mpich/commit/c597c8d79deea220a42751fda0f01ce7076… >> >> >> https://github.com/pmodels/mpich/commit/8edabc7373b82dd660019e53d2461317658… >> >> >> and thanks to everybody who helped: >> >> Satish >> Min >> Wesley >> Ken >> Rob >> Jed >> >> :) >> >> Eric >> >> On 17/04/18 04:58 PM, Min Si wrote: >>> Hi all, >>> >>> Thanks for narrowing down the problem. I checked the MPICH code and >>> believe this is a bug in MPICH. I just created a PR to fix it: >>> https://github.com/pmodels/mpich/pull/3097 >>> >>> It should be merged into MPICH master branch soon. >>> >>> Thanks, >>> Min >>> >>> On 2018/04/17 14:10, Eric Chamberland wrote: >>>> Hi, >>>> >>>> are we talking about the "tag" passed to MPI_Isend for example? >>>> >>>> but does that mean there is something to change for any MPI call >>>> which involves tags usage or is it only a PETSc "bad" tag usage? >>>> >>>> thanks Satish for your finding! >>>> >>>> Eric >>>> >>>> On 16/04/18 11:31 PM, Satish Balay wrote: >>>>> On Tue, 13 Mar 2018, Eric Chamberland wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> each night we are testing mpich/master with our petsc-based code. >>>>>> I don't >>>>>> know if PETSc team is doing the same thing with mpich/master? >>>>>> (Maybe it is a >>>>>> good idea?) >>>>>> >>>>>> Everything was fine (except the issue >>>>>> https://github.com/pmodels/mpich/issues/2892) up to commit >>>>>> 7b8d64debd, but >>>>>> since commit mpich:a8a2b30fd21), I have a segfault on a any >>>>>> parallel nightly >>>>>> test. >>>>> >>>>> I attempted a bisect of the above range of commits - and narrowed >>>>> down to: >>>>> >>>>>>>>>>>> >>>>> db11d4c4a70e39a28be88ed32f00542301699e08 is the first bad commit >>>>> <<<<<<< >>>>>>>>>>>>> >>>>> balay@asterix /home/balay/soft/build/mpich ((db11d4c4a...)|BISECTING) >>>>> $ git show db11d4c4a70e39a28be88ed32f00542301699e08 >>>>> commit db11d4c4a70e39a28be88ed32f00542301699e08 (HEAD, refs/bisect/bad) >>>>> Author: Ken Raffenetti <raffenet(a)mcs.anl.gov> >>>>> Date: Thu Feb 15 11:37:59 2018 -0600 >>>>> >>>>> init: Fix tag upper limit initialization >>>>> The starting point for this value is equivalent to the >>>>> usable tag bits >>>>> macro. This value should be set before device initialization, >>>>> otherwise devices will assume they have more bits than are >>>>> actually >>>>> available. >>>>> Signed-off-by: Wesley Bland <wesley.bland(a)intel.com> >>>>> >>>>> diff --git a/src/mpi/init/initthread.c b/src/mpi/init/initthread.c >>>>> index cbc41f4d5..b31ae2f07 100644 >>>>> --- a/src/mpi/init/initthread.c >>>>> +++ b/src/mpi/init/initthread.c >>>>> @@ -403,7 +403,7 @@ int MPIR_Init_thread(int *argc, char ***argv, >>>>> int required, int *provided) >>>>> MPIR_Process.attrs.host = MPI_PROC_NULL; >>>>> MPIR_Process.attrs.io = MPI_PROC_NULL; >>>>> MPIR_Process.attrs.lastusedcode = MPI_ERR_LASTCODE; >>>>> - MPIR_Process.attrs.tag_ub = 0; >>>>> + MPIR_Process.attrs.tag_ub = MPIR_TAG_USABLE_BITS; >>>>> MPIR_Process.attrs.universe = MPIR_UNIVERSE_SIZE_NOT_SET; >>>>> MPIR_Process.attrs.wtime_is_global = 0; >>>>> @@ -531,13 +531,6 @@ int MPIR_Init_thread(int *argc, char ***argv, >>>>> int required, int *provided) >>>>> MPIR_Assert(((unsigned) MPIR_Process. >>>>> attrs.tag_ub & ((unsigned) >>>>> MPIR_Process.attrs.tag_ub + 1)) == 0); >>>>> - /* Set aside tag space for tagged collectives and failure >>>>> notification */ >>>>> -#ifdef HAVE_TAG_ERROR_BITS >>>>> - MPIR_Process.attrs.tag_ub >>= 3; >>>>> -#else >>>>> - MPIR_Process.attrs.tag_ub >>= 1; >>>>> -#endif >>>>> - >>>>> /* Assert: tag_ub is at least the minimum asked for in the MPI >>>>> spec */ >>>>> MPIR_Assert(MPIR_Process.attrs.tag_ub >= 32767); >>>>> <<<<<<<<<<<<<<<<< >>>>> >>>>> Reverthing this patch gets mpich-3.3b2 working with petsc >>>>> >>>>> Satish >>>>>

1 0

Re: [mpich-discuss] mpich-master: final link failed: nonrepresentable section on output
by Siegmar Gross 07 Nov '18

07 Nov '18

Hi, back in my office I tried some more versions: --enable-fortran=fc --enable-fortran=f77 --enable-fortran=no All versions break with an error. loki mpich-master_2018-11-07_Linux.x86_64.64_fc 121 head -7 config.log | tail -1 $ ../mpich/configure --prefix=/usr/local/mpich-master_64_cc --libdir=/usr/local/mpich-master_64_cc/lib64 --includedir=/usr/local/mpich-master_64_cc/include64 CC=cc CXX=CC F77=f77 FC=f95 CFLAGS=-m64 CXXFLAGS=-m64 FFLAGS=-m64 FCFLAGS=-m64 LDFLAGS=-m64 -L/usr/lib64 -R/usr/lib64 --enable-fortran=fc --enable-cxx --enable-romio --enable-debuginfo --enable-smpcoll --enable-threads=multiple --with-thread-package=posix --enable-shared fd1026 has logged off pts/4 from :0. loki mpich-master_2018-11-07_Linux.x86_64.64_fc 122 loki mpich-master_2018-11-07_Linux.x86_64.64_fc 123 tail log.configure.Linux.x86_64.64_cc checking whether to build static libraries... yes checking for f95 option to produce PIC... -KPIC checking if f95 PIC flag -KPIC works... yes checking if f95 static flag -Bstatic works... yes checking if f95 supports -c -o file.o... yes checking if f95 supports -c -o file.o... (cached) yes checking whether the f95 linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking dynamic linker characteristics... (cached) GNU/Linux ld.so checking how to hardcode library paths into programs... immediate configure: error: Fortran 90 support requires enabling Fortran 77 loki mpich-master_2018-11-07_Linux.x86_64.64_fc 12 "configure" shouldn't break, especially because "--enable-f77" is deprecated. loki mpich-master_2018-11-07_Linux.x86_64.64_f77 121 head -7 config.log | tail -1 $ ../mpich/configure --prefix=/usr/local/mpich-master_64_cc --libdir=/usr/local/mpich-master_64_cc/lib64 --includedir=/usr/local/mpich-master_64_cc/include64 CC=cc CXX=CC F77=f77 FC=f95 CFLAGS=-m64 CXXFLAGS=-m64 FFLAGS=-m64 FCFLAGS=-m64 LDFLAGS=-m64 -L/usr/lib64 -R/usr/lib64 --enable-fortran=f77 --enable-cxx --enable-romio --enable-debuginfo --enable-smpcoll --enable-threads=multiple --with-thread-package=posix --enable-shared fd1026 has logged off pts/2 from :0. loki mpich-master_2018-11-07_Linux.x86_64.64_f77 122 loki mpich-master_2018-11-07_Linux.x86_64.64_f77 125 tail -33 log.make.Linux.x86_64.64_cc | grep -v NOTICE CC src/pmi/simple/lib_libmpi_la-simple_pmi.lo CCLD src/mpi/debugger/libdbginitdummy.la CXX src/binding/cxx/initcxx.lo CC src/env/mpichversion.o "/export2/src/mpich-master/mpich/src/openpa/src/primitives/opa_gcc_intel_32_64_ops.h", line 98: warning: parameter in inline asm statement unused: %2 "/export2/src/mpich-master/mpich/src/openpa/src/primitives/opa_gcc_intel_32_64_ops.h", line 105: warning: parameter in inline asm statement unused: %1 "/export2/src/mpich-master/mpich/src/openpa/src/primitives/opa_gcc_intel_32_64_ops.h", line 112: warning: parameter in inline asm statement unused: %1 "/export2/src/mpich-master/mpich/src/openpa/src/primitives/opa_gcc_intel_32_64_ops.h", line 121: warning: parameter in inline asm statement unused: %2 "/export2/src/mpich-master/mpich/src/openpa/src/primitives/opa_gcc_intel_32_64_ops.h", line 128: warning: parameter in inline asm statement unused: %3 "/export2/src/mpich-master/mpich/src/openpa/src/primitives/opa_gcc_intel_32_64_ops.h", line 140: warning: parameter in inline asm statement unused: %1 "/export2/src/mpich-master/mpich/src/openpa/src/primitives/opa_gcc_intel_32_64_ops.h", line 148: warning: parameter in inline asm statement unused: %1 "/export2/src/mpich-master/mpich/src/openpa/src/primitives/opa_gcc_intel_32_64_ops.h", line 155: warning: parameter in inline asm statement unused: %3 "/export2/src/mpich-master/mpich/src/openpa/src/primitives/opa_gcc_intel_32_64_ops.h", line 162: warning: parameter in inline asm statement unused: %3 CC src/env/mpivars.o CCLD lib/libtvmpich.la "../mpich/src/include/mpir_handlemem.h", line 61: warning: null dimension: indirect "../mpich/src/include/mpir_handlemem.h", line 122: warning: null dimension: indirect GEN lib/libmpi.la "./src/include/mpichconf.h", line 1718: Warning (Anachronism): Attempt to redefine __restrict__ without using #undef. 1 Warning(s) detected. f90: Warning: Option -shared passed to ld, if ld is invoked, ignored otherwise /usr/bin/ld: /opt/sun/developerstudio12.6/lib/compilers/amd64/crt1x.o: relocation R_X86_64_32 against hidden symbol `__dso_handle' can not be used when making a shared object /usr/bin/ld: final link failed: nonrepresentable section on output postopt: error: ld failed to link the binary Makefile:17006: recipe for target 'lib/libmpi.la' failed make[2]: *** [lib/libmpi.la] Error 2 make[2]: Leaving directory '/export2/src/mpich-master/mpich-master_2018-11-07_Linux.x86_64.64_f77' Makefile:40516: recipe for target 'all-recursive' failed make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory '/export2/src/mpich-master/mpich-master_2018-11-07_Linux.x86_64.64_f77' Makefile:10582: recipe for target 'all' failed make: *** [all] Error 2 loki mpich-master_2018-11-07_Linux.x86_64.64_f77 126 loki mpich-master_2018-11-07_Linux.x86_64.64_no 122 head -7 config.log | tail -1 $ ../mpich/configure --prefix=/usr/local/mpich-master_64_cc --libdir=/usr/local/mpich-master_64_cc/lib64 --includedir=/usr/local/mpich-master_64_cc/include64 CC=cc CXX=CC F77=f77 FC=f95 CFLAGS=-m64 CXXFLAGS=-m64 FFLAGS=-m64 FCFLAGS=-m64 LDFLAGS=-m64 -L/usr/lib64 -R/usr/lib64 --enable-fortran=no --enable-cxx --enable-romio --enable-debuginfo --enable-smpcoll --enable-threads=multiple --with-thread-package=posix --enable-shared loki mpich-master_2018-11-07_Linux.x86_64.64_no 123 loki mpich-master_2018-11-07_Linux.x86_64.64_no 124 tail -35 log.make.Linux.x86_64.64_cc "./src/include/mpichconf.h", line 1718: Warning (Anachronism): Attempt to redefine __restrict__ without using #undef. 1 Warning(s) detected. CXXLD lib/libmpicxx.la CCLD src/env/mpichversion CCLD src/env/mpivars CC: -library=Crun cannot be used with -std=c++03. To use this library you need to switch to -std=sun03 CC: -library=Cstd cannot be used with -std=c++03. To use this library you need to switch to -std=sun03 Makefile:10787: recipe for target 'lib/libmpicxx.la' failed make[2]: *** [lib/libmpicxx.la] Error 1 make[2]: *** Waiting for unfinished jobs.... /usr/bin/ld: src/env/mpichversion.o: in function `hwloc_distrib': mpichversion.c:(.text+0xb0): undefined reference to `hwloc_bitmap_weight' /usr/bin/ld: mpichversion.c:(.text+0x13d): undefined reference to `hwloc_bitmap_weight' /usr/bin/ld: mpichversion.c:(.text+0x1e4): undefined reference to `hwloc_bitmap_dup' /usr/bin/ld: mpichversion.c:(.text+0x219): undefined reference to `hwloc_bitmap_or' /usr/bin/ld: src/env/mpichversion.o: in function `MPIR_Datatype_free_contents': mpichversion.c:(.text+0x33b): undefined reference to `MPIR_Datatype_builtin' /usr/bin/ld: mpichversion.c:(.text+0x35a): undefined reference to `MPIR_Datatype_direct' /usr/bin/ld: mpichversion.c:(.text+0x369): undefined reference to `MPIR_Datatype_mem' /usr/bin/ld: mpichversion.c:(.text+0x3b0): undefined reference to `MPIR_Process' /usr/bin/ld: mpichversion.c:(.text+0x42a): undefined reference to `MPIR_Assert_fail' /usr/bin/ld: mpichversion.c:(.text+0x447): undefined reference to `MPIR_Assert_fail' /usr/bin/ld: src/env/mpichversion.o: in function `MPIR_Datatype_free': mpichversion.c:(.text+0x47e): undefined reference to `MPIR_Dataloop_free' /usr/bin/ld: mpichversion.c:(.text+0x485): undefined reference to `MPIR_Datatype_mem' postopt: error: ld failed to link the binary cc: postopt failed for src/env/.libs/mpichversion Makefile:20520: recipe for target 'src/env/mpichversion' failed make[2]: *** [src/env/mpichversion] Error 2 make[2]: Leaving directory '/export2/src/mpich-master/mpich-master_2018-11-07_Linux.x86_64.64_no' Makefile:40516: recipe for target 'all-recursive' failed make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory '/export2/src/mpich-master/mpich-master_2018-11-07_Linux.x86_64.64_no' Makefile:10582: recipe for target 'all' failed make: *** [all] Error 2 loki mpich-master_2018-11-07_Linux.x86_64.64_no 125 I would be grateful, if somebody can fix the problems. Do you need anything else? Thank you very much for any help in advance. Kind regards Siegmar On 11/7/18 10:28 AM, Siegmar Gross wrote: > Hi, > > I've updated the software with the following commands. > > cd mpich > git pull https://github.com/pmodels/mpich.git master > git submodule update --init --recursive > autogen.sh > > I got the same error as before building the package once more > with the following configure command. > > loki mpich-master_2018-11-07_Linux.x86_64.64_cc 112 head -7 config.log | tail -1 > $ ../mpich/configure --prefix=/usr/local/mpich-master_64_cc > --libdir=/usr/local/mpich-master_64_cc/lib64 > --includedir=/usr/local/mpich-master_64_cc/include64 CC=cc CXX=CC F77=f77 FC=f95 > CFLAGS=-m64 CXXFLAGS=-m64 FFLAGS=-m64 FCFLAGS=-m64 LDFLAGS=-m64 -L/usr/lib64 > -R/usr/lib64 --enable-fortran=yes --enable-cxx --enable-romio --enable-debuginfo > --enable-smpcoll --enable-threads=multiple --with-thread-package=posix > --enable-shared > loki mpich-master_2018-11-07_Linux.x86_64.64_cc 112 > > > Best regards and thank you very much for your help > > Siegmar > > > Am 06.11.2018 um 18:21 schrieb Madhu, Kavitha Tiptur: >> Hi >> Looks like your linker is failing for fortran90. You could disable fortran >> builds during configuration if you don’t need this. Alternatively, this issue >> seems to be fixed in https://github.com/pmodels/mpich/pull/3353 and merged >> into MPICH master. Can you pull from master and try this one more time? >> >>> On Nov 1, 2018, at 2:11 AM, Siegmar Gross >>> <siegmar.gross(a)informatik.hs-fulda.de >>> <mailto:[email protected]>> wrote: >>> >>> Hi, >>> >>> thank you very much your help. I'm waiting for your solution. >>> >>> Kind regards >>> >>> Siegmar >>> >>> >>>> Sorry I was away for some time. I have created an >>>> issue(https://github.com/pmodels/mpich/issues/3358) and am looking into it. >>>> I will get back to you with updates as soon as possible. Sorry about the >>>> delays. >>>>> On Oct 31, 2018, at 2:57 AM, Siegmar Gross via discuss <discuss(a)mpich.org >>>>> <mailto:[email protected]> <mailto:[email protected]>> wrote: >>>>> >>>>> Hi, >>>>> >>>>> I tried to build mpich-master-v3.3b3-357-gdcd523e and the latest version >>>>> of mpich-master from github on my "SUSE Linux Enterprise Server 12.3 >>>>> (x86_64)" with Sun C 5.15 (Oracle Developer Studio 12.6). Unfortunately, >>>>> I get the following error for both versions. >>>>> >>>>> loki mpich-master-v3.3b3-357-gdcd523e-Linux.x86_64.64_cc 138 tail -30 >>>>> log.make.Linux.x86_64.64_cc | grep -v NOTICE >>>>> "../mpich-master-v3.3b3-357-gdcd523e/src/include/mpir_handlemem.h", line >>>>> 122: warning: null dimension: indirect >>>>> CC src/pmi/simple/lib_libmpi_la-simple_pmiutil.lo >>>>> CC src/pmi/simple/lib_libmpi_la-simple_pmi.lo >>>>> CC src/mpi/debugger/src_mpi_debugger_libdbginitdummy_la-dbginit.lo >>>>> "/export2/src/mpich-3.3/mpich-master-v3.3b3-357-gdcd523e/src/openpa/src/primitives/opa_gcc_intel_32_64_ops.h", >>>>> line 98: warning: parameter in inline asm statement unused: %2 >>>>> "/export2/src/mpich-3.3/mpich-master-v3.3b3-357-gdcd523e/src/openpa/src/primitives/opa_gcc_intel_32_64_ops.h", >>>>> line 105: warning: parameter in inline asm statement unused: %1 >>>>> "/export2/src/mpich-3.3/mpich-master-v3.3b3-357-gdcd523e/src/openpa/src/primitives/opa_gcc_intel_32_64_ops.h", >>>>> line 112: warning: parameter in inline asm statement unused: %1 >>>>> "/export2/src/mpich-3.3/mpich-master-v3.3b3-357-gdcd523e/src/openpa/src/primitives/opa_gcc_intel_32_64_ops.h", >>>>> line 121: warning: parameter in inline asm statement unused: %2 >>>>> "/export2/src/mpich-3.3/mpich-master-v3.3b3-357-gdcd523e/src/openpa/src/primitives/opa_gcc_intel_32_64_ops.h", >>>>> line 128: warning: parameter in inline asm statement unused: %3 >>>>> "/export2/src/mpich-3.3/mpich-master-v3.3b3-357-gdcd523e/src/openpa/src/primitives/opa_gcc_intel_32_64_ops.h", >>>>> line 140: warning: parameter in inline asm statement unused: %1 >>>>> "/export2/src/mpich-3.3/mpich-master-v3.3b3-357-gdcd523e/src/openpa/src/primitives/opa_gcc_intel_32_64_ops.h", >>>>> line 148: warning: parameter in inline asm statement unused: %1 >>>>> "/export2/src/mpich-3.3/mpich-master-v3.3b3-357-gdcd523e/src/openpa/src/primitives/opa_gcc_intel_32_64_ops.h", >>>>> line 155: warning: parameter in inline asm statement unused: %3 >>>>> "/export2/src/mpich-3.3/mpich-master-v3.3b3-357-gdcd523e/src/openpa/src/primitives/opa_gcc_intel_32_64_ops.h", >>>>> line 162: warning: parameter in inline asm statement unused: %3 >>>>> "../mpich-master-v3.3b3-357-gdcd523e/src/include/mpir_handlemem.h", line >>>>> 61: warning: null dimension: indirect >>>>> "../mpich-master-v3.3b3-357-gdcd523e/src/include/mpir_handlemem.h", line >>>>> 122: warning: null dimension: indirect >>>>> CCLD src/mpi/debugger/libdbginitdummy.la <http://libdbginitdummy.la> >>>>> <http://libdbginitdummy.la> >>>>> GEN lib/libmpi.la <http://libmpi.la> <http://libmpi.la> >>>>> f90: Warning: Option -shared passed to ld, if ld is invoked, ignored otherwise >>>>> /usr/bin/ld: /opt/sun/developerstudio12.6/lib/compilers/amd64/crt1x.o: >>>>> relocation R_X86_64_32 against hidden symbol `__dso_handle' can not be used >>>>> when making a shared object >>>>> /usr/bin/ld: final link failed: nonrepresentable section on output >>>>> postopt: error: ld failed to link the binary >>>>> Makefile:17007: recipe for target 'lib/libmpi.la <http://libmpi.la> >>>>> <http://libmpi.la>' failed >>>>> make[2]: *** [lib/libmpi.la <http://libmpi.la> <http://libmpi.la>] Error 2 >>>>> make[2]: Leaving directory >>>>> '/export2/src/mpich-3.3/mpich-master-v3.3b3-357-gdcd523e-Linux.x86_64.64_cc' >>>>> Makefile:40517: recipe for target 'all-recursive' failed >>>>> make[1]: *** [all-recursive] Error 1 >>>>> make[1]: Leaving directory >>>>> '/export2/src/mpich-3.3/mpich-master-v3.3b3-357-gdcd523e-Linux.x86_64.64_cc' >>>>> Makefile:10583: recipe for target 'all' failed >>>>> make: *** [all] Error 2 >>>>> loki mpich-master-v3.3b3-357-gdcd523e-Linux.x86_64.64_cc 138 >>>>> >>>>> >>>>> loki mpich-master-v3.3b3-357-gdcd523e-Linux.x86_64.64_cc 141 head -7 >>>>> config.log | tail -1 >>>>> $ ../mpich-master-v3.3b3-357-gdcd523e/configure >>>>> --prefix=/usr/local/mpich-3.3_64_cc >>>>> --libdir=/usr/local/mpich-3.3_64_cc/lib64 >>>>> --includedir=/usr/local/mpich-3.3_64_cc/include64 CC=cc CXX=CC F77=f77 >>>>> FC=f95 CFLAGS=-m64 CXXFLAGS=-m64 FFLAGS=-m64 FCFLAGS=-m64 LDFLAGS=-m64 >>>>> -L/usr/lib64 -R/usr/lib64 --enable-fortran=yes --enable-cxx --enable-romio >>>>> --enable-debuginfo --enable-smpcoll --enable-threads=multiple >>>>> --with-thread-package=posix --enable-shared >>>>> loki mpich-master-v3.3b3-357-gdcd523e-Linux.x86_64.64_cc 142 >>>>> >>>>> >>>>> I would be grateful, if somebody can fix the problem. Do you need anything >>>>> else? Thank you very much for any help in advance. >>>>> >>>>> >>>>> Kind regards >>>>> >>>>> Siegmar >>>>> _______________________________________________ >>>>> discuss mailing list discuss(a)mpich.org <mailto:[email protected]> >>>>> <mailto:[email protected]> >>>>> To manage subscription options or unsubscribe: >>>>> https://lists.mpich.org/mailman/listinfo/discuss >>

1 0