- discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] BLCR kernel module not present
by michael 26 Apr '13

26 Apr '13

Thanks Pavan Just to be clear, you're saying if I use mvapich with blcr then a running multi-node MPI job when killed (eg out of time) by a batch scheduler can be restarted (from which checkpoint?) presuming it doesn't have open files? Many thanks, M On Fri, 2013-04-26 at 15:12 -0500, Pavan Balaji wrote: > Michael, > > BLCR support for mpich should work fine; if something is broken, please > let us know. > > However, the core BLCR group itself hadn't released updates in a while, > primarily because they didn't have direct funding for it. I believe > that's fixed now and they are working on newer releases. > > -- Pavan > > On 04/26/2013 03:07 PM US Central Time, michael wrote: > > Hi folks > > I was wondering what the state of BLCR for mpich/mvapich is? eg how > > reliably can one presume it to be? > > Thanks, Michael > > > > > > On Fri, 2013-04-26 at 14:33 -0500, Wesley Bland wrote: > >> It looks like you might have missed installing the kernel module for > >> BLCR. What is the output of `lsmod`? > >> > >> > >> Alternatively, if you installed BLCR by using apt-get in Ubuntu, you > >> should be able to use dkms to manage your kernel modules > >> automatically. Make sure you have the package 'blcr-dkms' installed > >> (you should be able to check this by typing `dims status`. > >> > >> > >> Do either of those solutions solve your issue? > >> > >> > >> Wesley > >> > >> On Apr 26, 2013, at 1:55 PM, basma a.azeem > >> <basmaabdelazeem(a)hotmail.com <mailto:[email protected]>> wrote: > >> > >>> > >>> > >>> Thank you for your help > >>> > >>> > >>> i installed BLCR 0.8.5 on my ubuntu 12.10 to be used for MPICH -3.0.3 > >>> this version of blcr should support to kernels through 3.7.1 > >>> > >>> when i run the command : > >>> basma@basma-Satellite-A500:~$ mpiexec --info > >>> > >>> results: > >>> > >>> HYDRA build details: > >>> Version: 3.0.3 > >>> Release Date: Thu Mar 28 16:01:21 CDT 2013 > >>> CC: gcc > >>> CXX: c++ > >>> F77: no > >>> F90: no > >>> Configure options: > >>> '--disable-option-checking' '--prefix=/home/basma/mpich2-install' > >>> '--disable-f77' '--disable-fc' '--enable-checkpointing' > >>> '--with-hydra-ckpointlib=blcr' '--cache-file=/dev/null' '--srcdir=.' > >>> 'CC=gcc' 'CFLAGS= -O2' 'LDFLAGS= ' 'LIBS=-lrt -lcr -lpthread ' > >>> 'CPPFLAGS= -I/home/basma/libraries/mpich-3.0.3/src/mpl/include > >>> -I/home/basma/libraries/mpich-3.0.3/src/mpl/include > >>> -I/home/basma/libraries/mpich-3.0.3/src/openpa/src > >>> -I/home/basma/libraries/mpich-3.0.3/src/openpa/src > >>> -I/home/basma/libraries/mpich-3.0.3/src/mpi/romio/include' > >>> Process Manager: pmi > >>> Launchers available: ssh rsh fork slurm ll > >>> lsf sge manual persist > >>> Topology libraries available: hwloc > >>> Resource management kernels available: user slurm ll lsf sge > >>> pbs cobalt > >>> Checkpointing libraries available: blcr > >>> Demux engines available: poll select > >>> > >>> so i thought that every thing is ok but when i try to rum mpiexec it > >>> failed: > >>> > >>> basma@basma-Satellite-A500:~$ mpiexec -ckpointlib blcr > >>> -ckpoint-prefix /home/business/ckpts/app.ckpoint -ckpoint-interval > >>> 3600 -n 4 /home/basma/libraries/mpich-3.0.3/examples/cpi > >>> > >>> results: > >>> > >>> Fatal error in MPI_Init: Other MPI error, error stack: > >>> MPIR_Init_thread(433)...: > >>> MPID_Init(151)..........: channel initialization failed > >>> MPIDI_CH3_Init(70)......: > >>> MPID_nem_init(379)......: > >>> MPIDI_nem_ckpt_init(153): BLCR kernel module not present > >>> Fatal error in MPI_Init: Other MPI error, error stack: > >>> MPIR_Init_thread(433)...: > >>> MPID_Init(151)..........: channel initialization failed > >>> MPIDI_CH3_Init(70)......: > >>> MPID_nem_init(379)......: > >>> MPIDI_nem_ckpt_init(153): BLCR kernel module not present > >>> Fatal error in MPI_Init: Other MPI error, error stack: > >>> MPIR_Init_thread(433)...: > >>> MPID_Init(151)..........: channel initialization failed > >>> MPIDI_CH3_Init(70)......: > >>> MPID_nem_init(379)......: > >>> MPIDI_nem_ckpt_init(153): BLCR kernel module not present > >>> Fatal error in MPI_Init: Other MPI error, error stack: > >>> MPIR_Init_thread(433)...: > >>> MPID_Init(151)..........: channel initialization failed > >>> MPIDI_CH3_Init(70)......: > >>> MPID_nem_init(379)......: > >>> MPIDI_nem_ckpt_init(153): BLCR kernel module not present > >>> > >>> =================================================================================== > >>> = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES > >>> = EXIT CODE: 1 > >>> = CLEANING UP REMAINING PROCESSES > >>> = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES > >>> =================================================================================== > >>> > >>> > >>> > >>> i am a Linux and parallel programming beginner > >>> > >>> Thank you > >>> > >>> > >>> > >>> _______________________________________________ > >>> discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > >>> To manage subscription options or unsubscribe: > >>> https://lists.mpich.org/mailman/listinfo/discuss > >> > >> > >> _______________________________________________ > >> discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > >> To manage subscription options or unsubscribe: > >> https://lists.mpich.org/mailman/listinfo/discuss > > > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > >

1 0

Re: [mpich-discuss] BLCR kernel module not present
by Pavan Balaji 26 Apr '13

26 Apr '13

Michael, BLCR support for mpich should work fine; if something is broken, please let us know. However, the core BLCR group itself hadn't released updates in a while, primarily because they didn't have direct funding for it. I believe that's fixed now and they are working on newer releases. -- Pavan On 04/26/2013 03:07 PM US Central Time, michael wrote: > Hi folks > I was wondering what the state of BLCR for mpich/mvapich is? eg how > reliably can one presume it to be? > Thanks, Michael > > > On Fri, 2013-04-26 at 14:33 -0500, Wesley Bland wrote: >> It looks like you might have missed installing the kernel module for >> BLCR. What is the output of `lsmod`? >> >> >> Alternatively, if you installed BLCR by using apt-get in Ubuntu, you >> should be able to use dkms to manage your kernel modules >> automatically. Make sure you have the package 'blcr-dkms' installed >> (you should be able to check this by typing `dims status`. >> >> >> Do either of those solutions solve your issue? >> >> >> Wesley >> >> On Apr 26, 2013, at 1:55 PM, basma a.azeem >> <basmaabdelazeem(a)hotmail.com <mailto:[email protected]>> wrote: >> >>> >>> >>> Thank you for your help >>> >>> >>> i installed BLCR 0.8.5 on my ubuntu 12.10 to be used for MPICH -3.0.3 >>> this version of blcr should support to kernels through 3.7.1 >>> >>> when i run the command : >>> basma@basma-Satellite-A500:~$ mpiexec --info >>> >>> results: >>> >>> HYDRA build details: >>> Version: 3.0.3 >>> Release Date: Thu Mar 28 16:01:21 CDT 2013 >>> CC: gcc >>> CXX: c++ >>> F77: no >>> F90: no >>> Configure options: >>> '--disable-option-checking' '--prefix=/home/basma/mpich2-install' >>> '--disable-f77' '--disable-fc' '--enable-checkpointing' >>> '--with-hydra-ckpointlib=blcr' '--cache-file=/dev/null' '--srcdir=.' >>> 'CC=gcc' 'CFLAGS= -O2' 'LDFLAGS= ' 'LIBS=-lrt -lcr -lpthread ' >>> 'CPPFLAGS= -I/home/basma/libraries/mpich-3.0.3/src/mpl/include >>> -I/home/basma/libraries/mpich-3.0.3/src/mpl/include >>> -I/home/basma/libraries/mpich-3.0.3/src/openpa/src >>> -I/home/basma/libraries/mpich-3.0.3/src/openpa/src >>> -I/home/basma/libraries/mpich-3.0.3/src/mpi/romio/include' >>> Process Manager: pmi >>> Launchers available: ssh rsh fork slurm ll >>> lsf sge manual persist >>> Topology libraries available: hwloc >>> Resource management kernels available: user slurm ll lsf sge >>> pbs cobalt >>> Checkpointing libraries available: blcr >>> Demux engines available: poll select >>> >>> so i thought that every thing is ok but when i try to rum mpiexec it >>> failed: >>> >>> basma@basma-Satellite-A500:~$ mpiexec -ckpointlib blcr >>> -ckpoint-prefix /home/business/ckpts/app.ckpoint -ckpoint-interval >>> 3600 -n 4 /home/basma/libraries/mpich-3.0.3/examples/cpi >>> >>> results: >>> >>> Fatal error in MPI_Init: Other MPI error, error stack: >>> MPIR_Init_thread(433)...: >>> MPID_Init(151)..........: channel initialization failed >>> MPIDI_CH3_Init(70)......: >>> MPID_nem_init(379)......: >>> MPIDI_nem_ckpt_init(153): BLCR kernel module not present >>> Fatal error in MPI_Init: Other MPI error, error stack: >>> MPIR_Init_thread(433)...: >>> MPID_Init(151)..........: channel initialization failed >>> MPIDI_CH3_Init(70)......: >>> MPID_nem_init(379)......: >>> MPIDI_nem_ckpt_init(153): BLCR kernel module not present >>> Fatal error in MPI_Init: Other MPI error, error stack: >>> MPIR_Init_thread(433)...: >>> MPID_Init(151)..........: channel initialization failed >>> MPIDI_CH3_Init(70)......: >>> MPID_nem_init(379)......: >>> MPIDI_nem_ckpt_init(153): BLCR kernel module not present >>> Fatal error in MPI_Init: Other MPI error, error stack: >>> MPIR_Init_thread(433)...: >>> MPID_Init(151)..........: channel initialization failed >>> MPIDI_CH3_Init(70)......: >>> MPID_nem_init(379)......: >>> MPIDI_nem_ckpt_init(153): BLCR kernel module not present >>> >>> =================================================================================== >>> = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES >>> = EXIT CODE: 1 >>> = CLEANING UP REMAINING PROCESSES >>> = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES >>> =================================================================================== >>> >>> >>> >>> i am a Linux and parallel programming beginner >>> >>> Thank you >>> >>> >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org <mailto:[email protected]> >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org <mailto:[email protected]> >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] BLCR kernel module not present
by michael 26 Apr '13

26 Apr '13

Hi folks I was wondering what the state of BLCR for mpich/mvapich is? eg how reliably can one presume it to be? Thanks, Michael On Fri, 2013-04-26 at 14:33 -0500, Wesley Bland wrote: > It looks like you might have missed installing the kernel module for > BLCR. What is the output of `lsmod`? > > > > Alternatively, if you installed BLCR by using apt-get in Ubuntu, you > should be able to use dkms to manage your kernel modules > automatically. Make sure you have the package 'blcr-dkms' installed > (you should be able to check this by typing `dims status`. > > > Do either of those solutions solve your issue? > > > Wesley > > > On Apr 26, 2013, at 1:55 PM, basma a.azeem > <basmaabdelazeem(a)hotmail.com> wrote: > > > > > > > > > > > Thank you for your help > > > > > > i installed BLCR 0.8.5 on my ubuntu 12.10 to be used for > > MPICH -3.0.3 > > this version of blcr should support to kernels through 3.7.1 > > > > when i run the command : > > basma@basma-Satellite-A500:~$ mpiexec --info > > > > results: > > > > HYDRA build details: > > Version: 3.0.3 > > Release Date: Thu Mar 28 16:01:21 CDT > > 2013 > > CC: gcc > > CXX: c++ > > F77: no > > F90: no > > Configure options: > > '--disable-option-checking' '--prefix=/home/basma/mpich2-install' > > '--disable-f77' '--disable-fc' '--enable-checkpointing' > > '--with-hydra-ckpointlib=blcr' '--cache-file=/dev/null' '--srcdir=.' > > 'CC=gcc' 'CFLAGS= -O2' 'LDFLAGS= ' 'LIBS=-lrt -lcr -lpthread ' > > 'CPPFLAGS= -I/home/basma/libraries/mpich-3.0.3/src/mpl/include > > -I/home/basma/libraries/mpich-3.0.3/src/mpl/include > > -I/home/basma/libraries/mpich-3.0.3/src/openpa/src > > -I/home/basma/libraries/mpich-3.0.3/src/openpa/src > > -I/home/basma/libraries/mpich-3.0.3/src/mpi/romio/include' > > Process Manager: pmi > > Launchers available: ssh rsh fork slurm ll > > lsf sge manual persist > > Topology libraries available: hwloc > > Resource management kernels available: user slurm ll lsf sge > > pbs cobalt > > Checkpointing libraries available: blcr > > Demux engines available: poll select > > > > so i thought that every thing is ok but when i try to rum mpiexec it > > failed: > > > > basma@basma-Satellite-A500:~$ mpiexec -ckpointlib blcr > > -ckpoint-prefix /home/business/ckpts/app.ckpoint -ckpoint-interval > > 3600 -n 4 /home/basma/libraries/mpich-3.0.3/examples/cpi > > > > results: > > > > Fatal error in MPI_Init: Other MPI error, error stack: > > MPIR_Init_thread(433)...: > > MPID_Init(151)..........: channel initialization failed > > MPIDI_CH3_Init(70)......: > > MPID_nem_init(379)......: > > MPIDI_nem_ckpt_init(153): BLCR kernel module not present > > Fatal error in MPI_Init: Other MPI error, error stack: > > MPIR_Init_thread(433)...: > > MPID_Init(151)..........: channel initialization failed > > MPIDI_CH3_Init(70)......: > > MPID_nem_init(379)......: > > MPIDI_nem_ckpt_init(153): BLCR kernel module not present > > Fatal error in MPI_Init: Other MPI error, error stack: > > MPIR_Init_thread(433)...: > > MPID_Init(151)..........: channel initialization failed > > MPIDI_CH3_Init(70)......: > > MPID_nem_init(379)......: > > MPIDI_nem_ckpt_init(153): BLCR kernel module not present > > Fatal error in MPI_Init: Other MPI error, error stack: > > MPIR_Init_thread(433)...: > > MPID_Init(151)..........: channel initialization failed > > MPIDI_CH3_Init(70)......: > > MPID_nem_init(379)......: > > MPIDI_nem_ckpt_init(153): BLCR kernel module not present > > > > =================================================================================== > > = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES > > = EXIT CODE: 1 > > = CLEANING UP REMAINING PROCESSES > > = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES > > =================================================================================== > > > > > > > > i am a Linux and parallel programming beginner > > > > Thank you > > > > > > > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] BLCR kernel module not present
by Wesley Bland 26 Apr '13

26 Apr '13

It looks like you might have missed installing the kernel module for BLCR. What is the output of `lsmod`? Alternatively, if you installed BLCR by using apt-get in Ubuntu, you should be able to use dkms to manage your kernel modules automatically. Make sure you have the package 'blcr-dkms' installed (you should be able to check this by typing `dims status`. Do either of those solutions solve your issue? Wesley On Apr 26, 2013, at 1:55 PM, basma a.azeem <basmaabdelazeem(a)hotmail.com> wrote: > > > Thank you for your help > > > i installed BLCR 0.8.5 on my ubuntu 12.10 to be used for MPICH -3.0.3 > this version of blcr should support to kernels through 3.7.1 > > when i run the command : > basma@basma-Satellite-A500:~$ mpiexec --info > > results: > > HYDRA build details: > Version: 3.0.3 > Release Date: Thu Mar 28 16:01:21 CDT 2013 > CC: gcc > CXX: c++ > F77: no > F90: no > Configure options: '--disable-option-checking' '--prefix=/home/basma/mpich2-install' '--disable-f77' '--disable-fc' '--enable-checkpointing' '--with-hydra-ckpointlib=blcr' '--cache-file=/dev/null' '--srcdir=.' 'CC=gcc' 'CFLAGS= -O2' 'LDFLAGS= ' 'LIBS=-lrt -lcr -lpthread ' 'CPPFLAGS= -I/home/basma/libraries/mpich-3.0.3/src/mpl/include -I/home/basma/libraries/mpich-3.0.3/src/mpl/include -I/home/basma/libraries/mpich-3.0.3/src/openpa/src -I/home/basma/libraries/mpich-3.0.3/src/openpa/src -I/home/basma/libraries/mpich-3.0.3/src/mpi/romio/include' > Process Manager: pmi > Launchers available: ssh rsh fork slurm ll lsf sge manual persist > Topology libraries available: hwloc > Resource management kernels available: user slurm ll lsf sge pbs cobalt > Checkpointing libraries available: blcr > Demux engines available: poll select > > so i thought that every thing is ok but when i try to rum mpiexec it failed: > > basma@basma-Satellite-A500:~$ mpiexec -ckpointlib blcr -ckpoint-prefix /home/business/ckpts/app.ckpoint -ckpoint-interval 3600 -n 4 /home/basma/libraries/mpich-3.0.3/examples/cpi > > results: > > Fatal error in MPI_Init: Other MPI error, error stack: > MPIR_Init_thread(433)...: > MPID_Init(151)..........: channel initialization failed > MPIDI_CH3_Init(70)......: > MPID_nem_init(379)......: > MPIDI_nem_ckpt_init(153): BLCR kernel module not present > Fatal error in MPI_Init: Other MPI error, error stack: > MPIR_Init_thread(433)...: > MPID_Init(151)..........: channel initialization failed > MPIDI_CH3_Init(70)......: > MPID_nem_init(379)......: > MPIDI_nem_ckpt_init(153): BLCR kernel module not present > Fatal error in MPI_Init: Other MPI error, error stack: > MPIR_Init_thread(433)...: > MPID_Init(151)..........: channel initialization failed > MPIDI_CH3_Init(70)......: > MPID_nem_init(379)......: > MPIDI_nem_ckpt_init(153): BLCR kernel module not present > Fatal error in MPI_Init: Other MPI error, error stack: > MPIR_Init_thread(433)...: > MPID_Init(151)..........: channel initialization failed > MPIDI_CH3_Init(70)......: > MPID_nem_init(379)......: > MPIDI_nem_ckpt_init(153): BLCR kernel module not present > > =================================================================================== > = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES > = EXIT CODE: 1 > = CLEANING UP REMAINING PROCESSES > = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES > =================================================================================== > > > > i am a Linux and parallel programming beginner > > Thank you > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

FW: BLCR kernel module not present
by basma a.azeem 26 Apr '13

26 Apr '13

Thank you for your help i installed BLCR 0.8.5 on my ubuntu 12.10 to be used for MPICH -3.0.3 this version of blcr should support to kernels through 3.7.1 when i run the command : basma@basma-Satellite-A500:~$ mpiexec --info results: HYDRA build details: Version: 3.0.3 Release Date: Thu Mar 28 16:01:21 CDT 2013 CC: gcc CXX: c++ F77: no F90: no Configure options: '--disable-option-checking' '--prefix=/home/basma/mpich2-install' '--disable-f77' '--disable-fc' '--enable-checkpointing' '--with-hydra-ckpointlib=blcr' '--cache-file=/dev/null' '--srcdir=.' 'CC=gcc' 'CFLAGS= -O2' 'LDFLAGS= ' 'LIBS=-lrt -lcr -lpthread ' 'CPPFLAGS= -I/home/basma/libraries/mpich-3.0.3/src/mpl/include -I/home/basma/libraries/mpich-3.0.3/src/mpl/include -I/home/basma/libraries/mpich-3.0.3/src/openpa/src -I/home/basma/libraries/mpich-3.0.3/src/openpa/src -I/home/basma/libraries/mpich-3.0.3/src/mpi/romio/include' Process Manager: pmi Launchers available: ssh rsh fork slurm ll lsf sge manual persist Topology libraries available: hwloc Resource management kernels available: user slurm ll lsf sge pbs cobalt Checkpointing libraries available: blcr Demux engines available: poll select so i thought that every thing is ok but when i try to rum mpiexec it failed: basma@basma-Satellite-A500:~$ mpiexec -ckpointlib blcr -ckpoint-prefix /home/business/ckpts/app.ckpoint -ckpoint-interval 3600 -n 4 /home/basma/libraries/mpich-3.0.3/examples/cpi results: Fatal error in MPI_Init: Other MPI error, error stack: MPIR_Init_thread(433)...: MPID_Init(151)..........: channel initialization failed MPIDI_CH3_Init(70)......: MPID_nem_init(379)......: MPIDI_nem_ckpt_init(153): BLCR kernel module not present Fatal error in MPI_Init: Other MPI error, error stack: MPIR_Init_thread(433)...: MPID_Init(151)..........: channel initialization failed MPIDI_CH3_Init(70)......: MPID_nem_init(379)......: MPIDI_nem_ckpt_init(153): BLCR kernel module not present Fatal error in MPI_Init: Other MPI error, error stack: MPIR_Init_thread(433)...: MPID_Init(151)..........: channel initialization failed MPIDI_CH3_Init(70)......: MPID_nem_init(379)......: MPIDI_nem_ckpt_init(153): BLCR kernel module not present Fatal error in MPI_Init: Other MPI error, error stack: MPIR_Init_thread(433)...: MPID_Init(151)..........: channel initialization failed MPIDI_CH3_Init(70)......: MPID_nem_init(379)......: MPIDI_nem_ckpt_init(153): BLCR kernel module not present =================================================================================== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 1 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =================================================================================== i am a Linux and parallel programming beginner Thank you

1 0

Re: [mpich-discuss] segment fault when using knem
by Dave Goodell 26 Apr '13

26 Apr '13

On Apr 25, 2013, at 8:03 PM CDT, M Xie <xmxmxie(a)gmail.com> wrote: > I am doing some tests with mpich in a dual-processor SMP server. > Now I use mpich-3.0.3, I also use knem for accelerating intra-node communication. > > The channel I used is nemesis:tcp, and nemesis-local-lmt=knem. > But when I use osu_benchmarks to do the bandwidth test, osu_bw will segment > fault at the 2MB, which is the default value of MPICH_NEM_LMT_DMA_THRESHOLD. > It seems when DMA channel in knem is used, the segment fault occurs. > When I set MPICH_NEM_LMT_DMA_THRESHOLD to a smaller value, such as > 131072, osu_bw will segment fault at 131072. > > I also test NAS Parallell Benchmark. I noticed when DMA channel in knem is used, > sometimes the NPB tests will be freezed after running for a while. > > In the attachment, I list some config and core dump files. > > Did anyone met or solved the similar problems. (I saw this over on the knem list but didn't have time until now to respond) Our knem support hasn't been touched in a long time. I think the version we ship in "contrib" is very old (0.5.0), before the full RDMA-style interface existed. Our side of the code probably hasn't been updated since that timeframe, modulo a patch that Brice contributed. Also, we don't run it in our nightly tests, so it's possible it has bit-rotted for another reason. It looks like there's a bad request handle being dereferenced, though just from the stack trace it's hard to tell whether that's a corrupted handle (garbage) or a stale handle value which has been freed. We could be hitting this longstanding LMT bug: https://trac.mpich.org/projects/mpich/ticket/1039 Let me try to setup an environment where I can reproduce this and see if there's an easy/obvious fix. -Dave

1 0

segment fault when using knem
by M Xie 25 Apr '13

25 Apr '13

Hello, I am doing some tests with mpich in a dual-processor SMP server. Now I use mpich-3.0.3, I also use knem for accelerating intra-node communication. The channel I used is nemesis:tcp, and nemesis-local-lmt=knem. But when I use osu_benchmarks to do the bandwidth test, osu_bw will segment fault at the 2MB, which is the default value of MPICH_NEM_LMT_DMA_THRESHOLD. It seems when DMA channel in knem is used, the segment fault occurs. When I set MPICH_NEM_LMT_DMA_THRESHOLD to a smaller value, such as 131072, osu_bw will segment fault at 131072. I also test NAS Parallell Benchmark. I noticed when DMA channel in knem is used, sometimes the NPB tests will be freezed after running for a while. In the attachment, I list some config and core dump files. Did anyone met or solved the similar problems. Thanks for your help.

1 0

Re: [mpich-discuss] MPICH 3.0.4 Ubuntu, Debian and Slackware packages (x86 and amd64)
by Pavan Balaji 25 Apr '13

25 Apr '13

1 0

MPICH 3.0.4 Ubuntu, Debian and Slackware packages (x86 and amd64)
by Marcelo Soares Souza 25 Apr '13

25 Apr '13

1 0

Retagged mpich-3.0.4
by Pavan Balaji 25 Apr '13

25 Apr '13

All, We accidentally tagged mpich-3.0.4 incorrectly in the mpich git repo to point to [85acf6c5], instead of the correct hash [0f1067da]. The release tarball is still fine; just the git tag was incorrect. We have fixed this in the git repo now. If you have pulled the 3.0.4 tag from the mpich git repo, please correct your cloned copy using: % git tag -d v3.0.4 % git fetch origin tag v3.0.4 You might also be able to do: % git pull --tags (though we are not entirely sure that'll work). Sorry about the mess-up. Once again, the release tarball is correct. Just the tag was incorrect. Regards, The MPICH Team -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0