May 2014 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] question on correct use of mpi_isend/mpi_irecv
by Lu, Huiwei 31 May '14

31 May '14

Hi, Iulian, The communication protocol seems fine to me. It looks A has allocated some buffer for sending but forgets to free it. You may want to check if all malloc’ed memory in your program have been properly freed. You may also use valgrind to detect memory leak. See details in http://wiki.mpich.org/mpich/index.php/Support_for_Debugging_Memory_Allocati… — Huiwei Lu Postdoc Appointee Mathematics and Computer Science Division, Argonne National Laboratory http://www.mcs.anl.gov/~huiweilu/ On May 27, 2014, at 11:15 AM, Grindeanu, Iulian R. <iulian(a)mcs.anl.gov> wrote: > Hello, > We are trying to track down a memory leak, and we are not sure if we are using correctly MPI_ISend and MPI_Irecv > > We would like to use non-blocking communication, between pairs of processes. > > (it happens during a computation, at every time step; every processor has a list of processors it needs to communicate with; what we do not know, is the size of the messages, in advance) > > Let's assume A needs to send to B > Because we do not know the size in advance, we send first a fixed size message from A to B; this first message has info about how big the total message needs to be; A knows it has to send more. When B receives the first part, it sends an ack (small size, int 4) to A, acknowledging, and also resizing the local buffer, to fill more data; then A sends the rest. > > All these sends / receives use Isend and Irecv, and we try to match the messages, using proper tags. We use different tags for fixed size, ack, and rest of the message. > > Before sending the code, I would like to know is what we try is feasible and doable? > > We notice that the memory use on processor A keeps increasing, when we do this in a loop, on the order of about 1000-2000 bytes per iteration. > > I think we are matching correctly isends and ireceives, and that our buffers are not leaking (we may be wrong) > > So again, an iteration is like this: > A sends a fixed size message to B; B sends back an ack, and when A receives it, sends the rest, because B has now the proper buffer to receive. > > Next iteration, the size of the message is again unknown, so we do again the dance. > > Do you have an example like that we can use? Should we use other types of sends/receives? > > Thanks, > Iulian > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] What is the difference between MPI_THREAD_SINGLE and MPI_THREAD_FUNNELED
by Boisvert, Sebastien 30 May '14

30 May '14

On Friday, May 30, 2014 12:26 PM, discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of Rajeev Thakur [thakur(a)mcs.anl.gov] wrote: > To: discuss(a)mpich.org > Subject: Re: [mpich-discuss] What is the difference between MPI_THREAD_SINGLE and MPI_THREAD_FUNNELED > > It shouldn't, but what does it return in your program? I am currently using MPI_Init (I thought it was OK if only the main thread calls MPI functions). Code-> https://github.com/GeneAssembly/biosal/blob/master/engine/node.c#L33 > > Rajeev > > On May 30, 2014, at 11:33 AM, "Boisvert, Sebastien" <boisvert(a)anl.gov> wrote: > >> >> On Friday, May 30, 2014 11:22 AM, discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of Rajeev Thakur [thakur(a)mcs.anl.gov] wrote: >>> To: discuss(a)mpich.org >>> Subject: Re: [mpich-discuss] What is the difference between MPI_THREAD_SINGLE and MPI_THREAD_FUNNELED >>> >>> You should use MPI_THREAD_FUNNELED. MPI_THREAD_SINGLE should be used if your program has one thread total. >> >> Is it possible with MPICH that >> MPI_Init_thread(&argc, &argv, MPI_THREAD_FUNNELED, &provided) >> sets provided to MPI_THREAD_SINGLE ? >> >> >> Thanks >> >>> >>> Rajeev >>> >>> >>> On May 30, 2014, at 10:49 AM, "Boisvert, Sebastien" <boisvert(a)anl.gov> wrote: >>> >>>> Hi MPICH community: >>>> >>>> With MPICH, if my MPI process has 10 threads (including the main thread) and only the main thread >>>> calls MPI functions, do I need to use MPI_THREAD_SINGLE or MPI_THREAD_FUNNELED >>>> when I invoke MPI_Init_thread ? >>>> >>>> Thanks a ton ! >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] What is the difference between MPI_THREAD_SINGLE and MPI_THREAD_FUNNELED
by Rajeev Thakur 30 May '14

30 May '14

It shouldn't, but what does it return in your program? Rajeev On May 30, 2014, at 11:33 AM, "Boisvert, Sebastien" <boisvert(a)anl.gov> wrote: > > On Friday, May 30, 2014 11:22 AM, discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of Rajeev Thakur [thakur(a)mcs.anl.gov] wrote: >> To: discuss(a)mpich.org >> Subject: Re: [mpich-discuss] What is the difference between MPI_THREAD_SINGLE and MPI_THREAD_FUNNELED >> >> You should use MPI_THREAD_FUNNELED. MPI_THREAD_SINGLE should be used if your program has one thread total. > > Is it possible with MPICH that > MPI_Init_thread(&argc, &argv, MPI_THREAD_FUNNELED, &provided) > sets provided to MPI_THREAD_SINGLE ? > > > Thanks > >> >> Rajeev >> >> >> On May 30, 2014, at 10:49 AM, "Boisvert, Sebastien" <boisvert(a)anl.gov> wrote: >> >>> Hi MPICH community: >>> >>> With MPICH, if my MPI process has 10 threads (including the main thread) and only the main thread >>> calls MPI functions, do I need to use MPI_THREAD_SINGLE or MPI_THREAD_FUNNELED >>> when I invoke MPI_Init_thread ? >>> >>> Thanks a ton ! >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] What is the difference between MPI_THREAD_SINGLE and MPI_THREAD_FUNNELED
by Boisvert, Sebastien 30 May '14

30 May '14

On Friday, May 30, 2014 11:22 AM, discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of Rajeev Thakur [thakur(a)mcs.anl.gov] wrote: > To: discuss(a)mpich.org > Subject: Re: [mpich-discuss] What is the difference between MPI_THREAD_SINGLE and MPI_THREAD_FUNNELED > > You should use MPI_THREAD_FUNNELED. MPI_THREAD_SINGLE should be used if your program has one thread total. Is it possible with MPICH that MPI_Init_thread(&argc, &argv, MPI_THREAD_FUNNELED, &provided) sets provided to MPI_THREAD_SINGLE ? Thanks > > Rajeev > > > On May 30, 2014, at 10:49 AM, "Boisvert, Sebastien" <boisvert(a)anl.gov> wrote: > >> Hi MPICH community: >> >> With MPICH, if my MPI process has 10 threads (including the main thread) and only the main thread >> calls MPI functions, do I need to use MPI_THREAD_SINGLE or MPI_THREAD_FUNNELED >> when I invoke MPI_Init_thread ? >> >> Thanks a ton ! >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] What is the difference between MPI_THREAD_SINGLE and MPI_THREAD_FUNNELED
by Rajeev Thakur 30 May '14

30 May '14

You should use MPI_THREAD_FUNNELED. MPI_THREAD_SINGLE should be used if your program has one thread total. Rajeev On May 30, 2014, at 10:49 AM, "Boisvert, Sebastien" <boisvert(a)anl.gov> wrote: > Hi MPICH community: > > With MPICH, if my MPI process has 10 threads (including the main thread) and only the main thread > calls MPI functions, do I need to use MPI_THREAD_SINGLE or MPI_THREAD_FUNNELED > when I invoke MPI_Init_thread ? > > Thanks a ton ! > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

What is the difference between MPI_THREAD_SINGLE and MPI_THREAD_FUNNELED
by Boisvert, Sebastien 30 May '14

30 May '14

Hi MPICH community: With MPICH, if my MPI process has 10 threads (including the main thread) and only the main thread calls MPI functions, do I need to use MPI_THREAD_SINGLE or MPI_THREAD_FUNNELED when I invoke MPI_Init_thread ? Thanks a ton !

1 0

Re: [mpich-discuss] per-object build issue
by Balaji, Pavan 29 May '14

29 May '14

It looks correct (I’m surprised other compilers didn’t catch it), though it should be moved to the existing "#if defined(MPICH_IS_THREADED) && !defined(MPID_DEVICE_DEFINES_THREAD_CS)” block in mpiimplthread.h. Can someone at Argonne create a patch and push to mpich-review? I’ll review it more formally. Thanks, — Pavan On May 29, 2014, at 2:09 PM, Archer, Charles J <charles.j.archer(a)intel.com> wrote: > No problem. > > Anyone from ANL brave enough to patch this in? > I’m not sure what it will break, I definitely didn’t test it extensively across all threading types. > > -C > > On May 29, 2014, at 12:03 PM, Greenberg, Hugh N <hng(a)lanl.gov> wrote: > >> Charles, >> >> It seems to work. Thanks a lot! >> >> On May 29, 2014, at 12:41 PM, Archer, Charles J <charles.j.archer(a)intel.com> wrote: >> >>> I hit this problem at some point as well, and proposed a patch…I kept the patch around in my git stash: >>> Hopefully it helps you. >>> >>> diff --git a/src/include/mpiimplthread.h b/src/include/mpiimplthread.h >>> index f0141c6..c2e1cee 100644 >>> --- a/src/include/mpiimplthread.h >>> +++ b/src/include/mpiimplthread.h >>> @@ -1039,5 +1039,18 @@ do { \ >>> #define MPIU_THREAD_MPI_OBJ_FINALIZE(objptr_) do {} while (0) >>> #endif >>> >>> +#if defined(MPICH_IS_THREADED) && !defined(MPID_DEVICE_DEFINES_THREAD_CS) >>> +static void >>> +MPIU_Thread_CS_enter_lockname_impl_(enum MPIU_Nest_mutexes kind, >>> + const char *lockname, >>> + MPID_Thread_mutex_t *mutex); >>> + >>> + >>> +static void >>> +MPIU_Thread_CS_exit_lockname_impl_(enum MPIU_Nest_mutexes kind, >>> + const char *lockname, >>> + MPID_Thread_mutex_t *mutex); >>> +#endif >>> + >>> #endif /* !defined(MPIIMPLTHREAD_H_INCLUDED) */ >>> >>> >>> >>> On May 29, 2014, at 11:29 AM, Greenberg, Hugh N <hng(a)lanl.gov> wrote: >>> >>>> I’m having a problem building mpich 3.1 with the enable-thread-cs=per-object build option. I get a lot of errors like these: >>>> >>>> /src/include/mpiimplthreadpost.h:27: warning: conflicting types for 'MPIU_Thread_CS_enter_lockname_impl_' >>>> ./src/include/mpiimplthreadpost.h:27: error: static declaration of 'MPIU_Thread_CS_enter_lockname_impl_' follows non-static declaration >>>> ./src/include/mpiimpl.h:1280: note: previous implicit declaration of 'MPIU_Thread_CS_enter_lockname_impl_' was here >>>> >>>> Is there a fix for this? Thanks. >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] mpich and nmap
by Kenneth Raffenetti 29 May '14

29 May '14

Steve, I tested this with the latest MPICH development snapshot and the behavior is still the same as you describe. I will file a ticket and cc you to track progress on a possible fix. Ken On 05/28/2014 02:47 PM, Steve Krueger wrote: > I am using MPICH 1.4.1p1 with ch3:sock device, and have noticed that it is vulnerable > to being scanned by nmap. > > With a rank sitting on an MPI_Recv, if you use nmap to scan the rank's listen ports > with a command like: > > nmap -p <port> -T4 -A -v <host> > > MPI_Recv with detect an error and exit. > > Has this been fixed in newer versions of MPICH? > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss >

1 0

Re: [mpich-discuss] per-object build issue
by Archer, Charles J 29 May '14

29 May '14

No problem. Anyone from ANL brave enough to patch this in? I’m not sure what it will break, I definitely didn’t test it extensively across all threading types. -C On May 29, 2014, at 12:03 PM, Greenberg, Hugh N <hng(a)lanl.gov> wrote: > Charles, > > It seems to work. Thanks a lot! > > On May 29, 2014, at 12:41 PM, Archer, Charles J <charles.j.archer(a)intel.com> wrote: > >> I hit this problem at some point as well, and proposed a patch…I kept the patch around in my git stash: >> Hopefully it helps you. >> >> diff --git a/src/include/mpiimplthread.h b/src/include/mpiimplthread.h >> index f0141c6..c2e1cee 100644 >> --- a/src/include/mpiimplthread.h >> +++ b/src/include/mpiimplthread.h >> @@ -1039,5 +1039,18 @@ do { \ >> #define MPIU_THREAD_MPI_OBJ_FINALIZE(objptr_) do {} while (0) >> #endif >> >> +#if defined(MPICH_IS_THREADED) && !defined(MPID_DEVICE_DEFINES_THREAD_CS) >> +static void >> +MPIU_Thread_CS_enter_lockname_impl_(enum MPIU_Nest_mutexes kind, >> + const char *lockname, >> + MPID_Thread_mutex_t *mutex); >> + >> + >> +static void >> +MPIU_Thread_CS_exit_lockname_impl_(enum MPIU_Nest_mutexes kind, >> + const char *lockname, >> + MPID_Thread_mutex_t *mutex); >> +#endif >> + >> #endif /* !defined(MPIIMPLTHREAD_H_INCLUDED) */ >> >> >> >> On May 29, 2014, at 11:29 AM, Greenberg, Hugh N <hng(a)lanl.gov> wrote: >> >>> I’m having a problem building mpich 3.1 with the enable-thread-cs=per-object build option. I get a lot of errors like these: >>> >>> /src/include/mpiimplthreadpost.h:27: warning: conflicting types for 'MPIU_Thread_CS_enter_lockname_impl_' >>> ./src/include/mpiimplthreadpost.h:27: error: static declaration of 'MPIU_Thread_CS_enter_lockname_impl_' follows non-static declaration >>> ./src/include/mpiimpl.h:1280: note: previous implicit declaration of 'MPIU_Thread_CS_enter_lockname_impl_' was here >>> >>> Is there a fix for this? Thanks. >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] per-object build issue
by Greenberg, Hugh N 29 May '14

29 May '14

Charles, It seems to work. Thanks a lot! On May 29, 2014, at 12:41 PM, Archer, Charles J <charles.j.archer(a)intel.com> wrote: > I hit this problem at some point as well, and proposed a patch…I kept the patch around in my git stash: > Hopefully it helps you. > > diff --git a/src/include/mpiimplthread.h b/src/include/mpiimplthread.h > index f0141c6..c2e1cee 100644 > --- a/src/include/mpiimplthread.h > +++ b/src/include/mpiimplthread.h > @@ -1039,5 +1039,18 @@ do { \ > #define MPIU_THREAD_MPI_OBJ_FINALIZE(objptr_) do {} while (0) > #endif > > +#if defined(MPICH_IS_THREADED) && !defined(MPID_DEVICE_DEFINES_THREAD_CS) > +static void > +MPIU_Thread_CS_enter_lockname_impl_(enum MPIU_Nest_mutexes kind, > + const char *lockname, > + MPID_Thread_mutex_t *mutex); > + > + > +static void > +MPIU_Thread_CS_exit_lockname_impl_(enum MPIU_Nest_mutexes kind, > + const char *lockname, > + MPID_Thread_mutex_t *mutex); > +#endif > + > #endif /* !defined(MPIIMPLTHREAD_H_INCLUDED) */ > > > > On May 29, 2014, at 11:29 AM, Greenberg, Hugh N <hng(a)lanl.gov> wrote: > >> I’m having a problem building mpich 3.1 with the enable-thread-cs=per-object build option. I get a lot of errors like these: >> >> /src/include/mpiimplthreadpost.h:27: warning: conflicting types for 'MPIU_Thread_CS_enter_lockname_impl_' >> ./src/include/mpiimplthreadpost.h:27: error: static declaration of 'MPIU_Thread_CS_enter_lockname_impl_' follows non-static declaration >> ./src/include/mpiimpl.h:1280: note: previous implicit declaration of 'MPIU_Thread_CS_enter_lockname_impl_' was here >> >> Is there a fix for this? Thanks. >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0