I am re-running the global symbol checker. Rajeev

-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Dave Goodell Sent: Thursday, July 24, 2008 3:58 PM To: [email protected] Subject: Re: [mpich2-core] Re: Release housekeeping

Bill, I've been running some tests with this configuration (on Linux) and I don't get any complaints about memory leaks. Valgrind also doesn't suspect any leaks in the spot testing that I have done (it does have a problem with some uninitialized data, but that's a different issue to discuss). Is there a specific platform/ configuration/test that triggers the memory checker for you?

Also, when I look at the global symbol checker results, it looks like everything fails because nothing can get past the configure stage: http://www.mcs.anl.gov/research/projects/mpich2/todo/globsyms. htm and http://www.mcs.anl.gov/research/projects/mpich2/todo/c-gforker- ch3.log. I agree that the output was generated recently (looks like about 10 mins ago) and that it's building nemesis by default, but I don't see anything about socksm.o anywhere. Was yesterday's output more complete?

-Dave

On Jul 24, 2008, at 9:04 AM, William Gropp wrote:

...

(I'm resending this because ANL rejected my original reply)

Yes, I'm looking at those pages (I'm getting to them through http:// wiki.mcs.anl.gov/mpich2/index.php/Developer_Documentation ). The global symbols one has a date of yesterday, and the "ch3" results are clearly for nemesis (the socksm.o file is in the newtcp netmod). The style checker output is recent as well (no dcmfd items).

Special tests must be run manually. The reason that I know about the memory leaks is that I *always* run with

--enable-g=log,mem,dbg --enable-posix=strict

This gives me the earliest warning that there may be problems and great flexibility (because of enable-g=log,dbg) in getting more info about any problems (which is also why the various printfs should use the dbg or log macros - that way, there is a common and available way to enable them when you need them).

Bill

Thanks Bill, I can reproduce it now. The good news is that at first glance, these all appear to be memory allocated within the scope of MPI_Init that just needs a corresponding free within the scope of MPI_Finalize. Most (all?) of them are not actual memory leaks, rather they just need to be fixed so that we know when a real memory leak does come along. The ones you get in the test suites are more likely to be real leaks that we need to deal with, but we have to attenuate the noise first. I'm going to head home now, but I'll fix these up tomorrow morning. -Dave On Jul 24, 2008, at 4:35 PM, William Gropp wrote:

Dave,

Here's my exact configure line

/Users/gropp/projects/software/mpich2/configure --with- pm=gforker:mpd:remshell --enable-g=log,mem,dbg --enable- strict=posix --prefix=/Users/gropp/tmp/mpi2-inst-nemesis

Here's a sample of the output from cpi:

groppmac:~/tmp/mpich2/examples gropp$ ./cpi Process 0 of 1 is on groppmac.cs.uiuc.edu pi is approximately 3.1415926544231341, Error is 0.0000000008333410 wall clock time = 0.006257 [0] 24 at [0x00601368], /channels/nemesis/nemesis/src/ mpid_nem_mpich2.c[73] [0] 8 at [0x006012b8], /channels/nemesis/nemesis/src/ mpid_nem_mpich2.c[65] [0] 8 at [0x00602228], h3/channels/nemesis/nemesis/src/ mpid_nem_init.c[321] [0] 8 at [0x00602178], h3/channels/nemesis/nemesis/src/ mpid_nem_init.c[320] [0] 1024 at [0x01801698], esis/net_mod/newtcp_module/ newtcp_module_poll.c[22] [0] 160 at [0x00601948], /nemesis/nemesis/net_mod/newtcp_module/ socksm.c[118] [0] 880 at [0x01801098], /nemesis/nemesis/net_mod/newtcp_module/ socksm.c[116] [0] 8 at [0x00601808], h3/channels/nemesis/nemesis/src/ mpid_nem_init.c[242] [0] 8 at [0x00601658], h3/channels/nemesis/nemesis/src/ mpid_nem_init.c[241] [0] 0 at [0x006015b8], h3/channels/nemesis/nemesis/src/ mpid_nem_init.c[104] [0] 8 at [0x00600cf8], h3/channels/nemesis/nemesis/src/ mpid_nem_init.c[102] [0] 8 at [0x00601508], h3/channels/nemesis/nemesis/src/ mpid_nem_init.c[94] [0] 56 at [0x006001c8], h3/channels/nemesis/nemesis/src/ mpid_nem_init.c[93] [0] 8 at [0x00601758], h3/channels/nemesis/nemesis/src/ mpid_nem_init.c[525] [0] 8 at [0x00601458], h3/channels/nemesis/nemesis/src/ mpid_nem_init.c[461] groppmac:~/tmp/mpich2/examples gropp$

If I run the test suites, I get leaks in additional places as well.

I'm running under OS X 10.4 .

Bill

On Jul 24, 2008, at 3:57 PM, Dave Goodell wrote:

...

Bill, I've been running some tests with this configuration (on Linux) and I don't get any complaints about memory leaks. Valgrind also doesn't suspect any leaks in the spot testing that I have done (it does have a problem with some uninitialized data, but that's a different issue to discuss). Is there a specific platform/ configuration/test that triggers the memory checker for you?

Also, when I look at the global symbol checker results, it looks like everything fails because nothing can get past the configure stage: http://www.mcs.anl.gov/research/projects/mpich2/todo/globsyms.htm and http://www.mcs.anl.gov/research/projects/mpich2/todo/c-gforker- ch3.log. I agree that the output was generated recently (looks like about 10 mins ago) and that it's building nemesis by default, but I don't see anything about socksm.o anywhere. Was yesterday's output more complete?

-Dave

William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign

Certainly the corresponding init/finalize routines are the right place. By "within the scope of MPI_{Init,Finalize}" I meant those functions or functions called by them, including netmod level finalize calls. -Dave On Jul 25, 2008, at 8:56 AM, William Gropp wrote:

Thanks. Where possible, the corresponding frees should go in the matching routine, e.g., the MPID_xxx_Finalize (the nemesis and netmod versions). I did see some with "lmt" which I suspect are real leaks as well; I'l rerun the tests once these are fixed.

Bill

On Jul 24, 2008, at 5:55 PM, Dave Goodell wrote:

...

Thanks Bill, I can reproduce it now. The good news is that at first glance, these all appear to be memory allocated within the scope of MPI_Init that just needs a corresponding free within the scope of MPI_Finalize. Most (all?) of them are not actual memory leaks, rather they just need to be fixed so that we know when a real memory leak does come along. The ones you get in the test suites are more likely to be real leaks that we need to deal with, but we have to attenuate the noise first.

I'm going to head home now, but I'll fix these up tomorrow morning.

-Dave

On Jul 24, 2008, at 4:35 PM, William Gropp wrote:

...

Dave,

Here's my exact configure line

/Users/gropp/projects/software/mpich2/configure --with- pm=gforker:mpd:remshell --enable-g=log,mem,dbg --enable- strict=posix --prefix=/Users/gropp/tmp/mpi2-inst-nemesis

Here's a sample of the output from cpi:

groppmac:~/tmp/mpich2/examples gropp$ ./cpi Process 0 of 1 is on groppmac.cs.uiuc.edu pi is approximately 3.1415926544231341, Error is 0.0000000008333410 wall clock time = 0.006257 [0] 24 at [0x00601368], /channels/nemesis/nemesis/src/ mpid_nem_mpich2.c[73] [0] 8 at [0x006012b8], /channels/nemesis/nemesis/src/ mpid_nem_mpich2.c[65] [0] 8 at [0x00602228], h3/channels/nemesis/nemesis/src/ mpid_nem_init.c[321] [0] 8 at [0x00602178], h3/channels/nemesis/nemesis/src/ mpid_nem_init.c[320] [0] 1024 at [0x01801698], esis/net_mod/newtcp_module/ newtcp_module_poll.c[22] [0] 160 at [0x00601948], /nemesis/nemesis/net_mod/newtcp_module/ socksm.c[118] [0] 880 at [0x01801098], /nemesis/nemesis/net_mod/newtcp_module/ socksm.c[116] [0] 8 at [0x00601808], h3/channels/nemesis/nemesis/src/ mpid_nem_init.c[242] [0] 8 at [0x00601658], h3/channels/nemesis/nemesis/src/ mpid_nem_init.c[241] [0] 0 at [0x006015b8], h3/channels/nemesis/nemesis/src/ mpid_nem_init.c[104] [0] 8 at [0x00600cf8], h3/channels/nemesis/nemesis/src/ mpid_nem_init.c[102] [0] 8 at [0x00601508], h3/channels/nemesis/nemesis/src/ mpid_nem_init.c[94] [0] 56 at [0x006001c8], h3/channels/nemesis/nemesis/src/ mpid_nem_init.c[93] [0] 8 at [0x00601758], h3/channels/nemesis/nemesis/src/ mpid_nem_init.c[525] [0] 8 at [0x00601458], h3/channels/nemesis/nemesis/src/ mpid_nem_init.c[461] groppmac:~/tmp/mpich2/examples gropp$

If I run the test suites, I get leaks in additional places as well.

I'm running under OS X 10.4 .

Bill

On Jul 24, 2008, at 3:57 PM, Dave Goodell wrote:

...

Bill, I've been running some tests with this configuration (on Linux) and I don't get any complaints about memory leaks. Valgrind also doesn't suspect any leaks in the spot testing that I have done (it does have a problem with some uninitialized data, but that's a different issue to discuss). Is there a specific platform/ configuration/test that triggers the memory checker for you?

Also, when I look at the global symbol checker results, it looks like everything fails because nothing can get past the configure stage: http://www.mcs.anl.gov/research/projects/mpich2/todo/ globsyms.htm and http://www.mcs.anl.gov/research/projects/mpich2/todo/c-gforker- ch3.log. I agree that the output was generated recently (looks like about 10 mins ago) and that it's building nemesis by default, but I don't see anything about socksm.o anywhere. Was yesterday's output more complete?

-Dave

William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign

William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign

Results from the global symbol checker are up. Rajeev

-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Dave Goodell Sent: Thursday, July 24, 2008 3:58 PM To: [email protected] Subject: Re: [mpich2-core] Re: Release housekeeping

Bill, I've been running some tests with this configuration (on Linux) and I don't get any complaints about memory leaks. Valgrind also doesn't suspect any leaks in the spot testing that I have done (it does have a problem with some uninitialized data, but that's a different issue to discuss). Is there a specific platform/ configuration/test that triggers the memory checker for you?

Also, when I look at the global symbol checker results, it looks like everything fails because nothing can get past the configure stage: http://www.mcs.anl.gov/research/projects/mpich2/todo/globsyms. htm and http://www.mcs.anl.gov/research/projects/mpich2/todo/c-gforker- ch3.log. I agree that the output was generated recently (looks like about 10 mins ago) and that it's building nemesis by default, but I don't see anything about socksm.o anywhere. Was yesterday's output more complete?

-Dave

On Jul 24, 2008, at 9:04 AM, William Gropp wrote:

...

(I'm resending this because ANL rejected my original reply)

Yes, I'm looking at those pages (I'm getting to them through http:// wiki.mcs.anl.gov/mpich2/index.php/Developer_Documentation ). The global symbols one has a date of yesterday, and the "ch3" results are clearly for nemesis (the socksm.o file is in the newtcp netmod). The style checker output is recent as well (no dcmfd items).

Special tests must be run manually. The reason that I know about the memory leaks is that I *always* run with

--enable-g=log,mem,dbg --enable-posix=strict

This gives me the earliest warning that there may be problems and great flexibility (because of enable-g=log,dbg) in getting more info about any problems (which is also why the various printfs should use the dbg or log macros - that way, there is a common and available way to enable them when you need them).

Bill