- discuss - mailman3.cels.anl.gov

Hi Jayesh
by Javad Ataiy 17 Mar '13

17 Mar '13

I have an AMD Athlon PC x86 Desktop, and ThinkPad x220 Core i7 laptop. I want to increase my desktop processing power, can this be done with MPICH. I use desktop mostly for gaming. Thanks alot. Sincerely JAVAD ATAIY

1 0

Re: [mpich-discuss] Hydra fills in the MPIR_proctable[] incorrectly with multiple processes per node
by Pavan Balaji 15 Mar '13

15 Mar '13

Hi John, On 03/12/2013 11:32 AM US Central Time, John DelSignore wrote: > I look forward to your fix for this problem. This is fixed in the git master. http://git.mpich.org/mpich.git/commitdiff/e04dd4b64ff618f2df58789265b741a8e… It should appear in the upcoming 3.0.3 release. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] Hydra fills in the MPIR_proctable[] incorrectly with multiple processes per node
by Pavan Balaji 14 Mar '13

14 Mar '13

Hi John, On 03/12/2013 11:32 AM US Central Time, John DelSignore wrote: > If possible, it would be good is MPICH could be changed to share the > host and executable name character strings across multiple process > descriptor entries. This is a maintenance hassle with respect to chasing dangling pointers in the future. Also, I'm not sure how much space this saves. We are working with the MPI Forum on a more scalable interface where this information is distributed across various processes. That might be a better model here. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] MPI_set_view, for offset larger than an integer
by Rajeev Thakur 13 Mar '13

13 Mar '13

And if you are using Fortran, you have to define the offset variable as integer (kind=MPI_OFFSET_KIND). On Mar 13, 2013, at 10:57 AM, Jeff Hammond wrote: > The MPI standard, which is the only official documentation about the > function syntax of MPI_File_set_view, says this: > > int MPI_File_set_view(MPI_File fh, MPI_Offset disp, MPI_Datatype etype, > MPI_Datatype filetype, const char *datarep, MPI_Info info) > > The size of the disp type is specified as follows: > > 13.6.7 MPI_Offset Type > > MPI_Offset is an integer type of size sufficient to represent the size > (in bytes) of the largest file supported by MPI. Displacements and > offsets are always specified as values of type MPI_Offset. > > Thus, it should not be possible to overflow this value if you're > accessing the file in a correct way. > > Jeff > > On Wed, Mar 13, 2013 at 10:28 AM, Ana María Sánchez > <krazorange(a)gmail.com> wrote: >> Hi all, >> >> I need to read a file, using MPI_set_view. The problem occurs when the >> file is so large that the offset that I need to provide to the MPI_set_view >> is larger than the maximum value for an integer. For example I need to set >> the offset to 3826145886. In some documentation I've read the offset is a >> long, but in other it indicates its an integer. >> >> Is there any possible work around this? Could the offset be set to >> negative, and if so, will it start at the end of the file? >> >> I have installed mpich2-1.3.1. >> >> Thank you, >> >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > > > -- > Jeff Hammond > Argonne Leadership Computing Facility > University of Chicago Computation Institute > jhammond(a)alcf.anl.gov / (630) 252-5381 > http://www.linkedin.com/in/jeffhammond > https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPI_set_view, for offset larger than an integer
by Jeff Hammond 13 Mar '13

13 Mar '13

The MPI standard, which is the only official documentation about the function syntax of MPI_File_set_view, says this: int MPI_File_set_view(MPI_File fh, MPI_Offset disp, MPI_Datatype etype, MPI_Datatype filetype, const char *datarep, MPI_Info info) The size of the disp type is specified as follows: 13.6.7 MPI_Offset Type MPI_Offset is an integer type of size sufficient to represent the size (in bytes) of the largest file supported by MPI. Displacements and offsets are always specified as values of type MPI_Offset. Thus, it should not be possible to overflow this value if you're accessing the file in a correct way. Jeff On Wed, Mar 13, 2013 at 10:28 AM, Ana María Sánchez <krazorange(a)gmail.com> wrote: > Hi all, > > I need to read a file, using MPI_set_view. The problem occurs when the > file is so large that the offset that I need to provide to the MPI_set_view > is larger than the maximum value for an integer. For example I need to set > the offset to 3826145886. In some documentation I've read the offset is a > long, but in other it indicates its an integer. > > Is there any possible work around this? Could the offset be set to > negative, and if so, will it start at the end of the file? > > I have installed mpich2-1.3.1. > > Thank you, > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute jhammond(a)alcf.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond

1 0

MPI_set_view, for offset larger than an integer
by Ana María Sánchez 13 Mar '13

13 Mar '13

Hi all, I need to read a file, using MPI_set_view. The problem occurs when the file is so large that the offset that I need to provide to the MPI_set_view is larger than the maximum value for an integer. For example I need to set the offset to 3826145886. In some documentation I've read the offset is a long, but in other it indicates its an integer. Is there any possible work around this? Could the offset be set to negative, and if so, will it start at the end of the file? I have installed mpich2-1.3.1. Thank you,

1 0

Re: [mpich-discuss] Possible bug: --enable-g=all configure option does not include all other --enable-g options
by Dave Goodell 12 Mar '13

12 Mar '13

fixed: http://git.mpich.org/mpich.git/commitdiff/4b6df5cd -Dave On Mar 12, 2013, at 10:42 AM CDT, Dave Goodell <goodell(a)mcs.anl.gov> wrote: > I don't like the gcc model that much. > > The key is just to change the current assumption that "yes"=="all". I'll code up a patch to add a "most" and make "yes" (i.e., the default when you just pass "--enable-g") equivalent to that. Then I'll change "all" to really mean everything. > > -Dave > > On Mar 12, 2013, at 10:36 AM CDT, Jim Dinan <dinan(a)mcs.anl.gov> wrote: > >> How about, following the gcc model: >> >> --enable-g=all enable most debugging features >> --enable-g=extra enable most + additional debugging features >> >> ~Jim. >> >> On 3/12/13 8:59 AM, William Gropp wrote: >>> While I agree that "all" should mean "all", this unfortunate use is >>> consistent with gcc, which uses -Wall to mean "most of the warning >>> options, but not the ones that who ever is in charge of this release did >>> not like or forgot about." >>> >>> A better name might be --enable-g=most - and for gcc, -Wmost . Anything >>> else is either too long or misleading. >>> >>> Bill >>> >>> William Gropp >>> Director, Parallel Computing Institute >>> Deputy Director for Research >>> Institute for Advanced Computing Applications and Technologies >>> Thomas M. Siebel Chair in Computer Science >>> University of Illinois Urbana-Champaign >>> >>> >>> >>> >>> On Mar 11, 2013, at 8:14 PM, Jeff Hammond wrote: >>> >>>> --enable-g=all should enable all debugging options to satisfy the >>>> principle of least surprise as well as simple English language meaning >>>> of the word "all". >>>> >>>> perhaps there should be an option >>>> --enable-g=important/most/devel/useful/sane that enables all of the >>>> debug options that are reasonable for the "average hardcore user" / >>>> developer. >>>> >>>> jeff >>>> >>>> On Mon, Mar 11, 2013 at 4:33 PM, Jeremiah Willcock >>>> <jewillco(a)osl.iu.edu <mailto:[email protected]>> wrote: >>>>> On Mon, 11 Mar 2013, Dave Goodell wrote: >>>>> >>>>>> On Mar 11, 2013, at 3:36 PM CDT, Jeremiah Willcock >>>>>> <jewillco(a)osl.iu.edu <mailto:[email protected]>> >>>>>> wrote: >>>>>> >>>>>>> In MPICH 3.0.2, the help message for the --enable-g=all configure >>>>>>> option >>>>>>> states that it enables "All of the above choices", but in fact >>>>>>> (according to >>>>>>> configure.ac) it does not enable the "handle" or "memarena" >>>>>>> debugging flags. >>>>>>> Is that the intended behavior? Thank you for your help. >>>>>> >>>>>> >>>>>> The situation is unclear. I just did a little bit of git >>>>>> archaeology and >>>>>> it looks like these flags were added prior to MPICH2-1.0.6, which is >>>>>> as far >>>>>> back as I can look without having to sift through ancient CVS >>>>>> history. The >>>>>> pickaxe doesn't show any place where variables corresponding to >>>>>> those flags >>>>>> might have been removed from the "all|yes)" case. It looks like just a >>>>>> mistake or sloppiness on someone's part in the distant past. >>>>>> >>>>>> I'm not sure what the right "fix" is. The "memarena" option actually >>>>>> introduces quite a bit of overhead that we may not want enabled >>>>>> casually. >>>>>> The "handles" option is pretty lightweight, at least in stock MPICH. I >>>>>> suppose we could add both to the "all" option and then change the "yes" >>>>>> option to be a set of reasonable defaults instead. >>>>>> >>>>>> Do you have a preference? Or are you just observing a discrepancy? >>>>> >>>>> >>>>> I was both observing a discrepancy and trying to debug something and was >>>>> surprised that "all" didn't include "handles". >>>>> >>>>> -- Jeremiah Willcock >>>>> >>>>> _______________________________________________ >>>>> discuss mailing list discuss(a)mpich.org <mailto:[email protected]> >>>>> To manage subscription options or unsubscribe: >>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>> >>>> >>>> >>>> -- >>>> Jeff Hammond >>>> Argonne Leadership Computing Facility >>>> University of Chicago Computation Institute >>>> jhammond(a)alcf.anl.gov <mailto:[email protected]> / (630) 252-5381 >>>> http://www.linkedin.com/in/jeffhammond >>>> https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>> >>> >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >>> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Help with eclipse juno and mpich2
by Jayesh Krishna 12 Mar '13

12 Mar '13

Hi, If you are using MPICH2 you cannot use the Open MPI compiler wrapper to build your code. If you are using OpenMPI you should contact the OpenMPI developers. Regards, Jayesh ----- Original Message ----- From: "George Papageorgakis" <ngeopapas(a)gmail.com> To: "Jayesh Krishna" <jayesh(a)mcs.anl.gov> Sent: Monday, March 11, 2013 2:11:02 PM Subject: Re: [mpich-discuss] Help with eclipse juno and mpich2 I have run a hello_mpi on my pc once but since then i get The Open MPI wrapper compiler was unable to find the specified compiler cl.exe in your PATH. Note that this compiler was either specified at configure time or in one of several possible environment variables.

1 0

Re: [mpich-discuss] Hydra fills in the MPIR_proctable[] incorrectly with multiple processes per node
by John DelSignore 12 Mar '13

12 Mar '13

Hi Pavan, Thanks very much for your reply and for working on a patch. I tried it again with your suggested "-hosts" option, as follows, and indeed version 1.4.1p1 does work: Hello from rank 0 of 6, getpid()==21949 Hello from rank 1 of 6, getpid()==21950 Hello from rank 2 of 6, getpid()==21951 Hello from rank 3 of 6, getpid()==21945 Hello from rank 4 of 6, getpid()==21946 Hello from rank 5 of 6, getpid()==21947 mpir_proctable_t::create: extracting hostname/execname/pids for 6 processes mpir_proctable_t::create: MPIR_proctable[0]: host_name(0x005641c0)="127.0.0.2", executable_name(0x00575640)="./mpichbug", pid=21949 mpir_proctable_t::create: MPIR_proctable[1]: host_name(0x00575620)="127.0.0.2", executable_name(0x005757a0)="./mpichbug", pid=21950 mpir_proctable_t::create: MPIR_proctable[2]: host_name(0x005643a0)="127.0.0.2", executable_name(0x00575740)="./mpichbug", pid=21951 mpir_proctable_t::create: MPIR_proctable[3]: host_name(0x00564110)="127.0.0.3", executable_name(0x00575410)="./mpichbug", pid=21945 mpir_proctable_t::create: MPIR_proctable[4]: host_name(0x00575430)="127.0.0.3", executable_name(0x00575450)="./mpichbug", pid=21946 mpir_proctable_t::create: MPIR_proctable[5]: host_name(0x00575470)="127.0.0.3", executable_name(0x00575490)="./mpichbug", pid=21947 I look forward to your fix for this problem. On a different topic, the above output shows a potential performance and scalability problem with the way the MPIR_proctable[] is filled out. The MPIR_PROCDESC contain two pointer to null-terminated string members, as described in the MPIR spec: typedef struct { char *host_name; char *executable_name; int pid; } MPIR_PROCDESC; As an "advice to implementors" kind-of suggestion, the MPIR spec says: "The MPI implementation should share the host and executable name character strings across multiple process descriptor entries whenever possible. For example, if all of the MPI processes are executing ï¿½/path/a.outï¿½, then the executable name field in each process descriptor should point to the same null-terminated character string. Sharing the strings enhances the tools scalability by allowing it to cache data from the starter process and avoid reading redundant character strings." In the output above, we can see that there are exactly three unique strings: "127.0.0.2", "127.0.0.3", and "./mpichbug". However, each string is allocated at a unique address. This arrangement not only causes the MPIR proctable entries to require more space in the MPI starter process, it requires the tool to read each string individually, thus defeating any data-caching the tool might do, and slowing the tool down when reading the MPIR proctable. If possible, it would be good is MPICH could be changed to share the host and executable name character strings across multiple process descriptor entries. Cheers, John D. Pavan Balaji wrote: > John, > > This does seem like a bug. Specifically, this is a problem with the > wrap-around of hosts. For example, I don't expect this problem to show > up when you do: > > mpiexec -hosts 127.0.0.2:3,127.0.0.3:3 -n 6 ./mpichbug > > This should only show up when the number of cores is not sufficient in > the first round and mpiexec has to wrap around to the first host again: > > mpiexec -hosts 127.0.0.2,127.0.0.3 -n 6 ./mpichbug > > I'm working on a patch. I'll commit it in shortly. > > -- Pavan > > On 03/11/2013 02:59 PM US Central Time, John DelSignore wrote: >> Hi, >> >> I'm pretty sure this is a MPICH Hydra bug, but I wanted to ask this group before I go through the trouble of figuring out how to file a MPICH bug report, which I think requires me to create an MPICH Trac account, which I don't know how to do. >> >> As far as I can tell, Hydra fills in the MPIR_proctable[] incorrectly with multiple processes per node. The index into the MPIR_proctable[] is supposed to be the MPI process's rank in MPI_COMM_WORLD. To demonstrate this problem. I created a simple MPI "hello world" program where each MPI process prints out its rank and pid; I attached it to this email. >> >> This is the version of MPICH I am using: >> >> shell% /home/mware/argonne/mpich2/1.4.1p1/x86_64-linux/bin/mpirun --version|head -3 >> HYDRA build details: >> Version: 1.4.1p1 >> Release Date: Thu Sep 1 13:53:02 CDT 2011 >> shell% >> >> I ran the code under TotalView using 2 nodes and 6 processes (3 per node). I enabled logging so that TotalView would output the contents of the MPIR_proctable[] as it extracted it from the mpirun process. Here is the output of the run: >> >> shell% tv8cli \ >> -verbosity errors \ >> -x15 \ >> -parallel_stop no \ >> -debug_file debug.log \ >> -args \ >> /home/mware/argonne/mpich2/1.4.1p1/x86_64-linux/bin/mpirun \ >> -hosts 127.0.0.2,127.0.0.3 \ >> -n 6 \ >> ./mpichbug >> d1.<> dcont >> Hello from rank 0 of 6, getpid()==2691 >> Hello from rank 1 of 6, getpid()==2729 >> Hello from rank 2 of 6, getpid()==2693 >> Hello from rank 3 of 6, getpid()==2730 >> Hello from rank 4 of 6, getpid()==2694 >> Hello from rank 5 of 6, getpid()==2734 >> d1.<> quit -force >> shell% >> >> Grepping for "proctable" in the debugger's log file shows the contents of the MPIR_proctable[]: >> >> shell% grep proctable debug.log >> mpir_proctable_t::create: extracting hostname/execname/pids for 6 processes >> mpir_proctable_t::create: MPIR_proctable[0]: host_name(0x0056be20)="127.0.0.2", executable_name(0x0056be80)="./mpichbug", pid=2691 >> mpir_proctable_t::create: MPIR_proctable[1]: host_name(0x0056be00)="127.0.0.2", executable_name(0x0056bde0)="./mpichbug", pid=2693 >> mpir_proctable_t::create: MPIR_proctable[2]: host_name(0x005859a0)="127.0.0.2", executable_name(0x0056c080)="./mpichbug", pid=2694 >> mpir_proctable_t::create: MPIR_proctable[3]: host_name(0x005856b0)="127.0.0.3", executable_name(0x005856d0)="./mpichbug", pid=2729 >> mpir_proctable_t::create: MPIR_proctable[4]: host_name(0x005856f0)="127.0.0.3", executable_name(0x00585710)="./mpichbug", pid=2730 >> mpir_proctable_t::create: MPIR_proctable[5]: host_name(0x00585730)="127.0.0.3", executable_name(0x00585750)="./mpichbug", pid=2734 >> shell% >> >> Matching up the pid values shows that the MPIR_proctable[rank] does not match the rank returned to the program by MPI_Comm_rank() for some of the MPI processes. Here's the MPI rank to MPIR_proctable rank mapping: >> 0 => 0 >> 1 => 3 >> 2 => 1 >> 3 => 4 >> 4 => 2 >> 5 => 5 >> >> Do you agree that this is an MPICH Hydra bug? >> >> Any advice on how to create an MPICH Trac account so that I can report the bug? >> >> Thanks, John D. >> >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> >

1 0

Re: [mpich-discuss] Hydra fills in the MPIR_proctable[] incorrectly with multiple processes per node
by Pavan Balaji 12 Mar '13

12 Mar '13

John, This does seem like a bug. Specifically, this is a problem with the wrap-around of hosts. For example, I don't expect this problem to show up when you do: mpiexec -hosts 127.0.0.2:3,127.0.0.3:3 -n 6 ./mpichbug This should only show up when the number of cores is not sufficient in the first round and mpiexec has to wrap around to the first host again: mpiexec -hosts 127.0.0.2,127.0.0.3 -n 6 ./mpichbug I'm working on a patch. I'll commit it in shortly. -- Pavan On 03/11/2013 02:59 PM US Central Time, John DelSignore wrote: > Hi, > > I'm pretty sure this is a MPICH Hydra bug, but I wanted to ask this group before I go through the trouble of figuring out how to file a MPICH bug report, which I think requires me to create an MPICH Trac account, which I don't know how to do. > > As far as I can tell, Hydra fills in the MPIR_proctable[] incorrectly with multiple processes per node. The index into the MPIR_proctable[] is supposed to be the MPI process's rank in MPI_COMM_WORLD. To demonstrate this problem. I created a simple MPI "hello world" program where each MPI process prints out its rank and pid; I attached it to this email. > > This is the version of MPICH I am using: > > shell% /home/mware/argonne/mpich2/1.4.1p1/x86_64-linux/bin/mpirun --version|head -3 > HYDRA build details: > Version: 1.4.1p1 > Release Date: Thu Sep 1 13:53:02 CDT 2011 > shell% > > I ran the code under TotalView using 2 nodes and 6 processes (3 per node). I enabled logging so that TotalView would output the contents of the MPIR_proctable[] as it extracted it from the mpirun process. Here is the output of the run: > > shell% tv8cli \ > -verbosity errors \ > -x15 \ > -parallel_stop no \ > -debug_file debug.log \ > -args \ > /home/mware/argonne/mpich2/1.4.1p1/x86_64-linux/bin/mpirun \ > -hosts 127.0.0.2,127.0.0.3 \ > -n 6 \ > ./mpichbug > d1.<> dcont > Hello from rank 0 of 6, getpid()==2691 > Hello from rank 1 of 6, getpid()==2729 > Hello from rank 2 of 6, getpid()==2693 > Hello from rank 3 of 6, getpid()==2730 > Hello from rank 4 of 6, getpid()==2694 > Hello from rank 5 of 6, getpid()==2734 > d1.<> quit -force > shell% > > Grepping for "proctable" in the debugger's log file shows the contents of the MPIR_proctable[]: > > shell% grep proctable debug.log > mpir_proctable_t::create: extracting hostname/execname/pids for 6 processes > mpir_proctable_t::create: MPIR_proctable[0]: host_name(0x0056be20)="127.0.0.2", executable_name(0x0056be80)="./mpichbug", pid=2691 > mpir_proctable_t::create: MPIR_proctable[1]: host_name(0x0056be00)="127.0.0.2", executable_name(0x0056bde0)="./mpichbug", pid=2693 > mpir_proctable_t::create: MPIR_proctable[2]: host_name(0x005859a0)="127.0.0.2", executable_name(0x0056c080)="./mpichbug", pid=2694 > mpir_proctable_t::create: MPIR_proctable[3]: host_name(0x005856b0)="127.0.0.3", executable_name(0x005856d0)="./mpichbug", pid=2729 > mpir_proctable_t::create: MPIR_proctable[4]: host_name(0x005856f0)="127.0.0.3", executable_name(0x00585710)="./mpichbug", pid=2730 > mpir_proctable_t::create: MPIR_proctable[5]: host_name(0x00585730)="127.0.0.3", executable_name(0x00585750)="./mpichbug", pid=2734 > shell% > > Matching up the pid values shows that the MPIR_proctable[rank] does not match the rank returned to the program by MPI_Comm_rank() for some of the MPI processes. Here's the MPI rank to MPIR_proctable rank mapping: > 0 => 0 > 1 => 3 > 2 => 1 > 3 => 4 > 4 => 2 > 5 => 5 > > Do you agree that this is an MPICH Hydra bug? > > Any advice on how to create an MPICH Trac account so that I can report the bug? > > Thanks, John D. > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0