October 2014 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] buffer overflow
by Seo, Sangmin 27 Oct '14

27 Oct '14

Hi Amjad, As the error message, "25 bytes received but buffer size is 24”, says, your buffer size is smaller than the message size. You need to increase your buffer which is the memory pointed to by the first argument of MPI_Recv. Best regards, Sangmin On Oct 27, 2014, at 12:27 AM, Md. Amjad Hossain <mhossai2(a)kent.edu> wrote: > Hi, > > I am sending and receiving a user defined (struct) data type using MPI_Send and MPI_Recv function. I am getting following buffer overflow error. > > Fatal error in MPI_Recv: Message truncated, error stack: > MPI_Recv(187).....................: MPI_Recv(buf=0x7fff2248b930, count=1, dtype=USER<struct>, src=0, tag=1, MPI_COMM_WORLD, status=0x7fff2248b970) failed > MPIDI_CH3U_Receive_data_found(131): Message from rank 0 and tag 1 truncated; 25 bytes received but buffer size is 24 > > > If I just send one byte less, everything is working properly. Could anyone please let me know which buffer it is and how can I increase the buffer size to avoid the error? > > Regards > Amjad > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Announcing the availability of mpich-3.1.3
by Seo, Sangmin 27 Oct '14

27 Oct '14

Hi Aurelien, Thank you for the notice. We have updated the Downloads page in www.mpich.org<http://www.mpich.org>. Best regards, Sangmin On Oct 26, 2014, at 11:53 AM, Aurélien Larcher <aurelien.larcher(a)gmail.com<mailto:[email protected]>> wrote: Hi, MPICH 3.1.3 compiled with GCC 4.8.3 is now available in the OpenIndiana repository: http://pkg.openindiana.org/hipster-2014.1/info/0/library%2Fmpich%2Fgcc%403.… and modules files: http://pkg.openindiana.org/hipster-2014.1/info/0/library%2Fmpich%2Fgcc%2Fmo… Best regards, Aurelien On Wed, Oct 8, 2014 at 6:14 PM, Seo, Sangmin <sseo(a)anl.gov<mailto:[email protected]>> wrote: The MPICH team is pleased to announce the availability of a new stable release, mpich-3.1.3. This is a stable release that adds several enhancements to Portals4 support, PAMI, RMA, and ROMIO. It also contains a large number of bug fixes. All production environments are encouraged to upgrade to this release. More information about the release can be found on the MPICH website: http://www.mpich.org<http://www.mpich.org/> A list of changes in this release is included at the end of this email. Regards, The MPICH Team =============================================================================== Changes in 3.1.3 =============================================================================== # Several enhancements to Portals4 support. # Several enhancements to PAMI (thanks to IBM for the code contribution). # Several enhancements to the CH3 RMA implementation. # Several enhancements to ROMIO. # Fixed deadlock in multi-threaded MPI_Comm_idup. # Several other minor bug fixes, memory leak fixes, and code cleanup. A full list of changes is available at the following link: http://git.mpich.org/mpich.git/shortlog/v3.1.2..v3.1.3 A full list of bugs that have been fixed is available at the following link: https://trac.mpich.org/projects/mpich/query?status=closed&group=resolution&… _______________________________________________ discuss mailing list discuss(a)mpich.org<mailto:[email protected]> To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss -- ------------------------------------------------------------------------------- LARCHER Aurélien | KTH, School of Computer Science and Communication Work: +46 (0) 8 790 71 42 | Lindstedtsvägen 5, Plan 4, 100 44 Stockholm, SWEDEN ------------------------------------------------------------------------------- _______________________________________________ discuss mailing list discuss(a)mpich.org<mailto:[email protected]> To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re : MPI process killed and SIGUSR1
by Roy, Hirak 27 Oct '14

27 Oct '14

Hi Wesley, Here is my server.c and client.c. I have explicitly added 'assert(0);' in client.c. However my receive hangs in server and does not complete. Could you please enlighten on this more. Thanks, Hirak #include <sys/types.h> #include <unistd.h> #include "mpi.h" #include <stdio.h> #include <assert.h> #include <stdlib.h> #include <fcntl.h> int main (int argc, char* argv[]) { MPI_Init (&argc, &argv); MPI_Errhandler_set(MPI_COMM_WORLD,MPI_ERRORS_RETURN); char portname[MPI_MAX_PORT_NAME]; MPI_Open_port(MPI_INFO_NULL, portname); MPI_Publish_name("RandomSession", MPI_INFO_NULL, portname); printf ("Successfully published portname : %s\n", portname); MPI_Comm newComm ; MPI_Comm_accept (portname, MPI_INFO_NULL, 0, MPI_COMM_SELF, &newComm); printf ("Successfully connected to client\n"); MPI_Errhandler_set(newComm, MPI_ERRORS_RETURN); printf ("Waiting on receive\n"); int val = 0 ; MPI_Status status; int errcode = MPI_Recv(&val, 1, MPI_INT, 0, 99, newComm, &status); int errclass ; MPI_Error_class(errcode, &errclass); if (errclass == MPI_SUCCESS) { printf ("Received a message with tag 99\n"); MPI_Comm_disconnect (&newComm); printf ("Disconnected client\n"); } else { printf ("Receive error\n"); } MPI_Unpublish_name("RandomSession", MPI_INFO_NULL, portname); MPI_Close_port(portname); MPI_Finalize(); } #include <sys/types.h> #include <unistd.h> #include "mpi.h" #include <stdio.h> #include <assert.h> #include <stdlib.h> #include <fcntl.h> int main (int argc, char* argv[]) { MPI_Init (&argc, &argv); MPI_Errhandler_set(MPI_COMM_WORLD,MPI_ERRORS_RETURN); char portname[MPI_MAX_PORT_NAME]; if (MPI_Lookup_name("RandomSession", MPI_INFO_NULL, portname) == MPI_SUCCESS) { printf ("Successfully got portname : %s\n", portname); MPI_Comm newComm ; MPI_Comm_connect (portname, MPI_INFO_NULL, 0, MPI_COMM_SELF, &newComm); printf ("Successfully connected to server\n"); MPI_Errhandler_set(newComm, MPI_ERRORS_RETURN); assert (0); int val = 0 ; MPI_Send(&val, 1, MPI_INT, 0, 99, newComm); printf ("Send a message with tag 99\n"); MPI_Comm_disconnect (&newComm); printf ("Disconnected client\n"); } MPI_Finalize(); } MPICH Version: 3.0.4 MPICH Release date: Wed Apr 24 10:08:10 CDT 2013 MPICH Device: ch3:sock MPICH configure: --prefix /home/jlevitt/local/mpich-3.0.4/linux_x86_64 --disable-f77 --disable-fc --disable-f90modules --disable-cxx --enable-fast=nochkmsg --enable-fast=notiming --enable-fast=ndebug --enable-fast=O3 --with-device=ch3:sock CFLAGS=-O3 -fPIC CXXFLAGS=-O3 -fPIC MPICH CC: /u/prod/gnu/gcc/20121129/gcc-4.5.0-linux_x86_64/bin/gcc -O3 -fPIC -O3 MPICH CXX: no -O3 -fPIC MPICH F77: no MPICH FC: no _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

buffer overflow
by Md. Amjad Hossain 27 Oct '14

27 Oct '14

Hi, I am sending and receiving a user defined (struct) data type using MPI_Send and MPI_Recv function. I am getting following buffer overflow error. Fatal error in MPI_Recv: Message truncated, error stack: MPI_Recv(187).....................: MPI_Recv(buf=0x7fff2248b930, count=1, dtype=USER<struct>, src=0, tag=1, MPI_COMM_WORLD, status=0x7fff2248b970) failed MPIDI_CH3U_Receive_data_found(131): Message from rank 0 and tag 1 truncated; 25 bytes received but buffer size is 24 If I just send one byte less, everything is working properly. Could anyone please let me know which buffer it is and how can I increase the buffer size to avoid the error? Regards Amjad _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Announcing the availability of mpich-3.1.3
by Aurélien Larcher 26 Oct '14

26 Oct '14

Hi, MPICH 3.1.3 compiled with GCC 4.8.3 is now available in the OpenIndiana repository: http://pkg.openindiana.org/hipster-2014.1/info/0/library%2Fmpich%2Fgcc%403.… and modules files: http://pkg.openindiana.org/hipster-2014.1/info/0/library%2Fmpich%2Fgcc%2Fmo… Best regards, Aurelien On Wed, Oct 8, 2014 at 6:14 PM, Seo, Sangmin <sseo(a)anl.gov> wrote: > The MPICH team is pleased to announce the availability of a new stable > release, mpich-3.1.3. > > This is a stable release that adds several enhancements to Portals4 > support, PAMI, RMA, and ROMIO. It also contains a large number of bug fixes. > > All production environments are encouraged to upgrade to this release. > > More information about the release can be found on the MPICH website: > > http://www.mpich.org > > A list of changes in this release is included at the end of this email. > > Regards, > > The MPICH Team > > > =============================================================================== > Changes in 3.1.3 > > =============================================================================== > > # Several enhancements to Portals4 support. > > # Several enhancements to PAMI (thanks to IBM for the code contribution). > > # Several enhancements to the CH3 RMA implementation. > > # Several enhancements to ROMIO. > > # Fixed deadlock in multi-threaded MPI_Comm_idup. > > # Several other minor bug fixes, memory leak fixes, and code cleanup. > > A full list of changes is available at the following link: > > http://git.mpich.org/mpich.git/shortlog/v3.1.2..v3.1.3 > > A full list of bugs that have been fixed is available at the > following link: > > > https://trac.mpich.org/projects/mpich/query?status=closed&group=resolution&… > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- ------------------------------------------------------------------------------- LARCHER Aurélien | KTH, School of Computer Science and Communication Work: +46 (0) 8 790 71 42 | Lindstedtsvägen 5, Plan 4, 100 44 Stockholm, SWEDEN ------------------------------------------------------------------------------- _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] CPU usage versus Nodes, Threads
by Qiguo Jing 23 Oct '14

23 Oct '14

Hi Ruben, You are right! My algorithm does what you described. I will record the timestamp for each thread and every event. Thanks for your suggestions! Qiguo From: parasietje(a)gmail.com [mailto:[email protected]] On Behalf Of Ruben Faelens Sent: Thursday, October 23, 2014 10:58 AM To: discuss(a)mpich.org Subject: Re: [mpich-discuss] CPU usage versus Nodes, Threads Hi Qiguo, You should try to collect performance statistics, especially regarding your specific nodes and what they are doing at every moment in time. If I understand correctly, your algorithm does the following: - Master thread: read in data, split it up into pieces, transfer pieces to slaves - Slave thread: do calculation, transfer data back to master - Master: recombine data, do calculation, split data back up, transfer pieces to slaves - etc... The reason you do not see linear performance scaling could be due to the following: - The master thread recombining and splitting the data set may be responsible for a large part of the work (and therefore is the bottleneck) - Work is not divided equally. A significant part of the time is spent waiting on one slave node who has a more difficult problem (takes longer) than the rest. - There is a common dataset. I/O takes a larger part of the time when more slave nodes are used. The only way to know for sure, is to simply generate a log file that shows the time every process starts and ends a specific process step. Output the time when - the slave starts receiving data - starts calculation - starts sending results back - starts waiting for his next piece of data This will clearly show you what each node is doing at each moment in time, and should identify the bottleneck. / Ruben On Thu, Oct 23, 2014 at 5:27 PM, Qiguo Jing <qjing(a)trinityconsultants.com<mailto:[email protected]>> wrote: Hi Bob, Thanks for your suggestions. Here are more tests. We actually have three clusters. Cluster 1 and 2: 8 nodes, (2 Processors, 4 cores/processor, no HT – Total 8 Threads)/node Cluster 3: 8 nodes, (1 Processors, 4 cores /processor, HT – Total 8 Threads )/node We also have a standalone machine: 2 processors, 6 cores/processor, HT – total 24 threads. For one particular case: Cluster 1 and 2 take 48 min to finish with 8 nodes, 8 threads/node, 60% CPU usage; 53 min to finish with 3 nodes, 8 threads/node, 90% CPU usage; Cluster 3 takes 227 min to finish with 8 nodes, 8 threads/node, 20% CPU usage; 207 min to finish with 3 nodes, 8 threads/node, 50% CPU usage; Standalone machine takes 82 min to finish with 24 threads, 100% CPU usage. It looks like with 24 threads, they should be pretty busy? Could the above phenomena be a hardware issue more than software? Qiguo From: Bob Ilgner [mailto:[email protected]] Sent: Thursday, October 23, 2014 1:11 AM To: discuss(a)mpich.org<mailto:[email protected]> Subject: Re: [mpich-discuss] CPU usage versus Nodes, Threads Hi Qiguo,

1 0

Re: [mpich-discuss] CPU usage versus Nodes, Threads
by Ruben Faelens 23 Oct '14

23 Oct '14

Hi Qiguo, You should try to collect performance statistics, especially regarding your specific nodes and what they are doing at every moment in time. If I understand correctly, your algorithm does the following: - Master thread: read in data, split it up into pieces, transfer pieces to slaves - Slave thread: do calculation, transfer data back to master - Master: recombine data, do calculation, split data back up, transfer pieces to slaves - etc... The reason you do not see linear performance scaling could be due to the following: - The master thread recombining and splitting the data set may be responsible for a large part of the work (and therefore is the bottleneck) - Work is not divided equally. A significant part of the time is spent waiting on one slave node who has a more difficult problem (takes longer) than the rest. - There is a common dataset. I/O takes a larger part of the time when more slave nodes are used. The only way to know for sure, is to simply generate a log file that shows the time every process starts and ends a specific process step. Output the time when - the slave starts receiving data - starts calculation - starts sending results back - starts waiting for his next piece of data This will clearly show you what each node is doing at each moment in time, and should identify the bottleneck. / Ruben On Thu, Oct 23, 2014 at 5:27 PM, Qiguo Jing <qjing(a)trinityconsultants.com> wrote: > Hi Bob, > > > > Thanks for your suggestions. Here are more tests. We actually have > three clusters. > > > > Cluster 1 and 2: 8 nodes, (2 Processors, 4 cores/processor, no HT – Total > 8 Threads)/node > > Cluster 3: 8 nodes, (1 Processors, 4 cores /processor, HT – > Total 8 Threads )/node > > > > We also have a standalone machine: 2 processors, 6 cores/processor, HT – > total 24 threads. > > > > > > For one particular case: > > > > Cluster 1 and 2 take 48 min to finish with 8 nodes, 8 threads/node, 60% > CPU usage; 53 min to finish with 3 nodes, 8 threads/node, 90% CPU usage; > > > > Cluster 3 takes 227 min to finish with 8 nodes, 8 threads/node, 20% CPU > usage; 207 min to finish with 3 nodes, 8 threads/node, 50% CPU usage; > > > > Standalone machine takes 82 min to finish with 24 threads, 100% CPU usage. > > > > It looks like with 24 threads, they should be pretty busy? Could the > above phenomena be a hardware issue more than software? > > > > Qiguo > > > > *From:* Bob Ilgner [mailto:[email protected] <bobilgner(a)gmail.com>] > *Sent:* Thursday, October 23, 2014 1:11 AM > *To:* discuss(a)mpich.org > *Subject:* Re: [mpich-discuss] CPU usage versus Nodes, Threads > > > > Hi Qiguo, > > > > From the results table it looks as if you are using a computationally > sparse algorithm. i.e. the problem may not lie with the inefficiency of > your communication between threads but just that your algorithm is not > keeping the processor busy enough for a large number of threads. The only > way that you will know for sure whether this is a comms issue or an > algorithmic one is to use a profiling tool, such as Vampir or Paraver. > > > > With the profiling result you will be able to determine whether you need > to make algorithmic changes in your bulk processing and enhance comms as > per Huiwei's notes. > > > > I would be interested to know what your profiling shows. > > > > Regards, bob > > > > On Thu, Oct 23, 2014 at 12:02 AM, Qiguo Jing <qjing(a)trinityconsultants.com> > wrote: > > Hi All, > > > > We have a parallel program running on a cluster. We recently found a > case, which decreases the CPU usage and increase the run-time when > increases Nodes. Below is the results table. > > > > The particular run requires a lot of data communication between nodes. > > > > Any thoughts about this phenomena? Or is there any way we can improve the > CPU usage when using higher number of nodes? > > > > Average CPU Usage (%) > > Number of Nodes > > Number of Threads/Node > > 100 > > 1 > > 8 > > 92 > > 2 > > 8 > > 50 > > 3 > > 8 > > 40 > > 4 > > 8 > > 35 > > 5 > > 8 > > 30 > > 6 > > 8 > > 25 > > 7 > > 8 > > 20 > > 8 > > 8 > > 20 > > 8 > > 4 > > > > > > Thanks! > > > _________________________________________________________________________ > > The information transmitted is intended only for the person or entity to > which it is addressed and may contain confidential and/or privileged > material. Any review, retransmission, dissemination or other use of, or > taking of any action in reliance upon, this information by persons or > entities other than the intended recipient is prohibited. If you received > this in error, please contact the sender and delete the material from any > computer. > _________________________________________________________________________ > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > > > > _________________________________________________________________________ > > The information transmitted is intended only for the person or entity to > which it is addressed and may contain confidential and/or privileged > material. Any review, retransmission, dissemination or other use of, or > taking of any action in reliance upon, this information by persons or > entities other than the intended recipient is prohibited. If you received > this in error, please contact the sender and delete the material from any > computer. > _________________________________________________________________________ > > _________________________________________________________________________ > > The information transmitted is intended only for the person or entity to > which it is addressed and may contain confidential and/or privileged > material. Any review, retransmission, dissemination or other use of, or > taking of any action in reliance upon, this information by persons or > entities other than the intended recipient is prohibited. If you received > this in error, please contact the sender and delete the material from any > computer. > _________________________________________________________________________ > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- / Ruben FAELENS +32 494 06 72 59 _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] CPU usage versus Nodes, Threads
by Qiguo Jing 23 Oct '14

23 Oct '14

Hi Bob, Thanks for your suggestions. Here are more tests. We actually have three clusters. Cluster 1 and 2: 8 nodes, (2 Processors, 4 cores/processor, no HT – Total 8 Threads)/node Cluster 3: 8 nodes, (1 Processors, 4 cores /processor, HT – Total 8 Threads )/node We also have a standalone machine: 2 processors, 6 cores/processor, HT – total 24 threads. For one particular case: Cluster 1 and 2 take 48 min to finish with 8 nodes, 8 threads/node, 60% CPU usage; 53 min to finish with 3 nodes, 8 threads/node, 90% CPU usage; Cluster 3 takes 227 min to finish with 8 nodes, 8 threads/node, 20% CPU usage; 207 min to finish with 3 nodes, 8 threads/node, 50% CPU usage; Standalone machine takes 82 min to finish with 24 threads, 100% CPU usage. It looks like with 24 threads, they should be pretty busy? Could the above phenomena be a hardware issue more than software? Qiguo From: Bob Ilgner [mailto:[email protected]] Sent: Thursday, October 23, 2014 1:11 AM To: discuss(a)mpich.org<mailto:[email protected]> Subject: Re: [mpich-discuss] CPU usage versus Nodes, Threads Hi Qiguo,

1 0

Re: [mpich-discuss] Status of checkpointing mechanisms with Slurm + MPICH + BLCR
by Wesley Bland 23 Oct '14

23 Oct '14

Hi Manuel, Unfortunately, the situation for checkpointing hasn’t changed much since that last email. Checkpointing has been a relatively low priority item for us since full-system checkpointing it is used by so few people now (in favor of more application-centric checkpointing) and we have very limited bandwidth to figure out what needs to happen to support it again. I’m not sure of the last version of MPICH that did work with checkpointing (probably something in the MPICH2-1.4 series if I had to guess. You’re welcome to try it out, but there’s a possibility that you’ll run into issues running such an old version against the SLURM and BLCR. It’s possible that we’ll get around to fixing checkpointing in the future, but there’s no timeline for that happening. AFAIK, pretty much all of the big MPI implementations have dropped support for it for the same reasons (https://www.open-mpi.org/faq/?category=ft <https://www.open-mpi.org/faq/?category=ft>). Thanks, Wesley > On Oct 23, 2014, at 10:18 AM, Manuel Rodríguez Pascual <manuel.rodriguez.pascual(a)gmail.com> wrote: > > Good afternoon all, > > I am a newbie in this MPICH world. I am trying to install a cluster with MPICH, having the possibility of checkpoint parallel tasks. > > My original idea was a software stack based on SLURM 14.03.8 + MPICH mpich-3.1.3 + BLCR 0.8.5 . They are supposed to have good integration among them, and the configuration process has been quite smooth until now. > > I have found however that the checkpoint of MPICH tasks is not working. At first I though it was my fault (configuration issues or whatever) due to it can be read in MPICH home page that BLCR integration is possible https://wiki.mpich.org/mpich/index.php/Checkpointing <https://wiki.mpich.org/mpich/index.php/Checkpointing> > > However, when looking for the solution I found this thread in this same mailing list: > http://lists.mpich.org/pipermail/discuss/2014-April/002498.html <http://lists.mpich.org/pipermail/discuss/2014-April/002498.html> > > saying " BLCR checkpointing hasn't worked for a few versions now. It's something we're working to fix in a future version". > > My question is then, > > -Is it possible right now to checkpoint MPICH with BLCR? > > -If not, is there any working checkpoint mechanism that you can suggest me? > > -If not, are you aware of a previous MPICH version where BLCR does work? Are there any drawbacks on employing it while you get the new one fixed? (Are you getting the new one fixed?) > > Thanks for your attention. Best regards, > > > > Manuel > > > -- > Dr. Manuel Rodríguez-Pascual > skype: manuel.rodriguez.pascual > phone: (+34) 913466173 // (+34) 679925108 > > CIEMAT-Moncloa > Edificio 22, desp. 1.25 > Avenida Complutense, 40 > 28040- MADRID > SPAIN > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] CPU usage versus Nodes, Threads
by Qiguo Jing 23 Oct '14

23 Oct '14

Hi Bob, Thanks for your suggestions. Here are more tests. We actually have three clusters. Cluster 1 and 2: 8 nodes, (2 Processors, 4 cores/processor, no HT – Total 8 Threads)/node Cluster 3: 8 nodes, (1 Processors, 4 cores /processor, HT – Total 8 Threads )/node We also have a standalone machine: 2 processors, 6 cores/processor, HT – total 24 threads. For one particular case: Cluster 1 and 2 take 48 min to finish with 8 nodes, 8 threads/node, 60% CPU usage; 53 min to finish with 3 nodes, 8 threads/node, 90% CPU usage; Cluster 3 takes 227 min to finish with 8 nodes, 8 threads/node, 20% CPU usage; 207 min to finish with 3 nodes, 8 threads/node, 50% CPU usage; Standalone machine takes 24 min to finish with 24 threads, 100% CPU usage. It looks like with 24 threads, they should be pretty busy? Could the above phenomena be a hardware issue more than software? Qiguo From: Bob Ilgner [mailto:[email protected]] Sent: Thursday, October 23, 2014 1:11 AM To: discuss(a)mpich.org Subject: Re: [mpich-discuss] CPU usage versus Nodes, Threads Hi Qiguo,

1 0