December 2013 - discuss - mailman3.cels.anl.gov

Hardware / Performance Recommendations?
by Lou Picciano 18 Dec '13

18 Dec '13

MPICH Friends, Could we ask for some insights/recommendations on what hardware your sites are using in delivering these MPI(CH) workflows? We've been doing proof-of-concept work thus far on readily-available commodity systems; in the near future, would be doing all custom builds, so are able to keep an 'open mind'! At this point, we're looking into all SuperMicro, ECC memory, Intel Xeon E2nnn v2 series CPUs, and perhaps looking into leveraging nVidia GPUs as well. Are there any big gotchas? Or no-gos? What would opinions be on the Faster-ClockSpeed-vs.-More-Cores argument? Is there a good argument in favor of the most expensive Xeons over more commodity i7 'consumer grade' CPUs (ECC aside)? Or as compared to Athlon multicore chips? Is the negotiation over network (10GB) trivial enough that adding more (smaller) nodes might be roughly equivalent to building monster servers? Ie, are there any numbers comparing the More-nodes-fewer-cores-each to the 1-node, many-cores approach? (Think we know the answer to this one, roughly, but are wondering if there are any hard numbers we could use in our planning decisions). Are there any specific hardware pieces / controllers / interfaces to be avoided? I realize it's hard to make such assertions in a vacuum - ie, without more specifics about the workflow - but we don't have these fully nailed down yet. Nor will we have the opportunity in the very near term to apply much code optimization expertise. For the moment, we're projecting for clustered fortran workflows - but would like to hear observations/insights into how best keep options open. Many Thanks for Insights, Lou Picciano

1 0

Which mpich version?
by Roman Schiesser 18 Dec '13

18 Dec '13

Good day everybody! I would like to ask you, if there are preferences in th mpich version for a high performance cluster (in sum 64 cores) based on a windows server 2008 enterprise R2 operating system. Thanx for support and wish you a merry Christmas, Manno2009

1 0

Binding ranks to CPUs + Hydra-Bug
by Jan Bierbaum 18 Dec '13

18 Dec '13

Hi! Can somebody please explain the CPU binding in MPICH in more detail than 'mpiexec -bind-to -help' does? In particular I'd like to know how to use '-bind-to user' correctly. What I want to do is manually assign each rank to a CPU. Not all available CPUs will necessarily be used and there may be more ranks than CPUs. What I learned from the help text and some trial and error testing is the following: - The general syntax I need is '-bind-to user:0,1,2,...', which assigns CPU0 to rank 0, CPU1 to rank1 and so on. - Specifying more CPU ids than ranks does not matter - they are simply ignored. - Specifying too few CPU ids will leave the remaining ranks unbound and thus free to migrate from CPU to CPU. Now assume I want to run r ranks on a machine with c CPUs and utilize only c/2 CPUs. This works fine as long as r <= c. Otherwise, when I specify *exactly* c CPU ids, MPICH seems to "extend" the given list to the remaining ranks. Is this expected and reliable behavior or mere coincidence? The main problem is that as soon as I try to hand '-bind-to' more than c CPU ids it triggers the following assertion fault: > [mpiexec@local] control_cb (/home/user/mpich-3.0.4/src/pm/hydra/pm/pmiserv/pmiserv_cb.c:202): assert (!closed) failed > [mpiexec@local] HYDT_dmxu_poll_wait_for_event (/home/user/mpich-3.0.4/src/pm/hydra/tools/demux/demux_poll.c:77): callback returned error status > [mpiexec@local] HYD_pmci_wait_for_completion (/home/user/mpich-3.0.4/src/pm/hydra/pm/pmiserv/pmiserv_pmci.c:197): error waiting for event > [mpiexec@local] main (/home/user/mpich-3.0.4/src/pm/hydra/ui/mpich/mpiexec.c:331): process manager error waiting for completion As you can see from the output I'm running MPICH 3.0.4. During the 'configure' step I used '--with-pm=hydra --enable-hydra-procbind' to enable CPU binding. Regards, Jan

1 0

Re: [mpich-discuss] The max amount of data each process can send in MPI_Alltoallv
by Rob Latham 17 Dec '13

17 Dec '13

On Fri, Dec 13, 2013 at 07:02:47PM -0600, Rajeev Thakur wrote: > You can create a contiguous datatype of size 1GB and then send 1G items of that size :-). ... as long as you are running a very recent MPICH, such as a clone from git or one of the pre-releaseses. 3.0.4 is *not* new enough. Until my "large count / large datatype" stuff went in a couple months ago, this much-advertised approach would give MPICH headaches. ==rob > On Dec 13, 2013, at 5:18 PM, Huiwei Lu <huiweilu(a)mcs.anl.gov> wrote: > > > Hi, folks, > > > > What’s the max amount of data each process can send in MPI_Alltoallv? http://www.mcs.anl.gov/research/projects/mpi/www/www3/MPI_Alltoallv.html > > > > As I understand, the input parameter of sendcnts is an array of int, so the amount of data each process can send is limited to MAX_INT (approximately 2G). > > > > What if I want to use MPI_Alltoallv-like MPI routines to send a buffer of data that exceeds 2GB? Does it need to cut the data buffers to smaller sizes? Is there an easy way to do this? The reason I am asking is one of my distributed sort algorithm uses MPI_Alltoallv to distribute data after partitioning; it crashed when I tried to send in each process more than 2GB data. -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA

1 0

Re: [mpich-discuss] warning: assignment makes pointer from integer without a cast
by Antoine Rougier 17 Dec '13

17 Dec '13

Hello, I think your issue is because in line 38 you cast the return of malloc as a int and not a int*. Best, Antoine Rougier On 12/17/2013 05:52 AM, Soheil Hooshdaran wrote: > Hello. > When 'mpicc'ing' my program, I get these warnings: > > /media/TOSHIBA/Hooshdaran_p2.c: In function ‘main’: > /media/TOSHIBA/Hooshdaran_p2.c:38:5: warning: assignment makes pointer > from integer without a cast [enabled by default] > /media/TOSHIBA/Hooshdaran_p2.c:39:14: warning: initialization makes > pointer from integer without a cast [enabled by default] > /media/TOSHIBA/Hooshdaran_p2.c:84:11: warning: assignment makes pointer > from integer without a cast [enabled by default] > > Why are they produced? I attach the program file for you > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss >

1 0

warning: assignment makes pointer from integer without a cast
by Soheil Hooshdaran 16 Dec '13

16 Dec '13

Hello. When 'mpicc'ing' my program, I get these warnings: /media/TOSHIBA/Hooshdaran_p2.c: In function ‘main’: /media/TOSHIBA/Hooshdaran_p2.c:38:5: warning: assignment makes pointer from integer without a cast [enabled by default] /media/TOSHIBA/Hooshdaran_p2.c:39:14: warning: initialization makes pointer from integer without a cast [enabled by default] /media/TOSHIBA/Hooshdaran_p2.c:84:11: warning: assignment makes pointer from integer without a cast [enabled by default] Why are they produced? I attach the program file for you

1 0

Re: [mpich-discuss] hi
by "Antonio J. Peña" 16 Dec '13

16 Dec '13

You can subscribe here: https://lists.mpich.org/mailman/listinfo/discuss Antonio On 12/16/2013 05:36 AM, fereshteh komijani wrote: > dear > if it is possible i want to intercept daily post from mpich forum > please guide me how i can do that > cheers > fereshte > > -- > ***Angel*** > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Antonio J. Peï¿½a Postdoctoral Appointee Mathematics and Computer Science Division Argonne National Laboratory 9700 South Cass Avenue, Bldg. 240, Of. 3148 Argonne, IL 60439-4847 apenya(a)mcs.anl.gov www.mcs.anl.gov/~apenya

1 0

hi
by fereshteh komijani 16 Dec '13

16 Dec '13

dear if it is possible i want to intercept daily post from mpich forum please guide me how i can do that cheers fereshte -- ***Angel***

1 0

Re: [mpich-discuss] The max amount of data each process can send in MPI_Alltoallv
by Jeff Hammond 15 Dec '13

15 Dec '13

On Sun, Dec 15, 2013 at 7:21 PM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: > > On Dec 16, 2013, at 9:49 AM, Huiwei Lu <huiweilu(a)mcs.anl.gov> wrote: >> @JT, yes, if the data continue to grow and there is no proper way to do it in MPI type, I will have to partition the buffer to smaller trunks. > > MPI (standard) can send infinitely large data using Rajeev’s suggestion. The internal MPICH stack can allow up to 64-bits (16 exabytes) because of its internal bookkeeping. When people start sending more than 16 exabytes in a single message, we’ll fix that. :-) Let us define Big Data to be the domain of computing wherein 64b indexing is not sufficient :-) Jeff -- Jeff Hammond jeff.science(a)gmail.com

1 0

Re: [mpich-discuss] The max amount of data each process can send in MPI_Alltoallv
by Pavan Balaji 15 Dec '13

15 Dec '13

On Dec 16, 2013, at 9:49 AM, Huiwei Lu <huiweilu(a)mcs.anl.gov> wrote: > @JT, yes, if the data continue to grow and there is no proper way to do it in MPI type, I will have to partition the buffer to smaller trunks. MPI (standard) can send infinitely large data using Rajeev’s suggestion. The internal MPICH stack can allow up to 64-bits (16 exabytes) because of its internal bookkeeping. When people start sending more than 16 exabytes in a single message, we’ll fix that. :-) -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0