March 2014 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] Mpich failing electric-fence check
by Jed Brown 07 Mar '14

07 Mar '14

Matthieu Dorier <matthieu.dorier(a)irisa.fr> writes: > ElectricFence Aborting: Allocating 0 bytes, probably a bug. I would argue that ElectricFence is wrong here. The standard guarantees that malloc(0) succeeds and it is useful to simplify code and to test for matching free in cases where a positive size is rare. C99 §7.20.3: If the size of the space requested is zero, the behavior is implementation- defined: either a null pointer is returned, or the behavior is as if the size were some nonzero value, except that the returned pointer shall not be used to access an object. Either way, the returned pointer can be passed to free. Are you aware of current malloc implementations which violate the above?

1 0

Re: [mpich-discuss] Mpich failing electric-fence check
by Balaji, Pavan 07 Mar '14

07 Mar '14

Thanks, we’ll look into it. We try to avoid malloc(0) because it’s not portable, but it sometimes slips through. https://trac.mpich.org/projects/mpich/ticket/2054 We also have our internal memory tracing integrated into MPICH, that is tested regularly. We should probably also update it to check for zero-byte allocation errors. — Pavan On Mar 7, 2014, at 2:45 AM, Matthieu Dorier <matthieu.dorier(a)irisa.fr> wrote: > Hi, > > I wanted to debug a memory corruption in an MPI program using the electric-fence tool, and noticed that electric-fence detects an error already in MPI_Init (thus the program stops and I cannot debug the actual memory corruption that happens later). The following program is a minimal one to exemplify the error: > > #include <mpi.h> > int main(int argc, char** argv) { > MPI_Init(&argc,&argv); > MPI_Finalize(); > return 0; > } > > The error output by electric-fence: > > ElectricFence Aborting: Allocating 0 bytes, probably a bug. > > And the backtrace output by gdb: > > Program received signal SIGILL, Illegal instruction. > 0x0012d422 in __kernel_vsyscall () > (gdb) backtrace > #0 0x0012d422 in __kernel_vsyscall () > #1 0x0040c976 in kill () at ../sysdeps/unix/syscall-template.S:82 > #2 0x0012fc54 in EF_Abort () from /usr/lib/libefence.so.0 > #3 0x0012f71b in memalign () from /usr/lib/libefence.so.0 > #4 0x0012f88b in malloc () from /usr/lib/libefence.so.0 > #5 0x001e3b6b in MPID_nem_init () from /home/mdorier/deploy/lib/libmpich.so.10 > #6 0x001d2f4c in MPIDI_CH3_Init () from /home/mdorier/deploy/lib/libmpich.so.10 > #7 0x001c8c57 in MPID_Init () from /home/mdorier/deploy/lib/libmpich.so.10 > #8 0x0029d435 in MPIR_Init_thread () from /home/mdorier/deploy/lib/libmpich.so.10 > #9 0x0029cd33 in PMPI_Init () from /home/mdorier/deploy/lib/libmpich.so.10 > #10 0x0804859f in main (argc=1, argv=0xbffff994) at m.c:4 > > The version of mpich is 3.0.4, gcc 4.6.4, on Ubuntu 10.4, linux kernel 2.6.32. > > I suspect a call to malloc with 0 as parameter, whose output is properly checked by Mpich, but makes electric-fence think there is an error. > > Matthieu Dorier > PhD student at ENS Rennes > http://people.irisa.fr/Matthieu.Dorier > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Problems with computation-communication overlap in non-blocking mode
by Velickovic Nikola 07 Mar '14

07 Mar '14

Dear all, I have a simple MPI program with two processes using non-blocking communication illustrated bellow: process 0: process 1: MPI_Isend MPI_Irecv compute stage compute stage MPI_Wait MPI_Wait Actual communication is performed by offloading it to another thread, or using DMA (KNEM module is used for this). Ideally what should happen is that process 0 issues a non-blocking send, process 1 receives the data and in the meantime (in parallel) the CPU cores where the processes run are doing the compute stage. When compute stage is completed calling MPI_Wait wraps up the communication. When I profile my application it turns out that actual communication is initiated with MPI_Wait (significant amount of time is spent there) and hence disables overlapping communication and computation since MPI_Wait is called after the compute stage. Computation in my test case takes more time than communication so MPI_Wait should not be consuming significant amount of time since the communication should be over by then. This I confirmed also by using MPI_Test instead of MPI_Wait. MPI_Test has the same effect as MPI_Wait (to the best of my knowledge) but is non-blocking. When placing MPI_Test strategically in the compute stage it initiates the communication and a certain communication-computation overlap is achieved. Could you please shed some light for me if I am doing something wrong with the library? Is it the way it should behave (MPI_Wait initiates the actual transfer)? How to achieve communication-computation overlap? Best, Nikola

1 0

Mpich failing electric-fence check
by Matthieu Dorier 07 Mar '14

07 Mar '14

Hi, I wanted to debug a memory corruption in an MPI program using the electric-fence tool, and noticed that electric-fence detects an error already in MPI_Init (thus the program stops and I cannot debug the actual memory corruption that happens later). The following program is a minimal one to exemplify the error: #include <mpi.h> int main(int argc, char** argv) { MPI_Init(&argc,&argv); MPI_Finalize(); return 0; } The error output by electric-fence: ElectricFence Aborting: Allocating 0 bytes, probably a bug. And the backtrace output by gdb: Program received signal SIGILL, Illegal instruction. 0x0012d422 in __kernel_vsyscall () (gdb) backtrace #0 0x0012d422 in __kernel_vsyscall () #1 0x0040c976 in kill () at ../sysdeps/unix/syscall-template.S:82 #2 0x0012fc54 in EF_Abort () from /usr/lib/libefence.so.0 #3 0x0012f71b in memalign () from /usr/lib/libefence.so.0 #4 0x0012f88b in malloc () from /usr/lib/libefence.so.0 #5 0x001e3b6b in MPID_nem_init () from /home/mdorier/deploy/lib/libmpich.so.10 #6 0x001d2f4c in MPIDI_CH3_Init () from /home/mdorier/deploy/lib/libmpich.so.10 #7 0x001c8c57 in MPID_Init () from /home/mdorier/deploy/lib/libmpich.so.10 #8 0x0029d435 in MPIR_Init_thread () from /home/mdorier/deploy/lib/libmpich.so.10 #9 0x0029cd33 in PMPI_Init () from /home/mdorier/deploy/lib/libmpich.so.10 #10 0x0804859f in main (argc=1, argv=0xbffff994) at m.c:4 The version of mpich is 3.0.4, gcc 4.6.4, on Ubuntu 10.4, linux kernel 2.6.32. I suspect a call to malloc with 0 as parameter, whose output is properly checked by Mpich, but makes electric-fence think there is an error. Matthieu Dorier PhD student at ENS Rennes http://people.irisa.fr/Matthieu.Dorier

1 0

Re: [mpich-discuss] Overwriting mpi_waitany_
by Matthieu Dorier 05 Mar '14

05 Mar '14

Great, thanks! Matthieu Dorier PhD student at ENS Rennes http://people.irisa.fr/Matthieu.Dorier ----- Mail original ----- > De: "Rajeev Thakur" <thakur(a)mcs.anl.gov> > À: discuss(a)mpich.org > Envoyé: Mercredi 5 Mars 2014 21:45:04 > Objet: Re: [mpich-discuss] Overwriting mpi_waitany_ > > The C function returns the index with a 0 base. You have to add one to it to > return the value in Fortran. See the last line of the Fortran binding for > Waitany in MPICH. > > Rajeev > > > On Mar 5, 2014, at 1:42 PM, Matthieu Dorier <matthieu.dorier(a)irisa.fr> > wrote: > > > Hi, > > > > I'm trying to overwrite the weak symbol mpi_waitany_ (fortran symbol), but > > I get an error (the function doesn't seem to work since it returns an > > index of 0). > > Here is my code, any idea why it doesn't work? > > > > void mpi_waitany_(MPI_Fint* count, MPI_Fint* array_of_requests, > > MPI_Fint* index, MPI_Fint* status, MPI_Fint* err) > > { > > MPI_Status cstatus; > > MPI_Status_f2c(status, &cstatus); > > MPI_Request* c_req = > > (MPI_Request*)malloc(sizeof(MPI_Request)*(*count)); > > int i; > > for(i=0;i<*count;i++) { > > c_req[i] = MPI_Request_f2c(array_of_requests[i]); > > } > > *err = PMPI_Waitany(*count,c_req,index,&cstatus); > > MPI_Status_c2f(&cstatus, status); > > for(i=0;i<*count;i++) { > > array_of_requests[i] = MPI_Request_c2f(c_req[i]); > > } > > free(c_req); > > } > > > > Thanks! > > > > Matthieu Dorier > > PhD student at ENS Rennes > > http://people.irisa.fr/Matthieu.Dorier > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss >

1 0

Re: [mpich-discuss] Overwriting mpi_waitany_
by Rajeev Thakur 05 Mar '14

05 Mar '14

The C function returns the index with a 0 base. You have to add one to it to return the value in Fortran. See the last line of the Fortran binding for Waitany in MPICH. Rajeev On Mar 5, 2014, at 1:42 PM, Matthieu Dorier <matthieu.dorier(a)irisa.fr> wrote: > Hi, > > I'm trying to overwrite the weak symbol mpi_waitany_ (fortran symbol), but I get an error (the function doesn't seem to work since it returns an index of 0). > Here is my code, any idea why it doesn't work? > > void mpi_waitany_(MPI_Fint* count, MPI_Fint* array_of_requests, > MPI_Fint* index, MPI_Fint* status, MPI_Fint* err) > { > MPI_Status cstatus; > MPI_Status_f2c(status, &cstatus); > MPI_Request* c_req = (MPI_Request*)malloc(sizeof(MPI_Request)*(*count)); > int i; > for(i=0;i<*count;i++) { > c_req[i] = MPI_Request_f2c(array_of_requests[i]); > } > *err = PMPI_Waitany(*count,c_req,index,&cstatus); > MPI_Status_c2f(&cstatus, status); > for(i=0;i<*count;i++) { > array_of_requests[i] = MPI_Request_c2f(c_req[i]); > } > free(c_req); > } > > Thanks! > > Matthieu Dorier > PhD student at ENS Rennes > http://people.irisa.fr/Matthieu.Dorier > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Overwriting mpi_waitany_
by Matthieu Dorier 05 Mar '14

05 Mar '14

Hi, I'm trying to overwrite the weak symbol mpi_waitany_ (fortran symbol), but I get an error (the function doesn't seem to work since it returns an index of 0). Here is my code, any idea why it doesn't work? void mpi_waitany_(MPI_Fint* count, MPI_Fint* array_of_requests, MPI_Fint* index, MPI_Fint* status, MPI_Fint* err) { MPI_Status cstatus; MPI_Status_f2c(status, &cstatus); MPI_Request* c_req = (MPI_Request*)malloc(sizeof(MPI_Request)*(*count)); int i; for(i=0;i<*count;i++) { c_req[i] = MPI_Request_f2c(array_of_requests[i]); } *err = PMPI_Waitany(*count,c_req,index,&cstatus); MPI_Status_c2f(&cstatus, status); for(i=0;i<*count;i++) { array_of_requests[i] = MPI_Request_c2f(c_req[i]); } free(c_req); } Thanks! Matthieu Dorier PhD student at ENS Rennes http://people.irisa.fr/Matthieu.Dorier

1 0

Re: [mpich-discuss] Help with Cray MPICH
by Marcin Zalewski 05 Mar '14

05 Mar '14

Nick, We got MPT 6.2.2 installed, and indeed, the initial tests don't cause the same problem anymore. Thank you for alerting me to that bug. It really saved me a lot of hair pulling. -m On Mon, Feb 3, 2014 at 2:46 PM, Nick Radcliffe <nradclif(a)cray.com> wrote: > Hi Marcin, > > I can't be certain, but this looks like a bug we fixed recently in Cray MPICH. The fix should be available in MPT 6.2.2 by Feb. 20. > > -Nick Radcliffe, > Cray MPT Team > > ________________________________________ > From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of Marcin Zalewski [marcin.zalewski(a)gmail.com] > Sent: Monday, February 03, 2014 1:27 PM > To: discuss(a)mpich.org > Subject: [mpich-discuss] Help with Cray MPICH > > I have an application I am trying to run on a Cray machine composed of > XE6 nodes. I have run this application previously using Open MPI and > MVAPICH on a few different machines, so I think it should be more or > less free of major bugs. However, when I run it on the Cray machine, I > get a segmentation fault with a stack trace that ends in this: > > #0 0x00002aaaafe265e6 in memcpy () from /lib64/libc.so.6 > #1 0x00002aaaae8fe023 in MPID_Segment_index_m2m () from > /opt/cray/lib64/libmpich_gnu_48.so.2 > #2 0x00002aaaae8fca18 in MPID_Segment_manipulate () from > /opt/cray/lib64/libmpich_gnu_48.so.2 > #3 0x00002aaaae9049d1 in MPID_Segment_unpack () from > /opt/cray/lib64/libmpich_gnu_48.so.2 > #4 0x00002aaaae8edd38 in MPID_nem_gni_complete_rdma_get () from > /opt/cray/lib64/libmpich_gnu_48.so.2 > #5 0x00002aaaae8e11c8 in MPID_nem_gni_check_localCQ () from > /opt/cray/lib64/libmpich_gnu_48.so.2 > #6 0x00002aaaae8e29fa in MPID_nem_gni_poll () from > /opt/cray/lib64/libmpich_gnu_48.so.2 > #7 0x00002aaaae8c3515 in MPIDI_CH3I_Progress () from > /opt/cray/lib64/libmpich_gnu_48.so.2 > #8 0x00002aaaae9a3d4d in PMPI_Testsome () from > /opt/cray/lib64/libmpich_gnu_48.so.2 > > I understand that this is very little information to go on, but I > really cannot deal with this problem, and I thought I would try this > list while I am trying to solicit help from our support team. > Currently, I do not have access to a debug version of the library (or > the source code). Has anyone here seen a similar error? What would be > a reason to get an error on memcpy in MPICH while calling MPI_Testsome > in general? I do have asserts in my code to make sure that the output > arrays are large enough, and I got it to work on other implementations > of MPI, so I am really clueless to what the problem might be. > > Thank you, > Marcin > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] very slow file writes independent of file size
by Geoffrey Irving 04 Mar '14

04 Mar '14

Thanks. I get the same non-Lustre error for scratch, but I can ask NERSC about that to confirm. I will be paranoid about stripe_count in future. :) Geoffrey On Mon, Mar 3, 2014 at 10:07 PM, Wei-keng Liao <wkliao(a)eecs.northwestern.edu> wrote: > > project directory is a GPFS. lfs command is for Lustre only. > > You can run lfs command on the directory where you wrote your files under $SCRATCH. > Files created in a Lustre directory inherit the striping settings of that directory. > The default stripe_count on Edison appears to be one. So, if you have large amount > of data to write, then changing stripe_count to a larger value should give a much > higher performance. All you need to do is to first create a new directory and then > set the stripe_count to say 96 using lfs command and write your files there. > For example, > mkdir $SCRATCH/new_directory > lfs setstripe -s 1M -c 96 -o -1 $SCRATCH/new_directory > > Using independent file points will most likely give you a much better performance, > as some implementations of shared file pointer MPI-IO functions use a temporary > file to store the pointer (so one process's change to the pointer can be immediately > visible to other processes). Because of this and other overheads, the use of > shared-pointer functions is generally discouraged. > > However, I think the poor performance in your case is mainly because of file > striping issue. > > > Wei-keng > > On Mar 3, 2014, at 11:36 PM, Geoffrey Irving wrote: > >> Unfortunately, I have moved the files since I ran the job (from >> scratch to project). Running lfs getstripe on files generated by a >> different run in the project directory produces >> >> edison04:restart-5% lfs getstripe sparse-15.npy >> can't find fs root for >> '/global/project/projectdirs/pentago/restart-5/sparse-15.npy': -19 >> 'sparse-15.npy' is not on a Lustre filesystem: No such device (19) >> cb_getstripe: 'sparse-15.npy' not on a Lustre fs?: Inappropriate ioctl >> for device (25) >> error: getstripe failed for sparse-15.npy. >> >> I will certainly use MPI_File_write_at_all in future, though it's >> unintuitive if accidentally using a different but still collective >> routine results in >3 minutes hangs. >> >> Thanks, >> Geoffrey >> >> >> >> On Mon, Mar 3, 2014 at 10:43 AM, Wei-keng Liao >> <wkliao(a)eecs.northwestern.edu> wrote: >>> Hi, Geoffrey >>> >>> Edison has a Lustre file system and I assume your program writes files >>> there. If this is the case, please check the file striping setting of your >>> output files, using command "lfs getstripe filename" >>> >>> Usually, a high-performance I/O can be obtained if you have configured >>> the lustre striping setting to a higher stripe_count (96 or 144 max on Edison). >>> >>> In addition, MPI_File_write_ordered is for shared file pointers. >>> It can have some impact to the performance. Using independent file pointers >>> often results in a better performance. You might want to consider to >>> change your program to use those functions (eg. MPI_File_write_all). >>> >>> >>> Wei-keng >>> >>> On Mar 3, 2014, at 12:21 AM, Geoffrey Irving wrote: >>> >>>> I'm doing postmortem on a 2048 node (16384 rank) job on Edison, and >>>> trying to understand why my I/O performance might have been slow. >>>> >>>> Here's the data: >>>> >>>> Measured I/O bandwidth: >>>> slice 35 write sparse bandwidth = 6082640 / (3.52519e+06 s / 16384) = >>>> 2.63287e-05 GB/s >>>> slice 34 write sparse bandwidth = 13824080 / (3.66608e+06 s / 16384) = >>>> 5.75379e-05 GB/s >>>> slice 33 write sparse bandwidth = 24754256 / (2.83647e+06 s / 16384) = >>>> 0.000133166 GB/s >>>> slice 32 write sparse bandwidth = 39370832 / (3.47016e+06 s / 16384) = >>>> 0.000173119 GB/s >>>> slice 31 write sparse bandwidth = 55812176 / (2.53623e+06 s / 16384) = >>>> 0.000335785 GB/s >>>> slice 30 write sparse bandwidth = 74741840 / (2.5714e+06 s / 16384) = >>>> 0.00044352 GB/s >>>> slice 29 write sparse bandwidth = 93560912 / (2.67336e+06 s / 16384) = >>>> 0.000534019 GB/s >>>> slice 28 write sparse bandwidth = 112803920 / (2.74639e+06 s / 16384) >>>> = 0.000626733 GB/s >>>> slice 27 write sparse bandwidth = 128194640 / (3.1603e+06 s / 16384) = >>>> 0.000618958 GB/s >>>> slice 26 write sparse bandwidth = 141281360 / (3.12754e+06 s / 16384) >>>> = 0.00068929 GB/s >>>> slice 25 write sparse bandwidth = 148193360 / (2.62376e+06 s / 16384) >>>> = 0.000861835 GB/s >>>> slice 24 write sparse bandwidth = 151861328 / (3.2145e+06 s / 16384) = >>>> 0.000720865 GB/s >>>> slice 23 write sparse bandwidth = 148193360 / (2.44736e+06 s / 16384) >>>> = 0.000923956 GB/s >>>> slice 22 write sparse bandwidth = 142055504 / (3.15962e+06 s / 16384) >>>> = 0.000686031 GB/s >>>> slice 21 write sparse bandwidth = 130388048 / (3.09774e+06 s / 16384) >>>> = 0.000642263 GB/s >>>> slice 20 write sparse bandwidth = 117964880 / (3.02676e+06 s / 16384) >>>> = 0.000594696 GB/s >>>> slice 19 write sparse bandwidth = 101560400 / (2.97198e+06 s / 16384) >>>> = 0.000521434 GB/s >>>> slice 18 write sparse bandwidth = 86372432 / (2.96247e+06 s / 16384) = >>>> 0.000444878 GB/s >>>> slice 18 write sections bandwidth = 1954957518434 / (1.83937e+07 s / >>>> 16384) = 1.62177 GB/s >>>> slice 17 write sparse bandwidth = 70170704 / (2.88973e+06 s / 16384) = >>>> 0.000370526 GB/s >>>> slice 17 write sections bandwidth = 1475380615039 / (1.36018e+07 s / >>>> 16384) = 1.65511 GB/s >>>> slice 17 read bandwidth (192 nodes) = 1475380615039 / 3383.36 s = 0.406122 GB/s >>>> per node: measured = 2.16598 MB/s, theoretical peak = 33.1341 MB/s >>>> >>>> Focusing on the "sparse" lines, the main point is that the time seems >>>> to be roughly independent of file size (plot attached). Each timing >>>> sample consists of (1) setup which I believe is negligible, (2) >>>> MPI_File_open, (3) MPI_File_write_ordered, (4) MPI_File_close. >>>> >>>> What might have caused these file writes to take so long? >>>> >>>> Geoffrey >>>> <2014-03-02-221950_1220x500.png>_______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Problems with running make
by Reuti 04 Mar '14

04 Mar '14

Am 04.03.2014 um 00:19 schrieb Ron Palmer: > Reuti, good comment about primary interface for mpi and secondary for the rest of the world. > > Yes, setting it up from scratch, and I would love a queuing system as well, I just had no idea such existed (learning by the minute!). These computers are dedicated to this parallelisation jobs I am running and will not run anything else. Would you be able to point my nose in a direction of a simple queuing system? > > All my parallel processing are controlled by my own perl scripts and commandline commands (provided binaries), nothing GUI at all (which is kinda nice). > > Do you think shutting down the xserver would make any difference to the performance of these computers? I am pretty sure that they al have a windows manager running by default... I only got them going last week so I have not (yet) gone in to this level of detail. As it will sleep most of the time, I don't think that it will consume much computing time. But I install and run my compute nodes in runlevel 3 (systemd speaking multi-user.target) because of this, and noone will ever log in directly to the nodes. -- Reuti > Thanks, > Ron > > On 4/03/2014 09:06, Reuti wrote: >> Am 03.03.2014 um 23:54 schrieb Gus Correa: >> >>> On 03/03/2014 04:36 PM, Ron Palmer wrote: >>>> Thanks Reuti for your comments. I will peruse that FAQ detail. >>>> >>>> I just thought of the fact that each of these rack computers have 4 >>>> ethernet sockets, eth0 - eth3... I could connect the cluster on a >>>> separate ethernet sockets via an extra switch not connected to the >>>> internet or any other computers, and accept all communication among >>>> them, and keep iptables up on the ethx connected to the outside world. I >>>> guess I would have to set up routing tables or something. Ah, more >>>> reading :-) >>>> >>>> Thanks for your help. >>>> Ron >>>> >>> Hi Ron >>> >>> If those extra interfaces are not in use, >>> and if you have a spare switch, >>> you can setup a separate private subnet exclusively for MPI. >>> You need to configure the interfaces consistently (IP, subnet mask, >>> perhaps a gateway). Configuring them statically is easy: >>> >>> https://access.redhat.com/site/documentation//en-US/Red_Hat_Enterprise_Linu… >>> >>> Use a subnet that doesn't intersect the existent/original IP range. >>> >>> http://en.wikipedia.org/wiki/Private_network >>> >>> You could also create host names associated to those IPs (say >>> node01, node02, node02), resolve them via /etc/hosts on each computer, >>> set passwordless ssh across these newly named "hosts". >>> This may simpler/safer than messing with the iptables. >>> >>> [Actually, the IP addresses you showed 192.168.X.Y, sound as a private >>> subnet already, not Internet, but that may be the subnet for your >>> organization/school/department already. So, you may set up a different >>> one on these three computers for MPI and very-local access.] >> Yep, maybe in the 10.0.0.0/8 range. >> >> BTW: are you setting up a cluster from scratch - will you also add any queuing system later on? >> >> >>> OpenMPI allows you to choose the interface that it will use, >>> so you can direct it to your very-local subnet: >>> >>> http://www.open-mpi.org/faq/?category=tcp#tcp-selection >> MPICH: >> >> http://wiki.mpich.org/mpich/index.php/Using_the_Hydra_Process_Manager#Hydra… >> >> It should work for other Ethernet interfaces too. >> >> Nevertheless: in any case it might be easier for the applications to use the primary interface solely for MPI, and any other one for the external access (this is usually my setup, as I don't have to provide the assignment of other interfaces this way). >> >> -- Reuti >> >> >> > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0