- discuss - mailman3.cels.anl.gov

MPI_Barrier
by pinak panigrahi 16 Jun '13

16 Jun '13

Hi, I would like to know what algorithms are used for MPI_Barrier implementation in MPICH ... specifically, for intra-node MPI calls ! -- Pinak Panigrahi pursuing Masters in Computer Science at Sri Sathya Sai Institute Of Higher Learning, Puttaparti, India. "Thank God for what you have, Trust Him for what you need !"

1 0

Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem
by Jeff Hammond 16 Jun '13

16 Jun '13

I do not own a Rasberry Pi. Is this thing self-hosted, hence you are driving the Hydra download+build from a wimpy ARM core? Or is your Pi attached to a real computer? In case it wasn't obvious from the shell prompt ("Jeffs-MacBook-Pro:tmp jhammond"), I was timing on a Macbook Pro, which happens to have a 2.5 GHz Intel Core i7 quad-core processor. When I time on the oldest processor to which I have access (PPC970MP 2.5 GHz circa 2007), the whole process takes twice as long (77 seconds). In both cases, I built in /tmp, which on Linux is a ramdisk (http://en.wikipedia.org/wiki/Tmpfs) If building Hydra is a bottleneck because the Pi is too slow for basic operations like download a file, unpack it and run gcc on its contents, then you should go back to your very first statement ("I am trying to use two raspberry-pi to sample and then process some data") and consider whether or not what you're doing makes sense. If a computer can't compile very simple software like Hydra in a reasonable amount of time, I fail to see how it can be used to process data quickly. While configure isn't necessarily a good model for data analysis, I imagine that your time is best spent implementing and running your data analysis on a computer with more substantial processing power. I guess I should also point out that you should be able to cross compile Hydra if you have the appropriate cross-compiler toolchain, hence you can download and build Hydra on an x86 laptop and then merely copy the resulting binary over to your toy processor. Best, Jeff On Sun, Jun 16, 2013 at 12:26 PM, Lee, Eibhlin <eibhlin.lee10(a)imperial.ac.uk> wrote: > Unfortunately my past experience with hydra is VERY different. > I'm baffled as to how you managed to do it so quickly. Surely you have to download the file from mpich.org then unzip (which takes at least 5 mins) then configure (up to an hour) make (a long time as well) and finally make install (another chance in which you can go have a cuppa). And then get the image and put it onto another device which first needs formatting; for me that takes anywhere between 10 minutes and an hour. > What did you do so that it took so little time? > Eibhlin > ________________________________________ > From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of Jeff Hammond [jhammond(a)alcf.anl.gov] > Sent: 16 June 2013 18:19 > To: discuss(a)mpich.org > Subject: Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem > >> I did not want to spend a whole day installing hydra instead of smpd if it would not work. > > It took me 41 seconds to install Hydra ex nihilo and I didn't even use > "make -j". The argument that installation time is a bottleneck is > invalid. smpd is not supported and you shouldn't use it. End of > discussion. > > Jeff > > <snip> > ALL DONE > Sun Jun 16 12:16:12 CDT 2013 > > real 0m40.912s > user 0m17.405s > sys 0m11.028s > > Jeffs-MacBook-Pro:tmp jhammond$ cat auto > #!/bin/bash > date > wget http://www.mpich.org/static/downloads/3.0.4/hydra-3.0.4.tar.gz && \ > tar -xzf hydra-3.0.4.tar.gz && \ > cd hydra-3.0.4 && \ > mkdir build && \ > cd build && \ > ../configure CC=gcc --prefix=/tmp/hydra-install && \ > make && \ > make install && \ > make check > echo "ALL DONE" > date > > -- > Jeff Hammond > Argonne Leadership Computing Facility > University of Chicago Computation Institute > jhammond(a)alcf.anl.gov / (630) 252-5381 > http://www.linkedin.com/in/jeffhammond > https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond > ALCF docs: http://www.alcf.anl.gov/user-guides > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute jhammond(a)alcf.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond ALCF docs: http://www.alcf.anl.gov/user-guides

1 0

Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem
by Lee, Eibhlin 16 Jun '13

16 Jun '13

Unfortunately my past experience with hydra is VERY different. I'm baffled as to how you managed to do it so quickly. Surely you have to download the file from mpich.org then unzip (which takes at least 5 mins) then configure (up to an hour) make (a long time as well) and finally make install (another chance in which you can go have a cuppa). And then get the image and put it onto another device which first needs formatting; for me that takes anywhere between 10 minutes and an hour. What did you do so that it took so little time? Eibhlin ________________________________________ From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of Jeff Hammond [jhammond(a)alcf.anl.gov] Sent: 16 June 2013 18:19 To: discuss(a)mpich.org Subject: Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem > I did not want to spend a whole day installing hydra instead of smpd if it would not work. It took me 41 seconds to install Hydra ex nihilo and I didn't even use "make -j". The argument that installation time is a bottleneck is invalid. smpd is not supported and you shouldn't use it. End of discussion. Jeff <snip> ALL DONE Sun Jun 16 12:16:12 CDT 2013 real 0m40.912s user 0m17.405s sys 0m11.028s Jeffs-MacBook-Pro:tmp jhammond$ cat auto #!/bin/bash date wget http://www.mpich.org/static/downloads/3.0.4/hydra-3.0.4.tar.gz && \ tar -xzf hydra-3.0.4.tar.gz && \ cd hydra-3.0.4 && \ mkdir build && \ cd build && \ ../configure CC=gcc --prefix=/tmp/hydra-install && \ make && \ make install && \ make check echo "ALL DONE" date -- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute jhammond(a)alcf.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond ALCF docs: http://www.alcf.anl.gov/user-guides _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem
by Jeff Hammond 16 Jun '13

16 Jun '13

> I did not want to spend a whole day installing hydra instead of smpd if it would not work. It took me 41 seconds to install Hydra ex nihilo and I didn't even use "make -j". The argument that installation time is a bottleneck is invalid. smpd is not supported and you shouldn't use it. End of discussion. Jeff <snip> ALL DONE Sun Jun 16 12:16:12 CDT 2013 real 0m40.912s user 0m17.405s sys 0m11.028s Jeffs-MacBook-Pro:tmp jhammond$ cat auto #!/bin/bash date wget http://www.mpich.org/static/downloads/3.0.4/hydra-3.0.4.tar.gz && \ tar -xzf hydra-3.0.4.tar.gz && \ cd hydra-3.0.4 && \ mkdir build && \ cd build && \ ../configure CC=gcc --prefix=/tmp/hydra-install && \ make && \ make install && \ make check echo "ALL DONE" date -- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute jhammond(a)alcf.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond ALCF docs: http://www.alcf.anl.gov/user-guides

1 0

Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem
by Lee, Eibhlin 16 Jun '13

16 Jun '13

First time I tried that was the case, I then started smpd in root instead of in user. This made the uid=0 as you said it should. So I can confirm that I can run mpiexec id as root and get the expected answers. Yes I am still running smpd but if someone can verify that the program I provided works with hydra I will configure MPICH with hydra and never look at smpd again. Eibhlin ________________________________________ From: Pavan Balaji [balaji(a)mcs.anl.gov] Sent: 16 June 2013 14:23 To: Lee, Eibhlin Cc: discuss(a)mpich.org Subject: Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem On 06/16/2013 08:19 AM, Lee, Eibhlin wrote: > Actually, I can run mpiexec id AS ROOT. It is only when I try to run > the program that contains /dev/mem in root that there is an issue. > Sorry that I wasn't so clear in my earlier post. Are you sure? Here's what you wrote in a previous email, which indicates that your processes are not running as root: ================================================== sorry when I do run mpiexec id the output is uid=1000(pi) gid=1000(pi) groups=1000(pi),4(adm),20(dialout),24(cdrom),27(sudo),29(audio),44(video),46(plugdev),60(games),100(users),105(netdev),999(input) regardless of whether I'm in root or my usual user. root@raspi or pi@raspi. Is this output what you would expect? ================================================== You further added that you are running this with smpd. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem
by Pavan Balaji 16 Jun '13

16 Jun '13

On 06/16/2013 08:19 AM, Lee, Eibhlin wrote: > Actually, I can run mpiexec id AS ROOT. It is only when I try to run > the program that contains /dev/mem in root that there is an issue. > Sorry that I wasn't so clear in my earlier post. Are you sure? Here's what you wrote in a previous email, which indicates that your processes are not running as root: ================================================== sorry when I do run mpiexec id the output is uid=1000(pi) gid=1000(pi) groups=1000(pi),4(adm),20(dialout),24(cdrom),27(sudo),29(audio),44(video),46(plugdev),60(games),100(users),105(netdev),999(input) regardless of whether I'm in root or my usual user. root@raspi or pi@raspi. Is this output what you would expect? ================================================== You further added that you are running this with smpd. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem
by Lee, Eibhlin 16 Jun '13

16 Jun '13

Actually, I can run mpiexec id AS ROOT. It is only when I try to run the program that contains /dev/mem in root that there is an issue. Sorry that I wasn't so clear in my earlier post. Please re open the ticket to see if the program that contains /dev/mem will work properly using hydra. Eibhlin ________________________________________ From: Pavan Balaji [balaji(a)mcs.anl.gov] Sent: 16 June 2013 14:13 To: Lee, Eibhlin Cc: discuss(a)mpich.org Subject: Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem On 06/16/2013 07:37 AM, Lee, Eibhlin wrote: > Just to clarify: you were able to run the program that accesses > /dev/mem not just mpiexec id? I didn't run your application, but it doesn't matter. As I pointed out the primary problem you have is that your application processes are not running as root. If you don't fix that, you can't expect your application to work. I confirmed that you can run Hydra as root correctly. Till you get to that stage, I don't think we can do anything to help you. > I did not want to spend a whole day installing hydra instead of smpd > if it would not work. Well, all I can do is discourage you from using it. It's finally your call. But we can't spend more time debugging this for you unless you follow our recommendations. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] MPI server setup issue
by Pavan Balaji 16 Jun '13

16 Jun '13

Hi Sufeng, On 06/14/2013 04:35 PM, Sufeng Niu wrote: > 1. when I run a simple MPI hello world on multiple nodes, (I already > installed mpich3 library on master node, mount the nfs, shared the > executable file and mpi library, set slave node to be keyless ssh), my > program was stoped there say: > bash: /mnt/mpi/mpich-install/bin/hydra_pmi_proxy: /lib/ld-linux.so.2: > bad ELF interpreter: No such file or directory. 1. Did you make sure /mnt/mpi/mpich-install/bin/hydra_pmi_proxy is available on each node? 2. Did you also make sure all libraries it is linked to are available on each node? You can check these libraries using "ldd /mnt/mpi/mpich-install/bin/hydra_pmi_proxy" -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem
by Pavan Balaji 16 Jun '13

16 Jun '13

On 06/16/2013 07:37 AM, Lee, Eibhlin wrote: > Just to clarify: you were able to run the program that accesses > /dev/mem not just mpiexec id? I didn't run your application, but it doesn't matter. As I pointed out the primary problem you have is that your application processes are not running as root. If you don't fix that, you can't expect your application to work. I confirmed that you can run Hydra as root correctly. Till you get to that stage, I don't think we can do anything to help you. > I did not want to spend a whole day installing hydra instead of smpd > if it would not work. Well, all I can do is discourage you from using it. It's finally your call. But we can't spend more time debugging this for you unless you follow our recommendations. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem
by Lee, Eibhlin 16 Jun '13

16 Jun '13

Thank you Pavan, Just to clarify: you were able to run the program that accesses /dev/mem not just mpiexec id? I did not want to spend a whole day installing hydra instead of smpd if it would not work. Thank you everyone for your help! Eibhlin ________________________________________ From: Pavan Balaji [balaji(a)mcs.anl.gov] Sent: 16 June 2013 13:14 To: Lee, Eibhlin Cc: discuss(a)mpich.org Subject: Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem On 06/16/2013 04:53 AM, Lee, Eibhlin wrote: > Antonio set up a ticket > http://trac.mpich.org/projects/mpich/ticket/1885 to check whether the > problem would still occur in hydra. Could you please run the test > provided there with hydra and see if it works? I've verified that it works correctly and resolved the ticket. Please only use hydra. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0