- discuss - mailman3.cels.anl.gov

MPI server setup issue
by Sufeng Niu 14 Jun '13

14 Jun '13

Hello, I am a beginner on MPI programming, and right now I am working on an MPI project. I got a few questions related to implementation issues: 1. when I run a simple MPI hello world on multiple nodes, (I already installed mpich3 library on master node, mount the nfs, shared the executable file and mpi library, set slave node to be keyless ssh), my program was stoped there say: bash: /mnt/mpi/mpich-install/bin/hydra_pmi_proxy: /lib/ld-linux.so.2: bad ELF interpreter: No such file or directory. I can not get rid of it for a long times. even though I reset everything (I already add PATH=/mnt/mpi/mpich-install/bin:$PATH in .bash_profile). Do you have any clues on this problems? 2. for multiple servers, each of them has 10G ethernet card. for example, one network card address is eth5: 10.0.5.55. So if I want to launch MPI communication through 10G network card. Should I set the hostfile as: 10.0.5.55:$(PROCESS_NUM)? Or using iface eth5 Thanks a lot! -- Best Regards, Sufeng Niu ECASP lab, ECE department, Illinois Institute of Technology Tel: 312-731-7219

1 0

Re: [mpich-discuss] ckpoint-num error
by john donald 14 Jun '13

14 Jun '13

the file size is 121.6 MB after i raised the interval to 20 sec my test application is MPI/c integer sort program with 5000 iterations sorry for the trivial question but how to know that the checkpoint file is empty or not how can i open it? 2013/6/11 Wesley Bland <wbland(a)mcs.anl.gov> > Did you check if there's actually anything in the checkpoint files? If > they're empty, that probably means that you're checkpointing too frequently. > > On Jun 10, 2013, at 5:17 PM, john donald <johnd9886(a)gmail.com> wrote: > > i raised it to 20 sec but same results > sorry i am new to checkpoint restart > i am trying this initially on one multicore pc > how should it look like if the restart succeed? should it work in the same > terminal in which i am running restart command > my test app has 5000 iterations , checkpoint is taken at iteration no 300 > for example , if i choose to restart from this checkpoint file should it > restart near this iteration no 300 > > > 2013/6/6 Wesley Bland <wbland(a)mcs.anl.gov> > >> Is there actually anything in those checkpoints? With a checkpoint >> happening every 4 seconds you may be overdoing it. >> >> Wesley >> >> On Jun 5, 2013, at 2:14 PM, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: >> >> > I don't know, but see if anything on this page helps: >> > http://wiki.mpich.org/mpich/index.php/Checkpointing >> > >> > On Jun 5, 2013, at 4:09 PM, john donald wrote: >> > >> >> >> >> >> >> ---------- Forwarded message ---------- >> >> From: john donald <johnd9886(a)gmail.com> >> >> Date: 2013/6/3 >> >> Subject: ckpoint-num error >> >> To: mpich-discuss(a)mcs.anl.gov >> >> >> >> >> >> i used mpiexec with checkpoint and created two checkpoint files: >> >> >> >> mpiexec -ckpointlib blcr -ckpoint-prefix /home/john/ckpts/app.ckpoint >> -ckpoint-interval 4 -n 4 /home/john/app/md >> >> >> >> context-num0-0-0 >> >> context-num1-0-0 >> >> >> >> >> >> i am trying to make a restart >> >> mpiexec -ckpointlib blcr -ckpoint-prefix /home/john/ckpts/app.ckpoint >> -n 4 -ckpoint-num 1 >> >> >> >> but nothing happened it just hangs >> >> i also tried: >> >> mpiexec -ckpointlib blcr -ckpoint-prefix /home/john/ckpts/app.ckpoint >> -n 4 -ckpoint-num 0-0-0 >> >> also hangs >> >> >> >> _______________________________________________ >> >> discuss mailing list discuss(a)mpich.org >> >> To manage subscription options or unsubscribe: >> >> https://lists.mpich.org/mailman/listinfo/discuss >> > >> > _______________________________________________ >> > discuss mailing list discuss(a)mpich.org >> > To manage subscription options or unsubscribe: >> > https://lists.mpich.org/mailman/listinfo/discuss >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> > > >

1 0

Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem
by Jim Dinan 14 Jun '13

14 Jun '13

I don't know if this has been suggested, but you could also add your user to the kmem group and chmod /dev/mem so that you have the access you need. ~Jim. On Fri, Jun 14, 2013 at 1:24 PM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: > > You can run mpich as root. There's no restriction on that. You still > haven't tried out my suggestion of running "id" to check what user ID you > are running your processes as. My guess is that you are not setting your > user ID correctly. > > -- Pavan > > > On 06/14/2013 06:27 AM, Lee, Eibhlin wrote: > >> I found that the reason we want to access /dev/mem is to setup memory >> regions to access the peripherals. (We are trying to read the output of an >> ADC). At this point it becomes more a linux/raspberry-pi specific problem >> than an MPICH problem. Although the fact that you can't run a program that >> needs access to memory mapping (even as the root user) seems something that >> MPICH could improve on for future versions. I know I am using smpd instead >> of hydra so this problem may already be solved. But if someone could >> confirm that, it would be really helpful. >> ______________________________**__________ >> From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of >> Lee, Eibhlin [eibhlin.lee10(a)imperial.ac.uk] >> Sent: 14 June 2013 11:20 >> To: discuss(a)mpich.org >> Subject: Re: [mpich-discuss] Running an mpi program that needs >> to access /dev/mem >> >> Gus, >> I tried running cpi, as is included in the installation of MPI, on two >> machines with two processes. The output message confirmed that it had >> started only 1 process instead of 2. >> Process 0 of 1 is on raspi >> pi is approximately... >> >> Then it just hung. I think this is because the other machine didn't know >> where to output the data? >> >> When I tried running two processes on the one machine using the wrapper >> you suggested the output was the same but doubled. It didn't hang. This >> confirms that every process was started with rank 0. >> >> I'm not entirely sure why /dev/mem is needed. I'm working in a group and >> another member set up io and gpio and it seemed it needed access to >> /dev/mem I am going to do a strace as suggested by Pavan Balaji to see >> where it is used and see if I can somehow work around it. >> >> Thank you for your help. >> Eibhlin >> ______________________________**__________ >> From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of >> Gus Correa [gus(a)ldeo.columbia.edu] >> Sent: 13 June 2013 21:11 >> To: Discuss Mpich >> Subject: Re: [mpich-discuss] Running an mpi program that needs to >> access /dev/mem >> >> Hi Eibhlin >> >> On 06/13/2013 12:59 PM, Lee, Eibhlin wrote: >> >>> Gus, >>> I believe your first assumption is correct. Unfortunately it just seemed >>> to hang. I think this might be because each one is being made to have the >>> same rank... >>> >> >> Darn! I was afraid that it might give only rank 0 to all MPI processes. >> So, with the script wrapper the process being launched by mpiexec may >> indeed be sudo, >> not the actual mpi executable (main) :( >> Then it may actually launch a bunch of separate rank 0 replicas of your >> program, >> instead of assigning to them different ranks. >> However, without any output or error message, it is hard to tell. >> >> No output at all? >> No error message, just hangs? >> Have you tried a verbose flag (-v) to mpiexec? >> (Not sure if it exists in MPICH mpiexec, you'd need to check.) >> >> Would you care to try it with another mpi program, >> one that doesn't deal with /dev/mem (a risky business), >> say cpi.c (in the examples directory), or an mpi version of Hello, world, >> just to see if the mpiexec+sudo_script_wrapper works as expected or >> if everybody gets rank 0? >> >> >> It may already be obvious but this is the first time I am using Linux. I >>> had tried sudo $(which mpiexec ....) and sudo $(which mpiexec) ... both >>> without success. >>> >> >> "which mpiexec" will return the path to mpiexec, but won't execute it. >> >> You could try this (with backquotes): >> >> `which mpiexec` -n 2 ~/main >> >> On a side note, make sure the mpiexec you're using matches the >> mpicc/mpif90/MPI library from the MPICH that >> you used to compile the program. >> Often times computers have several flavors of MPI installed, and mixing >> them just doesn't work. >> >> Is putting the full path to it similar to/is a symlink? (This still >>> doesn't make main have super user privileges though.) >>> >> >> No, nothing to do with sudo privileges. >> >> This suggestion was just to avoid messing up your /usr/bin, >> which is a directory that despite the somewhat misleading name (/usr, >> for historical reasons I think), >> is supposed to hold system (Linux) programs (that users can use), but >> not user-installed programs. >> Normally things are that are installed in /usr get there via some Linux >> package manager program >> (yum, rpm, apt-get, etc), to keep consistency with libraries, etc. >> >> I belive MPICH would install by default in /usr/local/ (and put mpiexec >> in /usr/local/bin), >> which is kind of a default location for non-system applications. >> >> The full path suggestion would be something like: >> /path/to/where/you/installed/**mpiexec -n 2 ~/main >> >> However, this won't solve the other problem w.r.t. sudo and /dev/mem. >> >> You must know what you are doing, but it made me wonder, >> even if your program were sequential, why would you want to mess with >> /dev/mem directly? >> Just curious about it. >> >> Gus Correa >> >> >> >> Eibhlin >>> ______________________________**__________ >>> From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf >>> of Gus Correa [gus(a)ldeo.columbia.edu] >>> Sent: 13 June 2013 15:37 >>> To: Discuss Mpich >>> Subject: Re: [mpich-discuss] Running an mpi program that needs to >>> access /dev/mem >>> >>> Hi Lee >>> >>> How about replacing "~/main" in the mpiexec command line >>> by one-liner script? >>> Say, "sudo_main.sh", something like this: >>> >>> #! /bin/bash >>> sudo ~/main >>> >>> After all, it is "main" that accesses /dev/mem, >>> and needs "sudo" permissions, not mpiexec, right? >>> [Or do the mpiexec-launched processes inherit >>> the "sudo" stuff from mpiexec?] >>> >>> Not related, but, instead of putting mpiexec in /usr/bin, >>> can't you just use the full path to it? >>> >>> I hope this helps, >>> Gus Correa >>> >>> On 06/13/2013 10:09 AM, Lee, Eibhlin wrote: >>> >>>> Pavan, >>>> I had a lot of trouble getting hydra to work without having to enter a >>>> password/passphrase. I saw the option to pass a phrase in the mpich >>>> installers guide. I eventually found that for that command you needed to >>>> use the smpd process manager. That's the only reason I chose smpd over >>>> hydra. >>>> As to your other suggestion. I ran ./main and the same error (Can't >>>> open /dev/mem...) appeared. sudo ./main works but of course without >>>> multiple processes. >>>> Eibhlin >>>> ______________________________**__________ >>>> From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf >>>> of Pavan Balaji [balaji(a)mcs.anl.gov] >>>> Sent: 13 June 2013 14:34 >>>> To: discuss(a)mpich.org >>>> Subject: Re: [mpich-discuss] Running an mpi program that needs to >>>> access /dev/mem >>>> >>>> I just saw your older email. Why are you using smpd instead of the >>>> default process manager (hydra)? >>>> >>>> -- Pavan >>>> >>>> On 06/13/2013 08:05 AM, Pavan Balaji wrote: >>>> >>>>> What's "-phrase"? That's not a recognized option. I'm not sure where >>>>> the /dev/mem check is coming from. Try running ~/main without mpiexec >>>>> first. >>>>> >>>>> -- Pavan >>>>> >>>>> On 06/13/2013 06:56 AM, Lee, Eibhlin wrote: >>>>> >>>>>> Hello all, >>>>>> >>>>>> I am trying to use two raspberry-pi to sample and then process some >>>>>> data. The first process samples while the second processes and vice >>>>>> versa. To do this I use gpio and also mpich-3.0.4 with the process >>>>>> manager smpd. I have successfully run cpi on both machines (from the >>>>>> master machine). I have also managed to run a similar program but >>>>>> without the MPI, this involved compiling with gcc and when running >>>>>> putting sudo in front of the binary file. >>>>>> >>>>>> When I combine these two processes I get various error messages. >>>>>> For input: >>>>>> mpiexec -phrase cat -machinefile machinefile -n 2 ~/main >>>>>> the error is: >>>>>> Can't open /dev/mem >>>>>> Did you forget to use 'sudo .. ?' >>>>>> >>>>>> For input: >>>>>> sudo mpiexec -phrase cat -machinefile machinefile -n 2 ~/main >>>>>> the error is: >>>>>> sudo: mpiexec: Command not found >>>>>> >>>>>> I therefore put mpiexec into /usr/bin >>>>>> >>>>>> now for input: >>>>>> sudo mpiexec -phrase cat -machinefile machinefile -n 2 ~/main >>>>>> the error is: >>>>>> Can't open /dev/mem >>>>>> Did you forget to use 'sudo .. ?' >>>>>> >>>>>> Does anyone know how I can work around this? >>>>>> Thanks, >>>>>> Eibhlin >>>>>> >>>>>> >>>>>> >>>>>> ______________________________**_________________ >>>>>> discuss mailing list discuss(a)mpich.org >>>>>> To manage subscription options or unsubscribe: >>>>>> https://lists.mpich.org/**mailman/listinfo/discuss<https://lists.mpich.org/mailman/listinfo/discuss> >>>>>> >>>>>> -- >>>> Pavan Balaji >>>> http://www.mcs.anl.gov/~balaji >>>> ______________________________**_________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/**mailman/listinfo/discuss<https://lists.mpich.org/mailman/listinfo/discuss> >>>> ______________________________**_________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/**mailman/listinfo/discuss<https://lists.mpich.org/mailman/listinfo/discuss> >>>> >>> ______________________________**_________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/**mailman/listinfo/discuss<https://lists.mpich.org/mailman/listinfo/discuss> >>> ______________________________**_________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/**mailman/listinfo/discuss<https://lists.mpich.org/mailman/listinfo/discuss> >>> >> >> ______________________________**_________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/**mailman/listinfo/discuss<https://lists.mpich.org/mailman/listinfo/discuss> >> ______________________________**_________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/**mailman/listinfo/discuss<https://lists.mpich.org/mailman/listinfo/discuss> >> ______________________________**_________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/**mailman/listinfo/discuss<https://lists.mpich.org/mailman/listinfo/discuss> >> >> > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > ______________________________**_________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/**mailman/listinfo/discuss<https://lists.mpich.org/mailman/listinfo/discuss> >

1 0

Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem
by Pavan Balaji 14 Jun '13

14 Jun '13

You can run mpich as root. There's no restriction on that. You still haven't tried out my suggestion of running "id" to check what user ID you are running your processes as. My guess is that you are not setting your user ID correctly. -- Pavan On 06/14/2013 06:27 AM, Lee, Eibhlin wrote: > I found that the reason we want to access /dev/mem is to setup memory regions to access the peripherals. (We are trying to read the output of an ADC). At this point it becomes more a linux/raspberry-pi specific problem than an MPICH problem. Although the fact that you can't run a program that needs access to memory mapping (even as the root user) seems something that MPICH could improve on for future versions. I know I am using smpd instead of hydra so this problem may already be solved. But if someone could confirm that, it would be really helpful. > ________________________________________ > From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of Lee, Eibhlin [eibhlin.lee10(a)imperial.ac.uk] > Sent: 14 June 2013 11:20 > To: discuss(a)mpich.org > Subject: Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem > > Gus, > I tried running cpi, as is included in the installation of MPI, on two machines with two processes. The output message confirmed that it had started only 1 process instead of 2. > Process 0 of 1 is on raspi > pi is approximately... > > Then it just hung. I think this is because the other machine didn't know where to output the data? > > When I tried running two processes on the one machine using the wrapper you suggested the output was the same but doubled. It didn't hang. This confirms that every process was started with rank 0. > > I'm not entirely sure why /dev/mem is needed. I'm working in a group and another member set up io and gpio and it seemed it needed access to /dev/mem I am going to do a strace as suggested by Pavan Balaji to see where it is used and see if I can somehow work around it. > > Thank you for your help. > Eibhlin > ________________________________________ > From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of Gus Correa [gus(a)ldeo.columbia.edu] > Sent: 13 June 2013 21:11 > To: Discuss Mpich > Subject: Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem > > Hi Eibhlin > > On 06/13/2013 12:59 PM, Lee, Eibhlin wrote: >> Gus, >> I believe your first assumption is correct. Unfortunately it just seemed to hang. I think this might be because each one is being made to have the same rank... > > Darn! I was afraid that it might give only rank 0 to all MPI processes. > So, with the script wrapper the process being launched by mpiexec may > indeed be sudo, > not the actual mpi executable (main) :( > Then it may actually launch a bunch of separate rank 0 replicas of your > program, > instead of assigning to them different ranks. > However, without any output or error message, it is hard to tell. > > No output at all? > No error message, just hangs? > Have you tried a verbose flag (-v) to mpiexec? > (Not sure if it exists in MPICH mpiexec, you'd need to check.) > > Would you care to try it with another mpi program, > one that doesn't deal with /dev/mem (a risky business), > say cpi.c (in the examples directory), or an mpi version of Hello, world, > just to see if the mpiexec+sudo_script_wrapper works as expected or > if everybody gets rank 0? > > >> It may already be obvious but this is the first time I am using Linux. I had tried sudo $(which mpiexec ....) and sudo $(which mpiexec) ... both without success. > > "which mpiexec" will return the path to mpiexec, but won't execute it. > > You could try this (with backquotes): > > `which mpiexec` -n 2 ~/main > > On a side note, make sure the mpiexec you're using matches the > mpicc/mpif90/MPI library from the MPICH that > you used to compile the program. > Often times computers have several flavors of MPI installed, and mixing > them just doesn't work. > >> Is putting the full path to it similar to/is a symlink? (This still doesn't make main have super user privileges though.) > > No, nothing to do with sudo privileges. > > This suggestion was just to avoid messing up your /usr/bin, > which is a directory that despite the somewhat misleading name (/usr, > for historical reasons I think), > is supposed to hold system (Linux) programs (that users can use), but > not user-installed programs. > Normally things are that are installed in /usr get there via some Linux > package manager program > (yum, rpm, apt-get, etc), to keep consistency with libraries, etc. > > I belive MPICH would install by default in /usr/local/ (and put mpiexec > in /usr/local/bin), > which is kind of a default location for non-system applications. > > The full path suggestion would be something like: > /path/to/where/you/installed/mpiexec -n 2 ~/main > > However, this won't solve the other problem w.r.t. sudo and /dev/mem. > > You must know what you are doing, but it made me wonder, > even if your program were sequential, why would you want to mess with > /dev/mem directly? > Just curious about it. > > Gus Correa > > > >> Eibhlin >> ________________________________________ >> From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of Gus Correa [gus(a)ldeo.columbia.edu] >> Sent: 13 June 2013 15:37 >> To: Discuss Mpich >> Subject: Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem >> >> Hi Lee >> >> How about replacing "~/main" in the mpiexec command line >> by one-liner script? >> Say, "sudo_main.sh", something like this: >> >> #! /bin/bash >> sudo ~/main >> >> After all, it is "main" that accesses /dev/mem, >> and needs "sudo" permissions, not mpiexec, right? >> [Or do the mpiexec-launched processes inherit >> the "sudo" stuff from mpiexec?] >> >> Not related, but, instead of putting mpiexec in /usr/bin, >> can't you just use the full path to it? >> >> I hope this helps, >> Gus Correa >> >> On 06/13/2013 10:09 AM, Lee, Eibhlin wrote: >>> Pavan, >>> I had a lot of trouble getting hydra to work without having to enter a password/passphrase. I saw the option to pass a phrase in the mpich installers guide. I eventually found that for that command you needed to use the smpd process manager. That's the only reason I chose smpd over hydra. >>> As to your other suggestion. I ran ./main and the same error (Can't open /dev/mem...) appeared. sudo ./main works but of course without multiple processes. >>> Eibhlin >>> ________________________________________ >>> From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of Pavan Balaji [balaji(a)mcs.anl.gov] >>> Sent: 13 June 2013 14:34 >>> To: discuss(a)mpich.org >>> Subject: Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem >>> >>> I just saw your older email. Why are you using smpd instead of the >>> default process manager (hydra)? >>> >>> -- Pavan >>> >>> On 06/13/2013 08:05 AM, Pavan Balaji wrote: >>>> What's "-phrase"? That's not a recognized option. I'm not sure where >>>> the /dev/mem check is coming from. Try running ~/main without mpiexec >>>> first. >>>> >>>> -- Pavan >>>> >>>> On 06/13/2013 06:56 AM, Lee, Eibhlin wrote: >>>>> Hello all, >>>>> >>>>> I am trying to use two raspberry-pi to sample and then process some >>>>> data. The first process samples while the second processes and vice >>>>> versa. To do this I use gpio and also mpich-3.0.4 with the process >>>>> manager smpd. I have successfully run cpi on both machines (from the >>>>> master machine). I have also managed to run a similar program but >>>>> without the MPI, this involved compiling with gcc and when running >>>>> putting sudo in front of the binary file. >>>>> >>>>> When I combine these two processes I get various error messages. >>>>> For input: >>>>> mpiexec -phrase cat -machinefile machinefile -n 2 ~/main >>>>> the error is: >>>>> Can't open /dev/mem >>>>> Did you forget to use 'sudo .. ?' >>>>> >>>>> For input: >>>>> sudo mpiexec -phrase cat -machinefile machinefile -n 2 ~/main >>>>> the error is: >>>>> sudo: mpiexec: Command not found >>>>> >>>>> I therefore put mpiexec into /usr/bin >>>>> >>>>> now for input: >>>>> sudo mpiexec -phrase cat -machinefile machinefile -n 2 ~/main >>>>> the error is: >>>>> Can't open /dev/mem >>>>> Did you forget to use 'sudo .. ?' >>>>> >>>>> Does anyone know how I can work around this? >>>>> Thanks, >>>>> Eibhlin >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> discuss mailing list discuss(a)mpich.org >>>>> To manage subscription options or unsubscribe: >>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>> >>> -- >>> Pavan Balaji >>> http://www.mcs.anl.gov/~balaji >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem
by Antonio J. Peña 14 Jun '13

14 Jun '13

I've created ticket #1885 to check this issue. Here it's the URL: http://trac.mpich.org/projects/mpich/ticket/1885 Antonio On Friday, June 14, 2013 04:18:06 PM Lee, Eibhlin wrote: > Thank you Antonio. > > http://www.skpang.co.uk/blog/archives/615 If you follow the steps to > download adc you will get 4 files that need to be included in compilation: > gb_common.h gb_spi.h gb_common.c and gb_spi.c I have included the files > here as well. > > I compile them using > mpicc example_for_MPICH.c gb_common.c gb_spi.c examp -lm -Lbbcm2835-1.25/src > -lbcm2835 and execute in the standard manner for smpd but this will be > different in hydra. I normally run on 2 machines and have 2 processes > started. > > If you could please reply with the output in full that would help me. (I do > NEED /dev/mem it is directly accessed in gb_common.h) > > Thanks, > Eibhlin > > ________________________________________ > From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of > Antonio J. Peña [apenya(a)mcs.anl.gov] Sent: 14 June 2013 16:10 > To: discuss(a)mpich.org > Subject: Re: [mpich-discuss] Running an mpi program that needs to > access /dev/mem > > Hi Eibhlin, > > If you share a piece of code to test your issue, I can run it for you using > hydra to check if that solves it. > > Antonio > > On Friday, June 14, 2013 11:27:25 AM Lee, Eibhlin wrote: > > I found that the reason we want to access /dev/mem is to setup memory > > regions to access the peripherals. (We are trying to read the output of an > > ADC). At this point it becomes more a linux/raspberry-pi specific problem > > than an MPICH problem. Although the fact that you can't run a program that > > needs access to memory mapping (even as the root user) seems something > > that > > MPICH could improve on for future versions. I know I am using smpd instead > > of hydra so this problem may already be solved. But if someone could > > confirm that, it would be really helpful. > > ________________________________________ > > From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of > > Lee, Eibhlin [eibhlin.lee10(a)imperial.ac.uk] Sent: 14 June 2013 11:20 > > To: discuss(a)mpich.org > > Subject: Re: [mpich-discuss] Running an mpi program that needs to > > > > access /dev/mem > > > > Gus, > > I tried running cpi, as is included in the installation of MPI, on two > > machines with two processes. The output message confirmed that it had > > started only 1 process instead of 2. Process 0 of 1 is on raspi > > pi is approximately... > > > > Then it just hung. I think this is because the other machine didn't know > > where to output the data? > > > > When I tried running two processes on the one machine using the wrapper > > you > > suggested the output was the same but doubled. It didn't hang. This > > confirms that every process was started with rank 0. > > > > I'm not entirely sure why /dev/mem is needed. I'm working in a group and > > another member set up io and gpio and it seemed it needed access to > > /dev/mem I am going to do a strace as suggested by Pavan Balaji to see > > where it is used and see if I can somehow work around it. > > > > Thank you for your help. > > Eibhlin > > ________________________________________ > > From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of > > Gus Correa [gus(a)ldeo.columbia.edu] Sent: 13 June 2013 21:11 > > To: Discuss Mpich > > Subject: Re: [mpich-discuss] Running an mpi program that needs to > > access /dev/mem > > > > Hi Eibhlin > > > > On 06/13/2013 12:59 PM, Lee, Eibhlin wrote: > > > Gus, > > > I believe your first assumption is correct. Unfortunately it just seemed > > > to hang. I think this might be because each one is being made to have > > > the > > > same rank... > > > > Darn! I was afraid that it might give only rank 0 to all MPI processes. > > So, with the script wrapper the process being launched by mpiexec may > > indeed be sudo, > > not the actual mpi executable (main) :( > > Then it may actually launch a bunch of separate rank 0 replicas of your > > program, > > instead of assigning to them different ranks. > > However, without any output or error message, it is hard to tell. > > > > No output at all? > > No error message, just hangs? > > Have you tried a verbose flag (-v) to mpiexec? > > (Not sure if it exists in MPICH mpiexec, you'd need to check.) > > > > Would you care to try it with another mpi program, > > one that doesn't deal with /dev/mem (a risky business), > > say cpi.c (in the examples directory), or an mpi version of Hello, world, > > just to see if the mpiexec+sudo_script_wrapper works as expected or > > if everybody gets rank 0? > > > > > It may already be obvious but this is the first time I am using Linux. I > > > had tried sudo $(which mpiexec ....) and sudo $(which mpiexec) ... both > > > without success. > > > > "which mpiexec" will return the path to mpiexec, but won't execute it. > > > > You could try this (with backquotes): > > > > `which mpiexec` -n 2 ~/main > > > > On a side note, make sure the mpiexec you're using matches the > > mpicc/mpif90/MPI library from the MPICH that > > you used to compile the program. > > Often times computers have several flavors of MPI installed, and mixing > > them just doesn't work. > > > > > Is putting the full path to it similar to/is a symlink? (This still > > > doesn't make main have super user privileges though.) > > > > No, nothing to do with sudo privileges. > > > > This suggestion was just to avoid messing up your /usr/bin, > > which is a directory that despite the somewhat misleading name (/usr, > > for historical reasons I think), > > is supposed to hold system (Linux) programs (that users can use), but > > not user-installed programs. > > Normally things are that are installed in /usr get there via some Linux > > package manager program > > (yum, rpm, apt-get, etc), to keep consistency with libraries, etc. > > > > I belive MPICH would install by default in /usr/local/ (and put mpiexec > > in /usr/local/bin), > > which is kind of a default location for non-system applications. > > > > The full path suggestion would be something like: > > /path/to/where/you/installed/mpiexec -n 2 ~/main > > > > However, this won't solve the other problem w.r.t. sudo and /dev/mem. > > > > You must know what you are doing, but it made me wonder, > > even if your program were sequential, why would you want to mess with > > /dev/mem directly? > > Just curious about it. > > > > Gus Correa > > > > > Eibhlin > > > ________________________________________ > > > From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of > > > Gus Correa [gus(a)ldeo.columbia.edu] Sent: 13 June 2013 15:37 > > > To: Discuss Mpich > > > Subject: Re: [mpich-discuss] Running an mpi program that needs to > > > access /dev/mem > > > > > > Hi Lee > > > > > > How about replacing "~/main" in the mpiexec command line > > > by one-liner script? > > > Say, "sudo_main.sh", something like this: > > > > > > #! /bin/bash > > > sudo ~/main > > > > > > After all, it is "main" that accesses /dev/mem, > > > and needs "sudo" permissions, not mpiexec, right? > > > [Or do the mpiexec-launched processes inherit > > > the "sudo" stuff from mpiexec?] > > > > > > Not related, but, instead of putting mpiexec in /usr/bin, > > > can't you just use the full path to it? > > > > > > I hope this helps, > > > Gus Correa > > > > > > On 06/13/2013 10:09 AM, Lee, Eibhlin wrote: > > >> Pavan, > > >> I had a lot of trouble getting hydra to work without having to enter a > > >> password/passphrase. I saw the option to pass a phrase in the mpich > > >> installers guide. I eventually found that for that command you needed > > >> to > > >> use the smpd process manager. That's the only reason I chose smpd over > > >> hydra. As to your other suggestion. I ran ./main and the same error > > >> (Can't open /dev/mem...) appeared. sudo ./main works but of course > > >> without multiple processes. Eibhlin > > >> ________________________________________ > > >> From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf > > >> of > > >> Pavan Balaji [balaji(a)mcs.anl.gov] Sent: 13 June 2013 14:34 > > >> To: discuss(a)mpich.org > > >> Subject: Re: [mpich-discuss] Running an mpi program that needs to > > >> access > > >> > > >> /dev/mem > > >> > > >> I just saw your older email. Why are you using smpd instead of the > > >> default process manager (hydra)? > > >> > > >> -- Pavan > > >> > > >> On 06/13/2013 08:05 AM, Pavan Balaji wrote: > > >>> What's "-phrase"? That's not a recognized option. I'm not sure where > > >>> the /dev/mem check is coming from. Try running ~/main without mpiexec > > >>> first. > > >>> > > >>> -- Pavan > > >>> > > >>> On 06/13/2013 06:56 AM, Lee, Eibhlin wrote: > > >>>> Hello all, > > >>>> > > >>>> I am trying to use two raspberry-pi to sample and then process some > > >>>> data. The first process samples while the second processes and vice > > >>>> versa. To do this I use gpio and also mpich-3.0.4 with the process > > >>>> manager smpd. I have successfully run cpi on both machines (from the > > >>>> master machine). I have also managed to run a similar program but > > >>>> without the MPI, this involved compiling with gcc and when running > > >>>> putting sudo in front of the binary file. > > >>>> > > >>>> When I combine these two processes I get various error messages. > > >>>> For input: > > >>>> mpiexec -phrase cat -machinefile machinefile -n 2 ~/main > > >>>> the error is: > > >>>> Can't open /dev/mem > > >>>> Did you forget to use 'sudo .. ?' > > >>>> > > >>>> For input: > > >>>> sudo mpiexec -phrase cat -machinefile machinefile -n 2 ~/main > > >>>> the error is: > > >>>> sudo: mpiexec: Command not found > > >>>> > > >>>> I therefore put mpiexec into /usr/bin > > >>>> > > >>>> now for input: > > >>>> sudo mpiexec -phrase cat -machinefile machinefile -n 2 ~/main > > >>>> the error is: > > >>>> Can't open /dev/mem > > >>>> Did you forget to use 'sudo .. ?' > > >>>> > > >>>> Does anyone know how I can work around this? > > >>>> Thanks, > > >>>> Eibhlin > > >>>> > > >>>> > > >>>> > > >>>> _______________________________________________ > > >>>> discuss mailing list discuss(a)mpich.org > > >>>> To manage subscription options or unsubscribe: > > >>>> https://lists.mpich.org/mailman/listinfo/discuss > > >> > > >> -- > > >> Pavan Balaji > > >> http://www.mcs.anl.gov/~balaji > > >> _______________________________________________ > > >> discuss mailing list discuss(a)mpich.org > > >> To manage subscription options or unsubscribe: > > >> https://lists.mpich.org/mailman/listinfo/discuss > > >> _______________________________________________ > > >> discuss mailing list discuss(a)mpich.org > > >> To manage subscription options or unsubscribe: > > >> https://lists.mpich.org/mailman/listinfo/discuss > > > > > > _______________________________________________ > > > discuss mailing list discuss(a)mpich.org > > > To manage subscription options or unsubscribe: > > > https://lists.mpich.org/mailman/listinfo/discuss > > > _______________________________________________ > > > discuss mailing list discuss(a)mpich.org > > > To manage subscription options or unsubscribe: > > > https://lists.mpich.org/mailman/listinfo/discuss > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem
by Lee, Eibhlin 14 Jun '13

14 Jun '13

Thank you Antonio. http://www.skpang.co.uk/blog/archives/615 If you follow the steps to download adc you will get 4 files that need to be included in compilation: gb_common.h gb_spi.h gb_common.c and gb_spi.c I have included the files here as well. I compile them using mpicc example_for_MPICH.c gb_common.c gb_spi.c examp -lm -Lbbcm2835-1.25/src -lbcm2835 and execute in the standard manner for smpd but this will be different in hydra. I normally run on 2 machines and have 2 processes started. If you could please reply with the output in full that would help me. (I do NEED /dev/mem it is directly accessed in gb_common.h) Thanks, Eibhlin ________________________________________ From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of Antonio J. Peña [apenya(a)mcs.anl.gov] Sent: 14 June 2013 16:10 To: discuss(a)mpich.org Subject: Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem Hi Eibhlin, If you share a piece of code to test your issue, I can run it for you using hydra to check if that solves it. Antonio On Friday, June 14, 2013 11:27:25 AM Lee, Eibhlin wrote: > I found that the reason we want to access /dev/mem is to setup memory > regions to access the peripherals. (We are trying to read the output of an > ADC). At this point it becomes more a linux/raspberry-pi specific problem > than an MPICH problem. Although the fact that you can't run a program that > needs access to memory mapping (even as the root user) seems something that > MPICH could improve on for future versions. I know I am using smpd instead > of hydra so this problem may already be solved. But if someone could > confirm that, it would be really helpful. > ________________________________________ > From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of > Lee, Eibhlin [eibhlin.lee10(a)imperial.ac.uk] Sent: 14 June 2013 11:20 > To: discuss(a)mpich.org > Subject: Re: [mpich-discuss] Running an mpi program that needs to > access /dev/mem > > Gus, > I tried running cpi, as is included in the installation of MPI, on two > machines with two processes. The output message confirmed that it had > started only 1 process instead of 2. Process 0 of 1 is on raspi > pi is approximately... > > Then it just hung. I think this is because the other machine didn't know > where to output the data? > > When I tried running two processes on the one machine using the wrapper you > suggested the output was the same but doubled. It didn't hang. This > confirms that every process was started with rank 0. > > I'm not entirely sure why /dev/mem is needed. I'm working in a group and > another member set up io and gpio and it seemed it needed access to > /dev/mem I am going to do a strace as suggested by Pavan Balaji to see > where it is used and see if I can somehow work around it. > > Thank you for your help. > Eibhlin > ________________________________________ > From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of Gus > Correa [gus(a)ldeo.columbia.edu] Sent: 13 June 2013 21:11 > To: Discuss Mpich > Subject: Re: [mpich-discuss] Running an mpi program that needs to > access /dev/mem > > Hi Eibhlin > > On 06/13/2013 12:59 PM, Lee, Eibhlin wrote: > > Gus, > > I believe your first assumption is correct. Unfortunately it just seemed > > to hang. I think this might be because each one is being made to have the > > same rank... > Darn! I was afraid that it might give only rank 0 to all MPI processes. > So, with the script wrapper the process being launched by mpiexec may > indeed be sudo, > not the actual mpi executable (main) :( > Then it may actually launch a bunch of separate rank 0 replicas of your > program, > instead of assigning to them different ranks. > However, without any output or error message, it is hard to tell. > > No output at all? > No error message, just hangs? > Have you tried a verbose flag (-v) to mpiexec? > (Not sure if it exists in MPICH mpiexec, you'd need to check.) > > Would you care to try it with another mpi program, > one that doesn't deal with /dev/mem (a risky business), > say cpi.c (in the examples directory), or an mpi version of Hello, world, > just to see if the mpiexec+sudo_script_wrapper works as expected or > if everybody gets rank 0? > > > It may already be obvious but this is the first time I am using Linux. I > > had tried sudo $(which mpiexec ....) and sudo $(which mpiexec) ... both > > without success. > "which mpiexec" will return the path to mpiexec, but won't execute it. > > You could try this (with backquotes): > > `which mpiexec` -n 2 ~/main > > On a side note, make sure the mpiexec you're using matches the > mpicc/mpif90/MPI library from the MPICH that > you used to compile the program. > Often times computers have several flavors of MPI installed, and mixing > them just doesn't work. > > > Is putting the full path to it similar to/is a symlink? (This still > > doesn't make main have super user privileges though.) > No, nothing to do with sudo privileges. > > This suggestion was just to avoid messing up your /usr/bin, > which is a directory that despite the somewhat misleading name (/usr, > for historical reasons I think), > is supposed to hold system (Linux) programs (that users can use), but > not user-installed programs. > Normally things are that are installed in /usr get there via some Linux > package manager program > (yum, rpm, apt-get, etc), to keep consistency with libraries, etc. > > I belive MPICH would install by default in /usr/local/ (and put mpiexec > in /usr/local/bin), > which is kind of a default location for non-system applications. > > The full path suggestion would be something like: > /path/to/where/you/installed/mpiexec -n 2 ~/main > > However, this won't solve the other problem w.r.t. sudo and /dev/mem. > > You must know what you are doing, but it made me wonder, > even if your program were sequential, why would you want to mess with > /dev/mem directly? > Just curious about it. > > Gus Correa > > > Eibhlin > > ________________________________________ > > From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of > > Gus Correa [gus(a)ldeo.columbia.edu] Sent: 13 June 2013 15:37 > > To: Discuss Mpich > > Subject: Re: [mpich-discuss] Running an mpi program that needs to > > access /dev/mem > > > > Hi Lee > > > > How about replacing "~/main" in the mpiexec command line > > by one-liner script? > > Say, "sudo_main.sh", something like this: > > > > #! /bin/bash > > sudo ~/main > > > > After all, it is "main" that accesses /dev/mem, > > and needs "sudo" permissions, not mpiexec, right? > > [Or do the mpiexec-launched processes inherit > > the "sudo" stuff from mpiexec?] > > > > Not related, but, instead of putting mpiexec in /usr/bin, > > can't you just use the full path to it? > > > > I hope this helps, > > Gus Correa > > > > On 06/13/2013 10:09 AM, Lee, Eibhlin wrote: > >> Pavan, > >> I had a lot of trouble getting hydra to work without having to enter a > >> password/passphrase. I saw the option to pass a phrase in the mpich > >> installers guide. I eventually found that for that command you needed to > >> use the smpd process manager. That's the only reason I chose smpd over > >> hydra. As to your other suggestion. I ran ./main and the same error > >> (Can't open /dev/mem...) appeared. sudo ./main works but of course > >> without multiple processes. Eibhlin > >> ________________________________________ > >> From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of > >> Pavan Balaji [balaji(a)mcs.anl.gov] Sent: 13 June 2013 14:34 > >> To: discuss(a)mpich.org > >> Subject: Re: [mpich-discuss] Running an mpi program that needs to access > >> /dev/mem > >> > >> I just saw your older email. Why are you using smpd instead of the > >> default process manager (hydra)? > >> > >> -- Pavan > >> > >> On 06/13/2013 08:05 AM, Pavan Balaji wrote: > >>> What's "-phrase"? That's not a recognized option. I'm not sure where > >>> the /dev/mem check is coming from. Try running ~/main without mpiexec > >>> first. > >>> > >>> -- Pavan > >>> > >>> On 06/13/2013 06:56 AM, Lee, Eibhlin wrote: > >>>> Hello all, > >>>> > >>>> I am trying to use two raspberry-pi to sample and then process some > >>>> data. The first process samples while the second processes and vice > >>>> versa. To do this I use gpio and also mpich-3.0.4 with the process > >>>> manager smpd. I have successfully run cpi on both machines (from the > >>>> master machine). I have also managed to run a similar program but > >>>> without the MPI, this involved compiling with gcc and when running > >>>> putting sudo in front of the binary file. > >>>> > >>>> When I combine these two processes I get various error messages. > >>>> For input: > >>>> mpiexec -phrase cat -machinefile machinefile -n 2 ~/main > >>>> the error is: > >>>> Can't open /dev/mem > >>>> Did you forget to use 'sudo .. ?' > >>>> > >>>> For input: > >>>> sudo mpiexec -phrase cat -machinefile machinefile -n 2 ~/main > >>>> the error is: > >>>> sudo: mpiexec: Command not found > >>>> > >>>> I therefore put mpiexec into /usr/bin > >>>> > >>>> now for input: > >>>> sudo mpiexec -phrase cat -machinefile machinefile -n 2 ~/main > >>>> the error is: > >>>> Can't open /dev/mem > >>>> Did you forget to use 'sudo .. ?' > >>>> > >>>> Does anyone know how I can work around this? > >>>> Thanks, > >>>> Eibhlin > >>>> > >>>> > >>>> > >>>> _______________________________________________ > >>>> discuss mailing list discuss(a)mpich.org > >>>> To manage subscription options or unsubscribe: > >>>> https://lists.mpich.org/mailman/listinfo/discuss > >> > >> -- > >> Pavan Balaji > >> http://www.mcs.anl.gov/~balaji > >> _______________________________________________ > >> discuss mailing list discuss(a)mpich.org > >> To manage subscription options or unsubscribe: > >> https://lists.mpich.org/mailman/listinfo/discuss > >> _______________________________________________ > >> discuss mailing list discuss(a)mpich.org > >> To manage subscription options or unsubscribe: > >> https://lists.mpich.org/mailman/listinfo/discuss > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem
by Antonio J. Peña 14 Jun '13

14 Jun '13

Hi Eibhlin, If you share a piece of code to test your issue, I can run it for you using hydra to check if that solves it. Antonio On Friday, June 14, 2013 11:27:25 AM Lee, Eibhlin wrote: > I found that the reason we want to access /dev/mem is to setup memory > regions to access the peripherals. (We are trying to read the output of an > ADC). At this point it becomes more a linux/raspberry-pi specific problem > than an MPICH problem. Although the fact that you can't run a program that > needs access to memory mapping (even as the root user) seems something that > MPICH could improve on for future versions. I know I am using smpd instead > of hydra so this problem may already be solved. But if someone could > confirm that, it would be really helpful. > ________________________________________ > From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of > Lee, Eibhlin [eibhlin.lee10(a)imperial.ac.uk] Sent: 14 June 2013 11:20 > To: discuss(a)mpich.org > Subject: Re: [mpich-discuss] Running an mpi program that needs to > access /dev/mem > > Gus, > I tried running cpi, as is included in the installation of MPI, on two > machines with two processes. The output message confirmed that it had > started only 1 process instead of 2. Process 0 of 1 is on raspi > pi is approximately... > > Then it just hung. I think this is because the other machine didn't know > where to output the data? > > When I tried running two processes on the one machine using the wrapper you > suggested the output was the same but doubled. It didn't hang. This > confirms that every process was started with rank 0. > > I'm not entirely sure why /dev/mem is needed. I'm working in a group and > another member set up io and gpio and it seemed it needed access to > /dev/mem I am going to do a strace as suggested by Pavan Balaji to see > where it is used and see if I can somehow work around it. > > Thank you for your help. > Eibhlin > ________________________________________ > From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of Gus > Correa [gus(a)ldeo.columbia.edu] Sent: 13 June 2013 21:11 > To: Discuss Mpich > Subject: Re: [mpich-discuss] Running an mpi program that needs to > access /dev/mem > > Hi Eibhlin > > On 06/13/2013 12:59 PM, Lee, Eibhlin wrote: > > Gus, > > I believe your first assumption is correct. Unfortunately it just seemed > > to hang. I think this might be because each one is being made to have the > > same rank... > Darn! I was afraid that it might give only rank 0 to all MPI processes. > So, with the script wrapper the process being launched by mpiexec may > indeed be sudo, > not the actual mpi executable (main) :( > Then it may actually launch a bunch of separate rank 0 replicas of your > program, > instead of assigning to them different ranks. > However, without any output or error message, it is hard to tell. > > No output at all? > No error message, just hangs? > Have you tried a verbose flag (-v) to mpiexec? > (Not sure if it exists in MPICH mpiexec, you'd need to check.) > > Would you care to try it with another mpi program, > one that doesn't deal with /dev/mem (a risky business), > say cpi.c (in the examples directory), or an mpi version of Hello, world, > just to see if the mpiexec+sudo_script_wrapper works as expected or > if everybody gets rank 0? > > > It may already be obvious but this is the first time I am using Linux. I > > had tried sudo $(which mpiexec ....) and sudo $(which mpiexec) ... both > > without success. > "which mpiexec" will return the path to mpiexec, but won't execute it. > > You could try this (with backquotes): > > `which mpiexec` -n 2 ~/main > > On a side note, make sure the mpiexec you're using matches the > mpicc/mpif90/MPI library from the MPICH that > you used to compile the program. > Often times computers have several flavors of MPI installed, and mixing > them just doesn't work. > > > Is putting the full path to it similar to/is a symlink? (This still > > doesn't make main have super user privileges though.) > No, nothing to do with sudo privileges. > > This suggestion was just to avoid messing up your /usr/bin, > which is a directory that despite the somewhat misleading name (/usr, > for historical reasons I think), > is supposed to hold system (Linux) programs (that users can use), but > not user-installed programs. > Normally things are that are installed in /usr get there via some Linux > package manager program > (yum, rpm, apt-get, etc), to keep consistency with libraries, etc. > > I belive MPICH would install by default in /usr/local/ (and put mpiexec > in /usr/local/bin), > which is kind of a default location for non-system applications. > > The full path suggestion would be something like: > /path/to/where/you/installed/mpiexec -n 2 ~/main > > However, this won't solve the other problem w.r.t. sudo and /dev/mem. > > You must know what you are doing, but it made me wonder, > even if your program were sequential, why would you want to mess with > /dev/mem directly? > Just curious about it. > > Gus Correa > > > Eibhlin > > ________________________________________ > > From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of > > Gus Correa [gus(a)ldeo.columbia.edu] Sent: 13 June 2013 15:37 > > To: Discuss Mpich > > Subject: Re: [mpich-discuss] Running an mpi program that needs to > > access /dev/mem > > > > Hi Lee > > > > How about replacing "~/main" in the mpiexec command line > > by one-liner script? > > Say, "sudo_main.sh", something like this: > > > > #! /bin/bash > > sudo ~/main > > > > After all, it is "main" that accesses /dev/mem, > > and needs "sudo" permissions, not mpiexec, right? > > [Or do the mpiexec-launched processes inherit > > the "sudo" stuff from mpiexec?] > > > > Not related, but, instead of putting mpiexec in /usr/bin, > > can't you just use the full path to it? > > > > I hope this helps, > > Gus Correa > > > > On 06/13/2013 10:09 AM, Lee, Eibhlin wrote: > >> Pavan, > >> I had a lot of trouble getting hydra to work without having to enter a > >> password/passphrase. I saw the option to pass a phrase in the mpich > >> installers guide. I eventually found that for that command you needed to > >> use the smpd process manager. That's the only reason I chose smpd over > >> hydra. As to your other suggestion. I ran ./main and the same error > >> (Can't open /dev/mem...) appeared. sudo ./main works but of course > >> without multiple processes. Eibhlin > >> ________________________________________ > >> From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of > >> Pavan Balaji [balaji(a)mcs.anl.gov] Sent: 13 June 2013 14:34 > >> To: discuss(a)mpich.org > >> Subject: Re: [mpich-discuss] Running an mpi program that needs to access > >> /dev/mem > >> > >> I just saw your older email. Why are you using smpd instead of the > >> default process manager (hydra)? > >> > >> -- Pavan > >> > >> On 06/13/2013 08:05 AM, Pavan Balaji wrote: > >>> What's "-phrase"? That's not a recognized option. I'm not sure where > >>> the /dev/mem check is coming from. Try running ~/main without mpiexec > >>> first. > >>> > >>> -- Pavan > >>> > >>> On 06/13/2013 06:56 AM, Lee, Eibhlin wrote: > >>>> Hello all, > >>>> > >>>> I am trying to use two raspberry-pi to sample and then process some > >>>> data. The first process samples while the second processes and vice > >>>> versa. To do this I use gpio and also mpich-3.0.4 with the process > >>>> manager smpd. I have successfully run cpi on both machines (from the > >>>> master machine). I have also managed to run a similar program but > >>>> without the MPI, this involved compiling with gcc and when running > >>>> putting sudo in front of the binary file. > >>>> > >>>> When I combine these two processes I get various error messages. > >>>> For input: > >>>> mpiexec -phrase cat -machinefile machinefile -n 2 ~/main > >>>> the error is: > >>>> Can't open /dev/mem > >>>> Did you forget to use 'sudo .. ?' > >>>> > >>>> For input: > >>>> sudo mpiexec -phrase cat -machinefile machinefile -n 2 ~/main > >>>> the error is: > >>>> sudo: mpiexec: Command not found > >>>> > >>>> I therefore put mpiexec into /usr/bin > >>>> > >>>> now for input: > >>>> sudo mpiexec -phrase cat -machinefile machinefile -n 2 ~/main > >>>> the error is: > >>>> Can't open /dev/mem > >>>> Did you forget to use 'sudo .. ?' > >>>> > >>>> Does anyone know how I can work around this? > >>>> Thanks, > >>>> Eibhlin > >>>> > >>>> > >>>> > >>>> _______________________________________________ > >>>> discuss mailing list discuss(a)mpich.org > >>>> To manage subscription options or unsubscribe: > >>>> https://lists.mpich.org/mailman/listinfo/discuss > >> > >> -- > >> Pavan Balaji > >> http://www.mcs.anl.gov/~balaji > >> _______________________________________________ > >> discuss mailing list discuss(a)mpich.org > >> To manage subscription options or unsubscribe: > >> https://lists.mpich.org/mailman/listinfo/discuss > >> _______________________________________________ > >> discuss mailing list discuss(a)mpich.org > >> To manage subscription options or unsubscribe: > >> https://lists.mpich.org/mailman/listinfo/discuss > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem
by Lee, Eibhlin 14 Jun '13

14 Jun '13

I found that the reason we want to access /dev/mem is to setup memory regions to access the peripherals. (We are trying to read the output of an ADC). At this point it becomes more a linux/raspberry-pi specific problem than an MPICH problem. Although the fact that you can't run a program that needs access to memory mapping (even as the root user) seems something that MPICH could improve on for future versions. I know I am using smpd instead of hydra so this problem may already be solved. But if someone could confirm that, it would be really helpful. ________________________________________ From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of Lee, Eibhlin [eibhlin.lee10(a)imperial.ac.uk] Sent: 14 June 2013 11:20 To: discuss(a)mpich.org Subject: Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem Gus, I tried running cpi, as is included in the installation of MPI, on two machines with two processes. The output message confirmed that it had started only 1 process instead of 2. Process 0 of 1 is on raspi pi is approximately... Then it just hung. I think this is because the other machine didn't know where to output the data? When I tried running two processes on the one machine using the wrapper you suggested the output was the same but doubled. It didn't hang. This confirms that every process was started with rank 0. I'm not entirely sure why /dev/mem is needed. I'm working in a group and another member set up io and gpio and it seemed it needed access to /dev/mem I am going to do a strace as suggested by Pavan Balaji to see where it is used and see if I can somehow work around it. Thank you for your help. Eibhlin ________________________________________ From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of Gus Correa [gus(a)ldeo.columbia.edu] Sent: 13 June 2013 21:11 To: Discuss Mpich Subject: Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem Hi Eibhlin On 06/13/2013 12:59 PM, Lee, Eibhlin wrote: > Gus, > I believe your first assumption is correct. Unfortunately it just seemed to hang. I think this might be because each one is being made to have the same rank... Darn! I was afraid that it might give only rank 0 to all MPI processes. So, with the script wrapper the process being launched by mpiexec may indeed be sudo, not the actual mpi executable (main) :( Then it may actually launch a bunch of separate rank 0 replicas of your program, instead of assigning to them different ranks. However, without any output or error message, it is hard to tell. No output at all? No error message, just hangs? Have you tried a verbose flag (-v) to mpiexec? (Not sure if it exists in MPICH mpiexec, you'd need to check.) Would you care to try it with another mpi program, one that doesn't deal with /dev/mem (a risky business), say cpi.c (in the examples directory), or an mpi version of Hello, world, just to see if the mpiexec+sudo_script_wrapper works as expected or if everybody gets rank 0? > It may already be obvious but this is the first time I am using Linux. I had tried sudo $(which mpiexec ....) and sudo $(which mpiexec) ... both without success. "which mpiexec" will return the path to mpiexec, but won't execute it. You could try this (with backquotes): `which mpiexec` -n 2 ~/main On a side note, make sure the mpiexec you're using matches the mpicc/mpif90/MPI library from the MPICH that you used to compile the program. Often times computers have several flavors of MPI installed, and mixing them just doesn't work. > Is putting the full path to it similar to/is a symlink? (This still doesn't make main have super user privileges though.) No, nothing to do with sudo privileges. This suggestion was just to avoid messing up your /usr/bin, which is a directory that despite the somewhat misleading name (/usr, for historical reasons I think), is supposed to hold system (Linux) programs (that users can use), but not user-installed programs. Normally things are that are installed in /usr get there via some Linux package manager program (yum, rpm, apt-get, etc), to keep consistency with libraries, etc. I belive MPICH would install by default in /usr/local/ (and put mpiexec in /usr/local/bin), which is kind of a default location for non-system applications. The full path suggestion would be something like: /path/to/where/you/installed/mpiexec -n 2 ~/main However, this won't solve the other problem w.r.t. sudo and /dev/mem. You must know what you are doing, but it made me wonder, even if your program were sequential, why would you want to mess with /dev/mem directly? Just curious about it. Gus Correa > Eibhlin > ________________________________________ > From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of Gus Correa [gus(a)ldeo.columbia.edu] > Sent: 13 June 2013 15:37 > To: Discuss Mpich > Subject: Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem > > Hi Lee > > How about replacing "~/main" in the mpiexec command line > by one-liner script? > Say, "sudo_main.sh", something like this: > > #! /bin/bash > sudo ~/main > > After all, it is "main" that accesses /dev/mem, > and needs "sudo" permissions, not mpiexec, right? > [Or do the mpiexec-launched processes inherit > the "sudo" stuff from mpiexec?] > > Not related, but, instead of putting mpiexec in /usr/bin, > can't you just use the full path to it? > > I hope this helps, > Gus Correa > > On 06/13/2013 10:09 AM, Lee, Eibhlin wrote: >> Pavan, >> I had a lot of trouble getting hydra to work without having to enter a password/passphrase. I saw the option to pass a phrase in the mpich installers guide. I eventually found that for that command you needed to use the smpd process manager. That's the only reason I chose smpd over hydra. >> As to your other suggestion. I ran ./main and the same error (Can't open /dev/mem...) appeared. sudo ./main works but of course without multiple processes. >> Eibhlin >> ________________________________________ >> From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of Pavan Balaji [balaji(a)mcs.anl.gov] >> Sent: 13 June 2013 14:34 >> To: discuss(a)mpich.org >> Subject: Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem >> >> I just saw your older email. Why are you using smpd instead of the >> default process manager (hydra)? >> >> -- Pavan >> >> On 06/13/2013 08:05 AM, Pavan Balaji wrote: >>> What's "-phrase"? That's not a recognized option. I'm not sure where >>> the /dev/mem check is coming from. Try running ~/main without mpiexec >>> first. >>> >>> -- Pavan >>> >>> On 06/13/2013 06:56 AM, Lee, Eibhlin wrote: >>>> Hello all, >>>> >>>> I am trying to use two raspberry-pi to sample and then process some >>>> data. The first process samples while the second processes and vice >>>> versa. To do this I use gpio and also mpich-3.0.4 with the process >>>> manager smpd. I have successfully run cpi on both machines (from the >>>> master machine). I have also managed to run a similar program but >>>> without the MPI, this involved compiling with gcc and when running >>>> putting sudo in front of the binary file. >>>> >>>> When I combine these two processes I get various error messages. >>>> For input: >>>> mpiexec -phrase cat -machinefile machinefile -n 2 ~/main >>>> the error is: >>>> Can't open /dev/mem >>>> Did you forget to use 'sudo .. ?' >>>> >>>> For input: >>>> sudo mpiexec -phrase cat -machinefile machinefile -n 2 ~/main >>>> the error is: >>>> sudo: mpiexec: Command not found >>>> >>>> I therefore put mpiexec into /usr/bin >>>> >>>> now for input: >>>> sudo mpiexec -phrase cat -machinefile machinefile -n 2 ~/main >>>> the error is: >>>> Can't open /dev/mem >>>> Did you forget to use 'sudo .. ?' >>>> >>>> Does anyone know how I can work around this? >>>> Thanks, >>>> Eibhlin >>>> >>>> >>>> >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>>> >> -- >> Pavan Balaji >> http://www.mcs.anl.gov/~balaji >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem
by Lee, Eibhlin 14 Jun '13

14 Jun '13

Gus, I tried running cpi, as is included in the installation of MPI, on two machines with two processes. The output message confirmed that it had started only 1 process instead of 2. Process 0 of 1 is on raspi pi is approximately... Then it just hung. I think this is because the other machine didn't know where to output the data? When I tried running two processes on the one machine using the wrapper you suggested the output was the same but doubled. It didn't hang. This confirms that every process was started with rank 0. I'm not entirely sure why /dev/mem is needed. I'm working in a group and another member set up io and gpio and it seemed it needed access to /dev/mem I am going to do a strace as suggested by Pavan Balaji to see where it is used and see if I can somehow work around it. Thank you for your help. Eibhlin ________________________________________ From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of Gus Correa [gus(a)ldeo.columbia.edu] Sent: 13 June 2013 21:11 To: Discuss Mpich Subject: Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem Hi Eibhlin On 06/13/2013 12:59 PM, Lee, Eibhlin wrote: > Gus, > I believe your first assumption is correct. Unfortunately it just seemed to hang. I think this might be because each one is being made to have the same rank... Darn! I was afraid that it might give only rank 0 to all MPI processes. So, with the script wrapper the process being launched by mpiexec may indeed be sudo, not the actual mpi executable (main) :( Then it may actually launch a bunch of separate rank 0 replicas of your program, instead of assigning to them different ranks. However, without any output or error message, it is hard to tell. No output at all? No error message, just hangs? Have you tried a verbose flag (-v) to mpiexec? (Not sure if it exists in MPICH mpiexec, you'd need to check.) Would you care to try it with another mpi program, one that doesn't deal with /dev/mem (a risky business), say cpi.c (in the examples directory), or an mpi version of Hello, world, just to see if the mpiexec+sudo_script_wrapper works as expected or if everybody gets rank 0? > It may already be obvious but this is the first time I am using Linux. I had tried sudo $(which mpiexec ....) and sudo $(which mpiexec) ... both without success. "which mpiexec" will return the path to mpiexec, but won't execute it. You could try this (with backquotes): `which mpiexec` -n 2 ~/main On a side note, make sure the mpiexec you're using matches the mpicc/mpif90/MPI library from the MPICH that you used to compile the program. Often times computers have several flavors of MPI installed, and mixing them just doesn't work. > Is putting the full path to it similar to/is a symlink? (This still doesn't make main have super user privileges though.) No, nothing to do with sudo privileges. This suggestion was just to avoid messing up your /usr/bin, which is a directory that despite the somewhat misleading name (/usr, for historical reasons I think), is supposed to hold system (Linux) programs (that users can use), but not user-installed programs. Normally things are that are installed in /usr get there via some Linux package manager program (yum, rpm, apt-get, etc), to keep consistency with libraries, etc. I belive MPICH would install by default in /usr/local/ (and put mpiexec in /usr/local/bin), which is kind of a default location for non-system applications. The full path suggestion would be something like: /path/to/where/you/installed/mpiexec -n 2 ~/main However, this won't solve the other problem w.r.t. sudo and /dev/mem. You must know what you are doing, but it made me wonder, even if your program were sequential, why would you want to mess with /dev/mem directly? Just curious about it. Gus Correa > Eibhlin > ________________________________________ > From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of Gus Correa [gus(a)ldeo.columbia.edu] > Sent: 13 June 2013 15:37 > To: Discuss Mpich > Subject: Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem > > Hi Lee > > How about replacing "~/main" in the mpiexec command line > by one-liner script? > Say, "sudo_main.sh", something like this: > > #! /bin/bash > sudo ~/main > > After all, it is "main" that accesses /dev/mem, > and needs "sudo" permissions, not mpiexec, right? > [Or do the mpiexec-launched processes inherit > the "sudo" stuff from mpiexec?] > > Not related, but, instead of putting mpiexec in /usr/bin, > can't you just use the full path to it? > > I hope this helps, > Gus Correa > > On 06/13/2013 10:09 AM, Lee, Eibhlin wrote: >> Pavan, >> I had a lot of trouble getting hydra to work without having to enter a password/passphrase. I saw the option to pass a phrase in the mpich installers guide. I eventually found that for that command you needed to use the smpd process manager. That's the only reason I chose smpd over hydra. >> As to your other suggestion. I ran ./main and the same error (Can't open /dev/mem...) appeared. sudo ./main works but of course without multiple processes. >> Eibhlin >> ________________________________________ >> From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of Pavan Balaji [balaji(a)mcs.anl.gov] >> Sent: 13 June 2013 14:34 >> To: discuss(a)mpich.org >> Subject: Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem >> >> I just saw your older email. Why are you using smpd instead of the >> default process manager (hydra)? >> >> -- Pavan >> >> On 06/13/2013 08:05 AM, Pavan Balaji wrote: >>> What's "-phrase"? That's not a recognized option. I'm not sure where >>> the /dev/mem check is coming from. Try running ~/main without mpiexec >>> first. >>> >>> -- Pavan >>> >>> On 06/13/2013 06:56 AM, Lee, Eibhlin wrote: >>>> Hello all, >>>> >>>> I am trying to use two raspberry-pi to sample and then process some >>>> data. The first process samples while the second processes and vice >>>> versa. To do this I use gpio and also mpich-3.0.4 with the process >>>> manager smpd. I have successfully run cpi on both machines (from the >>>> master machine). I have also managed to run a similar program but >>>> without the MPI, this involved compiling with gcc and when running >>>> putting sudo in front of the binary file. >>>> >>>> When I combine these two processes I get various error messages. >>>> For input: >>>> mpiexec -phrase cat -machinefile machinefile -n 2 ~/main >>>> the error is: >>>> Can't open /dev/mem >>>> Did you forget to use 'sudo .. ?' >>>> >>>> For input: >>>> sudo mpiexec -phrase cat -machinefile machinefile -n 2 ~/main >>>> the error is: >>>> sudo: mpiexec: Command not found >>>> >>>> I therefore put mpiexec into /usr/bin >>>> >>>> now for input: >>>> sudo mpiexec -phrase cat -machinefile machinefile -n 2 ~/main >>>> the error is: >>>> Can't open /dev/mem >>>> Did you forget to use 'sudo .. ?' >>>> >>>> Does anyone know how I can work around this? >>>> Thanks, >>>> Eibhlin >>>> >>>> >>>> >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>>> >> -- >> Pavan Balaji >> http://www.mcs.anl.gov/~balaji >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem
by Gus Correa 13 Jun '13

13 Jun '13

Hi Eibhlin On 06/13/2013 12:59 PM, Lee, Eibhlin wrote: > Gus, > I believe your first assumption is correct. Unfortunately it just seemed to hang. I think this might be because each one is being made to have the same rank... Darn! I was afraid that it might give only rank 0 to all MPI processes. So, with the script wrapper the process being launched by mpiexec may indeed be sudo, not the actual mpi executable (main) :( Then it may actually launch a bunch of separate rank 0 replicas of your program, instead of assigning to them different ranks. However, without any output or error message, it is hard to tell. No output at all? No error message, just hangs? Have you tried a verbose flag (-v) to mpiexec? (Not sure if it exists in MPICH mpiexec, you'd need to check.) Would you care to try it with another mpi program, one that doesn't deal with /dev/mem (a risky business), say cpi.c (in the examples directory), or an mpi version of Hello, world, just to see if the mpiexec+sudo_script_wrapper works as expected or if everybody gets rank 0? > It may already be obvious but this is the first time I am using Linux. I had tried sudo $(which mpiexec ....) and sudo $(which mpiexec) ... both without success. "which mpiexec" will return the path to mpiexec, but won't execute it. You could try this (with backquotes): `which mpiexec` -n 2 ~/main On a side note, make sure the mpiexec you're using matches the mpicc/mpif90/MPI library from the MPICH that you used to compile the program. Often times computers have several flavors of MPI installed, and mixing them just doesn't work. > Is putting the full path to it similar to/is a symlink? (This still doesn't make main have super user privileges though.) No, nothing to do with sudo privileges. This suggestion was just to avoid messing up your /usr/bin, which is a directory that despite the somewhat misleading name (/usr, for historical reasons I think), is supposed to hold system (Linux) programs (that users can use), but not user-installed programs. Normally things are that are installed in /usr get there via some Linux package manager program (yum, rpm, apt-get, etc), to keep consistency with libraries, etc. I belive MPICH would install by default in /usr/local/ (and put mpiexec in /usr/local/bin), which is kind of a default location for non-system applications. The full path suggestion would be something like: /path/to/where/you/installed/mpiexec -n 2 ~/main However, this won't solve the other problem w.r.t. sudo and /dev/mem. You must know what you are doing, but it made me wonder, even if your program were sequential, why would you want to mess with /dev/mem directly? Just curious about it. Gus Correa > Eibhlin > ________________________________________ > From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of Gus Correa [gus(a)ldeo.columbia.edu] > Sent: 13 June 2013 15:37 > To: Discuss Mpich > Subject: Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem > > Hi Lee > > How about replacing "~/main" in the mpiexec command line > by one-liner script? > Say, "sudo_main.sh", something like this: > > #! /bin/bash > sudo ~/main > > After all, it is "main" that accesses /dev/mem, > and needs "sudo" permissions, not mpiexec, right? > [Or do the mpiexec-launched processes inherit > the "sudo" stuff from mpiexec?] > > Not related, but, instead of putting mpiexec in /usr/bin, > can't you just use the full path to it? > > I hope this helps, > Gus Correa > > On 06/13/2013 10:09 AM, Lee, Eibhlin wrote: >> Pavan, >> I had a lot of trouble getting hydra to work without having to enter a password/passphrase. I saw the option to pass a phrase in the mpich installers guide. I eventually found that for that command you needed to use the smpd process manager. That's the only reason I chose smpd over hydra. >> As to your other suggestion. I ran ./main and the same error (Can't open /dev/mem...) appeared. sudo ./main works but of course without multiple processes. >> Eibhlin >> ________________________________________ >> From: discuss-bounces(a)mpich.org [discuss-bounces(a)mpich.org] on behalf of Pavan Balaji [balaji(a)mcs.anl.gov] >> Sent: 13 June 2013 14:34 >> To: discuss(a)mpich.org >> Subject: Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem >> >> I just saw your older email. Why are you using smpd instead of the >> default process manager (hydra)? >> >> -- Pavan >> >> On 06/13/2013 08:05 AM, Pavan Balaji wrote: >>> What's "-phrase"? That's not a recognized option. I'm not sure where >>> the /dev/mem check is coming from. Try running ~/main without mpiexec >>> first. >>> >>> -- Pavan >>> >>> On 06/13/2013 06:56 AM, Lee, Eibhlin wrote: >>>> Hello all, >>>> >>>> I am trying to use two raspberry-pi to sample and then process some >>>> data. The first process samples while the second processes and vice >>>> versa. To do this I use gpio and also mpich-3.0.4 with the process >>>> manager smpd. I have successfully run cpi on both machines (from the >>>> master machine). I have also managed to run a similar program but >>>> without the MPI, this involved compiling with gcc and when running >>>> putting sudo in front of the binary file. >>>> >>>> When I combine these two processes I get various error messages. >>>> For input: >>>> mpiexec -phrase cat -machinefile machinefile -n 2 ~/main >>>> the error is: >>>> Can't open /dev/mem >>>> Did you forget to use 'sudo .. ?' >>>> >>>> For input: >>>> sudo mpiexec -phrase cat -machinefile machinefile -n 2 ~/main >>>> the error is: >>>> sudo: mpiexec: Command not found >>>> >>>> I therefore put mpiexec into /usr/bin >>>> >>>> now for input: >>>> sudo mpiexec -phrase cat -machinefile machinefile -n 2 ~/main >>>> the error is: >>>> Can't open /dev/mem >>>> Did you forget to use 'sudo .. ?' >>>> >>>> Does anyone know how I can work around this? >>>> Thanks, >>>> Eibhlin >>>> >>>> >>>> >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>>> >> -- >> Pavan Balaji >> http://www.mcs.anl.gov/~balaji >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0