September 2016 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] Why does MPICH perform busy wait?
by Michele De Stefano 30 Sep '16

30 Sep '16

Yes, you are right. select(), poll(), epoll() it's the way to go. Bye, Michele 2016-09-29 17:08 GMT+02:00 Rob Latham <robl(a)mcs.anl.gov>: > > Your proposal introduces latency. Data centers want power efficiency, but > several large classes of applications care about message rate. > > you don't need usleeep. There are several event-based system calls you > could use: sleect(), poll() -- which does not actually poll in most > implementations. > > Also, there are non-spinning channels you can use: e.g. ch3:sock > > It's been something we've been thinking about for a while! > http://trac.mpich.org/projects/mpich/ticket/79 > > > ==rob > > > > On 09/29/2016 09:07 AM, Tim Jammer wrote: > >> Hi >> >> Within my bachelor thesis I come across that the MPICH Implementation >> implements a busy wait. >> In our application, this leads to a waste of energy as all processes are >> spinning in a barrier while one process performs I/O. >> >> Therefore I want to propse to not wait as busy as it is done now. >> On the test system this saves 50% energy compared to busy wait. >> My basic prototype implementation causes only a negligible additional >> runtime overhead. >> >> My proposal is to sleep between polling if the communication has not >> finished yet. >> Within the first few iterations the implementation should not sleep, to >> guarantee maximum performance. >> But if the communication was not finished right away, the importance of >> saving energy grows. >> Therefore I propose to sleep and increase this sleeping time after every >> polling >> up to a maximum value in order to not waste much additional runtime on >> waiting. >> This will allow the CPU to power down and save energy. >> >> my basic prototype basically works as follows: >> >> int stime=0; >> while (/* Incoming data does not match the request*/) >> { >> usleep(stime); >> stime++; >> // do the actual polling... >> >> I can imagine, that the user can choose to activate this behaviour (i.e. >> with an environment variable) if he wants to save energy. >> >> I have attached the section of my thesis which describe it more in detail. >> >> >> My question is: was something like this considered before and are there >> any arguments why this was not implemented. >> >> Regards >> Tim Jammer >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> >> _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Michele De Stefano Linked In <http://it.linkedin.com/in/micheledestefano> mds-utils: a general purpose Open Source library <http://sourceforge.net/projects/mds-utils/> Personal Web Site <http://www.micheledestefano.altervista.org> _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Why does MPICH perform busy wait?
by Jeff Hammond 29 Sep '16

29 Sep '16

There are bunch of ways to address these issues. One can save power while still spinning. See e.g. http://mug.mvapich.cse.ohio-state.edu/static/media/mug/presentations/2015/m…. LLNL has done something similar. Certain types of passive waiting may not work with shared-memory communication. monitor-mwait would be interesting if not for the ring-0 issue. Jeff On Thu, Sep 29, 2016 at 8:08 AM, Rob Latham <robl(a)mcs.anl.gov> wrote: > > Your proposal introduces latency. Data centers want power efficiency, but > several large classes of applications care about message rate. > > you don't need usleeep. There are several event-based system calls you > could use: sleect(), poll() -- which does not actually poll in most > implementations. > > Also, there are non-spinning channels you can use: e.g. ch3:sock > > It's been something we've been thinking about for a while! > http://trac.mpich.org/projects/mpich/ticket/79 > > > ==rob > > > > On 09/29/2016 09:07 AM, Tim Jammer wrote: > >> Hi >> >> Within my bachelor thesis I come across that the MPICH Implementation >> implements a busy wait. >> In our application, this leads to a waste of energy as all processes are >> spinning in a barrier while one process performs I/O. >> >> Therefore I want to propse to not wait as busy as it is done now. >> On the test system this saves 50% energy compared to busy wait. >> My basic prototype implementation causes only a negligible additional >> runtime overhead. >> >> My proposal is to sleep between polling if the communication has not >> finished yet. >> Within the first few iterations the implementation should not sleep, to >> guarantee maximum performance. >> But if the communication was not finished right away, the importance of >> saving energy grows. >> Therefore I propose to sleep and increase this sleeping time after every >> polling >> up to a maximum value in order to not waste much additional runtime on >> waiting. >> This will allow the CPU to power down and save energy. >> >> my basic prototype basically works as follows: >> >> int stime=0; >> while (/* Incoming data does not match the request*/) >> { >> usleep(stime); >> stime++; >> // do the actual polling... >> >> I can imagine, that the user can choose to activate this behaviour (i.e. >> with an environment variable) if he wants to save energy. >> >> I have attached the section of my thesis which describe it more in detail. >> >> >> My question is: was something like this considered before and are there >> any arguments why this was not implemented. >> >> Regards >> Tim Jammer >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> >> _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/ _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Why does MPICH perform busy wait?
by Rob Latham 29 Sep '16

29 Sep '16

Your proposal introduces latency. Data centers want power efficiency, but several large classes of applications care about message rate. you don't need usleeep. There are several event-based system calls you could use: sleect(), poll() -- which does not actually poll in most implementations. Also, there are non-spinning channels you can use: e.g. ch3:sock It's been something we've been thinking about for a while! http://trac.mpich.org/projects/mpich/ticket/79 ==rob On 09/29/2016 09:07 AM, Tim Jammer wrote: > Hi > > Within my bachelor thesis I come across that the MPICH Implementation > implements a busy wait. > In our application, this leads to a waste of energy as all processes are > spinning in a barrier while one process performs I/O. > > Therefore I want to propse to not wait as busy as it is done now. > On the test system this saves 50% energy compared to busy wait. > My basic prototype implementation causes only a negligible additional > runtime overhead. > > My proposal is to sleep between polling if the communication has not > finished yet. > Within the first few iterations the implementation should not sleep, to > guarantee maximum performance. > But if the communication was not finished right away, the importance of > saving energy grows. > Therefore I propose to sleep and increase this sleeping time after every > polling > up to a maximum value in order to not waste much additional runtime on > waiting. > This will allow the CPU to power down and save energy. > > my basic prototype basically works as follows: > > int stime=0; > while (/* Incoming data does not match the request*/) > { > usleep(stime); > stime++; > // do the actual polling... > > I can imagine, that the user can choose to activate this behaviour (i.e. > with an environment variable) if he wants to save energy. > > I have attached the section of my thesis which describe it more in detail. > > > My question is: was something like this considered before and are there > any arguments why this was not implemented. > > Regards > Tim Jammer > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Why does MPICH perform busy wait?
by Michele De Stefano 29 Sep '16

29 Sep '16

I suggest, to whoever will address this issue, to have a look at how C++ >= 2011 handles the wait on a condition variable. I think the technique to be used should be analogous. Best regards, 2016-09-29 16:35 GMT+02:00 Maksym Planeta <mplaneta(a)os.inf.tu-dresden.de>: > Hi, > > have a look into this paper: > > A Case for Application-Oblivious Energy-Efficient MPI Runtime > A. Venkatesh , A. Vishnu , K. Hamidouche , N. Tallent , D. K. Panda , D. > Kerbyson , and A. Hoise > Supercomputing 2015, Nov 2015 > > That's about MVAPICH2, not MPICH, but they share a lot of code base. > > > On 09/29/2016 04:07 PM, Tim Jammer wrote: > >> Hi >> >> Within my bachelor thesis I come across that the MPICH Implementation >> implements a busy wait. >> In our application, this leads to a waste of energy as all processes are >> spinning in a barrier while one process performs I/O. >> >> Therefore I want to propse to not wait as busy as it is done now. >> On the test system this saves 50% energy compared to busy wait. >> My basic prototype implementation causes only a negligible additional >> runtime overhead. >> >> My proposal is to sleep between polling if the communication has not >> finished yet. >> Within the first few iterations the implementation should not sleep, to >> guarantee maximum performance. >> But if the communication was not finished right away, the importance of >> saving energy grows. >> Therefore I propose to sleep and increase this sleeping time after every >> polling >> up to a maximum value in order to not waste much additional runtime on >> waiting. >> This will allow the CPU to power down and save energy. >> >> my basic prototype basically works as follows: >> >> int stime=0; >> while (/* Incoming data does not match the request*/) >> { >> usleep(stime); >> stime++; >> // do the actual polling... >> >> I can imagine, that the user can choose to activate this behaviour (i.e. >> with an environment variable) if he wants to save energy. >> >> I have attached the section of my thesis which describe it more in detail. >> >> >> My question is: was something like this considered before and are there >> any arguments why this was not implemented. >> >> Regards >> Tim Jammer >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> >> > -- > Regards, > Maksym Planeta > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Michele De Stefano Linked In <http://it.linkedin.com/in/micheledestefano> mds-utils: a general purpose Open Source library <http://sourceforge.net/projects/mds-utils/> Personal Web Site <http://www.micheledestefano.altervista.org> _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Why does MPICH perform busy wait?
by Maksym Planeta 29 Sep '16

29 Sep '16

Hi, have a look into this paper: A Case for Application-Oblivious Energy-Efficient MPI Runtime A. Venkatesh , A. Vishnu , K. Hamidouche , N. Tallent , D. K. Panda , D. Kerbyson , and A. Hoise Supercomputing 2015, Nov 2015 That's about MVAPICH2, not MPICH, but they share a lot of code base. On 09/29/2016 04:07 PM, Tim Jammer wrote: > Hi > > Within my bachelor thesis I come across that the MPICH Implementation > implements a busy wait. > In our application, this leads to a waste of energy as all processes are > spinning in a barrier while one process performs I/O. > > Therefore I want to propse to not wait as busy as it is done now. > On the test system this saves 50% energy compared to busy wait. > My basic prototype implementation causes only a negligible additional > runtime overhead. > > My proposal is to sleep between polling if the communication has not > finished yet. > Within the first few iterations the implementation should not sleep, to > guarantee maximum performance. > But if the communication was not finished right away, the importance of > saving energy grows. > Therefore I propose to sleep and increase this sleeping time after every > polling > up to a maximum value in order to not waste much additional runtime on > waiting. > This will allow the CPU to power down and save energy. > > my basic prototype basically works as follows: > > int stime=0; > while (/* Incoming data does not match the request*/) > { > usleep(stime); > stime++; > // do the actual polling... > > I can imagine, that the user can choose to activate this behaviour (i.e. > with an environment variable) if he wants to save energy. > > I have attached the section of my thesis which describe it more in detail. > > > My question is: was something like this considered before and are there > any arguments why this was not implemented. > > Regards > Tim Jammer > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Regards, Maksym Planeta _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Why does MPICH perform busy wait?
by Tim Jammer 29 Sep '16

29 Sep '16

Hi Within my bachelor thesis I come across that the MPICH Implementation implements a busy wait. In our application, this leads to a waste of energy as all processes are spinning in a barrier while one process performs I/O. Therefore I want to propse to not wait as busy as it is done now. On the test system this saves 50% energy compared to busy wait. My basic prototype implementation causes only a negligible additional runtime overhead. My proposal is to sleep between polling if the communication has not finished yet. Within the first few iterations the implementation should not sleep, to guarantee maximum performance. But if the communication was not finished right away, the importance of saving energy grows. Therefore I propose to sleep and increase this sleeping time after every polling up to a maximum value in order to not waste much additional runtime on waiting. This will allow the CPU to power down and save energy. my basic prototype basically works as follows: int stime=0; while (/* Incoming data does not match the request*/) { usleep(stime); stime++; // do the actual polling... I can imagine, that the user can choose to activate this behaviour (i.e. with an environment variable) if he wants to save energy. I have attached the section of my thesis which describe it more in detail. My question is: was something like this considered before and are there any arguments why this was not implemented. Regards Tim Jammer _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPI I/O spilt collective implementation: MPI_File_write_all_begin blocking or non-blocking?
by Rob Latham 26 Sep '16

26 Sep '16

On 09/26/2016 11:15 AM, Pramod Kumbhar wrote: > > > On Tue, Feb 9, 2016 at 12:48 AM, pramod kumbhar > <pramod.s.kumbhar(a)gmail.com <mailto:[email protected]>> wrote: > > Hello All, > > I am testing MPI_FIle_write_all_begin/_end and it seems like > MPI_File_write_all_begin waits for i/o completion. > > Could someone confirm the implementation of these routines? that's right. ROMIO's split-collective implementation does not actually do any I/O in the background. You might find the non-blocking collective I/O routines, implemented in MPICH-3.2. or newer, a better fit for your use case? http://press3.mcs.anl.gov/romio/2015/05/06/non-blocking-collective-io/ ==rob > > Thanks, > Pramod > > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPI I/O spilt collective implementation: MPI_File_write_all_begin blocking or non-blocking?
by Pramod Kumbhar 26 Sep '16

26 Sep '16

> On Tue, Feb 9, 2016 at 12:48 AM, pramod kumbhar <pramod.s.kumbhar(a)gmail.com> wrote: > Hello All, > > I am testing MPI_FIle_write_all_begin/_end and it seems like > MPI_File_write_all_begin waits for i/o completion. > > Could someone confirm the implementation of these routines? > > Thanks, > Pramod _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Problem on Windows
by Halim Amer 23 Sep '16

23 Sep '16

As the error message says, mpiexec does not understand the -localonly option. You can remove it and try again. Note that we do not support Windows anymore (https://wiki.mpich.org/mpich/index.php/Frequently_Asked_Questions#Q:_Why_ca…) You will have better luck with MS-MPI or Intel MPI. Halim www.mcs.anl.gov/~aamer On 9/23/16 1:55 PM, Kanak Datta wrote: > Hi > > I am trying to use mpich2 1.4.1 alongside Quantum Espresso 5.3. I have been > using Windows 7 Enterprise version on Intel Xeon processor. > > I have followed instructions given in the following link to install mpich2 > for windows 7: > http://swash.sourceforge.net/online_doc/swashimp/node9.html > > However, when I am trying to execute QE in parallel formation I am getting > the following error: > > Unknown option: -localonly > > I use the following line to run QE using mpich2: > > mpiexec -localonly -np 8 pw <abc.scf.inp> > > I don't know what's wrong here. > > I can run single execution using QE. However, I have not been able to run > parallel execution. What should I do? > > Any help will be highly appreciated. > > Thanks > Kanak > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Problem on Windows
by Kanak Datta 23 Sep '16

23 Sep '16

Hi I am trying to use mpich2 1.4.1 alongside Quantum Espresso 5.3. I have been using Windows 7 Enterprise version on Intel Xeon processor. I have followed instructions given in the following link to install mpich2 for windows 7: http://swash.sourceforge.net/online_doc/swashimp/node9.html However, when I am trying to execute QE in parallel formation I am getting the following error: Unknown option: -localonly I use the following line to run QE using mpich2: mpiexec -localonly -np 8 pw <abc.scf.inp> I don't know what's wrong here. I can run single execution using QE. However, I have not been able to run parallel execution. What should I do? Any help will be highly appreciated. Thanks Kanak -- _______________________________________ Kanak Datta Graduate Student Electrical Engineering and Computer Science University of MIchigan, Ann Arbor _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0