May 2014 - devel - mailman3.cels.anl.gov

Re: [mpich-devel] Suboptimal MPI_Allreduce() for intercommunicators
by Lisandro Dalcin 01 May '14

01 May '14

On 1 May 2014 16:53, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: >>> (For example, on one side of the intercommunicator sendbuf=1 byte, recvbuf=1 gigabyte; on the other side, sendbuf=1 gigabyte, recvbuf=1 byte) >>> >> >> For Allreduce, you cannot have 1 byte/ 1gigabyte at each side. > > For intercommunicator collectives, one side gets the results from the other side. For allgather, it is specified that the communication can be of unequal amounts: MPI-3 pg 166, 9-15. I am assuming the same holds for allreduce, which just says "Both groups should provide count and datatype arguments that specify the same type signature." (188:23-24) > OK, you are right. After looking more closely to the code, I managed to write a patch and upload it to the trac ticket. I'm doing a memory allocation, but the original implementation also had to allocate memory at ranks 0 on left and right groups (look at MPIR_Reduce_inter() ) -- Lisandro Dalcin --------------- CIMEC (UNL/CONICET) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1016) Tel/Fax: +54-342-4511169

1 0

Re: [mpich-devel] Suboptimal MPI_Allreduce() for intercommunicators
by Rajeev Thakur 01 May '14

01 May '14

>> (For example, on one side of the intercommunicator sendbuf=1 byte, recvbuf=1 gigabyte; on the other side, sendbuf=1 gigabyte, recvbuf=1 byte) >> > > For Allreduce, you cannot have 1 byte/ 1gigabyte at each side. For intercommunicator collectives, one side gets the results from the other side. For allgather, it is specified that the communication can be of unequal amounts: MPI-3 pg 166, 9-15. I am assuming the same holds for allreduce, which just says "Both groups should provide count and datatype arguments that specify the same type signature." (188:23-24) Rajeev

1 0

Re: [mpich-devel] Suboptimal MPI_Allreduce() for intercommunicators
by Lisandro Dalcin 01 May '14

01 May '14

On 30 April 2014 18:56, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: > It was probably done that way to avoid the need for memory allocation, but can you add a ticket in Trac and we will look at it. > https://trac.mpich.org/projects/mpich/ticket/2074 > > (For example, on one side of the intercommunicator sendbuf=1 byte, recvbuf=1 gigabyte; on the other side, sendbuf=1 gigabyte, recvbuf=1 byte) > For Allreduce, you cannot have 1 byte/ 1gigabyte at each side. -- Lisandro Dalcin --------------- CIMEC (UNL/CONICET) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1016) Tel/Fax: +54-342-4511169

1 0

Re: [mpich-devel] progress in flushall
by Balaji, Pavan 30 Apr '14

30 Apr '14

Jeff, On Apr 30, 2014, at 4:29 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > On Wed, Apr 30, 2014 at 3:58 AM, Balaji, Pavan <balaji(a)anl.gov> wrote: >> >> On Apr 29, 2014, at 10:20 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: >>> Yeah, I know that's what MPICH gives me. My point is that is >>> undesirable. All sync operations should make progress, no matter >>> what. >> >> They are not required to make progress from the spec perspective. It’s a bad assumption to make that Win_flush will always make progress. I think it’s OK for it to be mostly a no-op when there are no operations posted. If you need progress, why don’t you use test or something similar? > > I want flushall to make progress because there might be incoming RMA > that won't be processed otherwise. While today MPICH makes progress > on all Ch3 ops in Barrier, if that ever changes, ARMCI_Barrier is > going to deadlock because it does win_flush_all then mpi_barrier. I don’t think that’ll change. Barrier will need to make progress on RMA operations (unless the RMA operations are asynchronous) in order to remain compliant with the standard. The following program has to always work correctly: P0: MPI_Put MPI_Win_flush(_all) MPI_Barrier P1: MPI_Barrier The only case that’ll change is if we decide to somehow do async progress for all RMA operations (e.g., using a separate thread). But in that case, this entire problem wouldn’t exist anyway. > I'm already adding Iprobe for progress inside of ARMCI-MPI, but it > seems silly to have to test for p2p to get progress in RMA and for RMA > sync ops to not guarantee progress. I'd much prefer the design where > p2p and rma progress are decoupled and my RMA sync ops are burdened > with progress on p2p and vice versa. I’m not sure how you can reliably do that, in a standard compliant way. — Pavan

1 0