March 2013 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] MPI_PUT Implementation Bug or Pilot Error?
by Timothy Stitt 29 Mar '13

29 Mar '13

Thanks Rajeev. That makes sense now...and a modified version using separate storage per iteration works perfectly. Tim. On Mar 29, 2013, at 1:55 PM, Rajeev Thakur <thakur(a)mcs.anl.gov<mailto:[email protected]>> wrote: MPI_Put is nonblocking. You are changing the contents of the source buffer before the put is complete. Rajeev On Mar 29, 2013, at 12:01 PM, Timothy Stitt wrote: Hi all, I've come across some strange behavior with a code that uses MPI_PUT(). I can confirm that this problem appears in older MPICH2 v1.4.x releases as well as the latest MPICH v3.0.3 release. I can also confirm that I receive the correct results when using Open MPI. I am attaching a reproducer code below that simulates the problem I am having (following the code will hopefully explain its purpose better than what I can describe here in a few lines). This case is hardcoded for a 4-process run. For some reason MPICH implementations are losing some data elements and duplicating others after the communication is complete. I would be grateful if someone can confirm whether this is indeed an implementation bug or a subtle problem with how I've implemented the one-sided communication. Thanks in advance, Tim. program reproducer use mpi implicit none integer :: error,dum_com(2),my_rank,i integer :: win_values_points, type_integer,NProc=4 INTEGER(KIND=MPI_ADDRESS_KIND) :: win_values_size, target_disp integer,allocatable :: neighbor_values(:), ncount_in(:) call MPI_INIT(error) call MPI_COMM_RANK(MPI_COMM_WORLD,my_rank,error) CALL MPI_Type_size(MPI_INTEGER,type_integer,error) ! Calculate Size Of Window (in bytes) win_values_size=2*type_integer*NProc ! Allocate Window Storage allocate(neighbor_values(0:2*Nproc-1)) neighbor_values=0 ! Define Input Values allocate(ncount_in(NProc)) if (my_rank .eq. 0) ncount_in = (/1,2,3,4/) if (my_rank .eq. 1) ncount_in = (/5,6,7,8/) if (my_rank .eq. 2) ncount_in = (/9,10,11,12/) if (my_rank .eq. 3) ncount_in = (/13,14,15,16/) ! Create Window CALL MPI_WIN_CREATE(neighbor_values, win_values_size, type_integer, MPI_INFO_NULL, MPI_COMM_WORLD, win_values_points, error) ! Create Neighbor Storage Win dow call MPI_Win_Fence(0,win_values_points,error) ! Print pre-communication data print*,my_rank,ncount_in call flush(5) call MPI_BARRIER(MPI_COMM_WORLD,error) ! Communicate Data Values to all proceses do i = 1,4 target_disp=2*my_rank dum_com(1)=ncount_in(i) dum_com(2)=99 CALL MPI_PUT(dum_com,2,MPI_INTEGER,Nproc-i,target_disp,2,MPI_INTEGER,win_values_points,error) end do call MPI_Win_Fence(0,win_values_points,error) ! Print post-communication values print * print *,my_rank,neighbor_values call flush(5) call MPI_BARRIER(MPI_COMM_WORLD,error) ! Free Window Storage call mpi_win_free(win_values_points,error) call MPI_FINALIZE(error) end program reproducer _______________________________________________ discuss mailing list discuss(a)mpich.org<mailto:[email protected]> To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org<mailto:[email protected]> To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss Tim Stitt PhD User Support Manager (CRC) Research Assistant Professor (Computer Science & Engineering) Room 108, Center for Research Computing | University of Notre Dame | P.O. Box 539, Notre Dame, IN 46556 | Phone: 574-631-5287 | Email: tstitt(a)nd.edu<mailto:[email protected]>

1 0

Re: [mpich-discuss] MPI_PUT Implementation Bug or Pilot Error?
by Rajeev Thakur 29 Mar '13

29 Mar '13

MPI_Put is nonblocking. You are changing the contents of the source buffer before the put is complete. Rajeev On Mar 29, 2013, at 12:01 PM, Timothy Stitt wrote: > Hi all, > > I've come across some strange behavior with a code that uses MPI_PUT(). I can confirm that this problem appears in older MPICH2 v1.4.x releases as well as the latest MPICH v3.0.3 release. I can also confirm that I receive the correct results when using Open MPI. > > I am attaching a reproducer code below that simulates the problem I am having (following the code will hopefully explain its purpose better than what I can describe here in a few lines). This case is hardcoded for a 4-process run. For some reason MPICH implementations are losing some data elements and duplicating others after the communication is complete. I would be grateful if someone can confirm whether this is indeed an implementation bug or a subtle problem with how I've implemented the one-sided communication. > > Thanks in advance, > > Tim. > > program reproducer > > use mpi > > implicit none > > integer :: error,dum_com(2),my_rank,i > integer :: win_values_points, type_integer,NProc=4 > INTEGER(KIND=MPI_ADDRESS_KIND) :: win_values_size, target_disp > integer,allocatable :: neighbor_values(:), ncount_in(:) > > call MPI_INIT(error) > call MPI_COMM_RANK(MPI_COMM_WORLD,my_rank,error) > > CALL MPI_Type_size(MPI_INTEGER,type_integer,error) > > ! Calculate Size Of Window (in bytes) > win_values_size=2*type_integer*NProc > > ! Allocate Window Storage > allocate(neighbor_values(0:2*Nproc-1)) > neighbor_values=0 > > ! Define Input Values > allocate(ncount_in(NProc)) > if (my_rank .eq. 0) ncount_in = (/1,2,3,4/) > if (my_rank .eq. 1) ncount_in = (/5,6,7,8/) > if (my_rank .eq. 2) ncount_in = (/9,10,11,12/) > if (my_rank .eq. 3) ncount_in = (/13,14,15,16/) > > ! Create Window > CALL MPI_WIN_CREATE(neighbor_values, win_values_size, type_integer, MPI_INFO_NULL, MPI_COMM_WORLD, win_values_points, error) ! Create Neighbor Storage Win > dow > call MPI_Win_Fence(0,win_values_points,error) > > ! Print pre-communication data > print*,my_rank,ncount_in > call flush(5) > call MPI_BARRIER(MPI_COMM_WORLD,error) > > ! Communicate Data Values to all proceses > do i = 1,4 > target_disp=2*my_rank > dum_com(1)=ncount_in(i) > dum_com(2)=99 > CALL MPI_PUT(dum_com,2,MPI_INTEGER,Nproc-i,target_disp,2,MPI_INTEGER,win_values_points,error) > end do > > call MPI_Win_Fence(0,win_values_points,error) > > ! Print post-communication values > print * > print *,my_rank,neighbor_values > call flush(5) > call MPI_BARRIER(MPI_COMM_WORLD,error) > > ! Free Window Storage > call mpi_win_free(win_values_points,error) > > call MPI_FINALIZE(error) > > end program reproducer > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

MPI_PUT Implementation Bug or Pilot Error?
by Timothy Stitt 29 Mar '13

29 Mar '13

Hi all, I've come across some strange behavior with a code that uses MPI_PUT(). I can confirm that this problem appears in older MPICH2 v1.4.x releases as well as the latest MPICH v3.0.3 release. I can also confirm that I receive the correct results when using Open MPI. I am attaching a reproducer code below that simulates the problem I am having (following the code will hopefully explain its purpose better than what I can describe here in a few lines). This case is hardcoded for a 4-process run. For some reason MPICH implementations are losing some data elements and duplicating others after the communication is complete. I would be grateful if someone can confirm whether this is indeed an implementation bug or a subtle problem with how I've implemented the one-sided communication. Thanks in advance, Tim. program reproducer use mpi implicit none integer :: error,dum_com(2),my_rank,i integer :: win_values_points, type_integer,NProc=4 INTEGER(KIND=MPI_ADDRESS_KIND) :: win_values_size, target_disp integer,allocatable :: neighbor_values(:), ncount_in(:) call MPI_INIT(error) call MPI_COMM_RANK(MPI_COMM_WORLD,my_rank,error) CALL MPI_Type_size(MPI_INTEGER,type_integer,error) ! Calculate Size Of Window (in bytes) win_values_size=2*type_integer*NProc ! Allocate Window Storage allocate(neighbor_values(0:2*Nproc-1)) neighbor_values=0 ! Define Input Values allocate(ncount_in(NProc)) if (my_rank .eq. 0) ncount_in = (/1,2,3,4/) if (my_rank .eq. 1) ncount_in = (/5,6,7,8/) if (my_rank .eq. 2) ncount_in = (/9,10,11,12/) if (my_rank .eq. 3) ncount_in = (/13,14,15,16/) ! Create Window CALL MPI_WIN_CREATE(neighbor_values, win_values_size, type_integer, MPI_INFO_NULL, MPI_COMM_WORLD, win_values_points, error) ! Create Neighbor Storage Win dow call MPI_Win_Fence(0,win_values_points,error) ! Print pre-communication data print*,my_rank,ncount_in call flush(5) call MPI_BARRIER(MPI_COMM_WORLD,error) ! Communicate Data Values to all proceses do i = 1,4 target_disp=2*my_rank dum_com(1)=ncount_in(i) dum_com(2)=99 CALL MPI_PUT(dum_com,2,MPI_INTEGER,Nproc-i,target_disp,2,MPI_INTEGER,win_values_points,error) end do call MPI_Win_Fence(0,win_values_points,error) ! Print post-communication values print * print *,my_rank,neighbor_values call flush(5) call MPI_BARRIER(MPI_COMM_WORLD,error) ! Free Window Storage call mpi_win_free(win_values_points,error) call MPI_FINALIZE(error) end program reproducer

1 0

Re: [mpich-discuss] Patch for MPE2 1.3.0 on MPI 3.0 implementations
by Jeremiah Willcock 28 Mar '13

28 Mar '13

On Thu, 28 Mar 2013, Jeremiah Willcock wrote: > Because of the addition of const to many function parameters, the logging and > tracing parts of MPE2 1.3.0 will not compile with an implementation of MPI 3 > such as the newest versions of MPICH and MVAPICH. I have attached a patch to > fix that. The main issue with it is that it would break compatibility with > MPI 2.* implementations that do not have the extra const modifiers; a version > of it to put into the MPE release would probably need a macro for the > newly-added consts that can enable them selectively. However, the patch > provided is likely to work for users who have this issue. It looks like a production-grade version of this is/was in the MPICH source tree at one time (see http://trac.mpich.org/projects/mpich/changeset/96bed1bda731799a15c4c8ab093b…, for example), but does not appear to be in the latest MPE release now that it is not included in MPICH. -- Jeremiah Willcock

1 0

Patch for MPE2 1.3.0 on MPI 3.0 implementations
by Jeremiah Willcock 28 Mar '13

28 Mar '13

Because of the addition of const to many function parameters, the logging and tracing parts of MPE2 1.3.0 will not compile with an implementation of MPI 3 such as the newest versions of MPICH and MVAPICH. I have attached a patch to fix that. The main issue with it is that it would break compatibility with MPI 2.* implementations that do not have the extra const modifiers; a version of it to put into the MPE release would probably need a macro for the newly-added consts that can enable them selectively. However, the patch provided is likely to work for users who have this issue. -- Jeremiah Willcock

1 0

Re: [mpich-discuss] MPI IO, reading MPI Writes without MPI read
by Ryan Crocker 28 Mar '13

28 Mar '13

I just went though the standard, i'm not doing anything to untoward. that i could see. I also would really not have to re-run these simulations. There has to be a way to get access to those MPI writes with an MPI read, and like i said, other programs have no trouble reading these in and using them (paravew, and ensight). I would just like to have my own stand alone in fortran. The writes come out of a fortran code compiled with mpich so there should be a way to read them back in with a fortran, mpich compiled code. Shouldn't there? On Mar 28, 2013, at 12:35 PM, Rajeev Thakur wrote: > MPI-IO just needs the address of the first location to write form (in C style). The amount written is specified by ncells_hexa_, MPI_REAL_SP > > On Mar 28, 2013, at 2:34 PM, Ryan Crocker wrote: > >> buffer3_hexa(1,1) would just write out the first point in the buffer3 array, while buffer3_hexa(:,1) tells it to write the whole vector. which is just a stacked version of my domain: >> >> do k >> do j >> do i >> m=m+1 >> buffer(m) = U(i,j,k) >> end do >> end do >> end do >> >> I think maybe my disp are messed up. If i'm writing from each process with the disp a function of the size of the number of nodes in the process does reading it in with the disp a function of all the nodes in all the process make sense? Like i said paraview and ensight have no problems reading these files, and the only information they get in the header is the total number of nodes in all the processes. >> >> >> On Mar 28, 2013, at 12:22 PM, Rajeev Thakur wrote: >> >>> The other thing I would try is use buffer3_hexa(1,1) instead of buffer3_hexa(:,1). Not sure what Fortran compilers do with the latter format, but there is discussion of issues with Fortran subscript notation on pgs 626-628 of the MPI-3 standard. >>> >>> On Mar 28, 2013, at 2:16 PM, Ryan Crocker wrote: >>> >>>> Yes. >>>> >>>> On Mar 28, 2013, at 12:11 PM, Rajeev Thakur wrote: >>>> >>>>> Have you declared disp as integer (kind=MPI_OFFSET_KIND) instead of just integer? >>>>> >>>>> On Mar 28, 2013, at 1:33 PM, Ryan Crocker wrote: >>>>> >>>>>> Okay, so i coded the MPI IO. All the characters and stuff show up, but when i try to read in my data buffers i just get the same number for the whole vector. All the characters in the header read in and are what they'er supposed to be, and the program finishes without any errors. >>>>>> >>>>>> my write is: >>>>>> >>>>>> call MPI_FILE_OPEN(comm,file,IOR(MPI_MODE_WRONLY,MPI_MODE_CREATE),mpi_info,iunit,ierr) >>>>>> >>>>>> ! Write header (only root) >>>>>> if (irank.eq.iroot) then >>>>>> buffer = trim(adjustl(name)) >>>>>> size = 80 >>>>>> call MPI_FILE_WRITE(iunit,buffer,size,MPI_CHARACTER,status,ierr) >>>>>> buffer = 'part' >>>>>> size = 80 >>>>>> call MPI_FILE_WRITE(iunit,buffer,size,MPI_CHARACTER,status,ierr) >>>>>> ibuffer = 1 >>>>>> size = 1 >>>>>> call MPI_FILE_WRITE(iunit,ibuffer,size,MPI_INTEGER,status,ierr) >>>>>> buffer = 'hexa8' >>>>>> size = 80 >>>>>> call MPI_FILE_WRITE(iunit,buffer,size,MPI_CHARACTER,status,ierr) >>>>>> end if >>>>>> >>>>>> ! Write the data >>>>>> disp = 3*80+4+0*ncells_hexa*4 >>>>>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>>>>> call MPI_FILE_WRITE_ALL(iunit,buffer3_hexa(:,1),ncells_hexa_,MPI_REAL_SP,status,ierr) >>>>>> disp = 3*80+4+1*ncells_hexa*4 >>>>>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>>>>> call MPI_FILE_WRITE_ALL(iunit,buffer3_hexa(:,2),ncells_hexa_,MPI_REAL_SP,status,ierr) >>>>>> disp = 3*80+4+2*ncells_hexa*4 >>>>>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>>>>> call MPI_FILE_WRITE_ALL(iunit,buffer3_hexa(:,3),ncells_hexa_,MPI_REAL_SP,status,ierr) >>>>>> >>>>>> ! Close the file >>>>>> call MPI_FILE_CLOSE(iunit,ierr) >>>>>> >>>>>> and the read is: >>>>>> >>>>>> ncells = parallel_sum(ncells_hexa_,ncells) >>>>>> >>>>>> allocate(buffer3(ncells,3)) >>>>>> >>>>>> openfile=trim(workdir)//'/'//'V/V.000002' >>>>>> call MPI_FILE_OPEN(comm,openfile,MPI_MODE_RDONLY,mpi_info,iunit,ierr) >>>>>> >>>>>> ! Read header >>>>>> bsize = 80 >>>>>> call MPI_FILE_READ(iunit,cbuffer,bsize,MPI_CHARACTER,status,ierr) >>>>>> print*,trim(cbuffer) >>>>>> bsize = 80 >>>>>> call MPI_FILE_READ(iunit,cbuffer,bsize,MPI_CHARACTER,status,ierr) >>>>>> print*,trim(cbuffer) >>>>>> bsize = 1 >>>>>> call MPI_FILE_READ(iunit,ibuffer,bsize,MPI_INTEGER,status,ierr) >>>>>> print*,ibuffer >>>>>> bsize = 80 >>>>>> call MPI_FILE_READ(iunit,cbuffer,bsize,MPI_CHARACTER,status,ierr) >>>>>> print*,trim(cbuffer),ncells >>>>>> >>>>>> ! Read the data >>>>>> disp = 3*80+4+0*ncells*4 >>>>>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>>>>> call MPI_FILE_READ_ALL(iunit,buffer3(:,1),ncells,MPI_REAL_SP,status,ierr) >>>>>> disp = 3*80+4+1*ncells*4 >>>>>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>>>>> call MPI_FILE_READ_ALL(iunit,buffer3(:,2),ncells,MPI_REAL_SP,status,ierr) >>>>>> disp = 3*80+4+2*ncells*4 >>>>>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>>>>> call MPI_FILE_READ_ALL(iunit,buffer3(:,3),ncells,MPI_REAL_SP,status,ierr) >>>>>> >>>>>> ! Close the file >>>>>> call MPI_FILE_CLOSE(iunit,ierr) >>>>>> >>>>>> On Mar 28, 2013, at 8:48 AM, Rob Latham wrote: >>>>>> >>>>>>> On Thu, Mar 28, 2013 at 08:19:23AM -0700, Ryan Crocker wrote: >>>>>>>> from the file those points are most definitely not in the same spot. We're using a set view so the memory is stored non-contiguously so you can't just read it back in, and i can see the offset in the data blocks. >>>>>>> >>>>>>> If you write the data with MPI_File_write_at, then you should read it >>>>>>> back with MPI_File_read_at. Fortran I/O will expect the input to have various >>>>>>> compiler-specific padding or ... well I don't properly know the >>>>>>> details, other than it's a mess. >>>>>>> >>>>>>> ==rob >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> On Mar 28, 2013, at 6:47 AM, Rajeev Thakur wrote: >>>>>>>> >>>>>>>>> Most MPI-IO implementations write plain binary files without any additional information. So the data should be exactly what you wrote and at the locations you specified. The disp is not stored in the file. It is just used to offset to the right location. The file won't be portable with the default "native" file format used for writing. In other words, it will be readable only on the same type of machine architecture. >>>>>>>>> >>>>>>>>> Try writing a simple MPI-IO program to write a file and then read it back using POSIX I/O calls (lseek, read) from a C program. I don't know if Fortran I/O expects files to be formatted in a particular way, which could cause a problem when reading from Fortran. >>>>>>>>> >>>>>>>>> Rajeev >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mar 28, 2013, at 2:00 AM, Ryan Crocker wrote: >>>>>>>>> >>>>>>>>>> So i'm not sure if this is crazy or not, but i have file outputs from my code that write ensight gold files in MPI. Here is the write, >>>>>>>>>> >>>>>>>>>> disp = 3*80+4+0*ncells_hexa*4 >>>>>>>>>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>>>>>>>>> call MPI_FILE_WRITE_ALL(iunit,buffer3_hexa(:,1),ncells_hexa_,MPI_REAL_SP,status,ierr) >>>>>>>>>> >>>>>>>>>> if i wanted to read that in binary in fortran, or C (preferably fortran) what exactly would i need to do? I can't seem to write code that reads these in and produces anything that looks like the plot i get in paraview. I know that MPI write puts out each processor data vector with that disp in between them, but i just can't make that structure make sense to me when i try to read it in. >>>>>>>>>> >>>>>>>>>> Thanks for the help, >>>>>>>>>> >>>>>>>>>> Ryan Crocker >>>>>>>>>> University of Vermont, School of Engineering >>>>>>>>>> Mechanical Engineering Department >>>>>>>>>> rcrocker(a)uvm.edu >>>>>>>>>> 315-212-7331 >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> discuss mailing list discuss(a)mpich.org >>>>>>>>>> To manage subscription options or unsubscribe: >>>>>>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> discuss mailing list discuss(a)mpich.org >>>>>>>>> To manage subscription options or unsubscribe: >>>>>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>>>>> >>>>>>>> Ryan Crocker >>>>>>>> University of Vermont, School of Engineering >>>>>>>> Mechanical Engineering Department >>>>>>>> rcrocker(a)uvm.edu >>>>>>>> 315-212-7331 >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> discuss mailing list discuss(a)mpich.org >>>>>>>> To manage subscription options or unsubscribe: >>>>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>>>> >>>>>>> -- >>>>>>> Rob Latham >>>>>>> Mathematics and Computer Science Division >>>>>>> Argonne National Lab, IL USA >>>>>>> _______________________________________________ >>>>>>> discuss mailing list discuss(a)mpich.org >>>>>>> To manage subscription options or unsubscribe: >>>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>>> >>>>>> Ryan Crocker >>>>>> University of Vermont, School of Engineering >>>>>> Mechanical Engineering Department >>>>>> rcrocker(a)uvm.edu >>>>>> 315-212-7331 >>>>>> >>>>>> _______________________________________________ >>>>>> discuss mailing list discuss(a)mpich.org >>>>>> To manage subscription options or unsubscribe: >>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>> >>>>> _______________________________________________ >>>>> discuss mailing list discuss(a)mpich.org >>>>> To manage subscription options or unsubscribe: >>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>> >>>> Ryan Crocker >>>> University of Vermont, School of Engineering >>>> Mechanical Engineering Department >>>> rcrocker(a)uvm.edu >>>> 315-212-7331 >>>> >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> Ryan Crocker >> University of Vermont, School of Engineering >> Mechanical Engineering Department >> rcrocker(a)uvm.edu >> 315-212-7331 >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss Ryan Crocker University of Vermont, School of Engineering Mechanical Engineering Department rcrocker(a)uvm.edu 315-212-7331

1 0

Re: [mpich-discuss] MPI IO, reading MPI Writes without MPI read
by Rajeev Thakur 28 Mar '13

28 Mar '13

MPI-IO just needs the address of the first location to write form (in C style). The amount written is specified by ncells_hexa_, MPI_REAL_SP On Mar 28, 2013, at 2:34 PM, Ryan Crocker wrote: > buffer3_hexa(1,1) would just write out the first point in the buffer3 array, while buffer3_hexa(:,1) tells it to write the whole vector. which is just a stacked version of my domain: > > do k > do j > do i > m=m+1 > buffer(m) = U(i,j,k) > end do > end do > end do > > I think maybe my disp are messed up. If i'm writing from each process with the disp a function of the size of the number of nodes in the process does reading it in with the disp a function of all the nodes in all the process make sense? Like i said paraview and ensight have no problems reading these files, and the only information they get in the header is the total number of nodes in all the processes. > > > On Mar 28, 2013, at 12:22 PM, Rajeev Thakur wrote: > >> The other thing I would try is use buffer3_hexa(1,1) instead of buffer3_hexa(:,1). Not sure what Fortran compilers do with the latter format, but there is discussion of issues with Fortran subscript notation on pgs 626-628 of the MPI-3 standard. >> >> On Mar 28, 2013, at 2:16 PM, Ryan Crocker wrote: >> >>> Yes. >>> >>> On Mar 28, 2013, at 12:11 PM, Rajeev Thakur wrote: >>> >>>> Have you declared disp as integer (kind=MPI_OFFSET_KIND) instead of just integer? >>>> >>>> On Mar 28, 2013, at 1:33 PM, Ryan Crocker wrote: >>>> >>>>> Okay, so i coded the MPI IO. All the characters and stuff show up, but when i try to read in my data buffers i just get the same number for the whole vector. All the characters in the header read in and are what they'er supposed to be, and the program finishes without any errors. >>>>> >>>>> my write is: >>>>> >>>>> call MPI_FILE_OPEN(comm,file,IOR(MPI_MODE_WRONLY,MPI_MODE_CREATE),mpi_info,iunit,ierr) >>>>> >>>>> ! Write header (only root) >>>>> if (irank.eq.iroot) then >>>>> buffer = trim(adjustl(name)) >>>>> size = 80 >>>>> call MPI_FILE_WRITE(iunit,buffer,size,MPI_CHARACTER,status,ierr) >>>>> buffer = 'part' >>>>> size = 80 >>>>> call MPI_FILE_WRITE(iunit,buffer,size,MPI_CHARACTER,status,ierr) >>>>> ibuffer = 1 >>>>> size = 1 >>>>> call MPI_FILE_WRITE(iunit,ibuffer,size,MPI_INTEGER,status,ierr) >>>>> buffer = 'hexa8' >>>>> size = 80 >>>>> call MPI_FILE_WRITE(iunit,buffer,size,MPI_CHARACTER,status,ierr) >>>>> end if >>>>> >>>>> ! Write the data >>>>> disp = 3*80+4+0*ncells_hexa*4 >>>>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>>>> call MPI_FILE_WRITE_ALL(iunit,buffer3_hexa(:,1),ncells_hexa_,MPI_REAL_SP,status,ierr) >>>>> disp = 3*80+4+1*ncells_hexa*4 >>>>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>>>> call MPI_FILE_WRITE_ALL(iunit,buffer3_hexa(:,2),ncells_hexa_,MPI_REAL_SP,status,ierr) >>>>> disp = 3*80+4+2*ncells_hexa*4 >>>>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>>>> call MPI_FILE_WRITE_ALL(iunit,buffer3_hexa(:,3),ncells_hexa_,MPI_REAL_SP,status,ierr) >>>>> >>>>> ! Close the file >>>>> call MPI_FILE_CLOSE(iunit,ierr) >>>>> >>>>> and the read is: >>>>> >>>>> ncells = parallel_sum(ncells_hexa_,ncells) >>>>> >>>>> allocate(buffer3(ncells,3)) >>>>> >>>>> openfile=trim(workdir)//'/'//'V/V.000002' >>>>> call MPI_FILE_OPEN(comm,openfile,MPI_MODE_RDONLY,mpi_info,iunit,ierr) >>>>> >>>>> ! Read header >>>>> bsize = 80 >>>>> call MPI_FILE_READ(iunit,cbuffer,bsize,MPI_CHARACTER,status,ierr) >>>>> print*,trim(cbuffer) >>>>> bsize = 80 >>>>> call MPI_FILE_READ(iunit,cbuffer,bsize,MPI_CHARACTER,status,ierr) >>>>> print*,trim(cbuffer) >>>>> bsize = 1 >>>>> call MPI_FILE_READ(iunit,ibuffer,bsize,MPI_INTEGER,status,ierr) >>>>> print*,ibuffer >>>>> bsize = 80 >>>>> call MPI_FILE_READ(iunit,cbuffer,bsize,MPI_CHARACTER,status,ierr) >>>>> print*,trim(cbuffer),ncells >>>>> >>>>> ! Read the data >>>>> disp = 3*80+4+0*ncells*4 >>>>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>>>> call MPI_FILE_READ_ALL(iunit,buffer3(:,1),ncells,MPI_REAL_SP,status,ierr) >>>>> disp = 3*80+4+1*ncells*4 >>>>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>>>> call MPI_FILE_READ_ALL(iunit,buffer3(:,2),ncells,MPI_REAL_SP,status,ierr) >>>>> disp = 3*80+4+2*ncells*4 >>>>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>>>> call MPI_FILE_READ_ALL(iunit,buffer3(:,3),ncells,MPI_REAL_SP,status,ierr) >>>>> >>>>> ! Close the file >>>>> call MPI_FILE_CLOSE(iunit,ierr) >>>>> >>>>> On Mar 28, 2013, at 8:48 AM, Rob Latham wrote: >>>>> >>>>>> On Thu, Mar 28, 2013 at 08:19:23AM -0700, Ryan Crocker wrote: >>>>>>> from the file those points are most definitely not in the same spot. We're using a set view so the memory is stored non-contiguously so you can't just read it back in, and i can see the offset in the data blocks. >>>>>> >>>>>> If you write the data with MPI_File_write_at, then you should read it >>>>>> back with MPI_File_read_at. Fortran I/O will expect the input to have various >>>>>> compiler-specific padding or ... well I don't properly know the >>>>>> details, other than it's a mess. >>>>>> >>>>>> ==rob >>>>>> >>>>>> >>>>>>> >>>>>>> On Mar 28, 2013, at 6:47 AM, Rajeev Thakur wrote: >>>>>>> >>>>>>>> Most MPI-IO implementations write plain binary files without any additional information. So the data should be exactly what you wrote and at the locations you specified. The disp is not stored in the file. It is just used to offset to the right location. The file won't be portable with the default "native" file format used for writing. In other words, it will be readable only on the same type of machine architecture. >>>>>>>> >>>>>>>> Try writing a simple MPI-IO program to write a file and then read it back using POSIX I/O calls (lseek, read) from a C program. I don't know if Fortran I/O expects files to be formatted in a particular way, which could cause a problem when reading from Fortran. >>>>>>>> >>>>>>>> Rajeev >>>>>>>> >>>>>>>> >>>>>>>> On Mar 28, 2013, at 2:00 AM, Ryan Crocker wrote: >>>>>>>> >>>>>>>>> So i'm not sure if this is crazy or not, but i have file outputs from my code that write ensight gold files in MPI. Here is the write, >>>>>>>>> >>>>>>>>> disp = 3*80+4+0*ncells_hexa*4 >>>>>>>>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>>>>>>>> call MPI_FILE_WRITE_ALL(iunit,buffer3_hexa(:,1),ncells_hexa_,MPI_REAL_SP,status,ierr) >>>>>>>>> >>>>>>>>> if i wanted to read that in binary in fortran, or C (preferably fortran) what exactly would i need to do? I can't seem to write code that reads these in and produces anything that looks like the plot i get in paraview. I know that MPI write puts out each processor data vector with that disp in between them, but i just can't make that structure make sense to me when i try to read it in. >>>>>>>>> >>>>>>>>> Thanks for the help, >>>>>>>>> >>>>>>>>> Ryan Crocker >>>>>>>>> University of Vermont, School of Engineering >>>>>>>>> Mechanical Engineering Department >>>>>>>>> rcrocker(a)uvm.edu >>>>>>>>> 315-212-7331 >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> discuss mailing list discuss(a)mpich.org >>>>>>>>> To manage subscription options or unsubscribe: >>>>>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> discuss mailing list discuss(a)mpich.org >>>>>>>> To manage subscription options or unsubscribe: >>>>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>>>> >>>>>>> Ryan Crocker >>>>>>> University of Vermont, School of Engineering >>>>>>> Mechanical Engineering Department >>>>>>> rcrocker(a)uvm.edu >>>>>>> 315-212-7331 >>>>>>> >>>>>>> _______________________________________________ >>>>>>> discuss mailing list discuss(a)mpich.org >>>>>>> To manage subscription options or unsubscribe: >>>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>>> >>>>>> -- >>>>>> Rob Latham >>>>>> Mathematics and Computer Science Division >>>>>> Argonne National Lab, IL USA >>>>>> _______________________________________________ >>>>>> discuss mailing list discuss(a)mpich.org >>>>>> To manage subscription options or unsubscribe: >>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>> >>>>> Ryan Crocker >>>>> University of Vermont, School of Engineering >>>>> Mechanical Engineering Department >>>>> rcrocker(a)uvm.edu >>>>> 315-212-7331 >>>>> >>>>> _______________________________________________ >>>>> discuss mailing list discuss(a)mpich.org >>>>> To manage subscription options or unsubscribe: >>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>> >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>> >>> Ryan Crocker >>> University of Vermont, School of Engineering >>> Mechanical Engineering Department >>> rcrocker(a)uvm.edu >>> 315-212-7331 >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > Ryan Crocker > University of Vermont, School of Engineering > Mechanical Engineering Department > rcrocker(a)uvm.edu > 315-212-7331 > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPI IO, reading MPI Writes without MPI read
by Ryan Crocker 28 Mar '13

28 Mar '13

buffer3_hexa(1,1) would just write out the first point in the buffer3 array, while buffer3_hexa(:,1) tells it to write the whole vector. which is just a stacked version of my domain: do k do j do i m=m+1 buffer(m) = U(i,j,k) end do end do end do I think maybe my disp are messed up. If i'm writing from each process with the disp a function of the size of the number of nodes in the process does reading it in with the disp a function of all the nodes in all the process make sense? Like i said paraview and ensight have no problems reading these files, and the only information they get in the header is the total number of nodes in all the processes. On Mar 28, 2013, at 12:22 PM, Rajeev Thakur wrote: > The other thing I would try is use buffer3_hexa(1,1) instead of buffer3_hexa(:,1). Not sure what Fortran compilers do with the latter format, but there is discussion of issues with Fortran subscript notation on pgs 626-628 of the MPI-3 standard. > > On Mar 28, 2013, at 2:16 PM, Ryan Crocker wrote: > >> Yes. >> >> On Mar 28, 2013, at 12:11 PM, Rajeev Thakur wrote: >> >>> Have you declared disp as integer (kind=MPI_OFFSET_KIND) instead of just integer? >>> >>> On Mar 28, 2013, at 1:33 PM, Ryan Crocker wrote: >>> >>>> Okay, so i coded the MPI IO. All the characters and stuff show up, but when i try to read in my data buffers i just get the same number for the whole vector. All the characters in the header read in and are what they'er supposed to be, and the program finishes without any errors. >>>> >>>> my write is: >>>> >>>> call MPI_FILE_OPEN(comm,file,IOR(MPI_MODE_WRONLY,MPI_MODE_CREATE),mpi_info,iunit,ierr) >>>> >>>> ! Write header (only root) >>>> if (irank.eq.iroot) then >>>> buffer = trim(adjustl(name)) >>>> size = 80 >>>> call MPI_FILE_WRITE(iunit,buffer,size,MPI_CHARACTER,status,ierr) >>>> buffer = 'part' >>>> size = 80 >>>> call MPI_FILE_WRITE(iunit,buffer,size,MPI_CHARACTER,status,ierr) >>>> ibuffer = 1 >>>> size = 1 >>>> call MPI_FILE_WRITE(iunit,ibuffer,size,MPI_INTEGER,status,ierr) >>>> buffer = 'hexa8' >>>> size = 80 >>>> call MPI_FILE_WRITE(iunit,buffer,size,MPI_CHARACTER,status,ierr) >>>> end if >>>> >>>> ! Write the data >>>> disp = 3*80+4+0*ncells_hexa*4 >>>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>>> call MPI_FILE_WRITE_ALL(iunit,buffer3_hexa(:,1),ncells_hexa_,MPI_REAL_SP,status,ierr) >>>> disp = 3*80+4+1*ncells_hexa*4 >>>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>>> call MPI_FILE_WRITE_ALL(iunit,buffer3_hexa(:,2),ncells_hexa_,MPI_REAL_SP,status,ierr) >>>> disp = 3*80+4+2*ncells_hexa*4 >>>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>>> call MPI_FILE_WRITE_ALL(iunit,buffer3_hexa(:,3),ncells_hexa_,MPI_REAL_SP,status,ierr) >>>> >>>> ! Close the file >>>> call MPI_FILE_CLOSE(iunit,ierr) >>>> >>>> and the read is: >>>> >>>> ncells = parallel_sum(ncells_hexa_,ncells) >>>> >>>> allocate(buffer3(ncells,3)) >>>> >>>> openfile=trim(workdir)//'/'//'V/V.000002' >>>> call MPI_FILE_OPEN(comm,openfile,MPI_MODE_RDONLY,mpi_info,iunit,ierr) >>>> >>>> ! Read header >>>> bsize = 80 >>>> call MPI_FILE_READ(iunit,cbuffer,bsize,MPI_CHARACTER,status,ierr) >>>> print*,trim(cbuffer) >>>> bsize = 80 >>>> call MPI_FILE_READ(iunit,cbuffer,bsize,MPI_CHARACTER,status,ierr) >>>> print*,trim(cbuffer) >>>> bsize = 1 >>>> call MPI_FILE_READ(iunit,ibuffer,bsize,MPI_INTEGER,status,ierr) >>>> print*,ibuffer >>>> bsize = 80 >>>> call MPI_FILE_READ(iunit,cbuffer,bsize,MPI_CHARACTER,status,ierr) >>>> print*,trim(cbuffer),ncells >>>> >>>> ! Read the data >>>> disp = 3*80+4+0*ncells*4 >>>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>>> call MPI_FILE_READ_ALL(iunit,buffer3(:,1),ncells,MPI_REAL_SP,status,ierr) >>>> disp = 3*80+4+1*ncells*4 >>>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>>> call MPI_FILE_READ_ALL(iunit,buffer3(:,2),ncells,MPI_REAL_SP,status,ierr) >>>> disp = 3*80+4+2*ncells*4 >>>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>>> call MPI_FILE_READ_ALL(iunit,buffer3(:,3),ncells,MPI_REAL_SP,status,ierr) >>>> >>>> ! Close the file >>>> call MPI_FILE_CLOSE(iunit,ierr) >>>> >>>> On Mar 28, 2013, at 8:48 AM, Rob Latham wrote: >>>> >>>>> On Thu, Mar 28, 2013 at 08:19:23AM -0700, Ryan Crocker wrote: >>>>>> from the file those points are most definitely not in the same spot. We're using a set view so the memory is stored non-contiguously so you can't just read it back in, and i can see the offset in the data blocks. >>>>> >>>>> If you write the data with MPI_File_write_at, then you should read it >>>>> back with MPI_File_read_at. Fortran I/O will expect the input to have various >>>>> compiler-specific padding or ... well I don't properly know the >>>>> details, other than it's a mess. >>>>> >>>>> ==rob >>>>> >>>>> >>>>>> >>>>>> On Mar 28, 2013, at 6:47 AM, Rajeev Thakur wrote: >>>>>> >>>>>>> Most MPI-IO implementations write plain binary files without any additional information. So the data should be exactly what you wrote and at the locations you specified. The disp is not stored in the file. It is just used to offset to the right location. The file won't be portable with the default "native" file format used for writing. In other words, it will be readable only on the same type of machine architecture. >>>>>>> >>>>>>> Try writing a simple MPI-IO program to write a file and then read it back using POSIX I/O calls (lseek, read) from a C program. I don't know if Fortran I/O expects files to be formatted in a particular way, which could cause a problem when reading from Fortran. >>>>>>> >>>>>>> Rajeev >>>>>>> >>>>>>> >>>>>>> On Mar 28, 2013, at 2:00 AM, Ryan Crocker wrote: >>>>>>> >>>>>>>> So i'm not sure if this is crazy or not, but i have file outputs from my code that write ensight gold files in MPI. Here is the write, >>>>>>>> >>>>>>>> disp = 3*80+4+0*ncells_hexa*4 >>>>>>>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>>>>>>> call MPI_FILE_WRITE_ALL(iunit,buffer3_hexa(:,1),ncells_hexa_,MPI_REAL_SP,status,ierr) >>>>>>>> >>>>>>>> if i wanted to read that in binary in fortran, or C (preferably fortran) what exactly would i need to do? I can't seem to write code that reads these in and produces anything that looks like the plot i get in paraview. I know that MPI write puts out each processor data vector with that disp in between them, but i just can't make that structure make sense to me when i try to read it in. >>>>>>>> >>>>>>>> Thanks for the help, >>>>>>>> >>>>>>>> Ryan Crocker >>>>>>>> University of Vermont, School of Engineering >>>>>>>> Mechanical Engineering Department >>>>>>>> rcrocker(a)uvm.edu >>>>>>>> 315-212-7331 >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> discuss mailing list discuss(a)mpich.org >>>>>>>> To manage subscription options or unsubscribe: >>>>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>>>> >>>>>>> _______________________________________________ >>>>>>> discuss mailing list discuss(a)mpich.org >>>>>>> To manage subscription options or unsubscribe: >>>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>>> >>>>>> Ryan Crocker >>>>>> University of Vermont, School of Engineering >>>>>> Mechanical Engineering Department >>>>>> rcrocker(a)uvm.edu >>>>>> 315-212-7331 >>>>>> >>>>>> _______________________________________________ >>>>>> discuss mailing list discuss(a)mpich.org >>>>>> To manage subscription options or unsubscribe: >>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>> >>>>> -- >>>>> Rob Latham >>>>> Mathematics and Computer Science Division >>>>> Argonne National Lab, IL USA >>>>> _______________________________________________ >>>>> discuss mailing list discuss(a)mpich.org >>>>> To manage subscription options or unsubscribe: >>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>> >>>> Ryan Crocker >>>> University of Vermont, School of Engineering >>>> Mechanical Engineering Department >>>> rcrocker(a)uvm.edu >>>> 315-212-7331 >>>> >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> Ryan Crocker >> University of Vermont, School of Engineering >> Mechanical Engineering Department >> rcrocker(a)uvm.edu >> 315-212-7331 >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss Ryan Crocker University of Vermont, School of Engineering Mechanical Engineering Department rcrocker(a)uvm.edu 315-212-7331

1 0

MPICH-3.0.2 Compilation error on 32 bit Arch Linux
by Mark E. Lee 28 Mar '13

28 Mar '13

To whom it may concern, I have failed to compile MPICH-3.0.2 on 32 bit Arch Linux with both single and multiple make threads. My configure options are : ====BEGIN CONFIGURE OPTIONS==== ./configure --prefix="/usr" \ --exec-prefix="/usr/lib" \ --bindir="/usr/bin" \ --sbindir="/usr/sbin" \ --sysconfdir="/etc/mpich" \ --includedir="/usr/include/mpich" \ --libdir="/usr/lib/mpich" \ --enable-shared \ --enable-fast=all,O3 ====END CONFIGURE OPTIONS==== See below for the error: ====BEGIN ERROR MESSAGE==== /home/mark/Downloads/mpich3/src/mpich-3.0.2/src/pm/hydra/tools/topo/hwloc/hwloc/include/private/cpuid.h: Assembler messages: /home/mark/Downloads/mpich3/src/mpich-3.0.2/src/pm/hydra/tools/topo/hwloc/hwloc/include/private/cpuid.h:39: Error: symbol `Lhwloc1' is already defined make[5]: *** [topology-x86.lo] Error 1 make[5]: Leaving directory `/home/mark/Downloads/mpich3/src/mpich-3.0.2/src/pm/hydra/tools/topo/hwloc/hwloc/src' make[4]: *** [all-recursive] Error 1 make[4]: Leaving directory `/home/mark/Downloads/mpich3/src/mpich-3.0.2/src/pm/hydra/tools/topo/hwloc/hwloc/src' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/home/mark/Downloads/mpich3/src/mpich-3.0.2/src/pm/hydra/tools/topo/hwloc/hwloc' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/home/mark/Downloads/mpich3/src/mpich-3.0.2/src/pm/hydra' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/home/mark/Downloads/mpich3/src/mpich-3.0.2' make: *** [all] Error 2 ====END ERROR MESSAGE==== Thanks, Mark -- Mark E. Lee <mark(a)markelee.com>

1 0

Re: [mpich-discuss] MPI IO, reading MPI Writes without MPI read
by Rajeev Thakur 28 Mar '13

28 Mar '13

The other thing I would try is use buffer3_hexa(1,1) instead of buffer3_hexa(:,1). Not sure what Fortran compilers do with the latter format, but there is discussion of issues with Fortran subscript notation on pgs 626-628 of the MPI-3 standard. On Mar 28, 2013, at 2:16 PM, Ryan Crocker wrote: > Yes. > > On Mar 28, 2013, at 12:11 PM, Rajeev Thakur wrote: > >> Have you declared disp as integer (kind=MPI_OFFSET_KIND) instead of just integer? >> >> On Mar 28, 2013, at 1:33 PM, Ryan Crocker wrote: >> >>> Okay, so i coded the MPI IO. All the characters and stuff show up, but when i try to read in my data buffers i just get the same number for the whole vector. All the characters in the header read in and are what they'er supposed to be, and the program finishes without any errors. >>> >>> my write is: >>> >>> call MPI_FILE_OPEN(comm,file,IOR(MPI_MODE_WRONLY,MPI_MODE_CREATE),mpi_info,iunit,ierr) >>> >>> ! Write header (only root) >>> if (irank.eq.iroot) then >>> buffer = trim(adjustl(name)) >>> size = 80 >>> call MPI_FILE_WRITE(iunit,buffer,size,MPI_CHARACTER,status,ierr) >>> buffer = 'part' >>> size = 80 >>> call MPI_FILE_WRITE(iunit,buffer,size,MPI_CHARACTER,status,ierr) >>> ibuffer = 1 >>> size = 1 >>> call MPI_FILE_WRITE(iunit,ibuffer,size,MPI_INTEGER,status,ierr) >>> buffer = 'hexa8' >>> size = 80 >>> call MPI_FILE_WRITE(iunit,buffer,size,MPI_CHARACTER,status,ierr) >>> end if >>> >>> ! Write the data >>> disp = 3*80+4+0*ncells_hexa*4 >>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>> call MPI_FILE_WRITE_ALL(iunit,buffer3_hexa(:,1),ncells_hexa_,MPI_REAL_SP,status,ierr) >>> disp = 3*80+4+1*ncells_hexa*4 >>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>> call MPI_FILE_WRITE_ALL(iunit,buffer3_hexa(:,2),ncells_hexa_,MPI_REAL_SP,status,ierr) >>> disp = 3*80+4+2*ncells_hexa*4 >>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>> call MPI_FILE_WRITE_ALL(iunit,buffer3_hexa(:,3),ncells_hexa_,MPI_REAL_SP,status,ierr) >>> >>> ! Close the file >>> call MPI_FILE_CLOSE(iunit,ierr) >>> >>> and the read is: >>> >>> ncells = parallel_sum(ncells_hexa_,ncells) >>> >>> allocate(buffer3(ncells,3)) >>> >>> openfile=trim(workdir)//'/'//'V/V.000002' >>> call MPI_FILE_OPEN(comm,openfile,MPI_MODE_RDONLY,mpi_info,iunit,ierr) >>> >>> ! Read header >>> bsize = 80 >>> call MPI_FILE_READ(iunit,cbuffer,bsize,MPI_CHARACTER,status,ierr) >>> print*,trim(cbuffer) >>> bsize = 80 >>> call MPI_FILE_READ(iunit,cbuffer,bsize,MPI_CHARACTER,status,ierr) >>> print*,trim(cbuffer) >>> bsize = 1 >>> call MPI_FILE_READ(iunit,ibuffer,bsize,MPI_INTEGER,status,ierr) >>> print*,ibuffer >>> bsize = 80 >>> call MPI_FILE_READ(iunit,cbuffer,bsize,MPI_CHARACTER,status,ierr) >>> print*,trim(cbuffer),ncells >>> >>> ! Read the data >>> disp = 3*80+4+0*ncells*4 >>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>> call MPI_FILE_READ_ALL(iunit,buffer3(:,1),ncells,MPI_REAL_SP,status,ierr) >>> disp = 3*80+4+1*ncells*4 >>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>> call MPI_FILE_READ_ALL(iunit,buffer3(:,2),ncells,MPI_REAL_SP,status,ierr) >>> disp = 3*80+4+2*ncells*4 >>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>> call MPI_FILE_READ_ALL(iunit,buffer3(:,3),ncells,MPI_REAL_SP,status,ierr) >>> >>> ! Close the file >>> call MPI_FILE_CLOSE(iunit,ierr) >>> >>> On Mar 28, 2013, at 8:48 AM, Rob Latham wrote: >>> >>>> On Thu, Mar 28, 2013 at 08:19:23AM -0700, Ryan Crocker wrote: >>>>> from the file those points are most definitely not in the same spot. We're using a set view so the memory is stored non-contiguously so you can't just read it back in, and i can see the offset in the data blocks. >>>> >>>> If you write the data with MPI_File_write_at, then you should read it >>>> back with MPI_File_read_at. Fortran I/O will expect the input to have various >>>> compiler-specific padding or ... well I don't properly know the >>>> details, other than it's a mess. >>>> >>>> ==rob >>>> >>>> >>>>> >>>>> On Mar 28, 2013, at 6:47 AM, Rajeev Thakur wrote: >>>>> >>>>>> Most MPI-IO implementations write plain binary files without any additional information. So the data should be exactly what you wrote and at the locations you specified. The disp is not stored in the file. It is just used to offset to the right location. The file won't be portable with the default "native" file format used for writing. In other words, it will be readable only on the same type of machine architecture. >>>>>> >>>>>> Try writing a simple MPI-IO program to write a file and then read it back using POSIX I/O calls (lseek, read) from a C program. I don't know if Fortran I/O expects files to be formatted in a particular way, which could cause a problem when reading from Fortran. >>>>>> >>>>>> Rajeev >>>>>> >>>>>> >>>>>> On Mar 28, 2013, at 2:00 AM, Ryan Crocker wrote: >>>>>> >>>>>>> So i'm not sure if this is crazy or not, but i have file outputs from my code that write ensight gold files in MPI. Here is the write, >>>>>>> >>>>>>> disp = 3*80+4+0*ncells_hexa*4 >>>>>>> call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) >>>>>>> call MPI_FILE_WRITE_ALL(iunit,buffer3_hexa(:,1),ncells_hexa_,MPI_REAL_SP,status,ierr) >>>>>>> >>>>>>> if i wanted to read that in binary in fortran, or C (preferably fortran) what exactly would i need to do? I can't seem to write code that reads these in and produces anything that looks like the plot i get in paraview. I know that MPI write puts out each processor data vector with that disp in between them, but i just can't make that structure make sense to me when i try to read it in. >>>>>>> >>>>>>> Thanks for the help, >>>>>>> >>>>>>> Ryan Crocker >>>>>>> University of Vermont, School of Engineering >>>>>>> Mechanical Engineering Department >>>>>>> rcrocker(a)uvm.edu >>>>>>> 315-212-7331 >>>>>>> >>>>>>> _______________________________________________ >>>>>>> discuss mailing list discuss(a)mpich.org >>>>>>> To manage subscription options or unsubscribe: >>>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>>> >>>>>> _______________________________________________ >>>>>> discuss mailing list discuss(a)mpich.org >>>>>> To manage subscription options or unsubscribe: >>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>> >>>>> Ryan Crocker >>>>> University of Vermont, School of Engineering >>>>> Mechanical Engineering Department >>>>> rcrocker(a)uvm.edu >>>>> 315-212-7331 >>>>> >>>>> _______________________________________________ >>>>> discuss mailing list discuss(a)mpich.org >>>>> To manage subscription options or unsubscribe: >>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>> >>>> -- >>>> Rob Latham >>>> Mathematics and Computer Science Division >>>> Argonne National Lab, IL USA >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>> >>> Ryan Crocker >>> University of Vermont, School of Engineering >>> Mechanical Engineering Department >>> rcrocker(a)uvm.edu >>> 315-212-7331 >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > Ryan Crocker > University of Vermont, School of Engineering > Mechanical Engineering Department > rcrocker(a)uvm.edu > 315-212-7331 > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0