Re: [mpich-discuss] MPI IO, reading MPI Writes without MPI read
Have you declared disp as integer (kind=MPI_OFFSET_KIND) instead of just integer? On Mar 28, 2013, at 1:33 PM, Ryan Crocker wrote:
Okay, so i coded the MPI IO. All the characters and stuff show up, but when i try to read in my data buffers i just get the same number for the whole vector. All the characters in the header read in and are what they'er supposed to be, and the program finishes without any errors.
my write is:
call MPI_FILE_OPEN(comm,file,IOR(MPI_MODE_WRONLY,MPI_MODE_CREATE),mpi_info,iunit,ierr)
! Write header (only root) if (irank.eq.iroot) then buffer = trim(adjustl(name)) size = 80 call MPI_FILE_WRITE(iunit,buffer,size,MPI_CHARACTER,status,ierr) buffer = 'part' size = 80 call MPI_FILE_WRITE(iunit,buffer,size,MPI_CHARACTER,status,ierr) ibuffer = 1 size = 1 call MPI_FILE_WRITE(iunit,ibuffer,size,MPI_INTEGER,status,ierr) buffer = 'hexa8' size = 80 call MPI_FILE_WRITE(iunit,buffer,size,MPI_CHARACTER,status,ierr) end if
! Write the data disp = 3*80+4+0*ncells_hexa*4 call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) call MPI_FILE_WRITE_ALL(iunit,buffer3_hexa(:,1),ncells_hexa_,MPI_REAL_SP,status,ierr) disp = 3*80+4+1*ncells_hexa*4 call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) call MPI_FILE_WRITE_ALL(iunit,buffer3_hexa(:,2),ncells_hexa_,MPI_REAL_SP,status,ierr) disp = 3*80+4+2*ncells_hexa*4 call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) call MPI_FILE_WRITE_ALL(iunit,buffer3_hexa(:,3),ncells_hexa_,MPI_REAL_SP,status,ierr)
! Close the file call MPI_FILE_CLOSE(iunit,ierr)
and the read is:
ncells = parallel_sum(ncells_hexa_,ncells)
allocate(buffer3(ncells,3))
openfile=trim(workdir)//'/'//'V/V.000002' call MPI_FILE_OPEN(comm,openfile,MPI_MODE_RDONLY,mpi_info,iunit,ierr)
! Read header bsize = 80 call MPI_FILE_READ(iunit,cbuffer,bsize,MPI_CHARACTER,status,ierr) print*,trim(cbuffer) bsize = 80 call MPI_FILE_READ(iunit,cbuffer,bsize,MPI_CHARACTER,status,ierr) print*,trim(cbuffer) bsize = 1 call MPI_FILE_READ(iunit,ibuffer,bsize,MPI_INTEGER,status,ierr) print*,ibuffer bsize = 80 call MPI_FILE_READ(iunit,cbuffer,bsize,MPI_CHARACTER,status,ierr) print*,trim(cbuffer),ncells
! Read the data disp = 3*80+4+0*ncells*4 call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) call MPI_FILE_READ_ALL(iunit,buffer3(:,1),ncells,MPI_REAL_SP,status,ierr) disp = 3*80+4+1*ncells*4 call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) call MPI_FILE_READ_ALL(iunit,buffer3(:,2),ncells,MPI_REAL_SP,status,ierr) disp = 3*80+4+2*ncells*4 call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) call MPI_FILE_READ_ALL(iunit,buffer3(:,3),ncells,MPI_REAL_SP,status,ierr)
! Close the file call MPI_FILE_CLOSE(iunit,ierr)
On Mar 28, 2013, at 8:48 AM, Rob Latham wrote:
On Thu, Mar 28, 2013 at 08:19:23AM -0700, Ryan Crocker wrote:
from the file those points are most definitely not in the same spot. We're using a set view so the memory is stored non-contiguously so you can't just read it back in, and i can see the offset in the data blocks.
If you write the data with MPI_File_write_at, then you should read it back with MPI_File_read_at. Fortran I/O will expect the input to have various compiler-specific padding or ... well I don't properly know the details, other than it's a mess.
==rob
On Mar 28, 2013, at 6:47 AM, Rajeev Thakur wrote:
Most MPI-IO implementations write plain binary files without any additional information. So the data should be exactly what you wrote and at the locations you specified. The disp is not stored in the file. It is just used to offset to the right location. The file won't be portable with the default "native" file format used for writing. In other words, it will be readable only on the same type of machine architecture.
Try writing a simple MPI-IO program to write a file and then read it back using POSIX I/O calls (lseek, read) from a C program. I don't know if Fortran I/O expects files to be formatted in a particular way, which could cause a problem when reading from Fortran.
Rajeev
On Mar 28, 2013, at 2:00 AM, Ryan Crocker wrote:
So i'm not sure if this is crazy or not, but i have file outputs from my code that write ensight gold files in MPI. Here is the write,
disp = 3*80+4+0*ncells_hexa*4 call MPI_FILE_SET_VIEW(iunit,disp,MPI_REAL_SP,fileview_hexa,"native",mpi_info,ierr) call MPI_FILE_WRITE_ALL(iunit,buffer3_hexa(:,1),ncells_hexa_,MPI_REAL_SP,status,ierr)
if i wanted to read that in binary in fortran, or C (preferably fortran) what exactly would i need to do? I can't seem to write code that reads these in and produces anything that looks like the plot i get in paraview. I know that MPI write puts out each processor data vector with that disp in between them, but i just can't make that structure make sense to me when i try to read it in.
Thanks for the help,
Ryan Crocker University of Vermont, School of Engineering Mechanical Engineering Department [email protected] 315-212-7331
_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss
_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss
Ryan Crocker University of Vermont, School of Engineering Mechanical Engineering Department [email protected] 315-212-7331
_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss
-- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA _______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss
Ryan Crocker University of Vermont, School of Engineering Mechanical Engineering Department [email protected] 315-212-7331
_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss
participants (1)
-
Rajeev Thakur