Error while reading/writing petsc file
Hello, I am writing a vector on a petsc binary file and subsequently reading the same in another part of the code. This works well for smaller vector sizes but fails for a large vector (vectorsize=2048000000; PetscScalar=PetscComplex). For a large vector, when the read vector is probed for values, I seems to me that it is populated with uninitialized values. This size is close to the int32 limit (2^31 -1) but does not really breach it. Hence i am nt using 64 bit integer. Can you please help me figure out what might be going wrong here.? I'm appending with this mail snippets of the code which read wnd write the petsc file. //********************************************************************************************************************************************* // // Snippet which writes v (a vector constructed using global size) to the file. Note that this snippet is part of a parallel code // PetscViewer viewer; error = PetscViewerBinaryOpen(PETSC_COMM_WORLD,fileName.c_str(),FILE_MODE_WRITE,&viewer); checkPETScError(error, "unbale to open binary file"); error = VecView(v,viewer); checkPETScError(error, "unable to use VecView"); error = PetscViewerDestroy(&viewer); checkPETScError(error,"unable to destry viewer"); // // cleanup // error = VecDestroy(&v); checkPETScError(error,"unable to destroy vector"); error = PetscOptionsSetValue(NULL,"-viewer_binary_mpiio",""); checkPETScError(error, "error in PetscOptionsSetValue"); //******************************************************************************************************************************* // // Snippet which reads v (this is part of a serial code) // PetscViewer viewer; error = PetscViewerBinaryOpen(PETSC_COMM_WORLD,fileName.c_str(),FILE_MODE_READ,&viewer); checkPETScError(error, "unbale to open binary file"); error = VecLoad(v,viewer); checkPETScError(error, "unable to use VecView"); error = PetscViewerDestroy(&viewer); checkPETScError(error,"unable to destry viewer"); // // write to console // //error = VecView(v,PETSC_VIEWER_STDOUT_WORLD); //checkPETScError(error, "unable to write to console"); // // copy petsc vec to stl vector // std::vector<std::complex<double> > dummy(vecSize); PetscScalar *pointerX; VecGetArray(v, &pointerX); std::copy(&(pointerX[0]),&(pointerX[vecSize]),dummy.begin()); VecRestoreArray(v, &pointerX); // subsequently dummy is written to an stl vector. //********************************************************************************************************************************************* Thanks, - Nelson David Rufus PhD Student Computational Materials Physics Group Mechanical Engineering University of Michigan
On Mon, Mar 5, 2018 at 1:52 PM, Nelson David Rufus <[email protected]> wrote:
Hello,
I am writing a vector on a petsc binary file and subsequently reading the same in another part of the code. This works well for smaller vector sizes but fails for a large vector (vectorsize=2048000000 <(204)%20800-0000>; PetscScalar=PetscComplex). For a large vector, when the read vector is probed for values, I seems to me that it is populated with uninitialized values.
The number of bytes written will not fit in a 32-bit integer. We do not exhaustively check for problems. I recommend reconfiguring with 64-bit ints and retrying. Thanks, Matt
This size is close to the int32 limit (2^31 -1) but does not really breach it. Hence i am nt using 64 bit integer. Can you please help me figure out what might be going wrong here.? I'm appending with this mail snippets of the code which read wnd write the petsc file. //********************************************************** ************************************************************ *********************** // // Snippet which writes v (a vector constructed using global size) to the file. Note that this snippet is part of a parallel code // PetscViewer viewer; error = PetscViewerBinaryOpen(PETSC_COMM_WORLD,fileName.c_str(), FILE_MODE_WRITE,&viewer); checkPETScError(error, "unbale to open binary file");
error = VecView(v,viewer); checkPETScError(error, "unable to use VecView");
error = PetscViewerDestroy(&viewer); checkPETScError(error,"unable to destry viewer");
// // cleanup // error = VecDestroy(&v); checkPETScError(error,"unable to destroy vector");
error = PetscOptionsSetValue(NULL,"-viewer_binary_mpiio",""); checkPETScError(error, "error in PetscOptionsSetValue");
//********************************************************** ********************************************************************* // // Snippet which reads v (this is part of a serial code) // PetscViewer viewer; error = PetscViewerBinaryOpen(PETSC_COMM_WORLD,fileName.c_str(), FILE_MODE_READ,&viewer); checkPETScError(error, "unbale to open binary file");
error = VecLoad(v,viewer); checkPETScError(error, "unable to use VecView");
error = PetscViewerDestroy(&viewer); checkPETScError(error,"unable to destry viewer");
// // write to console // //error = VecView(v,PETSC_VIEWER_STDOUT_WORLD); //checkPETScError(error, "unable to write to console");
// // copy petsc vec to stl vector // std::vector<std::complex<double> > dummy(vecSize); PetscScalar *pointerX; VecGetArray(v, &pointerX); std::copy(&(pointerX[0]),&(pointerX[vecSize]),dummy.begin()); VecRestoreArray(v, &pointerX);
// subsequently dummy is written to an stl vector. //********************************************************** ************************************************************ ***********************
Thanks, - Nelson David Rufus PhD Student Computational Materials Physics Group Mechanical Engineering University of Michigan
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.caam.rice.edu/~mk51/>
Thanks Matt. I will try to use a 32 bit integer to see if the problem goes away. I had a query about compatibility of while reading and writing files compiled with different int varaints. For example, in the past, I've noticed that a file written using an int32 petsc installation (using code similar to below) had trouble being read by an int64 petsc installation. Is this an issue? -- Nelson David Rufus PhD Student Computational Materials Physics Group Mechanical Engineering University of Michigan On Mon, Mar 5, 2018 at 1:56 PM, Matthew Knepley <[email protected]> wrote:
On Mon, Mar 5, 2018 at 1:52 PM, Nelson David Rufus <[email protected]> wrote:
Hello,
I am writing a vector on a petsc binary file and subsequently reading the same in another part of the code. This works well for smaller vector sizes but fails for a large vector (vectorsize=2048000000 <(204)%20800-0000>; PetscScalar=PetscComplex). For a large vector, when the read vector is probed for values, I seems to me that it is populated with uninitialized values.
The number of bytes written will not fit in a 32-bit integer. We do not exhaustively check for problems. I recommend reconfiguring with 64-bit ints and retrying.
Thanks,
Matt
This size is close to the int32 limit (2^31 -1) but does not really breach it. Hence i am nt using 64 bit integer. Can you please help me figure out what might be going wrong here.? I'm appending with this mail snippets of the code which read wnd write the petsc file. //********************************************************** ************************************************************ *********************** // // Snippet which writes v (a vector constructed using global size) to the file. Note that this snippet is part of a parallel code // PetscViewer viewer; error = PetscViewerBinaryOpen(PETSC_CO MM_WORLD,fileName.c_str(),FILE_MODE_WRITE,&viewer); checkPETScError(error, "unbale to open binary file");
error = VecView(v,viewer); checkPETScError(error, "unable to use VecView");
error = PetscViewerDestroy(&viewer); checkPETScError(error,"unable to destry viewer");
// // cleanup // error = VecDestroy(&v); checkPETScError(error,"unable to destroy vector");
error = PetscOptionsSetValue(NULL,"-viewer_binary_mpiio",""); checkPETScError(error, "error in PetscOptionsSetValue");
//********************************************************** ********************************************************************* // // Snippet which reads v (this is part of a serial code) // PetscViewer viewer; error = PetscViewerBinaryOpen(PETSC_COMM_WORLD,fileName.c_str(),FILE _MODE_READ,&viewer); checkPETScError(error, "unbale to open binary file");
error = VecLoad(v,viewer); checkPETScError(error, "unable to use VecView");
error = PetscViewerDestroy(&viewer); checkPETScError(error,"unable to destry viewer");
// // write to console // //error = VecView(v,PETSC_VIEWER_STDOUT_WORLD); //checkPETScError(error, "unable to write to console");
// // copy petsc vec to stl vector // std::vector<std::complex<double> > dummy(vecSize); PetscScalar *pointerX; VecGetArray(v, &pointerX); std::copy(&(pointerX[0]),&(pointerX[vecSize]),dummy.begin()); VecRestoreArray(v, &pointerX);
// subsequently dummy is written to an stl vector. //********************************************************** ************************************************************ ***********************
Thanks, - Nelson David Rufus PhD Student Computational Materials Physics Group Mechanical Engineering University of Michigan
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.caam.rice.edu/~mk51/>
On Mon, Mar 5, 2018 at 2:10 PM, Nelson David Rufus <[email protected]> wrote:
Thanks Matt. I will try to use a 32 bit integer to see if the problem goes away. I had a query about compatibility of while reading and writing files compiled with different int varaints. For example, in the past, I've noticed that a file written using an int32 petsc installation (using code similar to below) had trouble being read by an int64 petsc installation. Is this an issue?
Yes, definitely. What we write is not intraconvertible. We could write headers for things that have the integer size, but right now we do not. Thanks, Matt
-- Nelson David Rufus PhD Student Computational Materials Physics Group Mechanical Engineering University of Michigan
On Mon, Mar 5, 2018 at 1:56 PM, Matthew Knepley <[email protected]> wrote:
On Mon, Mar 5, 2018 at 1:52 PM, Nelson David Rufus <[email protected]> wrote:
Hello,
I am writing a vector on a petsc binary file and subsequently reading the same in another part of the code. This works well for smaller vector sizes but fails for a large vector (vectorsize=2048000000 <(204)%20800-0000>; PetscScalar=PetscComplex). For a large vector, when the read vector is probed for values, I seems to me that it is populated with uninitialized values.
The number of bytes written will not fit in a 32-bit integer. We do not exhaustively check for problems. I recommend reconfiguring with 64-bit ints and retrying.
Thanks,
Matt
This size is close to the int32 limit (2^31 -1) but does not really breach it. Hence i am nt using 64 bit integer. Can you please help me figure out what might be going wrong here.? I'm appending with this mail snippets of the code which read wnd write the petsc file. //********************************************************** ************************************************************ *********************** // // Snippet which writes v (a vector constructed using global size) to the file. Note that this snippet is part of a parallel code // PetscViewer viewer; error = PetscViewerBinaryOpen(PETSC_CO MM_WORLD,fileName.c_str(),FILE_MODE_WRITE,&viewer); checkPETScError(error, "unbale to open binary file");
error = VecView(v,viewer); checkPETScError(error, "unable to use VecView");
error = PetscViewerDestroy(&viewer); checkPETScError(error,"unable to destry viewer");
// // cleanup // error = VecDestroy(&v); checkPETScError(error,"unable to destroy vector");
error = PetscOptionsSetValue(NULL,"-viewer_binary_mpiio",""); checkPETScError(error, "error in PetscOptionsSetValue");
//********************************************************** ********************************************************************* // // Snippet which reads v (this is part of a serial code) // PetscViewer viewer; error = PetscViewerBinaryOpen(PETSC_COMM_WORLD,fileName.c_str(),FILE _MODE_READ,&viewer); checkPETScError(error, "unbale to open binary file");
error = VecLoad(v,viewer); checkPETScError(error, "unable to use VecView");
error = PetscViewerDestroy(&viewer); checkPETScError(error,"unable to destry viewer");
// // write to console // //error = VecView(v,PETSC_VIEWER_STDOUT_WORLD); //checkPETScError(error, "unable to write to console");
// // copy petsc vec to stl vector // std::vector<std::complex<double> > dummy(vecSize); PetscScalar *pointerX; VecGetArray(v, &pointerX); std::copy(&(pointerX[0]),&(pointerX[vecSize]),dummy.begin()); VecRestoreArray(v, &pointerX);
// subsequently dummy is written to an stl vector. //********************************************************** ************************************************************ ***********************
Thanks, - Nelson David Rufus PhD Student Computational Materials Physics Group Mechanical Engineering University of Michigan
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.caam.rice.edu/~mk51/>
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.caam.rice.edu/~mk51/>
On Mar 5, 2018, at 1:10 PM, Nelson David Rufus <[email protected]> wrote:
Thanks Matt. I will try to use a 32 bit integer to see if the problem goes away. I had a query about compatibility of while reading and writing files compiled with different int varaints. For example, in the past, I've noticed that a file written using an int32 petsc installation (using code similar to below) had trouble being read by an int64 petsc installation. Is this an issue?
You have to read and write with the same PETSc configure options, indice size and scalar type (complex or real) and precision much match.
-- Nelson David Rufus PhD Student Computational Materials Physics Group Mechanical Engineering University of Michigan
On Mon, Mar 5, 2018 at 1:56 PM, Matthew Knepley <[email protected]> wrote: On Mon, Mar 5, 2018 at 1:52 PM, Nelson David Rufus <[email protected]> wrote: Hello,
I am writing a vector on a petsc binary file and subsequently reading the same in another part of the code. This works well for smaller vector sizes but fails for a large vector (vectorsize=2048000000; PetscScalar=PetscComplex). For a large vector, when the read vector is probed for values, I seems to me that it is populated with uninitialized values.
The number of bytes written will not fit in a 32-bit integer. We do not exhaustively check for problems. I recommend reconfiguring with 64-bit ints and retrying.
Thanks,
Matt
This size is close to the int32 limit (2^31 -1) but does not really breach it. Hence i am nt using 64 bit integer. Can you please help me figure out what might be going wrong here.? I'm appending with this mail snippets of the code which read wnd write the petsc file. //********************************************************************************************************************************************* // // Snippet which writes v (a vector constructed using global size) to the file. Note that this snippet is part of a parallel code // PetscViewer viewer; error = PetscViewerBinaryOpen(PETSC_COMM_WORLD,fileName.c_str(),FILE_MODE_WRITE,&viewer); checkPETScError(error, "unbale to open binary file");
error = VecView(v,viewer); checkPETScError(error, "unable to use VecView");
error = PetscViewerDestroy(&viewer); checkPETScError(error,"unable to destry viewer");
// // cleanup // error = VecDestroy(&v); checkPETScError(error,"unable to destroy vector");
error = PetscOptionsSetValue(NULL,"-viewer_binary_mpiio",""); checkPETScError(error, "error in PetscOptionsSetValue");
//******************************************************************************************************************************* // // Snippet which reads v (this is part of a serial code) // PetscViewer viewer; error = PetscViewerBinaryOpen(PETSC_COMM_WORLD,fileName.c_str(),FILE_MODE_READ,&viewer); checkPETScError(error, "unbale to open binary file");
error = VecLoad(v,viewer); checkPETScError(error, "unable to use VecView");
error = PetscViewerDestroy(&viewer); checkPETScError(error,"unable to destry viewer");
// // write to console // //error = VecView(v,PETSC_VIEWER_STDOUT_WORLD); //checkPETScError(error, "unable to write to console");
// // copy petsc vec to stl vector // std::vector<std::complex<double> > dummy(vecSize); PetscScalar *pointerX; VecGetArray(v, &pointerX); std::copy(&(pointerX[0]),&(pointerX[vecSize]),dummy.begin()); VecRestoreArray(v, &pointerX);
// subsequently dummy is written to an stl vector. //*********************************************************************************************************************************************
Thanks, - Nelson David Rufus PhD Student Computational Materials Physics Group Mechanical Engineering University of Michigan
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
participants (3)
-
Matthew Knepley -
Nelson David Rufus -
Smith, Barry F.