November 2010 - petsc-users - mailman3.cels.anl.gov

Global DA index using DAGetCorners
by khalid ashraf 25 Nov '10

25 Nov '10

It doesn't seem like the problem is with the natural ordering. Since in Fig.9 of the manual, the highest number of the grid should be the same(30 in fig. 9) for natural and petsc ordering. But I get a lower number when I print it. I also tried the petsc to natural conversion using the following code, { ia[0]=(PetscInt)((k)*(my)*(mx)+(j)*(mx)+(i)); AOPetscToApplication(ao,1,ia); w_localptr[k][j][i] = ia[0]; PetscPrintf(PETSC_COMM_WORLD,"%d\n",ia[0]); } But this gives even lower number for the highest index. Anyway my goal is to be able to determine the global natural index of the grid within the for loops in the following way: for (k=zs; k<zs+zm; k++) { for (j=ys; j<ys+ym; j++) { for (i=xs; i<xs+xm; i++) { natural_global_coordinate(k',j',i')=f(k,j,i); if natural_global_coord[k'][j'][i'] > half of the grid along x {Do something} if natural_global_coord[k'][j'][i'] > half of the grid along y {Do something} etc.. }}} Could you please suggest what would be the simplest way to achieve this. Thanks in advance.

2 1

DAGetCorners; xs,ys,zs return 0 for all the processors
by khalid ashraf 25 Nov '10

25 Nov '10

Here is the call to DA creation: ierr = DACreate3d(PETSC_COMM_WORLD,DA_NONPERIODIC,DA_STENCIL_BOX,appctx.l,appctx.m,appctx.n, PETSC_DECIDE,PETSC_DECIDE,PETSC_DECIDE,1,1,PETSC_NULL,PETSC_NULL,PETSC_NULL, &appctx.da);CHKERRQ(ierr); Khalid

2 1

Global DA index using DAGetCorners
by khalid ashraf 25 Nov '10

25 Nov '10

Sorry for the spam. Just to illustrate the problem. The following code produces the following output when run on 4 processors. Code: ierr=DAGetCorners(appctx->da,&xs,&ys,&zs,&xm,&ym,&zm);CHKERRQ(ierr); for (k=zs; k<zs+zm; k++) { for (j=ys; j<ys+ym; j++) { for (i=xs; i<xs+xm; i++) { { Vec_A[k][j][i] = (PetscReal)(k*my*mx+j*mx+i); }}} Output: Vec_A (mx=16 X my=16 X mz=4): 0 1 2 34567 1617181920212223 3233343536373839 4849and so on I don't understand why does the array value jump to 16 after 7. Meaning why is the 8th element of the vector is 16. Thanks.

2 1

DAGetCorners; xs,ys,zs return 0 for all the processors
by khalid ashraf 25 Nov '10

25 Nov '10

Hi, I am having trouble with writing to global vector indexes using DA . I am using the following function to get the starting point of a particular processor. DAGetCorners(da,&xs,&ys,&zs,&xm,&ym,&zm); And then use it to iterate through the global indices. for (k=zs; k<zs+zm; k++) { for (j=ys; j<ys+ym; j++) { for (i=xs; i<xs+xm; i++){ if (i<(mx/2)) //left half of the grid { vector[k][j][i]=1; } else {vector[k][j][i]=0; } } } } But when I run this on multiple processors, xs, ys and zs come out to be 0 for all the processors. I expected them to be the starting point of the global DA index in a particular processor. Could anyone tell me whats the problem ? Am I missing something here. Thanks in advance. Best regards. Khalid

2 1

petsc rpath
by Jens-Malte Gottfried 24 Nov '10

24 Nov '10

Hi! I am trying to write a gentoo ebuild for petsc (my current work is already in the science overlay). Is there any possibility to disable writing the RPATH of the to-be-generated libpetsc.so? Currently, the library is built fine but RPATH is set to /usr/lib64:/usr/lib/gcc/x86_64-pc-linux-gnu/4.4.4:/usr/x86_64-pc-linux-gnu/lib which is not neccessary. Before trying to patch the build system, perhaps there is a simple configure option which I did not find. Thanks Jens-Malte Gottfried

4 5

Passing DMMG struct.
by Chris Bording 23 Nov '10

23 Nov '10

Hi I am trying to pass the DMMG struct to a function similar to what is in the examples. extern PetscErrorCode Myfunc(DMMG,Vec); int main(int argc,char **argv) { DMMG *dmmg; ...Vec b; Myfunc(dmmg,b) } PetscErrorCode Myfunc(DMMG dmmg, Vec b) { DA da = (DA)dmmg->dm; ..... } error: cannot convert ‘_n_DMMG**’ to ‘_n_DMMG*’ for argument ‘1’ to ‘PetscErrorCode Myfunc(_n_DMMG*, _p_Vec*) So how do I pass a DMMG defined structure into my function? Thanks Chris

5 6

Compile Errors in Make files
by Nunion 19 Nov '10

19 Nov '10

Hello, I recently started receiving the following error when trying to compile anything... makefile:16: /conf/variables: No such file or directory makefile:17: /conf/rules: No such file or directory makefile:858: /conf/test: No such file or directory make: *** No rule to make target `/conf/test'. Stop. The errors seem to be related to one another. I tried to hardcode this path, but the makefile is not picking up the path. I can't recall changing anything major Thanks! Mark

7 20

Trilinos-PETSc Interface
by Nunion 18 Nov '10

18 Nov '10

I am in the middle of configuring the PETSc/Trilinos interface. However, when I run 'make', the following error is being generated. I have testing and examples enabled. From my knowledge I have disabled all MPI features, i.e. "-D TPL_ENABLE_MPI:BOOL=OFF", however this error is happening while in the section, "Scanning dependencies of target epetraext". Any assistance is greatly appreciated. Thanks! Mark [ 57%] Building CXX object packages/epetraext/src/CMakeFiles/epetraext.dir/block/EpetraExt_MultiPointModelEvaluator.cpp.o [ 57%] Building CXX object packages/epetraext/src/CMakeFiles/epetraext.dir/block/EpetraExt_MultiSerialComm.cpp.o [ 57%] Building CXX object packages/epetraext/src/CMakeFiles/epetraext.dir/block/EpetraExt_BlockDiagMatrix.cpp.o [ 57%] Building CXX object packages/epetraext/src/CMakeFiles/epetraext.dir/block/EpetraExt_PointToBlockDiagPermute.cpp.o [ 57%] Building CXX object packages/epetraext/src/CMakeFiles/epetraext.dir/restrict/EpetraExt_RestrictedMultiVectorWrapper.cpp.o [ 57%] Building CXX object packages/epetraext/src/CMakeFiles/epetraext.dir/restrict/EpetraExt_RestrictedCrsMatrixWrapper.cpp.o [ 57%] Building CXX object packages/epetraext/src/CMakeFiles/epetraext.dir/petsc/EpetraExt_PETScAIJMatrix.cpp.o /home/walker/Desktop/PETSc/_NEW/petsc-3.1-p5/include/private/vecimpl.h:255: error: ‘MPI_Win’ does not name a type /home/walker/Desktop/trilinos/trilinos-10.4.1-Source/packages/epetraext/src/petsc/EpetraExt_PETScAIJMatrix.h: In member function ‘virtual int Epetra_PETScAIJMatrix::NumGlobalNonzeros() const’: /home/walker/Desktop/trilinos/trilinos-10.4.1-Source/packages/epetraext/src/petsc/EpetraExt_PETScAIJMatrix.h:232: warning: converting to ‘int’ from ‘const double’ /home/walker/Desktop/trilinos/trilinos-10.4.1-Source/packages/epetraext/src/petsc/EpetraExt_PETScAIJMatrix.cpp: In member function ‘virtual int Epetra_PETScAIJMatrix::ExtractDiagonalCopy(Epetra_Vector&) const’: /home/walker/Desktop/trilinos/trilinos-10.4.1-Source/packages/epetraext/src/petsc/EpetraExt_PETScAIJMatrix.cpp:243: error: ‘class Epetra_SerialComm’ has no member named ‘Comm’ /home/walker/Desktop/trilinos/trilinos-10.4.1-Source/packages/epetraext/src/petsc/EpetraExt_PETScAIJMatrix.cpp: In member function ‘virtual int Epetra_PETScAIJMatrix::Multiply(bool, const Epetra_MultiVector&, Epetra_MultiVector&) const’: /home/walker/Desktop/trilinos/trilinos-10.4.1-Source/packages/epetraext/src/petsc/EpetraExt_PETScAIJMatrix.cpp:292: error: ‘class Epetra_SerialComm’ has no member named ‘Comm’ /home/walker/Desktop/trilinos/trilinos-10.4.1-Source/packages/epetraext/src/petsc/EpetraExt_PETScAIJMatrix.cpp:293: error: ‘class Epetra_SerialComm’ has no member named ‘Comm’ /home/walker/Desktop/trilinos/trilinos-10.4.1-Source/packages/epetraext/src/petsc/EpetraExt_PETScAIJMatrix.cpp: In member function ‘virtual int Epetra_PETScAIJMatrix::LeftScale(const Epetra_Vector&)’: /home/walker/Desktop/trilinos/trilinos-10.4.1-Source/packages/epetraext/src/petsc/EpetraExt_PETScAIJMatrix.cpp:445: error: ‘class Epetra_SerialComm’ has no member named ‘Comm’ /home/walker/Desktop/trilinos/trilinos-10.4.1-Source/packages/epetraext/src/petsc/EpetraExt_PETScAIJMatrix.cpp: In member function ‘virtual int Epetra_PETScAIJMatrix::RightScale(const Epetra_Vector&)’: /home/walker/Desktop/trilinos/trilinos-10.4.1-Source/packages/epetraext/src/petsc/EpetraExt_PETScAIJMatrix.cpp:465: error: ‘class Epetra_SerialComm’ has no member named ‘Comm’ make[2]: *** [packages/epetraext/src/CMakeFiles/epetraext.dir/petsc/EpetraExt_PETScAIJMatrix.cpp.o] Error 1 make[1]: *** [packages/epetraext/src/CMakeFiles/epetraext.dir/all] Error 2 make: *** [all] Error 2 [walker@sys TB2]$

1 0

TS with inhomogeneous rhs
by Chetan Jhurani 18 Nov '10

18 Nov '10

Hello, Question in brief: what's a good way, using TS, to solve a system of ODEs which looks like this? M U_t = -K U + f(t) In addition, how can the code be kept nearly identical to allow different TS types to be used (EULER, BEULER, TSRUNGE_KUTTA, TSCRANK_NICHOLSON, etc)? I'd also like to take possible advantages of M and K being constants. Background and what I've tried: If f(t) were not present TSSetMatrices seems like the convenient way ahead. In fact, I can still use the same function in presence of f(t) for an explicit time-stepping method like RK or Euler if I use a MatShell for Arhs, and sneak in f(t) there. But I cannot use an implicit method like Backward Euler with a MatShell for Arhs easily. It needs MatFactor to be implemented for the shell matrix (at least). Is there something I'm missing? This is a common system of equation. Most TS examples solve u_t = u_xx, convenient for TSSetMatrices. ts/examples/tests/ex3.c looks like it should work for a system with f(t). But it needs the Jacobian matrix of RHS and it evaluates it using finite differences. In addition, for each time-step it computes an ILU factorization. Thanks, Chetan

3 3

Lengthy time to assemble the matrix
by Li, Zhisong (lizs) 18 Nov '10

18 Nov '10

Hi, Petsc Team, I noticed that the Petsc program often cost a lot of time to assemble a matrix, particularly when DOF is large. For example, I write a 3D structured CFD code with a linear system of 40 x 22 x 20 in dimension and DOF = 6. In a sequential computation, it takes more than 20 minutes to finish assembling the matrix before ksp operates. This same problem should take only a few seconds for a commercial CFD software to initialize. The processor is 2.67G Hz in speed with sufficient RAM (6GB). The largest matrix stencil is 19 points. I typically use the function: MatSetValuesStencil(A, 1, &abc, 19, def, val, INSERT_VALUES). I wonder the assembling can be faster if abc is an array of multiple matrix row coordinates. But in the case of DOF >1, this seems difficult to implement. Is this normal for Petsc or do we have any means to improve this? Thank you. Zhisong Li

3 2