August 2019 - petsc-users - mailman3.cels.anl.gov

Nodes coordinates in distributed dmplex
by Alejandro D Otero 16 Jul '22

16 Jul '22

Hi, sorry if this is an obvious question, but I cannot figure out how to recover finite element nodes coordinates once I have distributed a mesh stored as a dmplex. I am using petsc4py as interface to petsc rutines. I first created a dmplex using: dm.createFromCellList() In a sequential run I got the coordinates with: Coords = dm.getCoordinates() which gave a sequential vector with the coordinates of the mesh nodes. When I distribute the mesh with: dm.distribute() each mpi process has it own dm but the indexing of the vector resulting from getCoordinates() or getCoordinatesLocal() seems not consistent with the local numbering of the cells and nodes. Which is the correct way of doing this in PETSc philosophy? Thanks in advance, Alejandro

4 9

Problem coupling Petsc into OpenFOAM
by Vu Do Quoc 26 Jun '20

26 Jun '20

Hi all, I am trying to insert Petsc to OpenFOAM opensource software. I have been successfully compiling Petsc with an available solver in OpenFOAM by linking it with the shared library libpetsc.so. However, when I call the solver to run a test case, I got an error saying that: "libpetsc.so cannot be found", even though the library still exists in the $PETSC_ARCH/lib folder. I have been struggling for weeks but still, have not been able to figure it out. Therefore I would be very grateful for any suggestion to solve this problem. Thanks in advance for your time, Best regards, Vu Do -- *Vu Q. Do*------------------------------------------------ *Student of Aeronautical Engineering* Programme de Formation d'Ingénieurs d'Excellence au Vietnam *- PFIEV* School of Transportation Engineering Hanoi University of Science and Technology 01 Dai Co Viet Avenue, Hanoi, Vietnam E-mail: vu.doquochust(a)gmail.com

7 11

Converting complex PDE to real for KNL performance ?
by Sajid Ali 15 Apr '20

15 Apr '20

Hi, I'm able to solve the following equation using complex numbers (with ts_type cn and pc_type gamg) : u_t = A*u'' + F_t*u; (where A = -1j/(2k) amd u'' refers to u_xx+u_yy implemented with the familiar 5-point stencil) Now, I want to solve the same problem using real numbers. The equivalent equations are: u_t_real = 1/(2k) * u''_imag + F_real*u_real - F_imag*u_imag u_t_imag = -1/(2k) * u''_real + F_imag*u_real - F_real*u_imag Thus, if we now take our new u vector to have twice the length of the problem we're solving, keeping the first half as real and the second half as imaginary, we'd get a matrix that had matrices computing the laplacian via the 5-point stencil in the top-right and bottom-left corners and a diagonal [F_real+F_imag, F_real-F_imag] term. I tried doing this and the gamg preconditioner complains about an unsymmetric matrix. If i use the default preconditioner, I get DIVERGED_NONLINEAR_SOLVE. Is there a way to better organize the matrix ? PS: I'm trying to do this using only real numbers because I realized that the optimized avx-512 kernels for KNL are not implemented for complex numbers. Would that be implemented soon ? Thank You, Sajid Ali Applied Physics Northwestern University

7 15

MatGetDiagonal
by Nachiket Gokhale 09 Dec '19

09 Dec '19

Any chance of making this work in serial? http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetDiago… Not a show stopper, I am trying to get the diagonal of some small projected, dense matrices (which come from large sparse matrices). I am running in serial because 1) Since my projected matrices are small, and 2) PETSc does not do certain matrix multiplications involving a dense matrix in parallel, -Nachiket

4 5

CUDA-Aware MPI & PETSc
by David Gutzwiller 08 Oct '19

08 Oct '19

Hello, I'm currently using PETSc for the GPU acceleration of simple Krylov solver with GMRES, without preconditioning. This is within the framework of our in-house multigrid solver. I am getting a good GPU speedup on the finest grid level but progressively worse performance on each coarse level. This is not surprising, but I still hope to squeeze out some more performance, hopefully making it worthwhile to run some or all of the coarse grids on the GPU. I started investigating with nvprof / nsight and essentially came to the same conclusion that Xiangdong reported in a recent thread (July 16, "MemCpy (HtoD and DtoH) in Krylov solver"). My question is a follow-up to that thread: The MPI communication is staged from the host, which results in some H<->D transfers for every mat-vec operation. A CUDA-aware MPI implementation might avoid these transfers for communication between ranks that are assigned to the same accelerator. Has this been implemented or tested? In our solver we typically run with multiple MPI ranks all assigned to a single device, and running with a single rank is not really feasible as we still have a sizable amount of work for the CPU to chew through. Thus, I think quite a lot of the H<->D transfers could be avoided if I can skip the MPI staging on the host. I am quite new to PETSc so I wanted to ask around before blindly digging into this. Thanks for your help, David <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> Virus-free. www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> <#m_-3093947404852640465_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

2 5

Re: [petsc-users] Question about TSComputeRHSJacobianConstant
by Zhang, Hong 30 Sep '19

30 Sep '19

Sajid, I have also rested the simpler problem you provided. The branch hongzh/fix-computejacobian gives exactly the same numerical results as the master branch does, but runs much faster. So the solver seems to work correctly. To rule out the possible compiler issues, you might want to try a different compiler or different optimization flags. Also you might want to try smaller stepsizes. Hong On May 20, 2019, at 4:47 PM, Sajid Ali <sajidsyed2021(a)u.northwestern.edu<mailto:[email protected]>> wrote: Hi Hong, I tried running a simpler problem that solves the equation ` u_t = A*u_xx + A*u_yy; ` and the fix-computejacobian branch works for this on a coarse grid. The code for the same is here : https://github.com/s-sajid-ali/xwp_petsc/blob/master/2d/FD/free_space/ex_dm… and it requires no input file. It writes at all times steps and comparing the output at the last time step, everything looks fine. I want to eliminate a possible source of error which could be the fact that I installed both versions (3.11.2 and fix-computejacobian) with intel compilers and O3. Could floating point errors occur due to this ? I didn't specify -fp-model strict but since the results I got were reasonable I never bothered to run a test suite. Thank You, Sajid Ali Applied Physics Northwestern University

4 7

IMPORTANT PETSc repository changed from Bucketbit to GitLab
by Smith, Barry F. 06 Sep '19

06 Sep '19

PETSc folks. This announcement is for people who access PETSc from the BitBucket repository or post issues or have other activities with the Bitbucket repository We have changed the location of the PETSc repository from BitBucket to GitLab. For each copy of the repository you need to do git remote set-url origin git(a)gitlab.com<mailto:[email protected]>:petsc/petsc.git or git remote set-url origin https://gitlab.com/petsc/petsc.git You will likely also want to set up an account on gitlab and remember to set the ssh key information if you previously had write permission to the petsc repository and cannot write to the new repository please email bsmith(a)mcs.anl.gov<mailto:[email protected]> with your GitLab username and the email used Please do not make pull requests to the Gitlab site yet; we will be manually processing the PR from the BitBucket site over the next couple of days as we implement the testing. Please be patient, this is all new to use and it may take a few days to get out all the glitches. Thanks for your support Barry The reason for switching to GitLab is that it has a better testing system than BitBucket and Gitlab. We hope that will allow us to test and manage pull requests more rapidly, efficiently and accurately, thus allowing us to improve and add to PETSc more quickly.

4 9

Problem with TS and SNES VI
by Huck, Moritz 05 Sep '19

05 Sep '19

Hi, I am trying to solve a DAE with the ARKIMEX solver, which works mostly fine. The problem arises when some states go to unphysical values. I try to constrain my states with SNESVISetVariableBounds (through the petsc4py interface). But TS seems not respect this e.g. I have a state with is usually between 1 and 1e3 for which I set a lower bound of 1, but the state goes t0 -0.8 at some points. Are there some tolerances I have to set for VI or something like this? Best Regards, Moritz

2 1

Re: [petsc-users] Problem with TS and SNES VI
by Abhyankar, Shrirang G 02 Sep '19

02 Sep '19

For problems with constraints on the states, I would recommend trying the event functionality, TSEvent, that allows detection and location of discrete events, such as one that you have in your problem. https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/TS/TSSetEventH…. An example using TSEvent functionality: https://www.mcs.anl.gov/petsc/petsc-current/src/ts/examples/tutorials/ex40.… A brief intro to TSEvent can be found here<https://www.mcs.anl.gov/petsc/petsc-20/conference/Abhyankar_S.pdf>. Thanks, Shri From: petsc-users <petsc-users-bounces(a)mcs.anl.gov> on behalf of "Huck, Moritz via petsc-users" <petsc-users(a)mcs.anl.gov> Reply-To: "Huck, Moritz" <Moritz.Huck(a)rwth-aachen.de> Date: Monday, August 5, 2019 at 5:18 AM To: "petsc-users(a)mcs.anl.gov" <petsc-users(a)mcs.anl.gov> Subject: [petsc-users] Problem with TS and SNES VI Hi, I am trying to solve a DAE with the ARKIMEX solver, which works mostly fine. The problem arises when some states go to unphysical values. I try to constrain my states with SNESVISetVariableBounds (through the petsc4py interface). But TS seems not respect this e.g. I have a state with is usually between 1 and 1e3 for which I set a lower bound of 1, but the state goes t0 -0.8 at some points. Are there some tolerances I have to set for VI or something like this? Best Regards, Moritz

4 12

petsc on windows
by Sam Guo 30 Aug '19

30 Aug '19

Dear PETSc dev team, I am looking some tips porting petsc to windows. We have our mpi wrapper (so we can switch different mpi). I configure petsc using --with-mpi-lib and --with-mpi-include ./configure --with-cc="win32fe cl" --with-fc=0 --download-f2cblaslapack --with-mpi-lib=/home/xianzhongg/dev/star/lib/win64/intel18.3vc14/lib/StarMpiWrapper.lib --with-mpi-include=/home/xianzhongg/dev/star/base/src/mpi/include --with-shared-libaries=1 But I got error =============================================================================== Configuring PETSc to compile on your system =============================================================================== TESTING: check from config.libraries(config/BuildSystem/config/libraries.py:154) ******************************************************************************* UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details): ------------------------------------------------------------------------------- --with-mpi-lib=['/home/xianzhongg/dev/star/lib/win64/intel18.3vc14/lib/StarMpiWrapper.lib'] and --with-mpi-include=['/home/xianzhongg/dev/star/base/src/mpi/include'] did not work ******************************************************************************* To fix the configuration error, in config/BuildSystem/config/package.py, I removed self.executeTest(self.checkDependencies) self.executeTest(self.configureLibrary) self.executeTest(self.checkSharedLibrary) To link, I add my mpi wrapper to ${PTESTC_ARCH}/lib/petsc/conf/petscvariables: PCC_LINKER_FLAGS = -MD -wd4996 -Z7 /home/xianzhongg/dev/star/lib/win64/intel18.3vc14/lib/StarMpiWrapper.lib I got libpetstc.dll and libpetstc.lib. When I try to test it inside our code, PETSc somehow crates a duplicate of communicator with only 1 MPI process and PETSC_COMM_WORLD is set to 2. If I set PETSC_COMM_WORLD to 1 (our MPI_COMM_WORLD), PETSc is hanging. I am wondering if you could give me some tips how to debug this problem. BR, Sam

3 11