January 2024 - petsc-users - mailman3.cels.anl.gov

Spring 2024 PETSc meeting and release announcement 3.20
by Barry Smith 09 Nov '24

09 Nov '24

The May 23-24, 2024 PETSc annual meeting in Cologne Germany website is now available: https://cds.uni-koeln.de/en/workshops/petsc-2024/home Please register and start making your travel plans. We are pleased to announce the release of PETSc version 3.20.0 at https://petsc.org/release/download/ A list of the major changes and updates can be found at https://petsc.org/release/changes/320/ The release tarball location is changed from https://ftp.mcs.anl.gov/pub/petsc to https://web.cels.anl.gov/projects/petsc/download The final update to petsc-3.19 i.e petsc-3.19.6 is also available We recommend upgrading to PETSc 3.20.0 soon. As always, please report problems to petsc-maint(a)mcs.anl.gov and ask questions at petsc-users(a)mcs.anl.gov This release includes contributions from David Ham Iglesia Dolci Jack Betteridge Koki Sagiyama Marcin Rogowski Nathan Collier Aidan Hamilton Albert Cowie Barry Smith Blaise Bourdin Connor Ward David Andrs David Wells Duncan Campbell Hansol Suh Hong Zhang Jacob Faibussowitsch James Wright Jed Brown jeremy theler Joseph Pusztay Jose Roman Junchao Zhang Lisandro Dalcin Mark Adams Matthew Knepley Mr. Hong Zhang Pablo Brubeck Pierre Jolivet Pieter Ghysels Richard Tran Mills Satish Balay Sebastian Grimberg Stefano Zampini Suyash Tandon Toby Isaac YANG Zongze Zach Atkins and bug reports/proposed improvements received from Carl-Johan Thore Chris Douglas Daniel@OLCF Duan Junming Edoardo alinovi Philip Fackler Frederic Vi Getnet Betrie Aron Graham Hongrui Yu James Wright Jin Chen Jonas Lundgren Karsten Lettmann Boris Kaus LEONARDO MUTTI Maciej Lewkowicz maitri ksh Milan Pelletier Mr. Hong Zhang Ngoc Mai Monica Huynh Peng Wang Sébastien Gilles Daniel Taller Vibhas Mishra Wei Liu Li Xiaoye As always, thanks for your support, Barry

1 2

How to combine different element types into a single DMPlex?
by Eric Chamberland 01 Aug '24

01 Aug '24

Hi, while playing with DMPlexBuildFromCellListParallel, I noticed we have to specify "numCorners" which is a fixed value, then gives a fixed number of nodes for a series of elements. How can I then add, for example, triangles and quadrangles into a DMPlex? Thanks, Eric -- Eric Chamberland, ing., M. Ing Professionnel de recherche GIREF/Université Laval (418) 656-2131 poste 41 22 42

4 13

Help with Integrating PETSc into Fortran Groundwater Flow Simulation Code
by Shatanawi, Sawsan Muhammad 18 May '24

18 May '24

Hello everyone, I hope this email finds you well. My Name is Sawsan Shatanawi, and I am currently working on developing a Fortran code for simulating groundwater flow in a 3D system. The code involves solving a nonlinear system, and I have created the matrix to be solved using the PCG solver and Picard iteration. However, when I tried to assign it as a PETSc matrix I started getting a lot of error messages. I am kindly asking if someone can help me, I would be happy to share my code with him/her. Please find the attached file contains a list of errors I have gotten Thank you in advance for your time and assistance. Best regards, Sawsan

5 29

HDF5 corruption
by Adrian Croucher 13 May '24

13 May '24

hi, One of the users of my PETSc-based code has reported that HDF5 output files can be corrupted and unusable if e.g. the run is killed. I've just done a bit of reading about this and it appears to be a known issue with HDF5. Some people suggest flushing the HDF5 file periodically to help prevent data loss. I had a look at PetscViewerFlush() but it doesn't seem to be implemented for the HDF5 viewer- is that correct? I am currently calling PetscViewerHDF5Open() once at the start of the run and closing it at the end. Some other people suggest doing this before and after each write instead. Is there likely to be a significant performance penalty in doing that? I gather HDF5 journaling has been promised for a while to get around this problem but as far as I can see it hasn't materialised yet... Regards, Adrian -- Dr Adrian Croucher Senior Research Fellow Department of Engineering Science University of Auckland, New Zealand email: a.croucher(a)auckland.ac.nz tel: +64 (0)9 923 4611

4 17

Configure error while building PETSc with CUDA/MVAPICH2-GDR
by Sreeram R Venkat 19 Apr '24

19 Apr '24

I am trying to build PETSc with CUDA using the CUDA-Aware MVAPICH2-GDR. Here is my configure command: ./configure PETSC_ARCH=linux-c-debug-mvapich2-gdr --download-hypre --with-cuda=true --cuda-dir=$TACC_CUDA_DIR --with-hdf5=true --with-hdf5-dir=$TACC_PHDF5_DIR --download-elemental --download-metis --download-parmetis --with-cc=mpicc --with-cxx=mpicxx --with-fc=mpif90 which errors with: UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details): --------------------------------------------------------------------------------------------- CUDA compile failed with arch flags " -ccbin mpic++ -std=c++14 -Xcompiler -fPIC -Xcompiler -fvisibility=hidden -g -lineinfo -gencode arch=compute_80,code=sm_80" generated from "--with-cuda-arch=80" The same configure command works when I use the Intel MPI and I can build with CUDA. The full config.log file is attached. Please let me know if you need any other information. I appreciate your help with this. Thanks, Sreeram

6 15

Using PETSc GPU backend
by Ng, Cho-Kuen 13 Apr '24

13 Apr '24

I installed PETSc on Perlmutter using "spack install petsc+cuda+zoltan" and used it by "spack load petsc/fwge6pf". Then I compiled the application code (purely CPU code) linking to the petsc package, hoping that I can get performance improvement using the petsc GPU backend. However, the timing was the same using the same number of MPI tasks with and without GPU accelerators. Have I missed something in the process, for example, setting up PETSc options at runtime to use the GPU backend? Thanks, Cho

5 27

Hypre BoomerAMG settings options database
by Khurana, Parv 10 Apr '24

10 Apr '24

Hello PETSc users community, Happy new year! Thank you for the community support as always. I am using BoomerAMG for my research, and it is interfaced to my software via PETSc. I can only use options database keys as of now to tweak the settings I want for the AMG solve. I want to control the number of smoothener iterations at pre/post step for a given AMG cycle. I am looking for an options database key which helps me control this. I am not sure whether this is possible directly via the keys (Line 365: https://www.mcs.anl.gov/petsc/petsc-3.5.4/src/ksp/pc/impls/hypre/hypre.c.ht…) My comprehension of the current setup is that I have 1 smoothener iteration at every coarsening step. My aim is to do two pre and 2 post smoothening steps using the SSOR smoothener. BoomerAMG SOLVER PARAMETERS: Maximum number of cycles: 1 Stopping Tolerance: 0.000000e+00 Cycle type (1 = V, 2 = W, etc.): 1 Relaxation Parameters: Visiting Grid: down up coarse Number of sweeps: 1 1 1 Type 0=Jac, 3=hGS, 6=hSGS, 9=GE: 6 6 9 Point types, partial sweeps (1=C, -1=F): Pre-CG relaxation (down): 1 -1 Post-CG relaxation (up): -1 1 Coarsest grid: 0 PETSC settings I am using currently: -ksp_type preonly -pc_type hypre -pc_hypre_type boomeramg -pc_hypre_boomeramg_coarsen_type hmis -pc_hypre_boomeramg_relax_type_all symmetric-sor/jacobi -pc_hypre_boomeramg_strong_threshold 0.7 -pc_hypre_boomeramg_interp_type ext+i -pc_hypre_boomeramg_P_max 2 -pc_hypre_boomeramg_truncfactor 0.3 Thanks and Best Parv Khurana

4 5

Fortran Interface NULL object / Casting
by Nicholas Arnold-Medabalimi 23 Feb '24

23 Feb '24

Hi Petsc Users I am trying to use DMAddField in a Fortran code. I had some questions on casting/passing NULL. I follow how to pass NULL for standard types (INT, CHAR, etc). Is there a method/best practice for passing NULL for Petsc type arguments? (In this case DMAddLabel I'd want to pass NULL to a DMLabel Parameter not an intrinsic type) For the 2nd question, In some cases in C we would cast a more specific object to Petsc Object DMAddField(dm, NULL, (PetscObject)fvm); (where fvm is a PetscFVM type) Is there a method or practice to do the same in Fortran. Thanks Nicholas -- Nicholas Arnold-Medabalimi Ph.D. Candidate Computational Aeroscience Lab University of Michigan

3 2

Swarm view HDF5
by Mark Adams 21 Feb '24

21 Feb '24

I am trying to view a DMSwarm with: -weights_view hdf5:part.h5 Vec f; PetscCall(DMSetOutputSequenceNumber(sw, 0, 0.0)); PetscCall(DMSwarmCreateGlobalVectorFromField(sw, "w_q", &f)); PetscCall(PetscObjectSetName((PetscObject)f, "particle weights")); PetscCall(VecViewFromOptions(f, NULL, "-weights_view")); PetscCall(DMSwarmDestroyGlobalVectorFromField(sw, "w_q", &f)); And I get this error. I had this working once and did not set PetscViewerHDF5PushTimestepping, so I wanted to check. Thanks, Mark [0]PETSC ERROR: Object is in wrong state [0]PETSC ERROR: Timestepping has not been pushed yet. Call PetscViewerHDF5PushTimestepping() first [0]PETSC ERROR: WARNING! There are unused option(s) set! Could be the program crashed before usage or a spelling mistake, etc! [0]PETSC ERROR: Option left: name:-options_left (no value) source: command line [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting. [0]PETSC ERROR: Petsc Development GIT revision: v3.20.3-461-g585a01bd688 GIT Date: 2024-01-16 23:32:45 +0000 [0]PETSC ERROR: ./ex30k on a arch-macosx-gnu-O named MarksMac-302.local by markadams Thu Jan 18 10:05:53 2024 [0]PETSC ERROR: Configure options CFLAGS="-g -Wno-deprecated-declarations " CXXFLAGS="-g -Wno-deprecated-declarations " COPTFLAGS=-O CXXOPTFLAGS=-O --with-cc=/usr/local/opt/llvm/bin/clang --with-cxx=/usr/local/opt/llvm/bin/clang++ --download-mpich --with-strict-petscerrorcode --download-triangle=1 --with-debugging=0 --download-hdf5=1 PETSC_ARCH=arch-macosx-gnu-O [0]PETSC ERROR: #1 PetscViewerHDF5SetTimestep() at /Users/markadams/Codes/petsc/src/sys/classes/viewer/impls/hdf5/hdf5v.c:990 [0]PETSC ERROR: #2 VecView_Swarm_HDF5_Internal() at /Users/markadams/Codes/petsc/src/dm/impls/swarm/swarm.c:45 [0]PETSC ERROR: #3 VecView_Swarm() at /Users/markadams/Codes/petsc/src/dm/impls/swarm/swarm.c:86 [0]PETSC ERROR: #4 VecView() at /Users/markadams/Codes/petsc/src/vec/vec/interface/vector.c:806 [0]PETSC ERROR: #5 PetscObjectView() at /Users/markadams/Codes/petsc/src/sys/objects/destroy.c:76 [0]PETSC ERROR: #6 PetscObjectViewFromOptions() at /Users/markadams/Codes/petsc/src/sys/objects/destroy.c:128 [0]PETSC ERROR: #7 VecViewFromOptions() at /Users/markadams/Codes/petsc/src/vec/vec/interface/vector.c:691

2 4

Unique number in each element of a DMPlex mesh
by Berend van Wachem 09 Feb '24

09 Feb '24

Dear Petsc-Team, Is there a good way to define a unique integer number in each element (e.g. a cell) of a DMPlex mesh, which is in the same location, regardless of the number of processors or the distribution of the mesh over the processors? So, for instance, if I have a DMPlex box mesh, the top-right-front corner element (e.g. cell) will always have the same unique number, regardless of the number of processors the mesh is distributed over? I want to be able to link the results I have achieved with a mesh from DMPlex on a certain number of cores to the same mesh from a DMPlex on a different number of cores. Of course, I could make a tree based on the distance of each element to a certain point (based on the X,Y,Z co-ordinates of the element), and go through this tree in the same way and define an integer based on this, but that seems rather cumbersome. Thanks and best regards, Berend.

2 9