October 2025 - petsc-users - mailman3.cels.anl.gov

Question about Pseudo Time Stepping
by Aldo Bonfiglioli 20 Apr '26

20 Apr '26

Hi there, I am using -ts_type pseudo to find steady solutions of the Euler/NS PDEs. It looks like FormRHSFunction is called twice with the exactly same value of the dependent variable (u) before invoking TSRHSJacobianFn and then solving the linear system with KSP (u is then updated) > 1 TS dt 0.141197 time 0.141197 > Calling FormRHSFunction; ||u|| = 60.489405479528003 > 463.15635675129079 62.813532026019146 > 5.9789502719351155 > Calling FormRHSFunction; ||u|| = 60.489405479528003 > 463.1563567512907962.8135320260191465.9789502719351155 > Calling TSRHSJacobianFn with ||u|| = 60.489405479528003 > 463.15635675129079 62.813532026019146 > 5.9789502719351155 > Linear solve converged due to CONVERGED_RTOL iterations 9 > Calling FormRHSFunction; ||u|| = > 60.489405476878737463.1563567560218662.8135320316169655.9789502710519811 > 2 TS dt 0.155917 time 0.297115 > Calling FormRHSFunction; ||u|| = > 60.489405476878737463.1563567560218662.8135320316169655.9789502710519811 > Calling FormRHSFunction; ||u|| = > 60.489405476878737463.1563567560218662.8135320316169655.9789502710519811 > Calling TSRHSJacobianFn with ||u|| = > 60.489405476878737463.1563567560218662.8135320316169655.9789502710519811 > Why is it so? Am I possibly misusing (or missing) some options? My .petscrc file is attached. Thank you, Aldo -- Dr. Aldo Bonfiglioli Associate professor of Fluid Mechanics Dipartimento di Ingegneria Universita' della Basilicata V.le dell'Ateneo Lucano, 10 85100 Potenza ITALY tel:+39.0971.205203 fax:+39.0971.205215 web:https://urldefense.us/v3/__http://docenti.unibas.it/site/home/docente.html?m=002423__;!!G_uCfscf7eWS!ci_aySHdF2Qx_B-37IOzELFDtcQRthJ2NItK-JNbfXB0M2_21B_z97WIfjxZUyKCVRRhd1mzL6b-rnCed_i8LLA56JvFdz7JBrw$

3 5

[WARNING: UNSCANNABLE EXTRACTION FAILED]GMRES plus BlockJacobi behave differently for seemlingy identical matrices
by Nils Schween 27 Nov '25

27 Nov '25

Dear PETSc users, Dear PETSc developers, in our software we are solving a linear system with PETSc using GMRES in conjunction with a BlockJacobi preconditioner, i.e. the default of the KSP object. We have two versions of the system matrix, say A and B. The difference between them is the non-zero pattern. The non-zero pattern of matrix B is a subset of the one of matrix A. Their values should be identical. We solve the linear system, using A yields a solution after some iterations, whereas using B does not converge. I created binary files of the two matrices, the right-hand side, and wrote a small PETSc programm, which loads them and demonstrates the issue. I attach the files to this email. We would like to understand why the solver-preconditioner combination works in case A and not in case B. Can you help us finding this out? To test if the two matrices are identical, I substracted them and computed the Frobenius norm of the result. It is zero. To give you more context, we solve a system of partial differential equations that models astrophysical plasmas. It is essentially a system of advection-reaction equations. We use a discontinuous Galerkin (dG) method. Our code relies on the finite element library library deal.ii and its PETSc interface. The system matrices A and B are the result of the (dG) discretisation. We GMRES with a BlockJaboci preconditioner, because we do not know any better. I tested the code I sent with PETSc 3.24.0 and 3.19.1 on my workstation, i.e. Linux home-desktop 6.17.2-arch1-1 #1 SMP PREEMPT_DYNAMIC Sun, 12 Oct 2025 12:45:18 +0000 x86_64 GNU/Linux I use OpenMPI 5.0.8 and I compiled with mpicc, which in my cases use gcc. In case you need more information. Please let me know. Any help is appreciated. Thank you, Nils -- Nils Schween Phone: +49 6221 516 557 Mail: nils.schween(a)mpi-hd.mpg.de PGP-Key: 4DD3DCC0532EE96DB0C1F8B5368DBFA14CB81849 Max Planck Institute for Nuclear Physics Astrophysical Plasma Theory (APT) Saupfercheckweg 1, D-69117 Heidelberg https://www.mpi-hd.mpg.de/mpi/en/research/scientific-divisions-and-groups/i…

3 5

Petsc + nvhpc
by howen 17 Nov '25

17 Nov '25

Dear All, I am interfacing our CFD code (Fortran + OpenACC) to Petsc. Since we use OpenACC the natural choice for us is to use Nvidia´s nvhpc compiler. The Gnu compiler does not work well and we do not have access to the Cray compiler. I already know that the latest version of Petsc does not compile with nvhpc, I am therefore using version 3.21. I get good results on the CPU both in serial and parallel (MPI). However, the GPU implementation, that is what we are interested in, only work correctly for the serial version. In parallel, the results are different. Even for a CG solve. I would like to know, if you have experience with the Nvidia compiler. I am particularly interested if you have already observed issues with it. Your opinion on whether to put further effort into trying to find a bug I may have introduced during the interfacing is highly appreciated. Best, Herbert Owen Senior Researcher, Dpt. Computer Applications in Science and Engineering Barcelona Supercomputing Center (BSC-CNS) Tel: +34 93 413 4038 Skype: herbert.owen https://urldefense.us/v3/__https://scholar.google.es/citations?user=qe5O2IY…

5 8

How to map global vector to natural vector
by Xu, Donghui 13 Nov '25

13 Nov '25

Dear PETSc Team, I am working with petsc4py for my model. I had some experience of using PETSc in Fortran. In Fortran, I used the following subroutines: call DMPlexCreateNaturalVector(dm, natural, ierr) call DMPlexNaturalToGlobalBegin(dm,natural,X,ierr) call DMPlexNaturalToGlobalEnd(dm,natural,X,ierr) However, I found there are no such interfaces in petsc4py. Can you advise me on how to get the global vector in natural order with DMPLEX in petsc4py? Thanks, Donghui

2 3

Call for Survey Participants: Exploring Teamwork and Software in Scientific Computing
by McInnes, Lois Curfman 28 Oct '25

28 Oct '25

Hello PETSc users – Our research team is conducting survey research to understand perceptions of teamwork, AI, and software in scientific computing. As a member of the scientific computing community, your perspective is invaluable. We are reaching out to invite you to share your point of view on teamwork and software in scientific computing via an online survey. You can complete this survey at a time and place of your choosing. Participation in this research will take approximately 15 minutes. If you are interested in contributing your perspectives to this research, use the following link to access the survey: https://urldefense.us/v3/__https://umt.co1.qualtrics.com/jfe/form/SV_6uicew… <https://urldefense.us/v3/__https://gcc02.safelinks.protection.outlook.com/?… >. Please reach out to our research team with any questions you may have pertaining to this study. Feel free to share this call for participants with your network. The survey is open to all adults who work in scientific computing. This survey will remain open until mid-December. Thank you very much for your time and consideration, Research Team Olivia B. Newton, University of Montana (olivia.newton(a)umt.edu<mailto:[email protected]>) Lois Curfman McInnes, Argonne National Laboratory (curfman(a)anl.gov<mailto:[email protected]>) Anshu Dubey, Argonne National Laboratory (adubey(a)anl.gov<mailto:[email protected]>) Denice Ward Hood, University of Illinois (dwhood(a)illinois.edu<mailto:[email protected]>)

1 0

Options database in petsc4py
by SCOTTO Alexandre 27 Oct '25

27 Oct '25

Dear PETSc Community, In my developments, I am managing possibly several KSP solvers with options handled by the Options database. During my tests, I encountered the following behavior: Code: options = PETSc.Options("ksp_") options.setValue("atol", 7e-8) options.view() options.clear() options.view() Output: #PETSc Option Table entries: -ksp_atol 7e-08 # (source: code) #End of PETSc Option Table entries #PETSc Option Table entries: -ksp_atol 7e-08 # (source: code) #End of PETSc Option Table entries It seems that the clear() method does not really clear the Option database. To ensure that the several KSP I deal with are set with their own options (without getting options from a KSP previously set), the only way I found was to explicitly call the delValue() method for all the option keys passed: 1. Iterate over a dictionary of options and use setValue(name, value) 2. Set the KSP with option database: KSP.setFromOptions() 3. Iterate over a the keys of the dictionary and use delValue(name) to effectively clear the option database. Does it seem normal to you, is there something I am missing out? Regards, Alexandre Scotto.

3 6

interpreting petsc streams result
by Klaij, Christiaan 23 Oct '25

23 Oct '25

Attached is a petsc streams result kindly provided by a hardware vendor for a single compute node, dual socket, with two AMD epyc 9355 processors. Each processor has 32 cores, 12 DDR5 memory channels and mem BW around 600 GB/s. * It is not immediately clear which line corresponds to which y-axis. Could future versions of petsc please color the axis label with the matching line color? * Why would the achieved bandwidth be roughly 0.9 x 1e6 MB/s = 900 GB/s and not closer to 1200 GB/s? * The speed-up seems to be 12 out of 64, provided multiples of 8 cores are used. As expected given 12 memory channels? * Does the zig-zag pattern indicate a pinning problem, or is it unavoidable given the 8 core building block of these type of processors? Chris dr. ir. Christiaan Klaij | senior researcher Research & Development | CFD Development T +31 317 49 33 44 | https://urldefense.us/v3/__http://www.marin.nl__;!!G_uCfscf7eWS!fqSBpN3Ld5f…

3 7

Recommended workflow for continuing a TSSolve after making a change to mesh topology
by Ross Williams 22 Oct '25

22 Oct '25

Hello, I am asking for guidance and/or clarification on how one should best handle changes to the mesh topology within TSSetPreStep of a TSSolve. Out current methodology after making the change is: 1. TSReset(ts); 2. DMSetUp(dm); 3. TSSetSolution(ts, sol); 4. TSSetDM(ts, dm); 5. TSSetUp(ts); I have not found any relevant examples in the documentation so if there is one or an example in another code base, please feel free to direct me towards it. Kind regards, Ross. ________________________________ Ross Williams PhD Research Associate Glasgow Computational Engineering Centre (GCEC) E: ross.williams(a)glasgow.ac.uk W: https://urldefense.us/v3/__https://www.gla.ac.uk/schools/engineering/staff/… Pearce Lodge, James Watt School of Engineering, University of Glasgow, G12 8QQ, Glasgow, United Kingdom.

2 1

[GPU] Jacobi preconditioner
by LEDAC Pierre 22 Oct '25

22 Oct '25

Hello all, We are solving with PETSc a linear system updated every time step (constant stencil but coefficients changing). The matrix is preallocated once with MatSetPreallocationCOO() then filled each time step with MatSetValuesCOO() and we use device pointers for coo_i, coo_j, and coefficients values. It is working fine with a GMRES Ksp solver and PC Jacobi but we are surprised to see that every time step, during PCSetUp, MatGetDiagonal_SeqAIJ is called whereas the matrix is on the device. Looking at the API, it seems there is no MatGetDiagonal_SeqAIJCUSPARSE() but a MatGetDiagonal_SeqAIJKOKKOS(). Does it mean we should use Kokkos backend in PETSc to have Jacobi preconditioner built directly on device ? Or I am doing something wrong ? NB: Gmres is running well on device. I could use -ksp_reuse_preconditioner to avoid Jacobi being recreated each solve on host but it increases significantly the number of iterations. Thanks, [cid:efad8a7f-43b5-4363-9e63-818e26a804bd] Pierre LEDAC Commissariat à l’énergie atomique et aux énergies alternatives Centre de SACLAY DES/ISAS/DM2S/SGLS/LCAN Bâtiment 451 – point courrier n°41 F-91191 Gif-sur-Yvette +33 1 69 08 04 03 +33 6 83 42 05 79

3 22

source download with EasyBuild fails
by Jörg Saßmannshausen 21 Oct '25

21 Oct '25

Dear all, we are using EasyBuild for our software installation and as part of the installation process, EasyBuild is downloading the source-package automatically, and checks the checksum of the downloaded package. However, that seems to be broken for some time now, as explained in this issue: https://github.com/easybuilders/easybuild-framework/issues/4925 In short, EasyBuild is using the http-header: headers = {'User-Agent': 'EasyBuild', 'Accept': '*/*'} which seems to be blocked. I was wondering if it is possible to add an exception to that rule so the automated builds are working again. I personally would prefer this route, simply as it also allows you to get some statistics about how often the software was downloaded and installed via EasyBuild. So for me, that would be a win-win situation. Please let me know. Kind regards Jörg -- Dr. Jörg Saßmannshausen, MRSC (he/him) Senior Research Computing Analyst Information and Communication Technologies Imperial College London White City Campus Level 1 Mediaworks London, W12 7FP

2 1