September 2021 - petsc-users - mailman3.cels.anl.gov

PETSc 3.16 release
by Barry Smith 30 Sep '21

30 Sep '21

We are pleased to announce the release of PETSc version 3.16 at https://petsc.org/release/download/ <https://petsc.org/release/download/> A list of the major changes and updates can be found at https://petsc.org/release/docs/changes/316 <https://petsc.org/release/docs/changes/316> The final update to petsc-3.15 i.e petsc-3.15.5 is also available We recommend upgrading to PETSc 3.16 soon. As always, please report problems to petsc-maint(a)mcs.anl.gov <mailto:[email protected]> and ask questions at petsc-users(a)mcs.anl.gov <mailto:[email protected]> This release includes contributions from Albert Cowie Alexei Colin Barry Smith Blaise Bourdin Carsten Uphoff Connor Ward Daniel Finn Daniel Shapero Erik Schnetter Fande Kong Hong Zhang Jacob Faibussowitsch Jed Brown Jeremy Tillay Joe Wallwork Joseph Pusztay Jose Roman Junchao Zhang Koki Sagiyama Kyle Gerard Felker Lawrence Mitchell Leila Ghaffari Lisandro Dalcin Mark Adams Martin Diehl Matthew Knepley Matt McGurn Moritz Huck Mr. Hong Zhang nathawani olivecha Pablo Brubeck Patrick Sanan pbrubeck Pierre Jolivet Richard Tran Mills Rylee Sundermann Sajid Ali Sam Reynolds Satish Balay Scott Kruger Stefano Zampini Toby Isaac Todd Munson Vaclav Hapla Yang Zongze Zhao Gang and bug reports/patches/proposed improvements received from Adrian Croucher Alexandre Halbach Bret K. Stanford Chonglin Zhang (@zhangchonglin) cleaf "Constantinescu, Emil M." Damian Marek Daniel Stone Danyang Su David Salac dazza simplythebest Drew Parsons (@RizzerAtGitLab) edgar(a)openmail.cc <mailto:[email protected]> Emily Jakobs Emmanuel Ayala Eric Chamberland (@eric.chamberland) Fande Kong Getnet Betrie Haplav hg Iman Datta "Isaac, Tobin G" Jacob Faibussowitsch Jeremy Kozdon Jin Chen Junchao Zhang Lars Corbijn Lawrence Mitchell Lisandro Dalcin "Lundvick, Nick" Mark Adams Martin Diehl Matthew Otten Milan Pelletier Mr. Hong Zhang Nan Ding Pierre Jolivet Pieter Ghysels Qi Tang @qiuchangkai Rezgar Shakeri Rory Johnston Sam Fagbemi Saransh Saxena Sergio Bengoechea Stephen Jardin TAY wee-beng Victor Eijkhout Xiaoye S. Li Yang Liu As always, thanks for your support, Barry

1 0

Possibilities to VecScatter to a sparse Vector-Format
by Hannes Phil Niklas Brandt 30 Sep '21

30 Sep '21

Hello, I intend to compute a parallel Matrix-Vector-Product (via MPI) and therefore would like to scatter the entries of the input MPI-Vec v to a local vector containing all entries relevant to the current process. To achieve this I tried defining a VecScatter, which scatters from v to a sequential Vec v_seq (each process has it's own version of v_seq). However, storing v_seq (which has one entry for each global row, thus containing a large amount of zero-entries) may demand too much storage space (in comparison to my data-sparse Matrix-Storage-Format). I am interested in possibilities to scatter v to a sparse Vec-type to avoid storing unnecessary large amounts of zero-entries. Is there a sparse Vector format in Petsc compatible to the VecScatter procedure or is there another efficient way to compute Matrix-Vector-Products without usinglarge amounts of storage space on each process? Best Regards Hannes Brandt

3 2

pass a member function to MatShellSetOperation
by Michael Wick 30 Sep '21

30 Sep '21

Hi: I want to have the shell matrix-vector multiplication written as a class member function and pass it to the shell matrix via MatShellSetOperation. MatShellSetOperation(A, MATOP_MULT, (void (*)(void))(&Global_Assem::MyMatMult)); Perhaps I have a wrong understanding of function pointers, and I am constantly getting warnings that say I cannot convert a member function to a void type. The warning indeed makes sense to me, as the function pointer passed in the above manner is independent of an instance. Perhaps there are other ways of passing a member function that I don't know of. If you know how to address this, I would appreciate it a lot! Thanks, Mike

3 2

FGMRES and BCGS
by Marco Cisternino 29 Sep '21

29 Sep '21

Good Morning, I usually solve a non-symmetric discretization of the Poisson equation using GAMG+FGMRES. In the last days I tried to use BCGS in place of FGMRES, still using GAMG as preconditioner. No problem in finding the solution but I'm experiencing something I didn't expect. The test case is a 25 millions cells domain with Dirichlet and Neumann boundary conditions. Both the solvers are able to solve the problem with an increasing number of MPI processes, but: * FGMRES is about 25% faster than BCGS for all the processes number * Both solvers have the same scalability from 48 to 384 processes * Both solvers almost use the same amount of memory (FGMRES use a restart=30) Am I wrong expecting less memory consumption and more performance from BCGS with respect to FGMRES? Thank you in advance for any help. Best regards, Marco Cisternino

3 2

%T (percent time in this phase)
by Karthikeyan Chockalingam - STFC UKRI 29 Sep '21

29 Sep '21

Hello, I ran ex45 in the KPS tutorial, which is a 3D finite-difference Poisson problem. I noticed from the output from using the flag -log_summary that for various events their respective %T (percent time in this phase) do not add up to 100 but rather exceeds 100. So, I gather there is some overlap among these events. I am primarily looking at the events KSPSetUp, KSPSolve, PCSetUp and PCSolve. Is it possible to get a mutually exclusive %T or Time for these individual events? I have attached the log_summary output file from my run for your reference. Thanks! Karthik. This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses.

3 9

Turn off CUDA Devices information
by Yiyang Li 28 Sep '21

28 Sep '21

Hello, I have CUDA aware MPI, and I have upgraded from PETSc 3.12 to PETSc 3.15.4 and petsc4py 3.15.4. Now, when I call PETSc.KSP().solve(..., ...) The information of GPU is always printed to stdout by every MPI rank, like CUDA version: v 11040 CUDA Devices: 0 : Quadro P4000 6 1 Global memory: 8105 mb Shared memory: 48 kb Constant memory: 64 kb Block registers: 65536 CUDA version: v 11040 CUDA Devices: 0 : Quadro P4000 6 1 Global memory: 8105 mb Shared memory: 48 kb Constant memory: 64 kb Block registers: 6553 ... I wonder if there is an option to turn that off? I have tried including -cuda_device NONE in command options, but that did not work. Best regards, Yiyang

2 4

Using MATLAB on Windows with PETSc on WSL
by Lucas Banting 28 Sep '21

28 Sep '21

Hello, My overall goal is to send a sparse matrix to PETSc (in WSL)from MATLAB (in Windows) so I can use SLEPc for some eigenvalue routines, and send those eigenvectors back to MATLAB, as the MATLAB eigs() struggles with my matrix and I was looking to experiment with different eigenvalue algorithms. I was trying to configure PETSc on Windows Subsystem for Linux (WSL2). Configuring without '--with-matlab' works fine. I tried the configure command: ./configure --with-scalar-type=complex --with-openblas-dir=~/software/OpenBLAS/ --with-matlab-dir=/mnt/e/Program\ Files/MATLAB/R2020b I was wondering if there is a fundamental reason this configure won't work, or if it is just the space in 'Program Files' that is breaking the configure command. I think the only thing configuring with MATLAB is 'sopen' and 'sclose'? Is it possible I could just remake these on my own for a WSL compatible version? Thanks, Lucas

3 2

GMRES Breakdown
by TARDIEU Nicolas 28 Sep '21

28 Sep '21

Dear PETSc Team, We, code_aster's development team, are using PETSc in our application for years now, mainly for the KSP et PC features. We run tests cases every week without problems. After upgrading from PETSc 3.12.3 to 3.15.3, a test case using GMRES failed with KSP_DIVERGED_BREAKDOWN. By rolling back to 3.12.3, we have checked that the upgrade is the origin of the failure. We know this can occur with GMRES but it is the first time we are facing this situation. Is it a bug ? If yes, how can we help to fix it ? If no, what can we do ? Could the flexible version of GMRES be an alternative ? Best regards, Nicolas -- Nicolas Tardieu Ing, PhD Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message. Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free.

3 3

Interaction between petsc4py and application Fortran library
by WUKIE, NATHAN A DR-02 USAF AFMC AFRL/RQVC 27 Sep '21

27 Sep '21

How should petsc initialization be handled for a python application utilizing petsc4py and a Fortran library application also using petsc? The petsc documentation states that PetscInitializeFortran "should be called soon AFTER the call to PetscInitialize<https://petsc.org/release/docs/manualpages/Sys/PetscInitialize.html#PetscIn…>() if one is using a C main program that calls Fortran routines that in turn call PETSc routines". Does petsc4py.init(...) call PetscInitializeFortran? Is it permissible to call PetscInitializeFortran from the fortran library application itself? Or must PetscInitializeFortran be called from the C main program? Thank you, Nathan

2 1

Petsc memory consumption keep increasing in my loop
by Medane TCHAKOROM 24 Sep '21

24 Sep '21

Hello, I have problem with a code i'am working on. To illustrate my problem, here is an example: int main(int argc, char *argv[]) { PetscErrorCode ierr; ierr = PetscInitialize(&argc, &argv, (char *)0, NULL); if (ierr) return ierr; int i = 0; for (i = 0; i < 1; i++) { Mat A; ierr = MatCreate(PETSC_COMM_WORLD, &A); CHKERRQ(ierr); ierr = MatSetSizes(A, 16, 16, PETSC_DECIDE,PETSC_DECIDE); CHKERRQ(ierr); ierr = MatSetFromOptions(A); CHKERRQ(ierr); ierr = MatSetUp(A); CHKERRQ(ierr); ierr = MatAssemblyBegin(A, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); ierr = MatAssemblyEnd(A, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); /// SOME CODE HERE.... MatDestroy(&A); } FILE *fPtr; fPtr = fopen("petsc_dump_file.txt", "a"); PetscMallocDump(fPtr); fclose(fPtr); ierr = PetscFinalize(); CHKERRQ(ierr); return 0; } The problem is , in the loop, the memory consumption keep increasing till the end of the program. I checked memory leak with PetscMallocDump, and found out that the problem may be due to matrix creation. I'am new to Petsc and i don't know if i'am doing something wrong. Thanks Médane

2 4