June 2021 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] kokkos test fail in branch
by Barry Smith 05 Jun '21

05 Jun '21

Looks like the MPI libraries are not being passed to the NVCC internal compiler (gcc). This would normally be setup in MPI.py, please send configure.log Barry > On Jun 5, 2021, at 1:18 PM, Mark Adams <mfadams(a)lbl.gov> wrote: > > Ah, there was an old one there. I removed it and I can not compile ex3k. I thought check always rebuilt executables. Now I get a link error: > > 14:12 barry/2020-11-11/cleanup-matsetvaluesdevice *= /gpfs/alpine/csc314/scratch/adams/petsc/src/snes/tutorials$ make PETSC_DIR=/gpfs/alpine/csc314/scratch/adams/petsc PETSC_ARCH=arch-summit-opt-gnu-kokkos-notpl-cuda10 ex3k > PATH=/sw/sources/lsf-tools/2.0/summit/bin:/sw/summit/xalt/1.2.1/bin:/sw/summit/forge/20.0.1/bin:/autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-6.4.0/spectrum-mpi-10.3.1.2-20200121-awz2q5brde7wgdqqw4ugalrkukeub4eb/bin:/sw/summit/gcc/6.4.0/bin:/sw/summit/cuda/10.1.243/bin:/autofs/nccs-svm1_sw/summit/.swci/0-core/opt/spack/20180914/linux-rhel7-ppc64le/gcc-4.8.5/cmake-3.20.2-24ualfzy6em6ws5sbiu7rlgcuionodrm/bin:/autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-4.8.5/darshan-runtime-3.1.7-cnvxicgf5j4ap64qi6v5gxp67hmrjz43/bin:/sw/sources/hpss/bin:/opt/ibm/spectrumcomputing/lsf/10.1.0.9/linux3.10-glibc2.17-ppc64le-csm/etc:/opt/ibm/spectrumcomputing/lsf/10.1.0.9/linux3.10-glibc2.17-ppc64le-csm/bin:/opt/ibm/csm/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/ibm/flightlog/bin:/opt/ibutils/bin:/opt/ibm/spectrum_mpi/jsm_pmix/bin:/opt/puppetlabs/bin:/usr/lpp/mmfs/bin:`dirname <http://10.1.0.9/linux3.10-glibc2.17-ppc64le-csm/etc:/opt/ibm/spectrumcomput…> nvcc` NVCC_WRAPPER_DEFAULT_COMPILER=gcc /gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/bin/nvcc_wrapper --expt-extended-lambda -Xcompiler -rdynamic -lineinfo -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DPETSC_HAVE_CUDA_ATOMIC -DLANDAU_MAX_Q=4 -Xcompiler -fPIC -O3 -gencode arch=compute_70,code=sm_70 -I/autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-6.4.0/spectrum-mpi-10.3.1.2-20200121-awz2q5brde7wgdqqw4ugalrkukeub4eb/include -Wno-deprecated-gpu-targets -I/gpfs/alpine/csc314/scratch/adams/petsc/include -I/gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/include -I/sw/summit/cuda/10.1.243/include -I/autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-6.4.0/spectrum-mpi-10.3.1.2-20200121-awz2q5brde7wgdqqw4ugalrkukeub4eb/include -fPIC -g -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DPETSC_HAVE_CUDA_ATOMIC -DLANDAU_MAX_Q=4 -Werror=maybe-uninitialized -O0 ex3k.kokkos.cxx -Wl,-rpath,/gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/lib -L/gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/lib -Wl,-rpath,/gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/lib -L/gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/lib -L/autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-6.4.0/netlib-lapack-3.8.0-wcabdyqhdi5rooxbkqa6x5d7hxyxwdkm/lib64 -Wl,-rpath,/sw/summit/cuda/10.1.243/lib64 -L/sw/summit/cuda/10.1.243/lib64 -lpetsc -lkokkoskernels -lkokkoscontainers -lkokkoscore -lp4est -lsc -lblas -llapack -ltriangle -lm -lz -lcudart -lcufft -lcublas -lcusparse -lcusolver -lcurand -lstdc++ -ldl -o ex3k > nvcc_wrapper - *warning* you have set multiple optimization flags (-O*), only the last is used because nvcc can only accept a single optimization setting. > /usr/bin/ld: /tmp/tmpxft_0000e0c2_00000000-10_ex3k.kokkos.o: undefined reference to symbol 'ompi_mpi_comm_self' > /autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-6.4.0/spectrum-mpi-10.3.1.2-20200121-awz2q5brde7wgdqqw4ugalrkukeub4eb/lib/libmpi_ibm.so.3: error adding symbols: DSO missing from command line > collect2: error: ld returned 1 exit status > make: *** [ex3k] Error 1 > > On Sat, Jun 5, 2021 at 12:16 PM Junchao Zhang <junchao.zhang(a)gmail.com <mailto:[email protected]>> wrote: > $rm ex3k > $make ex3k > and run again? > > --Junchao Zhang > > > On Sat, Jun 5, 2021 at 10:25 AM Mark Adams <mfadams(a)lbl.gov <mailto:[email protected]>> wrote: > This is posted in Barry's MR, but I get this error with Kokkos-cuda on Summit. Failing to open a shared lib. > Thoughts? > Mark > > 11:15 barry/2020-11-11/cleanup-matsetvaluesdevice= /gpfs/alpine/csc314/scratch/adams/petsc$ make PETSC_DIR=/gpfs/alpine/csc314/scratch/adams/petsc PETSC_ARCH=arch-summit-opt-gnu-kokkos-notpl-cuda10 check > Running check examples to verify correct installation > Using PETSC_DIR=/gpfs/alpine/csc314/scratch/adams/petsc and PETSC_ARCH=arch-summit-opt-gnu-kokkos-notpl-cuda10 > C/C++ example src/snes/tutorials/ex19 run successfully with 1 MPI process > C/C++ example src/snes/tutorials/ex19 run successfully with 2 MPI processes > C/C++ example src/snes/tutorials/ex19 run successfully with cuda > gmake[3]: [runex3k_kokkos] Error 127 (ignored) > 1,25c1,2 > < atol=1e-50, rtol=1e-08, stol=1e-08, maxit=50, maxf=10000 > < Vec Object: Exact Solution 2 MPI processes > < type: mpikokkos > < Process [0] > < 0. > < 0.015625 > < 0.125 > < Process [1] > < 0.421875 > < 1. > < Vec Object: Forcing function 2 MPI processes > < type: mpikokkos > < Process [0] > < 1e-72 > < 1.50024 > < 3.01563 > < Process [1] > < 4.67798 > < 7. > < 0 SNES Function norm 5.414682427127e+00 > < 1 SNES Function norm 2.952582418265e-01 > < 2 SNES Function norm 4.502293658739e-04 > < 3 SNES Function norm 1.389665806646e-09 > < Number of SNES iterations = 3 > < Norm of error 1.49752e-10 Iterations 3 > --- > > ./ex3k: error while loading shared libraries: libpetsc.so.3.015: cannot open shared object file: No such file or directory > > ./ex3k: error while loading shared libraries: libpetsc.so.3.015: cannot open shared object file: No such file or directory > /gpfs/alpine/csc314/scratch/adams/petsc/src/snes/tutorials

1 0

Re: [petsc-dev] kokkos test fail in branch
by Mark Adams 05 Jun '21

05 Jun '21

Ah, there was an old one there. I removed it and I can not compile ex3k. I thought check always rebuilt executables. Now I get a link error: 14:12 barry/2020-11-11/cleanup-matsetvaluesdevice *= /gpfs/alpine/csc314/scratch/adams/petsc/src/snes/tutorials$ make PETSC_DIR=/gpfs/alpine/csc314/scratch/adams/petsc PETSC_ARCH=arch-summit-opt-gnu-kokkos-notpl-cuda10 ex3k PATH=/sw/sources/lsf-tools/2.0/summit/bin:/sw/summit/xalt/1.2.1/bin:/sw/summit/forge/20.0.1/bin:/autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-6.4.0/spectrum-mpi-10.3.1.2-20200121-awz2q5brde7wgdqqw4ugalrkukeub4eb/bin:/sw/summit/gcc/6.4.0/bin:/sw/summit/cuda/10.1.243/bin:/autofs/nccs-svm1_sw/summit/.swci/0-core/opt/spack/20180914/linux-rhel7-ppc64le/gcc-4.8.5/cmake-3.20.2-24ualfzy6em6ws5sbiu7rlgcuionodrm/bin:/autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-4.8.5/darshan-runtime-3.1.7-cnvxicgf5j4ap64qi6v5gxp67hmrjz43/bin:/sw/sources/hpss/bin:/opt/ibm/spectrumcomputing/lsf/ 10.1.0.9/linux3.10-glibc2.17-ppc64le-csm/etc:/opt/ibm/spectrumcomputing/lsf/10.1.0.9/linux3.10-glibc2.17-ppc64le-csm/bin:/opt/ibm/csm/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/ibm/flightlog/bin:/opt/ibutils/bin:/opt/ibm/spectrum_mpi/jsm_pmix/bin:/opt/puppetlabs/bin:/usr/lpp/mmfs/bin:`dirname nvcc` NVCC_WRAPPER_DEFAULT_COMPILER=gcc /gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/bin/nvcc_wrapper --expt-extended-lambda -Xcompiler -rdynamic -lineinfo -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DPETSC_HAVE_CUDA_ATOMIC -DLANDAU_MAX_Q=4 -Xcompiler -fPIC -O3 -gencode arch=compute_70,code=sm_70 -I/autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-6.4.0/spectrum-mpi-10.3.1.2-20200121-awz2q5brde7wgdqqw4ugalrkukeub4eb/include -Wno-deprecated-gpu-targets -I/gpfs/alpine/csc314/scratch/adams/petsc/include -I/gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/include -I/sw/summit/cuda/10.1.243/include -I/autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-6.4.0/spectrum-mpi-10.3.1.2-20200121-awz2q5brde7wgdqqw4ugalrkukeub4eb/include -fPIC -g -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DPETSC_HAVE_CUDA_ATOMIC -DLANDAU_MAX_Q=4 -Werror=maybe-uninitialized -O0 ex3k.kokkos.cxx -Wl,-rpath,/gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/lib -L/gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/lib -Wl,-rpath,/gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/lib -L/gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/lib -L/autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-6.4.0/netlib-lapack-3.8.0-wcabdyqhdi5rooxbkqa6x5d7hxyxwdkm/lib64 -Wl,-rpath,/sw/summit/cuda/10.1.243/lib64 -L/sw/summit/cuda/10.1.243/lib64 -lpetsc -lkokkoskernels -lkokkoscontainers -lkokkoscore -lp4est -lsc -lblas -llapack -ltriangle -lm -lz -lcudart -lcufft -lcublas -lcusparse -lcusolver -lcurand -lstdc++ -ldl -o ex3k nvcc_wrapper - *warning* you have set multiple optimization flags (-O*), only the last is used because nvcc can only accept a single optimization setting. */usr/bin/ld: /tmp/tmpxft_0000e0c2_00000000-10_ex3k.kokkos.o: undefined reference to symbol 'ompi_mpi_comm_self'*/autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-6.4.0/spectrum-mpi-10.3.1.2-20200121-awz2q5brde7wgdqqw4ugalrkukeub4eb/lib/libmpi_ibm.so.3: error adding symbols: DSO missing from command line collect2: error: ld returned 1 exit status make: *** [ex3k] Error 1 On Sat, Jun 5, 2021 at 12:16 PM Junchao Zhang <junchao.zhang(a)gmail.com> wrote: > $rm ex3k > $make ex3k > and run again? > > --Junchao Zhang > > > On Sat, Jun 5, 2021 at 10:25 AM Mark Adams <mfadams(a)lbl.gov> wrote: > >> This is posted in Barry's MR, but I get this error with Kokkos-cuda on >> Summit. Failing to open a shared lib. >> Thoughts? >> Mark >> >> 11:15 barry/2020-11-11/cleanup-matsetvaluesdevice= >> /gpfs/alpine/csc314/scratch/adams/petsc$ make >> PETSC_DIR=/gpfs/alpine/csc314/scratch/adams/petsc >> PETSC_ARCH=arch-summit-opt-gnu-kokkos-notpl-cuda10 check >> Running check examples to verify correct installation >> Using PETSC_DIR=/gpfs/alpine/csc314/scratch/adams/petsc and >> PETSC_ARCH=arch-summit-opt-gnu-kokkos-notpl-cuda10 >> C/C++ example src/snes/tutorials/ex19 run successfully with 1 MPI process >> C/C++ example src/snes/tutorials/ex19 run successfully with 2 MPI >> processes >> C/C++ example src/snes/tutorials/ex19 run successfully with cuda >> gmake[3]: [runex3k_kokkos] Error 127 (ignored) >> 1,25c1,2 >> < atol=1e-50, rtol=1e-08, stol=1e-08, maxit=50, maxf=10000 >> < Vec Object: Exact Solution 2 MPI processes >> < type: mpikokkos >> < Process [0] >> < 0. >> < 0.015625 >> < 0.125 >> < Process [1] >> < 0.421875 >> < 1. >> < Vec Object: Forcing function 2 MPI processes >> < type: mpikokkos >> < Process [0] >> < 1e-72 >> < 1.50024 >> < 3.01563 >> < Process [1] >> < 4.67798 >> < 7. >> < 0 SNES Function norm 5.414682427127e+00 >> < 1 SNES Function norm 2.952582418265e-01 >> < 2 SNES Function norm 4.502293658739e-04 >> < 3 SNES Function norm 1.389665806646e-09 >> < Number of SNES iterations = 3 >> < Norm of error 1.49752e-10 Iterations 3 >> --- >> >> >> *> ./ex3k: error while loading shared libraries: libpetsc.so.3.015: >> cannot open shared object file: No such file or directory> ./ex3k: error >> while loading shared libraries: libpetsc.so.3.015: cannot open shared >> object file: No such file or directory* >> /gpfs/alpine/csc314/scratch/adams/petsc/src/snes/tutorials >> >

1 0

Re: [petsc-dev] kokkos test fail in branch
by Junchao Zhang 05 Jun '21

05 Jun '21

$rm ex3k $make ex3k and run again? --Junchao Zhang On Sat, Jun 5, 2021 at 10:25 AM Mark Adams <mfadams(a)lbl.gov> wrote: > This is posted in Barry's MR, but I get this error with Kokkos-cuda on > Summit. Failing to open a shared lib. > Thoughts? > Mark > > 11:15 barry/2020-11-11/cleanup-matsetvaluesdevice= > /gpfs/alpine/csc314/scratch/adams/petsc$ make > PETSC_DIR=/gpfs/alpine/csc314/scratch/adams/petsc > PETSC_ARCH=arch-summit-opt-gnu-kokkos-notpl-cuda10 check > Running check examples to verify correct installation > Using PETSC_DIR=/gpfs/alpine/csc314/scratch/adams/petsc and > PETSC_ARCH=arch-summit-opt-gnu-kokkos-notpl-cuda10 > C/C++ example src/snes/tutorials/ex19 run successfully with 1 MPI process > C/C++ example src/snes/tutorials/ex19 run successfully with 2 MPI processes > C/C++ example src/snes/tutorials/ex19 run successfully with cuda > gmake[3]: [runex3k_kokkos] Error 127 (ignored) > 1,25c1,2 > < atol=1e-50, rtol=1e-08, stol=1e-08, maxit=50, maxf=10000 > < Vec Object: Exact Solution 2 MPI processes > < type: mpikokkos > < Process [0] > < 0. > < 0.015625 > < 0.125 > < Process [1] > < 0.421875 > < 1. > < Vec Object: Forcing function 2 MPI processes > < type: mpikokkos > < Process [0] > < 1e-72 > < 1.50024 > < 3.01563 > < Process [1] > < 4.67798 > < 7. > < 0 SNES Function norm 5.414682427127e+00 > < 1 SNES Function norm 2.952582418265e-01 > < 2 SNES Function norm 4.502293658739e-04 > < 3 SNES Function norm 1.389665806646e-09 > < Number of SNES iterations = 3 > < Norm of error 1.49752e-10 Iterations 3 > --- > > > *> ./ex3k: error while loading shared libraries: libpetsc.so.3.015: cannot > open shared object file: No such file or directory> ./ex3k: error while > loading shared libraries: libpetsc.so.3.015: cannot open shared object > file: No such file or directory* > /gpfs/alpine/csc314/scratch/adams/petsc/src/snes/tutorials >

1 0

kokkos test fail in branch
by Mark Adams 05 Jun '21

05 Jun '21

This is posted in Barry's MR, but I get this error with Kokkos-cuda on Summit. Failing to open a shared lib. Thoughts? Mark 11:15 barry/2020-11-11/cleanup-matsetvaluesdevice= /gpfs/alpine/csc314/scratch/adams/petsc$ make PETSC_DIR=/gpfs/alpine/csc314/scratch/adams/petsc PETSC_ARCH=arch-summit-opt-gnu-kokkos-notpl-cuda10 check Running check examples to verify correct installation Using PETSC_DIR=/gpfs/alpine/csc314/scratch/adams/petsc and PETSC_ARCH=arch-summit-opt-gnu-kokkos-notpl-cuda10 C/C++ example src/snes/tutorials/ex19 run successfully with 1 MPI process C/C++ example src/snes/tutorials/ex19 run successfully with 2 MPI processes C/C++ example src/snes/tutorials/ex19 run successfully with cuda gmake[3]: [runex3k_kokkos] Error 127 (ignored) 1,25c1,2 < atol=1e-50, rtol=1e-08, stol=1e-08, maxit=50, maxf=10000 < Vec Object: Exact Solution 2 MPI processes < type: mpikokkos < Process [0] < 0. < 0.015625 < 0.125 < Process [1] < 0.421875 < 1. < Vec Object: Forcing function 2 MPI processes < type: mpikokkos < Process [0] < 1e-72 < 1.50024 < 3.01563 < Process [1] < 4.67798 < 7. < 0 SNES Function norm 5.414682427127e+00 < 1 SNES Function norm 2.952582418265e-01 < 2 SNES Function norm 4.502293658739e-04 < 3 SNES Function norm 1.389665806646e-09 < Number of SNES iterations = 3 < Norm of error 1.49752e-10 Iterations 3 --- *> ./ex3k: error while loading shared libraries: libpetsc.so.3.015: cannot open shared object file: No such file or directory> ./ex3k: error while loading shared libraries: libpetsc.so.3.015: cannot open shared object file: No such file or directory* /gpfs/alpine/csc314/scratch/adams/petsc/src/snes/tutorials

1 0

Re: [petsc-dev] [petsc-users] ex5k.kokkos compile error
by Mark Adams 03 Jun '21

03 Jun '21

I was doing 'make ex5k.kokkos' It would be nice if that failed, but it building now. Thanks, On Thu, Jun 3, 2021 at 10:57 AM Junchao Zhang <junchao.zhang(a)gmail.com> wrote: > (Moved to petsc-dev) > > Mark, > Is it because your branch is out of date? petsc should use nvcc_wapper > to compile ex5k.kokkos.cxx. See mine > $cd ~/petsc/src/mat/tutorials > $ make ex5k > PATH=/home/jczhang/spack/opt/spack/linux-ubuntu18.04-skylake_avx512/gcc-8.3.0/gdb-9.1-bzjzolog57gom5anscfcb3oe6uqr6s6m/bin:/home/jczhang/spack/opt/spack/linux-ubuntu18.04-skylake_avx512/gcc-8.3.0/bear-2.2.0-xhfk3moyh7jled6o62f3om557iia6oun/bin:/home/jczhang/spack/opt/spack/linux-ubuntu18.04-skylake_avx512/gcc-8.3.0/perl-5.30.3-2boymujkmxezayez4emfw5sw5wuqgar6/bin:/home/jczhang/spack/opt/spack/linux-ubuntu18.04-skylake_avx512/gcc-8.3.0/openmpi-4.0.2-7vmyqiyk4iyvdeoqpux7fyoce6mjt7iw/bin:/home/jczhang/spack/opt/spack/linux-ubuntu18.04-skylake_avx512/gcc-8.3.0/cuda-10.2.89-uy4hlcnd7svrcahaguwx4bzl7ujoqx2v/bin:/nfs/gce/software/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.3.0/cmake-3.20.0-vov726r/bin:/nfs/gce/software/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.3.0/gcc-8.3.0-fjpc5ys/bin:/home/jczhang/soft/bin:/home/jczhang/spack/bin:/home/jczhang/arm/forge/20.2/bin:/home/jczhang/.vscode-server/data/User/globalStorage/llvm-vs-code-extensions.vscode-clangd/install/10.0.0/clangd_10.0.0/bin:/home/jczhang/.vscode-server/bin/054a9295330880ed74ceaedda236253b4f39a335/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:`dirname > nvcc` NVCC_WRAPPER_DEFAULT_COMPILER=gcc > /home/jczhang/petsc/linux-kokkos-dbg/bin/nvcc_wrapper --expt-extended-lambda > -Xcompiler -fPIC -g -gencode arch=compute_70,code=sm_70 > -I/home/jczhang/spack/opt/spack/linux-ubuntu18.04-skylake_avx512/gcc-8.3.0/openmpi-4.0.2-7vmyqiyk4iyvdeoqpux7fyoce6mjt7iw/include > -Wno-deprecated-gpu-targets -I/home/jczhang/petsc/include > -I/home/jczhang/petsc/linux-kokkos-dbg/include > -I/home/jczhang/spack/opt/spack/linux-ubuntu18.04-skylake_avx512/gcc-8.3.0/openmpi-4.0.2-7vmyqiyk4iyvdeoqpux7fyoce6mjt7iw/include > -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas > -fstack-protector -fvisibility=hidden -g -O0 ex5k.kokkos.cxx > -Wl,-rpath,/home/jczhang/petsc/linux-kokkos-dbg/lib > -L/home/jczhang/petsc/linux-kokkos-dbg/lib > -Wl,-rpath,/home/jczhang/petsc/linux-kokkos-dbg/lib > -L/home/jczhang/petsc/linux-kokkos-dbg/lib -lpetsc -lkokkoskernels > -lkokkoscontainers -lkokkoscore -llapack -lblas -lm -lcudart -lcufft > -lcublas -lcusparse -lcusolver -lcurand -lX11 -lquadmath -lstdc++ -ldl -o > ex5k > > --Junchao Zhang > > > On Thu, Jun 3, 2021 at 8:32 AM Mark Adams <mfadams(a)lbl.gov> wrote: > >> I am getting this error: >> >> 09:22 adams/landau-mass-opt= >> /gpfs/alpine/csc314/scratch/adams/petsc/src/mat/tutorials$ make >> PETSC_DIR=/gpfs/alpine/csc314/scratch/adams/petsc >> PETSC_ARCH=arch-summit-opt-gnu-kokkos-notpl-cuda10 ex5k.kokkos >> mpicxx -fPIC -g -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 >> -DPETSC_HAVE_CUDA_ATOMIC -DLANDAU_MAX_Q=4 -O0 -fPIC -g -DLANDAU_DIM=2 >> -DLANDAU_MAX_SPECIES=10 -DPETSC_HAVE_CUDA_ATOMIC -DLANDAU_MAX_Q=4 -O0 >> -fPIC -I/gpfs/alpine/csc314/scratch/adams/petsc/include >> -I/gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/include >> -I/sw/summit/cuda/10.1.243/include ex5k.kokkos.cxx >> -Wl,-rpath,/gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/lib >> -L/gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/lib >> -Wl,-rpath,/gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/lib >> -L/gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/lib >> -L/autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-6.4.0/netlib-lapack-3.8.0-wcabdyqhdi5rooxbkqa6x5d7hxyxwdkm/lib64 >> -Wl,-rpath,/sw/summit/cuda/10.1.243/lib64 -L/sw/summit/cuda/10.1.243/lib64 >> -lpetsc -lkokkoskernels -lkokkoscontainers -lkokkoscore -lp4est -lsc -lblas >> -llapack -ltriangle -lm -lz -lcudart -lcufft -lcublas -lcusparse -lcusolver >> -lcurand -lstdc++ -ldl -o ex5k.kokkos >> In file included from >> /gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/include/KokkosCore_Config_SetupBackend.hpp:47:0, >> from >> /gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/include/Kokkos_Macros.hpp:109, >> from >> /gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/include/Kokkos_Core_fwd.hpp:52, >> from >> /gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/include/Kokkos_Core.hpp:51, >> from ex5k.kokkos.cxx:10: >> /gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/include/setup/Kokkos_Setup_Cuda.hpp:54:2: >> error: #error "KOKKOS_ENABLE_CUDA defined but the compiler is not defining >> the __CUDACC__ macro as expected" >> #error \ >> ^~~~~ >> In file included from >> /sw/summit/cuda/10.1.243/include/crt/common_functions.h:267:0, >> from /sw/summit/cuda/10.1.243/include/cuda_runtime.h:115, >> from >> /gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/include/setup/Kokkos_Setup_Cuda.hpp:68, >> from >> /gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/include/KokkosCore_Config_SetupBackend.hpp:47, >> from >> /gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/include/Kokkos_Macros.hpp:109, >> from >> /gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/include/Kokkos_Core_fwd.hpp:52, >> from >> /gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/include/Kokkos_Core.hpp:51, >> from ex5k.kokkos.cxx:10: >> /sw/summit/cuda/10.1.243/include/crt/math_functions.h:8944:72: error: >> 'int std::isnan(double)' conflicts with a previous declaration >> __DEVICE_FUNCTIONS_DECL__ __cudart_builtin__ int isnan(double x) throw(); >> ^ >> Here is my config file: >> >> #!/usr/bin/env python >> # -lnvToolsExt >> >> if __name__ == '__main__': >> import sys >> import os >> sys.path.insert(0, os.path.abspath('config')) >> import configure >> configure_options = [ >> '--with-fc=0', >> '--CFLAGS=-fPIC -g -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 >> -DPETSC_HAVE_CUDA_ATOMIC -DLANDAU_MAX_Q=4', >> '--CXXFLAGS=-fPIC -g -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 >> -DPETSC_HAVE_CUDA_ATOMIC -DLANDAU_MAX_Q=4', >> '--COPTFLAGS=-O0', >> '--CXXOPTFLAGS=-O0', >> '--CUDAOPTFLAGS=-O0', >> # 'LIBS= -L'+os.environ['CUDAPATH']+'/lib64 -lnvToolsExt', >> '--CUDAFLAGS=-Xcompiler -rdynamic -lineinfo -DLANDAU_DIM=2 >> -DLANDAU_MAX_SPECIES=10 -DPETSC_HAVE_CUDA_ATOMIC -DLANDAU_MAX_Q=4', >> '--with-ssl=0', >> '--with-batch=0', >> '--with-cxx=mpicxx', >> '--with-mpiexec=jsrun -g1 ', >> '--with-openmp=0', >> # '--with-threadsafety=1', >> # '--with-log=0', >> '--with-cuda=1', >> '--with-cuda-gencodearch=70', >> '--with-cudac=nvcc', >> '--download-p4est=1', >> '--download-zlib', >> # '--download-hdf5=1', >> # '--download-metis', >> # '--download-superlu_dist', >> # '--download-superlu_dist-commit=HEAD', >> # '--download-hypre-configure-arguments=HYPRE_CUDA_SM=70', >> #'--with-hwloc=0', >> # '--download-parmetis', >> #'--download-hypre', >> '--download-triangle', >> #'--download-amgx', >> #'--download-fblaslapack', >> '--with-blaslapack-lib=-L' + os.environ['OLCF_NETLIB_LAPACK_ROOT'] + >> '/lib64 -lblas -llapack', >> #'--download-openblas', >> '--with-cc=mpicc', >> #'--with-fc=mpif90', >> '--with-shared-libraries=1', >> # '--known-mpi-shared-libraries=1', >> '--with-x=0', >> '--with-64-bit-indices=0', >> '--with-debugging=1', >> '--download-kokkos', >> '--with-ctable=0', >> '--with-make-np=8', >> '--download-kokkos-kernels', >> # '--with-kokkos-cuda-arch=VOLTA70', >> '--with-kokkos-kernels-tpl=0', >> >> '--download-kokkos-cmake-arguments=-DKokkos_ENABLE_AGGRESSIVE_VECTORIZATION=ON >> -DKokkos_ARCH_POWER9=ON', >> # '--download-kokkos-commit=origin/develop', >> # '--download-kokkos-kernels-commit=origin/develop', >> 'PETSC_ARCH=arch-summit-opt-gnu-kokkos-notpl-cuda10', >> ] >> configure.petsc_configure(configure_options) >> >> ~ >> ~ >> ~ >> ~ >> ~ >> ~ >> ~ >> >

1 0

Re: [petsc-dev] Auto-testing Failure?
by Scott Kruger 03 Jun '21

03 Jun '21

It's a type of special filter: test: filter: Error: true If you look in `src/sys/tests` there are a couple examples of this. The `Error:` enables the ignoring of the error code. Scott On 2021-06-03 09:54, Jacob Faibussowitsch did write: > Hello All, > > All of our unit tests look for some kind of positive result, but is there any established way in the harness to test for failure of a particular kind? > > Best regards, > > Jacob Faibussowitsch > (Jacob Fai - booss - oh - vitch) > -- Scott Kruger Tech-X Corporation kruger(a)txcorp.com 5621 Arapahoe Ave, Suite A Phone: (720) 466-3196 Boulder, CO 80303 Fax: (303) 448-7756

1 0

Re: [petsc-dev] [petsc-users] ex5k.kokkos compile error
by Junchao Zhang 03 Jun '21

03 Jun '21

(Moved to petsc-dev) Mark, Is it because your branch is out of date? petsc should use nvcc_wapper to compile ex5k.kokkos.cxx. See mine $cd ~/petsc/src/mat/tutorials $ make ex5k PATH=/home/jczhang/spack/opt/spack/linux-ubuntu18.04-skylake_avx512/gcc-8.3.0/gdb-9.1-bzjzolog57gom5anscfcb3oe6uqr6s6m/bin:/home/jczhang/spack/opt/spack/linux-ubuntu18.04-skylake_avx512/gcc-8.3.0/bear-2.2.0-xhfk3moyh7jled6o62f3om557iia6oun/bin:/home/jczhang/spack/opt/spack/linux-ubuntu18.04-skylake_avx512/gcc-8.3.0/perl-5.30.3-2boymujkmxezayez4emfw5sw5wuqgar6/bin:/home/jczhang/spack/opt/spack/linux-ubuntu18.04-skylake_avx512/gcc-8.3.0/openmpi-4.0.2-7vmyqiyk4iyvdeoqpux7fyoce6mjt7iw/bin:/home/jczhang/spack/opt/spack/linux-ubuntu18.04-skylake_avx512/gcc-8.3.0/cuda-10.2.89-uy4hlcnd7svrcahaguwx4bzl7ujoqx2v/bin:/nfs/gce/software/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.3.0/cmake-3.20.0-vov726r/bin:/nfs/gce/software/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-7.3.0/gcc-8.3.0-fjpc5ys/bin:/home/jczhang/soft/bin:/home/jczhang/spack/bin:/home/jczhang/arm/forge/20.2/bin:/home/jczhang/.vscode-server/data/User/globalStorage/llvm-vs-code-extensions.vscode-clangd/install/10.0.0/clangd_10.0.0/bin:/home/jczhang/.vscode-server/bin/054a9295330880ed74ceaedda236253b4f39a335/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:`dirname nvcc` NVCC_WRAPPER_DEFAULT_COMPILER=gcc /home/jczhang/petsc/linux-kokkos-dbg/bin/nvcc_wrapper --expt-extended-lambda -Xcompiler -fPIC -g -gencode arch=compute_70,code=sm_70 -I/home/jczhang/spack/opt/spack/linux-ubuntu18.04-skylake_avx512/gcc-8.3.0/openmpi-4.0.2-7vmyqiyk4iyvdeoqpux7fyoce6mjt7iw/include -Wno-deprecated-gpu-targets -I/home/jczhang/petsc/include -I/home/jczhang/petsc/linux-kokkos-dbg/include -I/home/jczhang/spack/opt/spack/linux-ubuntu18.04-skylake_avx512/gcc-8.3.0/openmpi-4.0.2-7vmyqiyk4iyvdeoqpux7fyoce6mjt7iw/include -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -fstack-protector -fvisibility=hidden -g -O0 ex5k.kokkos.cxx -Wl,-rpath,/home/jczhang/petsc/linux-kokkos-dbg/lib -L/home/jczhang/petsc/linux-kokkos-dbg/lib -Wl,-rpath,/home/jczhang/petsc/linux-kokkos-dbg/lib -L/home/jczhang/petsc/linux-kokkos-dbg/lib -lpetsc -lkokkoskernels -lkokkoscontainers -lkokkoscore -llapack -lblas -lm -lcudart -lcufft -lcublas -lcusparse -lcusolver -lcurand -lX11 -lquadmath -lstdc++ -ldl -o ex5k --Junchao Zhang On Thu, Jun 3, 2021 at 8:32 AM Mark Adams <mfadams(a)lbl.gov> wrote: > I am getting this error: > > 09:22 adams/landau-mass-opt= > /gpfs/alpine/csc314/scratch/adams/petsc/src/mat/tutorials$ make > PETSC_DIR=/gpfs/alpine/csc314/scratch/adams/petsc > PETSC_ARCH=arch-summit-opt-gnu-kokkos-notpl-cuda10 ex5k.kokkos > mpicxx -fPIC -g -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 > -DPETSC_HAVE_CUDA_ATOMIC -DLANDAU_MAX_Q=4 -O0 -fPIC -g -DLANDAU_DIM=2 > -DLANDAU_MAX_SPECIES=10 -DPETSC_HAVE_CUDA_ATOMIC -DLANDAU_MAX_Q=4 -O0 > -fPIC -I/gpfs/alpine/csc314/scratch/adams/petsc/include > -I/gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/include > -I/sw/summit/cuda/10.1.243/include ex5k.kokkos.cxx > -Wl,-rpath,/gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/lib > -L/gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/lib > -Wl,-rpath,/gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/lib > -L/gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/lib > -L/autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-6.4.0/netlib-lapack-3.8.0-wcabdyqhdi5rooxbkqa6x5d7hxyxwdkm/lib64 > -Wl,-rpath,/sw/summit/cuda/10.1.243/lib64 -L/sw/summit/cuda/10.1.243/lib64 > -lpetsc -lkokkoskernels -lkokkoscontainers -lkokkoscore -lp4est -lsc -lblas > -llapack -ltriangle -lm -lz -lcudart -lcufft -lcublas -lcusparse -lcusolver > -lcurand -lstdc++ -ldl -o ex5k.kokkos > In file included from > /gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/include/KokkosCore_Config_SetupBackend.hpp:47:0, > from > /gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/include/Kokkos_Macros.hpp:109, > from > /gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/include/Kokkos_Core_fwd.hpp:52, > from > /gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/include/Kokkos_Core.hpp:51, > from ex5k.kokkos.cxx:10: > /gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/include/setup/Kokkos_Setup_Cuda.hpp:54:2: > error: #error "KOKKOS_ENABLE_CUDA defined but the compiler is not defining > the __CUDACC__ macro as expected" > #error \ > ^~~~~ > In file included from > /sw/summit/cuda/10.1.243/include/crt/common_functions.h:267:0, > from /sw/summit/cuda/10.1.243/include/cuda_runtime.h:115, > from > /gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/include/setup/Kokkos_Setup_Cuda.hpp:68, > from > /gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/include/KokkosCore_Config_SetupBackend.hpp:47, > from > /gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/include/Kokkos_Macros.hpp:109, > from > /gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/include/Kokkos_Core_fwd.hpp:52, > from > /gpfs/alpine/csc314/scratch/adams/petsc/arch-summit-opt-gnu-kokkos-notpl-cuda10/include/Kokkos_Core.hpp:51, > from ex5k.kokkos.cxx:10: > /sw/summit/cuda/10.1.243/include/crt/math_functions.h:8944:72: error: 'int > std::isnan(double)' conflicts with a previous declaration > __DEVICE_FUNCTIONS_DECL__ __cudart_builtin__ int isnan(double x) throw(); > ^ > Here is my config file: > > #!/usr/bin/env python > # -lnvToolsExt > > if __name__ == '__main__': > import sys > import os > sys.path.insert(0, os.path.abspath('config')) > import configure > configure_options = [ > '--with-fc=0', > '--CFLAGS=-fPIC -g -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 > -DPETSC_HAVE_CUDA_ATOMIC -DLANDAU_MAX_Q=4', > '--CXXFLAGS=-fPIC -g -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 > -DPETSC_HAVE_CUDA_ATOMIC -DLANDAU_MAX_Q=4', > '--COPTFLAGS=-O0', > '--CXXOPTFLAGS=-O0', > '--CUDAOPTFLAGS=-O0', > # 'LIBS= -L'+os.environ['CUDAPATH']+'/lib64 -lnvToolsExt', > '--CUDAFLAGS=-Xcompiler -rdynamic -lineinfo -DLANDAU_DIM=2 > -DLANDAU_MAX_SPECIES=10 -DPETSC_HAVE_CUDA_ATOMIC -DLANDAU_MAX_Q=4', > '--with-ssl=0', > '--with-batch=0', > '--with-cxx=mpicxx', > '--with-mpiexec=jsrun -g1 ', > '--with-openmp=0', > # '--with-threadsafety=1', > # '--with-log=0', > '--with-cuda=1', > '--with-cuda-gencodearch=70', > '--with-cudac=nvcc', > '--download-p4est=1', > '--download-zlib', > # '--download-hdf5=1', > # '--download-metis', > # '--download-superlu_dist', > # '--download-superlu_dist-commit=HEAD', > # '--download-hypre-configure-arguments=HYPRE_CUDA_SM=70', > #'--with-hwloc=0', > # '--download-parmetis', > #'--download-hypre', > '--download-triangle', > #'--download-amgx', > #'--download-fblaslapack', > '--with-blaslapack-lib=-L' + os.environ['OLCF_NETLIB_LAPACK_ROOT'] + > '/lib64 -lblas -llapack', > #'--download-openblas', > '--with-cc=mpicc', > #'--with-fc=mpif90', > '--with-shared-libraries=1', > # '--known-mpi-shared-libraries=1', > '--with-x=0', > '--with-64-bit-indices=0', > '--with-debugging=1', > '--download-kokkos', > '--with-ctable=0', > '--with-make-np=8', > '--download-kokkos-kernels', > # '--with-kokkos-cuda-arch=VOLTA70', > '--with-kokkos-kernels-tpl=0', > > '--download-kokkos-cmake-arguments=-DKokkos_ENABLE_AGGRESSIVE_VECTORIZATION=ON > -DKokkos_ARCH_POWER9=ON', > # '--download-kokkos-commit=origin/develop', > # '--download-kokkos-kernels-commit=origin/develop', > 'PETSC_ARCH=arch-summit-opt-gnu-kokkos-notpl-cuda10', > ] > configure.petsc_configure(configure_options) > > ~ > ~ > ~ > ~ > ~ > ~ > ~ >

1 0

Auto-testing Failure?
by Jacob Faibussowitsch 03 Jun '21

03 Jun '21

Hello All, All of our unit tests look for some kind of positive result, but is there any established way in the harness to test for failure of a particular kind? Best regards, Jacob Faibussowitsch (Jacob Fai - booss - oh - vitch)

1 0

Re: [petsc-dev] Petsc Parallel Matrix Vector Multiplication for Skew-Symmetric Matrix
by Barry Smith 02 Jun '21

02 Jun '21

The easiest way to find implementations on the web is to type the function name MatMult() in google and look at the bottom of the page where it has links to all the implementations. Of course you have to guess which implementation you need based on the shortened names. Here is the one you want https://petsc.org/release/src/mat/impls/sbaij/mpi/mpisbaij.c.html#MatMult_M… <https://petsc.org/release/src/mat/impls/sbaij/mpi/mpisbaij.c.html#MatMult_M…> All you need to do is flip the sign of the transpose multiple. The creation of the needed VecScatter and work vectors is done here https://petsc.org/release/src/mat/impls/sbaij/mpi/mmsbaij.c <https://petsc.org/release/src/mat/impls/sbaij/mpi/mmsbaij.c> it concatenates the two needed parallel communications into a singe VecScatter. > On Jun 2, 2021, at 3:00 PM, Mark Adams <mfadams(a)lbl.gov> wrote: > > I guess we still have a MATSBAIJ. > In theory you could modify the MatVec to do skew-symmetric, easily, but this is such a niche operation (never seen it requested in 20+ years) that I would not want to put it in the repo. > This would save <2x memory and <2x time. If you have a blocked system then it would be close to 2x. > You could do this in a branch and just rebase over main to update PETSc. Forever, or until you can convince us to merge it. > > On Wed, Jun 2, 2021 at 3:26 PM Matthew Knepley <knepley(a)gmail.com <mailto:[email protected]>> wrote: > On Wed, Jun 2, 2021 at 3:23 PM Yağmur Aksan <yagmuraksan(a)gmail.com <mailto:[email protected]>> wrote: > Hello, > > I need a function that parallelly multiplies skew-symmetric matrix and vector. Is there any function that Petsc provides? > To do so, I think of using matMult function in Petsc and changing it to work with skew-symmetric matrices. But, I haven't found the implementation of MatMult function. If there isn't any other better option that you can recommend me, could you please tell me the link for the implementation of MatMult function? > > Do you want to save memory by using the special format? It will not save computing, and the speed improvement can be very small. > > Thanks, > > Matt > > Thank you. > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>

1 0

Re: [petsc-dev] Petsc Parallel Matrix Vector Multiplication for Skew-Symmetric Matrix
by Mark Adams 02 Jun '21

02 Jun '21

I guess we still have a MATSBAIJ. In theory you could modify the MatVec to do skew-symmetric, easily, but this is such a niche operation (never seen it requested in 20+ years) that I would not want to put it in the repo. This would save <2x memory and <2x time. If you have a blocked system then it would be close to 2x. You could do this in a branch and just rebase over main to update PETSc. Forever, or until you can convince us to merge it. On Wed, Jun 2, 2021 at 3:26 PM Matthew Knepley <knepley(a)gmail.com> wrote: > On Wed, Jun 2, 2021 at 3:23 PM Yağmur Aksan <yagmuraksan(a)gmail.com> wrote: > >> Hello, >> >> I need a function that parallelly multiplies skew-symmetric matrix and >> vector. Is there any function that Petsc provides? >> To do so, I think of using matMult function in Petsc and changing it to >> work with skew-symmetric matrices. But, I haven't found the implementation >> of MatMult function. If there isn't any other better option that you can >> recommend me, could you please tell me the link for the implementation of >> MatMult function? >> > > Do you want to save memory by using the special format? It will not save > computing, and the speed improvement can be very small. > > Thanks, > > Matt > > >> Thank you. >> > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > <http://www.cse.buffalo.edu/~knepley/> >

1 0