March 2021 - petsc-dev - mailman3.cels.anl.gov

Are pipelines down?
by Matthew Knepley 11 Mar '21

11 Mar '21

https://gitlab.com/danfinn/petsc/-/pipelines/269297070 Thanks, Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>

1 0

Re: [petsc-dev] Petsc "make test" have more failures for --with-openmp=1
by Pierre Jolivet 11 Mar '21

11 Mar '21

> On 11 Mar 2021, at 8:46 AM, Pierre Jolivet <pierre(a)joliv.et> wrote: > >> On 11 Mar 2021, at 6:16 AM, Barry Smith <bsmith(a)petsc.dev <mailto:[email protected]>> wrote: >> >> Eric, >> >> Sorry about not being more immediate. We still have this in our active email so you don't need to submit individual issues. We'll try to get to them as soon as we can. > > Indeed, I’m still trying to figure this out. > I realized that some of my configure flags were different than yours, e.g., no --with-memalign. > I’ve also added SuperLU_DIST to my installation. > Still, I can’t reproduce any issue. > I will continue looking into this, it appears I’m seeing some valgrind errors, but I don’t know if this is some side effect of OpenMPI not being valgrind-clean (last time I checked, there was no error with MPICH). It looks like Valgrind + OpenMPI (+ OpenMP?) is complaining about uninitialized memory in PetscSFGetMultiSF(). Could you please try out the following branch https://gitlab.com/petsc/petsc/-/commits/jolivet/fix-valgrind-openmpi <https://gitlab.com/petsc/petsc/-/commits/jolivet/fix-valgrind-openmpi> ? I’m not sure why there would be such a warning with OpenMPI and not with MPICH, and it is unlikely to fix anything, but for good measure, after compilation, could you please try: $ make -f gmakefile test search='snes_tutorials-ex12_quad_hpddm_reuse_baij’ Thanks, Pierre > Thank you for your patience, > Pierre > > /usr/bin/gmake -f gmakefile test test-fail=1 > Using MAKEFLAGS: test-fail=1 > TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex12_quad_hpddm_reuse_baij.counts > ok snes_tutorials-ex12_quad_hpddm_reuse_baij > ok diff-snes_tutorials-ex12_quad_hpddm_reuse_baij > TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tests-ex33_superlu_dist_2.counts > ok ksp_ksp_tests-ex33_superlu_dist_2 > ok diff-ksp_ksp_tests-ex33_superlu_dist_2 > TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tests-ex49_superlu_dist.counts > ok ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-0_conv-0 > ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-0_conv-0 > ok ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-0_conv-1 > ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-0_conv-1 > ok ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-1_conv-0 > ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-1_conv-0 > ok ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-1_conv-1 > ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-1_conv-1 > ok ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-0_conv-0 > ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-0_conv-0 > ok ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-0_conv-1 > ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-0_conv-1 > ok ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-1_conv-0 > ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-1_conv-0 > ok ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-1_conv-1 > ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-1_conv-1 > TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex50_tut_2.counts > ok ksp_ksp_tutorials-ex50_tut_2 > ok diff-ksp_ksp_tutorials-ex50_tut_2 > TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tests-ex33_superlu_dist.counts > ok ksp_ksp_tests-ex33_superlu_dist > ok diff-ksp_ksp_tests-ex33_superlu_dist > TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex56_hypre.counts > ok snes_tutorials-ex56_hypre > ok diff-snes_tutorials-ex56_hypre > TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex56_2.counts > ok ksp_ksp_tutorials-ex56_2 > ok diff-ksp_ksp_tutorials-ex56_2 > TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex17_3d_q3_trig_elas.counts > ok snes_tutorials-ex17_3d_q3_trig_elas > ok diff-snes_tutorials-ex17_3d_q3_trig_elas > TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex12_quad_hpddm_reuse_threshold_baij.counts > ok snes_tutorials-ex12_quad_hpddm_reuse_threshold_baij > ok diff-snes_tutorials-ex12_quad_hpddm_reuse_threshold_baij > TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5_superlu_dist_3.counts > not ok ksp_ksp_tutorials-ex5_superlu_dist_3 # Error code: 1 > # srun: error: Unable to create step for job 1426755: More processors requested than permitted > ok ksp_ksp_tutorials-ex5_superlu_dist_3 # SKIP Command failed so no diff > TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5f_superlu_dist.counts > ok ksp_ksp_tutorials-ex5f_superlu_dist # SKIP Fortran required for this test > TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex12_tri_parmetis_hpddm_baij.counts > ok snes_tutorials-ex12_tri_parmetis_hpddm_baij > ok diff-snes_tutorials-ex12_tri_parmetis_hpddm_baij > TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex19_tut_3.counts > ok snes_tutorials-ex19_tut_3 > ok diff-snes_tutorials-ex19_tut_3 > TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex17_3d_q3_trig_vlap.counts > ok snes_tutorials-ex17_3d_q3_trig_vlap > ok diff-snes_tutorials-ex17_3d_q3_trig_vlap > TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5f_superlu_dist_3.counts > ok ksp_ksp_tutorials-ex5f_superlu_dist_3 # SKIP Fortran required for this test > TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex19_superlu_dist.counts > ok snes_tutorials-ex19_superlu_dist > ok diff-snes_tutorials-ex19_superlu_dist > TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex56_attach_mat_nearnullspace-1_bddc_approx_hypre.counts > ok snes_tutorials-ex56_attach_mat_nearnullspace-1_bddc_approx_hypre > ok diff-snes_tutorials-ex56_attach_mat_nearnullspace-1_bddc_approx_hypre > TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex49_hypre_nullspace.counts > ok ksp_ksp_tutorials-ex49_hypre_nullspace > ok diff-ksp_ksp_tutorials-ex49_hypre_nullspace > TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex19_superlu_dist_2.counts > ok snes_tutorials-ex19_superlu_dist_2 > ok diff-snes_tutorials-ex19_superlu_dist_2 > TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5_superlu_dist_2.counts > not ok ksp_ksp_tutorials-ex5_superlu_dist_2 # Error code: 1 > # srun: error: Unable to create step for job 1426755: More processors requested than permitted > ok ksp_ksp_tutorials-ex5_superlu_dist_2 # SKIP Command failed so no diff > TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex56_attach_mat_nearnullspace-0_bddc_approx_hypre.counts > ok snes_tutorials-ex56_attach_mat_nearnullspace-0_bddc_approx_hypre > ok diff-snes_tutorials-ex56_attach_mat_nearnullspace-0_bddc_approx_hypre > TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex64_1.counts > ok ksp_ksp_tutorials-ex64_1 > ok diff-ksp_ksp_tutorials-ex64_1 > TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5_superlu_dist.counts > not ok ksp_ksp_tutorials-ex5_superlu_dist # Error code: 1 > # srun: error: Unable to create step for job 1426755: More processors requested than permitted > ok ksp_ksp_tutorials-ex5_superlu_dist # SKIP Command failed so no diff > TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5f_superlu_dist_2.counts > ok ksp_ksp_tutorials-ex5f_superlu_dist_2 # SKIP Fortran required for this test > >> Barry >> >> >>> On Mar 10, 2021, at 11:03 PM, Eric Chamberland <Eric.Chamberland(a)giref.ulaval.ca <mailto:[email protected]>> wrote: >>> >>> Barry, >>> >>> to get a some follow up on --with-openmp=1 failures, shall I open gitlab issues for: >>> >>> a) all hypre failures giving DIVERGED_INDEFINITE_PC >>> >>> b) all superlu_dist failures giving different results with initia and "Exceeded timeout limit of 60 s" >>> >>> c) hpddm failures "free(): invalid next size (fast)" and "Segmentation Violation" >>> >>> d) all tao's "Exceeded timeout limit of 60 s" >>> >>> I don't see how I could do all these debugging by myself... >>> >>> Thanks, >>> >>> Eric >>> >>> >> >

1 0

Re: [petsc-dev] Petsc "make test" have more failures for --with-openmp=1
by Eric Chamberland 11 Mar '21

11 Mar '21

Hi Pierre, ok, that's interesting! I will try to build a docker image until tomorrow and give you the exact recipe to reproduce the bugs. Eric On 2021-03-11 2:46 a.m., Pierre Jolivet wrote: > > >> On 11 Mar 2021, at 6:16 AM, Barry Smith <bsmith(a)petsc.dev >> <mailto:[email protected]>> wrote: >> >> >> Eric, >> >> Sorry about not being more immediate. We still have this in our >> active email so you don't need to submit individual issues. We'll try >> to get to them as soon as we can. > > Indeed, I’m still trying to figure this out. > I realized that some of my configure flags were different than yours, > e.g., no --with-memalign. > I’ve also added SuperLU_DIST to my installation. > Still, I can’t reproduce any issue. > I will continue looking into this, it appears I’m seeing some valgrind > errors, but I don’t know if this is some side effect of OpenMPI not > being valgrind-clean (last time I checked, there was no error with MPICH). > > Thank you for your patience, > Pierre > > /usr/bin/gmake -f gmakefile test test-fail=1 > Using MAKEFLAGS: test-fail=1 > TEST > arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex12_quad_hpddm_reuse_baij.counts > ok snes_tutorials-ex12_quad_hpddm_reuse_baij > ok diff-snes_tutorials-ex12_quad_hpddm_reuse_baij > TEST > arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tests-ex33_superlu_dist_2.counts > ok ksp_ksp_tests-ex33_superlu_dist_2 > ok diff-ksp_ksp_tests-ex33_superlu_dist_2 > TEST > arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tests-ex49_superlu_dist.counts > ok ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-0_conv-0 > ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-0_conv-0 > ok ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-0_conv-1 > ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-0_conv-1 > ok ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-1_conv-0 > ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-1_conv-0 > ok ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-1_conv-1 > ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-1_conv-1 > ok ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-0_conv-0 > ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-0_conv-0 > ok ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-0_conv-1 > ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-0_conv-1 > ok ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-1_conv-0 > ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-1_conv-0 > ok ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-1_conv-1 > ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-1_conv-1 > TEST > arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex50_tut_2.counts > ok ksp_ksp_tutorials-ex50_tut_2 > ok diff-ksp_ksp_tutorials-ex50_tut_2 > TEST > arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tests-ex33_superlu_dist.counts > ok ksp_ksp_tests-ex33_superlu_dist > ok diff-ksp_ksp_tests-ex33_superlu_dist > TEST > arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex56_hypre.counts > ok snes_tutorials-ex56_hypre > ok diff-snes_tutorials-ex56_hypre > TEST > arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex56_2.counts > ok ksp_ksp_tutorials-ex56_2 > ok diff-ksp_ksp_tutorials-ex56_2 > TEST > arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex17_3d_q3_trig_elas.counts > ok snes_tutorials-ex17_3d_q3_trig_elas > ok diff-snes_tutorials-ex17_3d_q3_trig_elas > TEST > arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex12_quad_hpddm_reuse_threshold_baij.counts > ok snes_tutorials-ex12_quad_hpddm_reuse_threshold_baij > ok diff-snes_tutorials-ex12_quad_hpddm_reuse_threshold_baij > TEST > arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5_superlu_dist_3.counts > not ok ksp_ksp_tutorials-ex5_superlu_dist_3 # Error code: 1 > #srun: error: Unable to create step for job 1426755: More processors > requested than permitted > ok ksp_ksp_tutorials-ex5_superlu_dist_3 # SKIP Command failed so no diff > TEST > arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5f_superlu_dist.counts > ok ksp_ksp_tutorials-ex5f_superlu_dist # SKIP Fortran required for > this test > TEST > arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex12_tri_parmetis_hpddm_baij.counts > ok snes_tutorials-ex12_tri_parmetis_hpddm_baij > ok diff-snes_tutorials-ex12_tri_parmetis_hpddm_baij > TEST > arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex19_tut_3.counts > ok snes_tutorials-ex19_tut_3 > ok diff-snes_tutorials-ex19_tut_3 > TEST > arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex17_3d_q3_trig_vlap.counts > ok snes_tutorials-ex17_3d_q3_trig_vlap > ok diff-snes_tutorials-ex17_3d_q3_trig_vlap > TEST > arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5f_superlu_dist_3.counts > ok ksp_ksp_tutorials-ex5f_superlu_dist_3 # SKIP Fortran required for > this test > TEST > arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex19_superlu_dist.counts > ok snes_tutorials-ex19_superlu_dist > ok diff-snes_tutorials-ex19_superlu_dist > TEST > arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex56_attach_mat_nearnullspace-1_bddc_approx_hypre.counts > ok snes_tutorials-ex56_attach_mat_nearnullspace-1_bddc_approx_hypre > ok diff-snes_tutorials-ex56_attach_mat_nearnullspace-1_bddc_approx_hypre > TEST > arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex49_hypre_nullspace.counts > ok ksp_ksp_tutorials-ex49_hypre_nullspace > ok diff-ksp_ksp_tutorials-ex49_hypre_nullspace > TEST > arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex19_superlu_dist_2.counts > ok snes_tutorials-ex19_superlu_dist_2 > ok diff-snes_tutorials-ex19_superlu_dist_2 > TEST > arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5_superlu_dist_2.counts > not ok ksp_ksp_tutorials-ex5_superlu_dist_2 # Error code: 1 > #srun: error: Unable to create step for job 1426755: More processors > requested than permitted > ok ksp_ksp_tutorials-ex5_superlu_dist_2 # SKIP Command failed so no diff > TEST > arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex56_attach_mat_nearnullspace-0_bddc_approx_hypre.counts > ok snes_tutorials-ex56_attach_mat_nearnullspace-0_bddc_approx_hypre > ok diff-snes_tutorials-ex56_attach_mat_nearnullspace-0_bddc_approx_hypre > TEST > arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex64_1.counts > ok ksp_ksp_tutorials-ex64_1 > ok diff-ksp_ksp_tutorials-ex64_1 > TEST > arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5_superlu_dist.counts > not ok ksp_ksp_tutorials-ex5_superlu_dist # Error code: 1 > #srun: error: Unable to create step for job 1426755: More processors > requested than permitted > ok ksp_ksp_tutorials-ex5_superlu_dist # SKIP Command failed so no diff > TEST > arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5f_superlu_dist_2.counts > ok ksp_ksp_tutorials-ex5f_superlu_dist_2 # SKIP Fortran required for > this test > >> Barry >> >> >>> On Mar 10, 2021, at 11:03 PM, Eric Chamberland >>> <Eric.Chamberland(a)giref.ulaval.ca >>> <mailto:[email protected]>> wrote: >>> >>> Barry, >>> >>> to get a some follow up on --with-openmp=1 failures, shall I open >>> gitlab issues for: >>> >>> a) all hypre failures giving DIVERGED_INDEFINITE_PC >>> >>> b) all superlu_dist failures giving different results with initia >>> and "Exceeded timeout limit of 60 s" >>> >>> c) hpddm failures "free(): invalid next size (fast)" and >>> "Segmentation Violation" >>> >>> d) all tao's "Exceeded timeout limit of 60 s" >>> >>> I don't see how I could do all these debugging by myself... >>> >>> Thanks, >>> >>> Eric >>> >>> >> > -- Eric Chamberland, ing., M. Ing Professionnel de recherche GIREF/Université Laval (418) 656-2131 poste 41 22 42

1 0

Re: [petsc-dev] Petsc "make test" have more failures for --with-openmp=1
by Pierre Jolivet 11 Mar '21

11 Mar '21

> On 11 Mar 2021, at 6:16 AM, Barry Smith <bsmith(a)petsc.dev> wrote: > > > Eric, > > Sorry about not being more immediate. We still have this in our active email so you don't need to submit individual issues. We'll try to get to them as soon as we can. Indeed, I’m still trying to figure this out. I realized that some of my configure flags were different than yours, e.g., no --with-memalign. I’ve also added SuperLU_DIST to my installation. Still, I can’t reproduce any issue. I will continue looking into this, it appears I’m seeing some valgrind errors, but I don’t know if this is some side effect of OpenMPI not being valgrind-clean (last time I checked, there was no error with MPICH). Thank you for your patience, Pierre /usr/bin/gmake -f gmakefile test test-fail=1 Using MAKEFLAGS: test-fail=1 TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex12_quad_hpddm_reuse_baij.counts ok snes_tutorials-ex12_quad_hpddm_reuse_baij ok diff-snes_tutorials-ex12_quad_hpddm_reuse_baij TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tests-ex33_superlu_dist_2.counts ok ksp_ksp_tests-ex33_superlu_dist_2 ok diff-ksp_ksp_tests-ex33_superlu_dist_2 TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tests-ex49_superlu_dist.counts ok ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-0_conv-0 ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-0_conv-0 ok ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-0_conv-1 ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-0_conv-1 ok ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-1_conv-0 ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-1_conv-0 ok ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-1_conv-1 ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-1_conv-1 ok ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-0_conv-0 ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-0_conv-0 ok ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-0_conv-1 ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-0_conv-1 ok ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-1_conv-0 ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-1_conv-0 ok ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-1_conv-1 ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-1_conv-1 TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex50_tut_2.counts ok ksp_ksp_tutorials-ex50_tut_2 ok diff-ksp_ksp_tutorials-ex50_tut_2 TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tests-ex33_superlu_dist.counts ok ksp_ksp_tests-ex33_superlu_dist ok diff-ksp_ksp_tests-ex33_superlu_dist TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex56_hypre.counts ok snes_tutorials-ex56_hypre ok diff-snes_tutorials-ex56_hypre TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex56_2.counts ok ksp_ksp_tutorials-ex56_2 ok diff-ksp_ksp_tutorials-ex56_2 TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex17_3d_q3_trig_elas.counts ok snes_tutorials-ex17_3d_q3_trig_elas ok diff-snes_tutorials-ex17_3d_q3_trig_elas TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex12_quad_hpddm_reuse_threshold_baij.counts ok snes_tutorials-ex12_quad_hpddm_reuse_threshold_baij ok diff-snes_tutorials-ex12_quad_hpddm_reuse_threshold_baij TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5_superlu_dist_3.counts not ok ksp_ksp_tutorials-ex5_superlu_dist_3 # Error code: 1 # srun: error: Unable to create step for job 1426755: More processors requested than permitted ok ksp_ksp_tutorials-ex5_superlu_dist_3 # SKIP Command failed so no diff TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5f_superlu_dist.counts ok ksp_ksp_tutorials-ex5f_superlu_dist # SKIP Fortran required for this test TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex12_tri_parmetis_hpddm_baij.counts ok snes_tutorials-ex12_tri_parmetis_hpddm_baij ok diff-snes_tutorials-ex12_tri_parmetis_hpddm_baij TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex19_tut_3.counts ok snes_tutorials-ex19_tut_3 ok diff-snes_tutorials-ex19_tut_3 TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex17_3d_q3_trig_vlap.counts ok snes_tutorials-ex17_3d_q3_trig_vlap ok diff-snes_tutorials-ex17_3d_q3_trig_vlap TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5f_superlu_dist_3.counts ok ksp_ksp_tutorials-ex5f_superlu_dist_3 # SKIP Fortran required for this test TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex19_superlu_dist.counts ok snes_tutorials-ex19_superlu_dist ok diff-snes_tutorials-ex19_superlu_dist TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex56_attach_mat_nearnullspace-1_bddc_approx_hypre.counts ok snes_tutorials-ex56_attach_mat_nearnullspace-1_bddc_approx_hypre ok diff-snes_tutorials-ex56_attach_mat_nearnullspace-1_bddc_approx_hypre TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex49_hypre_nullspace.counts ok ksp_ksp_tutorials-ex49_hypre_nullspace ok diff-ksp_ksp_tutorials-ex49_hypre_nullspace TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex19_superlu_dist_2.counts ok snes_tutorials-ex19_superlu_dist_2 ok diff-snes_tutorials-ex19_superlu_dist_2 TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5_superlu_dist_2.counts not ok ksp_ksp_tutorials-ex5_superlu_dist_2 # Error code: 1 # srun: error: Unable to create step for job 1426755: More processors requested than permitted ok ksp_ksp_tutorials-ex5_superlu_dist_2 # SKIP Command failed so no diff TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex56_attach_mat_nearnullspace-0_bddc_approx_hypre.counts ok snes_tutorials-ex56_attach_mat_nearnullspace-0_bddc_approx_hypre ok diff-snes_tutorials-ex56_attach_mat_nearnullspace-0_bddc_approx_hypre TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex64_1.counts ok ksp_ksp_tutorials-ex64_1 ok diff-ksp_ksp_tutorials-ex64_1 TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5_superlu_dist.counts not ok ksp_ksp_tutorials-ex5_superlu_dist # Error code: 1 # srun: error: Unable to create step for job 1426755: More processors requested than permitted ok ksp_ksp_tutorials-ex5_superlu_dist # SKIP Command failed so no diff TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5f_superlu_dist_2.counts ok ksp_ksp_tutorials-ex5f_superlu_dist_2 # SKIP Fortran required for this test > Barry > > >> On Mar 10, 2021, at 11:03 PM, Eric Chamberland <Eric.Chamberland(a)giref.ulaval.ca <mailto:[email protected]>> wrote: >> >> Barry, >> >> to get a some follow up on --with-openmp=1 failures, shall I open gitlab issues for: >> >> a) all hypre failures giving DIVERGED_INDEFINITE_PC >> >> b) all superlu_dist failures giving different results with initia and "Exceeded timeout limit of 60 s" >> >> c) hpddm failures "free(): invalid next size (fast)" and "Segmentation Violation" >> >> d) all tao's "Exceeded timeout limit of 60 s" >> >> I don't see how I could do all these debugging by myself... >> >> Thanks, >> >> Eric >> >> >

1 0

Re: [petsc-dev] Petsc "make test" have more failures for --with-openmp=1
by Barry Smith 10 Mar '21

10 Mar '21

Eric, Sorry about not being more immediate. We still have this in our active email so you don't need to submit individual issues. We'll try to get to them as soon as we can. Barry > On Mar 10, 2021, at 11:03 PM, Eric Chamberland <Eric.Chamberland(a)giref.ulaval.ca> wrote: > > Barry, > > to get a some follow up on --with-openmp=1 failures, shall I open gitlab issues for: > > a) all hypre failures giving DIVERGED_INDEFINITE_PC > > b) all superlu_dist failures giving different results with initia and "Exceeded timeout limit of 60 s" > > c) hpddm failures "free(): invalid next size (fast)" and "Segmentation Violation" > > d) all tao's "Exceeded timeout limit of 60 s" > > I don't see how I could do all these debugging by myself... > > Thanks, > > Eric > >

1 0

Re: [petsc-dev] Petsc "make test" have more failures for --with-openmp=1
by Eric Chamberland 10 Mar '21

10 Mar '21

Barry, to get a some follow up on --with-openmp=1 failures, shall I open gitlab issues for: a) all hypre failures giving DIVERGED_INDEFINITE_PC b) all superlu_dist failures giving different results with initia and "Exceeded timeout limit of 60 s" c) hpddm failures "free(): invalid next size (fast)" and "Segmentation Violation" d) all tao's "Exceeded timeout limit of 60 s" I don't see how I could do all these debugging by myself... Thanks, Eric

1 0

Re: [petsc-dev] petsc release plan for march 2021
by Satish Balay 09 Mar '21

09 Mar '21

On Tue, 9 Mar 2021, Patrick Sanan wrote: > > > > Am 09.03.2021 um 19:58 schrieb Satish Balay via petsc-dev <petsc-dev(a)mcs.anl.gov>: > > > > All, > > > > Its time for another PETSc release - due end of March. > > > > For this release [3.15], will work with the following dates: > > > > - feature freeze: March 28 say 5PM EST > > - release: March 30 say 5PM EST > > > > Merges after freeze should contain only fixes that would normally be acceptable to release workflow. > > > > I've created a new milestone 'v3.14-release'. So if you are working on a MR with the goal of merging before release - its best to use this tag with the MR. > v3.15-release ? Sorry - yes. I created v3.15-release tag Satish > > > > And it would be good to avoid merging large changes at the last minute. And not have merge requests stuck in need of reviews, testing and other necessary tasks. > > > > And I would think the testing/CI resources would get stressed in this timeframe - so it would be good to use them judiciously if possible. > > > > - if there are failures in stage-2 or 3 - and its no longer necessary to complete all the jobs - one can 'cancel' the pipeline. > > - if a fix needs to be tested - one can first test with only the failed jobs (if this is known) - before doing a full test pipeline. i.e: > > - use the automatically started and paused 'merge-request' pipeline (or start new 'web' pipeline, and cancel it immediately) > > - now toggle only the jobs that need to be run > > - [on success of the selected jobs] if one wants to run the full pipeleine - click 'retry' - and the remaining canceled jobs should now get scheduled. > > > > Thanks, > > Satish >

1 0

Re: [petsc-dev] petsc release plan for march 2021
by Patrick Sanan 09 Mar '21

09 Mar '21

> Am 09.03.2021 um 19:58 schrieb Satish Balay via petsc-dev <petsc-dev(a)mcs.anl.gov>: > > All, > > Its time for another PETSc release - due end of March. > > For this release [3.15], will work with the following dates: > > - feature freeze: March 28 say 5PM EST > - release: March 30 say 5PM EST > > Merges after freeze should contain only fixes that would normally be acceptable to release workflow. > > I've created a new milestone 'v3.14-release'. So if you are working on a MR with the goal of merging before release - its best to use this tag with the MR. v3.15-release ? > > And it would be good to avoid merging large changes at the last minute. And not have merge requests stuck in need of reviews, testing and other necessary tasks. > > And I would think the testing/CI resources would get stressed in this timeframe - so it would be good to use them judiciously if possible. > > - if there are failures in stage-2 or 3 - and its no longer necessary to complete all the jobs - one can 'cancel' the pipeline. > - if a fix needs to be tested - one can first test with only the failed jobs (if this is known) - before doing a full test pipeline. i.e: > - use the automatically started and paused 'merge-request' pipeline (or start new 'web' pipeline, and cancel it immediately) > - now toggle only the jobs that need to be run > - [on success of the selected jobs] if one wants to run the full pipeleine - click 'retry' - and the remaining canceled jobs should now get scheduled. > > Thanks, > Satish

1 0

petsc release plan for march 2021
by Satish Balay 09 Mar '21

09 Mar '21

All, Its time for another PETSc release - due end of March. For this release [3.15], will work with the following dates: - feature freeze: March 28 say 5PM EST - release: March 30 say 5PM EST Merges after freeze should contain only fixes that would normally be acceptable to release workflow. I've created a new milestone 'v3.14-release'. So if you are working on a MR with the goal of merging before release - its best to use this tag with the MR. And it would be good to avoid merging large changes at the last minute. And not have merge requests stuck in need of reviews, testing and other necessary tasks. And I would think the testing/CI resources would get stressed in this timeframe - so it would be good to use them judiciously if possible. - if there are failures in stage-2 or 3 - and its no longer necessary to complete all the jobs - one can 'cancel' the pipeline. - if a fix needs to be tested - one can first test with only the failed jobs (if this is known) - before doing a full test pipeline. i.e: - use the automatically started and paused 'merge-request' pipeline (or start new 'web' pipeline, and cancel it immediately) - now toggle only the jobs that need to be run - [on success of the selected jobs] if one wants to run the full pipeleine - click 'retry' - and the remaining canceled jobs should now get scheduled. Thanks, Satish

1 0

Re: [petsc-dev] Fwd: [MCS-PI] Cerebras AI Training workshop
by Dener, Alp 08 Mar '21

08 Mar '21

Hi Barry, The email caught my attention too and I’m considering registering. The focus on ML workloads is not a downside at least for me. I’m gradually moving towards ML training in TAO with pyTorch interfaces (part of LDRD with Richard and Hong) and I have a plasma physics app related to this that can be ported to this machine, so it seems like good timing to be part of this. Alp Dener Argonne National Laboratory Mathematics and Computer Science https://alp.dener.me On Mar 8, 2021, at 9:24 PM, Barry Smith <bsmith(a)petsc.dev> wrote: PETSc folks, This actually seems like a pretty interesting machine for PETSc. It seems ANL has had it for a very long time but kept it off the website and didn't provide much access except to chosen people. It would be great if at least one PETSc person could attend to see if there are possibilities, though the course may focus exclusively on ML type workloads. Barry Begin forwarded message: From: "Taylor, Valerie E." <vtaylor(a)anl.gov<mailto:[email protected]>> Subject: [MCS-PI] FW: Cerebras AI Training workshop Date: March 8, 2021 at 8:31:27 PM CST To: "mcs-pi(a)mcs.anl.gov<mailto:[email protected]>" <mcs-pi(a)mcs.anl.gov<mailto:[email protected]>>, "postdocs(a)mcs.anl.gov<mailto:[email protected]>" <postdocs(a)mcs.anl.gov<mailto:[email protected]>> Message-Id: <DF10FF2B-21D6-4C03-B0B2-08219115AAE0(a)anl.gov<mailto:[email protected]>> Hello Everyone: Below, please find details about a two-day Argonne user training workshop for Cerebras. Registration is required. There will be more training sessions in the near future. --Valerie From: Venkatram Vishwanath <venkat(a)anl.gov<mailto:[email protected]>> Date: Monday, March 8, 2021 at 1:14 PM To: "Taylor, Valerie E." <vtaylor(a)anl.gov<mailto:[email protected]>>, Ian Foster <foster(a)anl.gov<mailto:[email protected]>>, "Habib, Salman" <habib(a)anl.gov<mailto:[email protected]>>, "Papka, Michael E." <papka(a)anl.gov<mailto:[email protected]>>, Rick Stevens <stevens(a)anl.gov<mailto:[email protected]>> Cc: Venkatram Vishwanath <venkat(a)anl.gov<mailto:[email protected]>>, "Thakur, Rajeev" <thakur(a)anl.gov<mailto:[email protected]>>, "Emani, Murali Krishna" <memani(a)anl.gov<mailto:[email protected]>> Subject: Cerebras AI Training workshop Dear All, We will be hosting a two-day Argonne user training workshop for the Cerebras AI system on March 22 (Mon) and March 23 (Tuesday). This will be held from 12:00 PM to ~4:00 PM each day and will be virtual. Here is a link to the registration page and we will share the meeting link with the accepted registrants. https://events.cels.anl.gov/event/97/ For the training, we will provide an overview of the Cerebras hardware and the software stack. We will demonstrate how one can log into the system, run sample examples, and get started on the system. Detailed agenda is on the registration page. We expect attendees to be very hands-on, port their science deep-learning workloads to run on the system, provide feedback on the system and contribute to the training materials. We expect to have more training sessions in the near future as well. We plan to target this training session for users willing to be very much hands-on. We may need to filter attendees based on the number of registrants for improved workshop experience and given system constraints. Please do forward this to others who will benefit from attending this. Do let us know if you have any questions and feel free to reach out to us. Best regards, Murali, Rajeev and Venkat _______________________________________________ MCS-PI mailing list MCS-PI(a)lists.mcs.anl.gov<mailto:[email protected]> https://lists.mcs.anl.gov/mailman/listinfo/mcs-pi

1 0