April 2021 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] DMNetwork static sizing
by Matthew Knepley 07 Apr '21

07 Apr '21

On Tue, Apr 6, 2021 at 7:44 PM Abhyankar, Shrirang G < shrirang.abhyankar(a)pnnl.gov> wrote: > Hong, > > It was just to keep things simple. We can have an API (and a run-time > option) for setting the max. number of components instead of a configure > option. I can work on making this change. > I would advocate just carrying the size of those arrays in the struct. That way it can be completely flexible, and you can just check at runtime if there is a chance of out of bounds indexing. Thanks, Matt > > > Thanks, > > Shri > > *From: *petsc-dev <petsc-dev-bounces(a)mcs.anl.gov> on behalf of PETSc > Development <petsc-dev(a)mcs.anl.gov> > *Reply-To: *"Zhang, Hong" <hzhang(a)mcs.anl.gov> > *Date: *Tuesday, April 6, 2021 at 4:46 PM > *To: *Matthew Knepley <knepley(a)gmail.com>, PETSc Development < > petsc-dev(a)mcs.anl.gov> > *Cc: *"Abhyankar, Shrirang G" <shrirang.abhyankar(a)pnnl.gov> > *Subject: *Re: [petsc-dev] DMNetwork static sizing > > > > Shri, > > You designed this approach. Is it intended or out of implementation > convenience at the time? > > Hong > ------------------------------ > > *From:* petsc-dev <petsc-dev-bounces(a)mcs.anl.gov> on behalf of Matthew > Knepley <knepley(a)gmail.com> > *Sent:* Monday, April 5, 2021 5:47 AM > *To:* PETSc <petsc-dev(a)mcs.anl.gov> > *Subject:* [petsc-dev] DMNetwork static sizing > > > > Dowe really need a configure time constant for > > > > struct _p_DMNetworkComponentHeader { > PetscInt index; /* index for user input global edge and vertex */ > PetscInt subnetid; /* Id for subnetwork */ > PetscInt ndata; /* number of components */ > PetscInt size[PETSC_DMNETWORK_MAXIMUM_COMPONENTS_PER_POINT]; > PetscInt key[PETSC_DMNETWORK_MAXIMUM_COMPONENTS_PER_POINT]; > PetscInt offset[PETSC_DMNETWORK_MAXIMUM_COMPONENTS_PER_POINT]; > PetscInt nvar[PETSC_DMNETWORK_MAXIMUM_COMPONENTS_PER_POINT]; /* Number > of variables */ > PetscInt offsetvarrel[PETSC_DMNETWORK_MAXIMUM_COMPONENTS_PER_POINT]; /* > offset from the first variable of the network point */ > } PETSC_ATTRIBUTEALIGNED(PetscMax(sizeof(double),sizeof(PetscScalar))); > > > > Can't we just allocate this struct when needed and carry the size along? > > > > This design seem to go against the rest of what we do in PETSc? > > > > Thanks, > > > > Matt > > > > -- > > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > > > https://www.cse.buffalo.edu/~knepley/ > <https://gcc02.safelinks.protection.outlook.com/?url=http:%2F%2Fwww.cse.buff…> > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>

1 0

回复： Petsc: Error code 1
by Chen Gang 06 Apr '21

06 Apr '21

ARM: configure with the following: -ffp-contract=off ./configure -with-debugging=0 COPTFLAGS='-O3 -ffp-contract=off -march=armv8.2-a -mtune=tsv110' CXXOPTFLAGS='-O3 -ffp-contract=off -march=armv8.2-a -mtune=tsv110' FOPTFLAGS='-O3 -ffp-contract=off -march=armv8.2-a -mtune=tsv110' --with-x=1 -download-fblaslapack PETSC-KERNEL-USE-UNROLL-4 There are 3 failures, all related to FC code # ------------- #   Summary # ------------- # FAILED diff-tao_bound_tutorials-plate2f_2 diff-tao_bound_tutorials-plate2f_1 diff-vec_is_is_tutorials-ex2f_1 # success 7547/9528 tests (79.2%) # failed 3/9528 tests (0.0%) # todo 225/9528 tests (2.4%) # skip 1753/9528 tests (18.4%) # # Wall clock time for tests: 1371 sec # Approximate CPU time (not incl. build time): 106722.62999999999 sec # # To rerun failed tests: #     /opt/rh/devtoolset-9/root/usr/bin/gmake -f gmakefile test test-fail=1 # # Timing summary (actual test time / total CPU time): #   dm_tests-ex36_3dp1: 467.68 sec / 477.71 sec #   dm_impls_stag_tests-ex1_multidof_3: 395.16 sec / 398.45 sec #   ts_tutorials-ex29_1: 236.54 sec / 238.30 sec #   dm_impls_stag_tests-ex1_basic_2: 205.41 sec / 207.34 sec #   dm_tests-ex34_1: 178.94 sec / 180.26 sec ------------------ 原始邮件 ------------------ 发件人: "Chen Gang" <569615491(a)qq.com>; 发送时间: 2021年4月7日(星期三) 上午8:45 收件人: "petsc-dev"<petsc-dev(a)mcs.anl.gov>;"Barry Smith"<bsmith(a)petsc.dev>; 抄送: "Alp Dener"<alp.dener(a)gmail.com>;"petsc-dev"<petsc-dev(a)mcs.anl.gov>;"cglwdm"<cglwdm(a)scu.edu.cn>; 主题: 回复： [petsc-dev] Petsc: Error code 1 Here is my configure options: ./configure -with-debugging=0 COPTFLAGS='-O3 -march=armv8.2-a -mtune=tsv110' CXXOPTFLAGS='-O3 -march=armv8.2-a -mtune=tsv110' FOPTFLAGS='-O3 -march=armv8.2-a -mtune=tsv110' --with-x=1 -download-fblaslapack PETSC-KERNEL-USE-UNROLL-4 --force The gcc is 9.3.1, the mpich is 3.4.1.  （When I use openmpi 4.10 or mpich 4.0 alpha, there are more failures related to the mpi.） ------------------ 原始邮件 ------------------ 发件人: "petsc-dev" <balay(a)mcs.anl.gov>; 发送时间: 2021年4月7日(星期三) 凌晨1:06 收件人: "Barry Smith"<bsmith(a)petsc.dev>; 抄送: "Chen Gang"<569615491(a)qq.com>;"Alp Dener"<alp.dener(a)gmail.com>;"petsc-dev"<petsc-dev(a)mcs.anl.gov>;"cglwdm"<cglwdm(a)scu.edu.cn>; 主题: Re: [petsc-dev] Petsc: Error code 1 > See the attachements. alltest.log is on a machine with 96 cores, ARM, with FC,gcc9.3.5,mpich3.4.1,fblaslapack; 6 failures Perhaps this is an issue with ARM - and such diffs are expected - as we already have multiple alt files for some of these tests $ ls -lt src/tao/bound/tutorials/output/plate2f_* -rw-r--r--. 1 balay balay 1029 Mar 23 19:48 src/tao/bound/tutorials/output/plate2f_1_alt.out -rw-r--r--. 1 balay balay 1071 Mar 23 19:48 src/tao/bound/tutorials/output/plate2f_1.out -rw-r--r--. 1 balay balay 1029 Mar 23 19:48 src/tao/bound/tutorials/output/plate2f_2_alt.out -rw-r--r--. 1 balay balay 1071 Mar 23 19:48 src/tao/bound/tutorials/output/plate2f_2.out >>>>>> not ok diff-vec_is_is_tutorials-ex2f_1 # Error code: 1 #       16,24d15 #       <   5 #       <   7 #       <   9 #       <  11 #       <  13 #       <  15 #       <  17 #       <  19 #       <  21 <<<<< This one is puzzling - missing fortran stdout? Perhaps compile issue on ARM? [its a sequential example - so can't blame MPI] Or they are all related to the optimization flags used? What configure options were used for the build? Satish On Tue, 6 Apr 2021, Barry Smith wrote: > >     Alp, > >    Except for the first test, these are all optimization problems (mostly in Fortran). The function values are very different so I am sending it to our optimization expert to take a look at it. The differences could possibly be related to the use of real() and maybe the direct use of floating point numbers that the compiler first treats as single and then converts to double thus losing precision. > >    Chen Gang, I assume you compiled with the default standard precision PETSc configure options? > > > > On Apr 6, 2021, at 3:56 AM, Chen Gang <569615491(a)qq.com<mailto:[email protected]>> wrote: > > > See the attachements. alltest.log is on a machine with 96 cores, ARM, with FC,gcc9.3.5,mpich3.4.1,fblaslapack; 6 failures >                                  alltest2.log is on an intel machine with 40  cores,x86, without FC; icc&mkl& intel mpi; only 1 failure > > ------------------ 原始邮件 ------------------ > 发件人: "petsc-dev" <balay(a)mcs.anl.gov<mailto:[email protected]>>; > 发送时间: 2021年4月6日(星期二) 中午12:38 > 收件人: "petsc-dev"<petsc-dev(a)mcs.anl.gov<mailto:[email protected]>>; > 抄送: "Chen Gang"<569615491(a)qq.com<mailto:[email protected]>>;"cglwdm"<cglwdm(a)scu.edu.cn<mailto:[email protected]>>; > 主题: Re: [petsc-dev] Petsc: Error code 1 > > Note: do not use '-j' with alltests. > > And run alltests on both machines [but *not* at the same time on machines] and send us logs from both the runs. > > Satish > > > On Mon, 5 Apr 2021, Satish Balay wrote: > > > Try: > > > > make alltests TIMEOUT=600 > > > > And send us the complete log (alltests.log) > > > > Satish > > > > On Tue, 6 Apr 2021, Chen Gang wrote: > > > > > Dear sir, > > > > > > > > > The result of make check is OK. And I do set the timeout to a larger value, which keeps me from getting timeout error. The thing is I have two machines. And I get the error code 1 in different tests on different machines.I don’t know what is error code1. What case this? How can I fix the failure tests. > > > > > > > > > ------------------ Original ------------------ > > > From: Satish Balay <balay(a)mcs.anl.gov<mailto:[email protected]>&gt; > > > Date: Tue,Apr 6,2021 0:18 PM > > > To: Chen Gang <569615491(a)qq.com<mailto:[email protected]>&gt; > > > Cc: petsc-dev <petsc-dev(a)mcs.anl.gov<mailto:[email protected]>&gt;, cglwdm <cglwdm(a)scu.edu.cn<mailto:[email protected]>&gt; > > > Subject: Re: [petsc-dev] Petsc: Error code 1 > > > > <alltests2.log><alltests.log> > >

1 0

回复： Petsc: Error code 1
by Chen Gang 06 Apr '21

06 Apr '21

ARM: configure with the following: -ffp-contract=off ./configure -with-debugging=0 COPTFLAGS='-O3 -ffp-contract=off -march=armv8.2-a -mtune=tsv110' CXXOPTFLAGS='-O3 -ffp-contract=off -march=armv8.2-a -mtune=tsv110' FOPTFLAGS='-O3 -ffp-contract=off -march=armv8.2-a -mtune=tsv110' --with-x=1 -download-fblaslapack PETSC-KERNEL-USE-UNROLL-4 There are 3 failures, all related to FC code # ------------- #   Summary # ------------- # FAILED diff-tao_bound_tutorials-plate2f_2 diff-tao_bound_tutorials-plate2f_1 diff-vec_is_is_tutorials-ex2f_1 # success 7547/9528 tests (79.2%) # failed 3/9528 tests (0.0%) # todo 225/9528 tests (2.4%) # skip 1753/9528 tests (18.4%) # # Wall clock time for tests: 1371 sec # Approximate CPU time (not incl. build time): 106722.62999999999 sec # # To rerun failed tests: #     /opt/rh/devtoolset-9/root/usr/bin/gmake -f gmakefile test test-fail=1 # # Timing summary (actual test time / total CPU time): #   dm_tests-ex36_3dp1: 467.68 sec / 477.71 sec #   dm_impls_stag_tests-ex1_multidof_3: 395.16 sec / 398.45 sec #   ts_tutorials-ex29_1: 236.54 sec / 238.30 sec #   dm_impls_stag_tests-ex1_basic_2: 205.41 sec / 207.34 sec #   dm_tests-ex34_1: 178.94 sec / 180.26 sec ------------------ 原始邮件 ------------------ 发件人: "Chen Gang" <569615491(a)qq.com>; 发送时间: 2021年4月7日(星期三) 上午10:49 收件人: "petsc-dev"<petsc-dev(a)mcs.anl.gov>; 主题: 回复： [petsc-dev] Petsc: Error code 1 ARM: configure with the following: -ffp-contract=off ./configure -with-debugging=0 COPTFLAGS='-O3 -ffp-contract=off -march=armv8.2-a -mtune=tsv110' CXXOPTFLAGS='-O3 -ffp-contract=off -march=armv8.2-a -mtune=tsv110' FOPTFLAGS='-O3 -ffp-contract=off -march=armv8.2-a -mtune=tsv110' --with-x=1 -download-fblaslapack PETSC-KERNEL-USE-UNROLL-4 There are 3 failures, all related to FC code # ------------- #   Summary # ------------- # FAILED diff-tao_bound_tutorials-plate2f_2 diff-tao_bound_tutorials-plate2f_1 diff-vec_is_is_tutorials-ex2f_1 # success 7547/9528 tests (79.2%) # failed 3/9528 tests (0.0%) # todo 225/9528 tests (2.4%) # skip 1753/9528 tests (18.4%) # # Wall clock time for tests: 1371 sec # Approximate CPU time (not incl. build time): 106722.62999999999 sec # # To rerun failed tests: #     /opt/rh/devtoolset-9/root/usr/bin/gmake -f gmakefile test test-fail=1 # # Timing summary (actual test time / total CPU time): #   dm_tests-ex36_3dp1: 467.68 sec / 477.71 sec #   dm_impls_stag_tests-ex1_multidof_3: 395.16 sec / 398.45 sec #   ts_tutorials-ex29_1: 236.54 sec / 238.30 sec #   dm_impls_stag_tests-ex1_basic_2: 205.41 sec / 207.34 sec #   dm_tests-ex34_1: 178.94 sec / 180.26 sec ------------------ 原始邮件 ------------------ 发件人: "petsc-dev" <balay(a)mcs.anl.gov>; 发送时间: 2021年4月7日(星期三) 凌晨1:06 收件人: "Barry Smith"<bsmith(a)petsc.dev>; 抄送: "Chen Gang"<569615491(a)qq.com>;"Alp Dener"<alp.dener(a)gmail.com>;"petsc-dev"<petsc-dev(a)mcs.anl.gov>;"cglwdm"<cglwdm(a)scu.edu.cn>; 主题: Re: [petsc-dev] Petsc: Error code 1 > See the attachements. alltest.log is on a machine with 96 cores, ARM, with FC,gcc9.3.5,mpich3.4.1,fblaslapack; 6 failures Perhaps this is an issue with ARM - and such diffs are expected - as we already have multiple alt files for some of these tests $ ls -lt src/tao/bound/tutorials/output/plate2f_* -rw-r--r--. 1 balay balay 1029 Mar 23 19:48 src/tao/bound/tutorials/output/plate2f_1_alt.out -rw-r--r--. 1 balay balay 1071 Mar 23 19:48 src/tao/bound/tutorials/output/plate2f_1.out -rw-r--r--. 1 balay balay 1029 Mar 23 19:48 src/tao/bound/tutorials/output/plate2f_2_alt.out -rw-r--r--. 1 balay balay 1071 Mar 23 19:48 src/tao/bound/tutorials/output/plate2f_2.out >>>>>> not ok diff-vec_is_is_tutorials-ex2f_1 # Error code: 1 #       16,24d15 #       <   5 #       <   7 #       <   9 #       <  11 #       <  13 #       <  15 #       <  17 #       <  19 #       <  21 <<<<< This one is puzzling - missing fortran stdout? Perhaps compile issue on ARM? [its a sequential example - so can't blame MPI] Or they are all related to the optimization flags used? What configure options were used for the build? Satish On Tue, 6 Apr 2021, Barry Smith wrote: > >     Alp, > >    Except for the first test, these are all optimization problems (mostly in Fortran). The function values are very different so I am sending it to our optimization expert to take a look at it. The differences could possibly be related to the use of real() and maybe the direct use of floating point numbers that the compiler first treats as single and then converts to double thus losing precision. > >    Chen Gang, I assume you compiled with the default standard precision PETSc configure options? > > > > On Apr 6, 2021, at 3:56 AM, Chen Gang <569615491(a)qq.com<mailto:[email protected]>> wrote: > > > See the attachements. alltest.log is on a machine with 96 cores, ARM, with FC,gcc9.3.5,mpich3.4.1,fblaslapack; 6 failures >                                  alltest2.log is on an intel machine with 40  cores,x86, without FC; icc&mkl& intel mpi; only 1 failure > > ------------------ 原始邮件 ------------------ > 发件人: "petsc-dev" <balay(a)mcs.anl.gov<mailto:[email protected]>>; > 发送时间: 2021年4月6日(星期二) 中午12:38 > 收件人: "petsc-dev"<petsc-dev(a)mcs.anl.gov<mailto:[email protected]>>; > 抄送: "Chen Gang"<569615491(a)qq.com<mailto:[email protected]>>;"cglwdm"<cglwdm(a)scu.edu.cn<mailto:[email protected]>>; > 主题: Re: [petsc-dev] Petsc: Error code 1 > > Note: do not use '-j' with alltests. > > And run alltests on both machines [but *not* at the same time on machines] and send us logs from both the runs. > > Satish > > > On Mon, 5 Apr 2021, Satish Balay wrote: > > > Try: > > > > make alltests TIMEOUT=600 > > > > And send us the complete log (alltests.log) > > > > Satish > > > > On Tue, 6 Apr 2021, Chen Gang wrote: > > > > > Dear sir, > > > > > > > > > The result of make check is OK. And I do set the timeout to a larger value, which keeps me from getting timeout error. The thing is I have two machines. And I get the error code 1 in different tests on different machines.I don’t know what is error code1. What case this? How can I fix the failure tests. > > > > > > > > > ------------------ Original ------------------ > > > From: Satish Balay <balay(a)mcs.anl.gov<mailto:[email protected]>&gt; > > > Date: Tue,Apr 6,2021 0:18 PM > > > To: Chen Gang <569615491(a)qq.com<mailto:[email protected]>&gt; > > > Cc: petsc-dev <petsc-dev(a)mcs.anl.gov<mailto:[email protected]>&gt;, cglwdm <cglwdm(a)scu.edu.cn<mailto:[email protected]>&gt; > > > Subject: Re: [petsc-dev] Petsc: Error code 1 > > > > <alltests2.log><alltests.log> > >

1 0

回复： Petsc: Error code 1
by Chen Gang 06 Apr '21

06 Apr '21

ARM: configure with the following: -ffp-contract=off ./configure -with-debugging=0 COPTFLAGS='-O3 -ffp-contract=off -march=armv8.2-a -mtune=tsv110' CXXOPTFLAGS='-O3 -ffp-contract=off -march=armv8.2-a -mtune=tsv110' FOPTFLAGS='-O3 -ffp-contract=off -march=armv8.2-a -mtune=tsv110' --with-x=1 -download-fblaslapack PETSC-KERNEL-USE-UNROLL-4 There are 3 failures, all related to FC code # ------------- #   Summary # ------------- # FAILED diff-tao_bound_tutorials-plate2f_2 diff-tao_bound_tutorials-plate2f_1 diff-vec_is_is_tutorials-ex2f_1 # success 7547/9528 tests (79.2%) # failed 3/9528 tests (0.0%) # todo 225/9528 tests (2.4%) # skip 1753/9528 tests (18.4%) # # Wall clock time for tests: 1371 sec # Approximate CPU time (not incl. build time): 106722.62999999999 sec # # To rerun failed tests: #     /opt/rh/devtoolset-9/root/usr/bin/gmake -f gmakefile test test-fail=1 # # Timing summary (actual test time / total CPU time): #   dm_tests-ex36_3dp1: 467.68 sec / 477.71 sec #   dm_impls_stag_tests-ex1_multidof_3: 395.16 sec / 398.45 sec #   ts_tutorials-ex29_1: 236.54 sec / 238.30 sec #   dm_impls_stag_tests-ex1_basic_2: 205.41 sec / 207.34 sec #   dm_tests-ex34_1: 178.94 sec / 180.26 sec ------------------ 原始邮件 ------------------ 发件人: "petsc-dev" <balay(a)mcs.anl.gov>; 发送时间: 2021年4月7日(星期三) 凌晨1:06 收件人: "Barry Smith"<bsmith(a)petsc.dev>; 抄送: "Chen Gang"<569615491(a)qq.com>;"Alp Dener"<alp.dener(a)gmail.com>;"petsc-dev"<petsc-dev(a)mcs.anl.gov>;"cglwdm"<cglwdm(a)scu.edu.cn>; 主题: Re: [petsc-dev] Petsc: Error code 1 > See the attachements. alltest.log is on a machine with 96 cores, ARM, with FC,gcc9.3.5,mpich3.4.1,fblaslapack; 6 failures Perhaps this is an issue with ARM - and such diffs are expected - as we already have multiple alt files for some of these tests $ ls -lt src/tao/bound/tutorials/output/plate2f_* -rw-r--r--. 1 balay balay 1029 Mar 23 19:48 src/tao/bound/tutorials/output/plate2f_1_alt.out -rw-r--r--. 1 balay balay 1071 Mar 23 19:48 src/tao/bound/tutorials/output/plate2f_1.out -rw-r--r--. 1 balay balay 1029 Mar 23 19:48 src/tao/bound/tutorials/output/plate2f_2_alt.out -rw-r--r--. 1 balay balay 1071 Mar 23 19:48 src/tao/bound/tutorials/output/plate2f_2.out >>>>>> not ok diff-vec_is_is_tutorials-ex2f_1 # Error code: 1 #       16,24d15 #       <   5 #       <   7 #       <   9 #       <  11 #       <  13 #       <  15 #       <  17 #       <  19 #       <  21 <<<<< This one is puzzling - missing fortran stdout? Perhaps compile issue on ARM? [its a sequential example - so can't blame MPI] Or they are all related to the optimization flags used? What configure options were used for the build? Satish On Tue, 6 Apr 2021, Barry Smith wrote: > >     Alp, > >    Except for the first test, these are all optimization problems (mostly in Fortran). The function values are very different so I am sending it to our optimization expert to take a look at it. The differences could possibly be related to the use of real() and maybe the direct use of floating point numbers that the compiler first treats as single and then converts to double thus losing precision. > >    Chen Gang, I assume you compiled with the default standard precision PETSc configure options? > > > > On Apr 6, 2021, at 3:56 AM, Chen Gang <569615491(a)qq.com<mailto:[email protected]>> wrote: > > > See the attachements. alltest.log is on a machine with 96 cores, ARM, with FC,gcc9.3.5,mpich3.4.1,fblaslapack; 6 failures >                                  alltest2.log is on an intel machine with 40  cores,x86, without FC; icc&mkl& intel mpi; only 1 failure > > ------------------ 原始邮件 ------------------ > 发件人: "petsc-dev" <balay(a)mcs.anl.gov<mailto:[email protected]>>; > 发送时间: 2021年4月6日(星期二) 中午12:38 > 收件人: "petsc-dev"<petsc-dev(a)mcs.anl.gov<mailto:[email protected]>>; > 抄送: "Chen Gang"<569615491(a)qq.com<mailto:[email protected]>>;"cglwdm"<cglwdm(a)scu.edu.cn<mailto:[email protected]>>; > 主题: Re: [petsc-dev] Petsc: Error code 1 > > Note: do not use '-j' with alltests. > > And run alltests on both machines [but *not* at the same time on machines] and send us logs from both the runs. > > Satish > > > On Mon, 5 Apr 2021, Satish Balay wrote: > > > Try: > > > > make alltests TIMEOUT=600 > > > > And send us the complete log (alltests.log) > > > > Satish > > > > On Tue, 6 Apr 2021, Chen Gang wrote: > > > > > Dear sir, > > > > > > > > > The result of make check is OK. And I do set the timeout to a larger value, which keeps me from getting timeout error. The thing is I have two machines. And I get the error code 1 in different tests on different machines.I don’t know what is error code1. What case this? How can I fix the failure tests. > > > > > > > > > ------------------ Original ------------------ > > > From: Satish Balay <balay(a)mcs.anl.gov<mailto:[email protected]>&gt; > > > Date: Tue,Apr 6,2021 0:18 PM > > > To: Chen Gang <569615491(a)qq.com<mailto:[email protected]>&gt; > > > Cc: petsc-dev <petsc-dev(a)mcs.anl.gov<mailto:[email protected]>&gt;, cglwdm <cglwdm(a)scu.edu.cn<mailto:[email protected]>&gt; > > > Subject: Re: [petsc-dev] Petsc: Error code 1 > > > > <alltests2.log><alltests.log> > >

1 0

回复： Petsc: Error code 1
by Chen Gang 06 Apr '21

06 Apr '21

On ARM， don't use O3， configure like this # ./configure  --download-fblaslapack No fail. So the failure is related to the Code Optimization level. I guess maybe it related to the fmadd of the ARM cpu. I will try O3 and shutdown the fmadd again to see how it will go. # ------------- #   Summary # ------------- # success 7554/9532 tests (79.2%) # failed 0/9532 tests (0.0%) # todo 225/9532 tests (2.4%) # skip 1753/9532 tests (18.4%) # # Wall clock time for tests: 1313 sec # Approximate CPU time (not incl. build time): 106895.54000000004 sec # # Timing summary (actual test time / total CPU time): #   dm_tests-ex36_3dp1: 468.83 sec / 485.43 sec #   dm_impls_stag_tests-ex1_basic_3: 390.64 sec / 419.42 sec #   dm_impls_stag_tests-ex1_zerodof_3: 388.13 sec / 401.36 sec #   ts_tutorials-ex29_1: 231.93 sec / 252.27 sec #   mat_tests-ex44_mpiio_15: 216.32 sec / 218.03 sec ------------------ 原始邮件 ------------------ 发件人: "petsc-dev" <balay(a)mcs.anl.gov>; 发送时间: 2021年4月7日(星期三) 凌晨1:06 收件人: "Barry Smith"<bsmith(a)petsc.dev>; 抄送: "Chen Gang"<569615491(a)qq.com>;"Alp Dener"<alp.dener(a)gmail.com>;"petsc-dev"<petsc-dev(a)mcs.anl.gov>;"cglwdm"<cglwdm(a)scu.edu.cn>; 主题: Re: [petsc-dev] Petsc: Error code 1 > See the attachements. alltest.log is on a machine with 96 cores, ARM, with FC,gcc9.3.5,mpich3.4.1,fblaslapack; 6 failures Perhaps this is an issue with ARM - and such diffs are expected - as we already have multiple alt files for some of these tests $ ls -lt src/tao/bound/tutorials/output/plate2f_* -rw-r--r--. 1 balay balay 1029 Mar 23 19:48 src/tao/bound/tutorials/output/plate2f_1_alt.out -rw-r--r--. 1 balay balay 1071 Mar 23 19:48 src/tao/bound/tutorials/output/plate2f_1.out -rw-r--r--. 1 balay balay 1029 Mar 23 19:48 src/tao/bound/tutorials/output/plate2f_2_alt.out -rw-r--r--. 1 balay balay 1071 Mar 23 19:48 src/tao/bound/tutorials/output/plate2f_2.out >>>>>> not ok diff-vec_is_is_tutorials-ex2f_1 # Error code: 1 #       16,24d15 #       <   5 #       <   7 #       <   9 #       <  11 #       <  13 #       <  15 #       <  17 #       <  19 #       <  21 <<<<< This one is puzzling - missing fortran stdout? Perhaps compile issue on ARM? [its a sequential example - so can't blame MPI] Or they are all related to the optimization flags used? What configure options were used for the build? Satish On Tue, 6 Apr 2021, Barry Smith wrote: > >     Alp, > >    Except for the first test, these are all optimization problems (mostly in Fortran). The function values are very different so I am sending it to our optimization expert to take a look at it. The differences could possibly be related to the use of real() and maybe the direct use of floating point numbers that the compiler first treats as single and then converts to double thus losing precision. > >    Chen Gang, I assume you compiled with the default standard precision PETSc configure options? > > > > On Apr 6, 2021, at 3:56 AM, Chen Gang <569615491(a)qq.com<mailto:[email protected]>> wrote: > > > See the attachements. alltest.log is on a machine with 96 cores, ARM, with FC,gcc9.3.5,mpich3.4.1,fblaslapack; 6 failures >                                  alltest2.log is on an intel machine with 40  cores,x86, without FC; icc&mkl& intel mpi; only 1 failure > > ------------------ 原始邮件 ------------------ > 发件人: "petsc-dev" <balay(a)mcs.anl.gov<mailto:[email protected]>>; > 发送时间: 2021年4月6日(星期二) 中午12:38 > 收件人: "petsc-dev"<petsc-dev(a)mcs.anl.gov<mailto:[email protected]>>; > 抄送: "Chen Gang"<569615491(a)qq.com<mailto:[email protected]>>;"cglwdm"<cglwdm(a)scu.edu.cn<mailto:[email protected]>>; > 主题: Re: [petsc-dev] Petsc: Error code 1 > > Note: do not use '-j' with alltests. > > And run alltests on both machines [but *not* at the same time on machines] and send us logs from both the runs. > > Satish > > > On Mon, 5 Apr 2021, Satish Balay wrote: > > > Try: > > > > make alltests TIMEOUT=600 > > > > And send us the complete log (alltests.log) > > > > Satish > > > > On Tue, 6 Apr 2021, Chen Gang wrote: > > > > > Dear sir, > > > > > > > > > The result of make check is OK. And I do set the timeout to a larger value, which keeps me from getting timeout error. The thing is I have two machines. And I get the error code 1 in different tests on different machines.I don’t know what is error code1. What case this? How can I fix the failure tests. > > > > > > > > > ------------------ Original ------------------ > > > From: Satish Balay <balay(a)mcs.anl.gov<mailto:[email protected]>&gt; > > > Date: Tue,Apr 6,2021 0:18 PM > > > To: Chen Gang <569615491(a)qq.com<mailto:[email protected]>&gt; > > > Cc: petsc-dev <petsc-dev(a)mcs.anl.gov<mailto:[email protected]>&gt;, cglwdm <cglwdm(a)scu.edu.cn<mailto:[email protected]>&gt; > > > Subject: Re: [petsc-dev] Petsc: Error code 1 > > > > <alltests2.log><alltests.log> > >

1 0

回复： Petsc: Error code 1
by Chen Gang 06 Apr '21

06 Apr '21

Here is my configure options: ./configure -with-debugging=0 COPTFLAGS='-O3 -march=armv8.2-a -mtune=tsv110' CXXOPTFLAGS='-O3 -march=armv8.2-a -mtune=tsv110' FOPTFLAGS='-O3 -march=armv8.2-a -mtune=tsv110' --with-x=1 -download-fblaslapack PETSC-KERNEL-USE-UNROLL-4 --force The gcc is 9.3.1, the mpich is 3.4.1.  （When I use openmpi 4.10 or mpich 4.0 alpha, there are more failures related to the mpi.） ------------------ 原始邮件 ------------------ 发件人: "petsc-dev" <balay(a)mcs.anl.gov>; 发送时间: 2021年4月7日(星期三) 凌晨1:06 收件人: "Barry Smith"<bsmith(a)petsc.dev>; 抄送: "Chen Gang"<569615491(a)qq.com>;"Alp Dener"<alp.dener(a)gmail.com>;"petsc-dev"<petsc-dev(a)mcs.anl.gov>;"cglwdm"<cglwdm(a)scu.edu.cn>; 主题: Re: [petsc-dev] Petsc: Error code 1 > See the attachements. alltest.log is on a machine with 96 cores, ARM, with FC,gcc9.3.5,mpich3.4.1,fblaslapack; 6 failures Perhaps this is an issue with ARM - and such diffs are expected - as we already have multiple alt files for some of these tests $ ls -lt src/tao/bound/tutorials/output/plate2f_* -rw-r--r--. 1 balay balay 1029 Mar 23 19:48 src/tao/bound/tutorials/output/plate2f_1_alt.out -rw-r--r--. 1 balay balay 1071 Mar 23 19:48 src/tao/bound/tutorials/output/plate2f_1.out -rw-r--r--. 1 balay balay 1029 Mar 23 19:48 src/tao/bound/tutorials/output/plate2f_2_alt.out -rw-r--r--. 1 balay balay 1071 Mar 23 19:48 src/tao/bound/tutorials/output/plate2f_2.out >>>>>> not ok diff-vec_is_is_tutorials-ex2f_1 # Error code: 1 #       16,24d15 #       <   5 #       <   7 #       <   9 #       <  11 #       <  13 #       <  15 #       <  17 #       <  19 #       <  21 <<<<< This one is puzzling - missing fortran stdout? Perhaps compile issue on ARM? [its a sequential example - so can't blame MPI] Or they are all related to the optimization flags used? What configure options were used for the build? Satish On Tue, 6 Apr 2021, Barry Smith wrote: > >     Alp, > >    Except for the first test, these are all optimization problems (mostly in Fortran). The function values are very different so I am sending it to our optimization expert to take a look at it. The differences could possibly be related to the use of real() and maybe the direct use of floating point numbers that the compiler first treats as single and then converts to double thus losing precision. > >    Chen Gang, I assume you compiled with the default standard precision PETSc configure options? > > > > On Apr 6, 2021, at 3:56 AM, Chen Gang <569615491(a)qq.com<mailto:[email protected]>> wrote: > > > See the attachements. alltest.log is on a machine with 96 cores, ARM, with FC,gcc9.3.5,mpich3.4.1,fblaslapack; 6 failures >                                  alltest2.log is on an intel machine with 40  cores,x86, without FC; icc&mkl& intel mpi; only 1 failure > > ------------------ 原始邮件 ------------------ > 发件人: "petsc-dev" <balay(a)mcs.anl.gov<mailto:[email protected]>>; > 发送时间: 2021年4月6日(星期二) 中午12:38 > 收件人: "petsc-dev"<petsc-dev(a)mcs.anl.gov<mailto:[email protected]>>; > 抄送: "Chen Gang"<569615491(a)qq.com<mailto:[email protected]>>;"cglwdm"<cglwdm(a)scu.edu.cn<mailto:[email protected]>>; > 主题: Re: [petsc-dev] Petsc: Error code 1 > > Note: do not use '-j' with alltests. > > And run alltests on both machines [but *not* at the same time on machines] and send us logs from both the runs. > > Satish > > > On Mon, 5 Apr 2021, Satish Balay wrote: > > > Try: > > > > make alltests TIMEOUT=600 > > > > And send us the complete log (alltests.log) > > > > Satish > > > > On Tue, 6 Apr 2021, Chen Gang wrote: > > > > > Dear sir, > > > > > > > > > The result of make check is OK. And I do set the timeout to a larger value, which keeps me from getting timeout error. The thing is I have two machines. And I get the error code 1 in different tests on different machines.I don’t know what is error code1. What case this? How can I fix the failure tests. > > > > > > > > > ------------------ Original ------------------ > > > From: Satish Balay <balay(a)mcs.anl.gov<mailto:[email protected]>&gt; > > > Date: Tue,Apr 6,2021 0:18 PM > > > To: Chen Gang <569615491(a)qq.com<mailto:[email protected]>&gt; > > > Cc: petsc-dev <petsc-dev(a)mcs.anl.gov<mailto:[email protected]>&gt;, cglwdm <cglwdm(a)scu.edu.cn<mailto:[email protected]>&gt; > > > Subject: Re: [petsc-dev] Petsc: Error code 1 > > > > <alltests2.log><alltests.log> > >

1 0

Re: [petsc-dev] DMNetwork static sizing
by Abhyankar, Shrirang G 06 Apr '21

06 Apr '21

Hong, It was just to keep things simple. We can have an API (and a run-time option) for setting the max. number of components instead of a configure option. I can work on making this change. Thanks, Shri From: petsc-dev <petsc-dev-bounces(a)mcs.anl.gov> on behalf of PETSc Development <petsc-dev(a)mcs.anl.gov> Reply-To: "Zhang, Hong" <hzhang(a)mcs.anl.gov> Date: Tuesday, April 6, 2021 at 4:46 PM To: Matthew Knepley <knepley(a)gmail.com>, PETSc Development <petsc-dev(a)mcs.anl.gov> Cc: "Abhyankar, Shrirang G" <shrirang.abhyankar(a)pnnl.gov> Subject: Re: [petsc-dev] DMNetwork static sizing Shri, You designed this approach. Is it intended or out of implementation convenience at the time? Hong ________________________________ From: petsc-dev <petsc-dev-bounces(a)mcs.anl.gov> on behalf of Matthew Knepley <knepley(a)gmail.com> Sent: Monday, April 5, 2021 5:47 AM To: PETSc <petsc-dev(a)mcs.anl.gov> Subject: [petsc-dev] DMNetwork static sizing Dowe really need a configure time constant for struct _p_DMNetworkComponentHeader { PetscInt index; /* index for user input global edge and vertex */ PetscInt subnetid; /* Id for subnetwork */ PetscInt ndata; /* number of components */ PetscInt size[PETSC_DMNETWORK_MAXIMUM_COMPONENTS_PER_POINT]; PetscInt key[PETSC_DMNETWORK_MAXIMUM_COMPONENTS_PER_POINT]; PetscInt offset[PETSC_DMNETWORK_MAXIMUM_COMPONENTS_PER_POINT]; PetscInt nvar[PETSC_DMNETWORK_MAXIMUM_COMPONENTS_PER_POINT]; /* Number of variables */ PetscInt offsetvarrel[PETSC_DMNETWORK_MAXIMUM_COMPONENTS_PER_POINT]; /* offset from the first variable of the network point */ } PETSC_ATTRIBUTEALIGNED(PetscMax(sizeof(double),sizeof(PetscScalar))); Can't we just allocate this struct when needed and carry the size along? This design seem to go against the rest of what we do in PETSc? Thanks, Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/<https://gcc02.safelinks.protection.outlook.com/?url=http:%2F%2Fwww.cse.buff…>

1 0

Re: [petsc-dev] (arm64) could not find object file symbol for symbol ___muldc3
by Barry Smith 06 Apr '21

06 Apr '21

Blaise, Thanks, can you try cd src/snes/tutorials and then cut and paste the huge line that you sent earlier that begins with mpif90 -Wl,-bind_at_load ..... and add at the end -lgcc and see if that line successfully links the Fortran code? Thanks Barry > On Apr 6, 2021, at 12:22 PM, Blaise A Bourdin <bourdin(a)lsu.edu> wrote: > > Hi Barry, > >> >> Please send configure.log > See attached > >> >> It would also be helpful if you could run >> >> nm -o /usr/lib/lib* | grep muldc3 > > That is uneventful (I have no idea how these broken symbolic link got there and can’t delete them because of SIP)… > SiMini:lib $ nm -o /usr/lib/lib* | grep muldc3 > /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/nm: error: /usr/lib/libhunspell-1.2.0.dylib: No such file or directory. > /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/nm: error: /usr/lib/libhunspell-1.2.dylib: No such file or directory. > /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/nm: error: /usr/lib/libiodbc.2.dylib: No such file or directory. > /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/nm: error: /usr/lib/libiodbc.dylib: No such file or directory. > /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/nm: error: /usr/lib/libiodbcinst.2.dylib: No such file or directory. > /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/nm: error: /usr/lib/libiodbcinst.dylib: No such file or directory. > /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/nm: error: /usr/lib/libstdc++.6.dylib: No such file or directory. > /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/nm: error: /usr/lib/libstdc++.dylib: No such file or directory. > > Blaise > > > >> >> Our cross language linker tests are not adding all the libraries they need to add when using Fortran for linking. >> >> Barry >> >> >> >>> On Apr 6, 2021, at 11:00 AM, Matthew Knepley <knepley(a)gmail.com <mailto:[email protected]>> wrote: >>> >>> On Tue, Apr 6, 2021 at 11:04 AM Blaise A Bourdin <bourdin(a)lsu.edu <mailto:[email protected]>> wrote: >>> Hi, >>> >>> I am having the following warning when compiling any fortran example (currently on main, but it’s been going on for a little while) on a ARM mac. >>> >>> >>> *******************Error detected during compile or link!******************* >>> See http://www.mcs.anl.gov/petsc/documentation/faq.html <https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.mcs.an…> >>> /opt/HPC/petsc-main/src/snes/tutorials ex5f >>> ********************************************************* >>> mpif90 -Wl,-bind_at_load -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress -Wl,-commons,use_dylibs -Wl,-search_paths_first -Wl,-no_compact_unwind -ffree-line-length-none -fallow-argument-mismatch -fPIC -g -ffree-line-length-none -fallow-argument-mismatch -fPIC -g -I/opt/HPC/petsc-main/include -I/opt/HPC/petsc-main/bigsur-gcc10.2-arm64-reallybasic-g/include -I/opt/X11/include ex5f.F90 -Wl,-rpath,/opt/HPC/petsc-main/bigsur-gcc10.2-arm64-reallybasic-g/lib -L/opt/HPC/petsc-main/bigsur-gcc10.2-arm64-reallybasic-g/lib -Wl,-rpath,/opt/X11/lib -L/opt/X11/lib -Wl,-rpath,/opt/homebrew/Cellar/mpich/3.4.1_2/lib -L/opt/homebrew/Cellar/mpich/3.4.1_2/lib -Wl,-rpath,/opt/homebrew/Cellar/gcc/10.2.0_4/lib/gcc/10/gcc/aarch64-apple-darwin20/10.2.1 -L/opt/homebrew/Cellar/gcc/10.2.0_4/lib/gcc/10/gcc/aarch64-apple-darwin20/10.2.1 -Wl,-rpath,/opt/homebrew/Cellar/gcc/10.2.0_4/lib/gcc/10 -L/opt/homebrew/Cellar/gcc/10.2.0_4/lib/gcc/10 -lpetsc -llapack -lblas -lX11 -lc++ -ldl -lmpifort -lmpi -lpmpi -lgfortran -lemutls_w -lm -lc++ -ldl -o ex5f >>> warning: (arm64) could not find object file symbol for symbol muldc3 >>> Fortran example src/snes/tutorials/ex5f run successfully with 1 MPI process >>> >>> >>> For reference, here is how I configured petsc: >>> ./configure --CFLAGS='-Wimplicit-function-declaration' --FFLAGS="-ffree-line-length-none -fallow-argument-mismatch" --with-debugging=1 --with-shared-libraries=1 --with-x11=1 >>> >>> It looks like this symbol is referenced in: sfpack.o >>> >>> SiMini:petsc-main (main)$ grep -ri muldc3 bigsur-gcc10.2-arm64-reallybasic-g/* >>> Binary file bigsur-gcc10.2-arm64-reallybasic-g/lib/libpetsc.3.015.0.dylib matches >>> Binary file bigsur-gcc10.2-arm64-reallybasic-g/lib/libpetsc.3.015.dylib matches >>> bigsur-gcc10.2-arm64-reallybasic-g/lib/petsc/conf/check.log:warning: (arm64) could not find object file symbol for symbol ___muldc3 >>> Binary file bigsur-gcc10.2-arm64-reallybasic-g/lib/libpetsc.dylib matches >>> Binary file bigsur-gcc10.2-arm64-reallybasic-g/obj/vec/is/sf/impls/basic/sfpack.o matches >>> >>> SiMini:petsc-main (main)$ nm bigsur-gcc10.2-arm64-reallybasic-g/obj/vec/is/sf/impls/basic/sfpack.o | grep muldc3 >>> U ___muldc3 >>> >>> Google tells me it is a gcc thing for complex multiplication. Ah, I think this is the problem: >>> >>> https://lists.mcs.anl.gov/pipermail/mpich-discuss/2012-August/012920.html <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mcs…> >>> >>> It is in the C library and you are linking from Fortran, so the compatibility libraries are not correct. >>> >>> Thanks, >>> >>> Matt >>> >>> Any idea? >>> Blaise >>> >>> >>> -- >>> A.K. & Shirley Barton Professor of Mathematics >>> Adjunct Professor of Mechanical Engineering >>> Adjunct of the Center for Computation & Technology >>> Louisiana State University, Lockett Hall Room 344, Baton Rouge, LA 70803, USA >>> Tel. +1 (225) 578 1612, Fax +1 (225) 578 4276 Web http://www.math.lsu.edu/~bourdin <http://www.math.lsu.edu/~bourdin> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. >>> -- Norbert Wiener >>> >>> https://www.cse.buffalo.edu/~knepley/ <https://nam04.safelinks.protection.outlook.com/?url=http:%2F%2Fwww.cse.buff…> >> > > -- > A.K. & Shirley Barton Professor of Mathematics > Adjunct Professor of Mechanical Engineering > Adjunct of the Center for Computation & Technology > Louisiana State University, Lockett Hall Room 344, Baton Rouge, LA 70803, USA > Tel. +1 (225) 578 1612, Fax +1 (225) 578 4276 Web http://www.math.lsu.edu/~bourdin <http://www.math.lsu.edu/~bourdin> > <configure.log>

1 0

Re: [petsc-dev] DMNetwork static sizing
by Zhang, Hong 06 Apr '21

06 Apr '21

Shri, You designed this approach. Is it intended or out of implementation convenience at the time? Hong ________________________________ From: petsc-dev <petsc-dev-bounces(a)mcs.anl.gov> on behalf of Matthew Knepley <knepley(a)gmail.com> Sent: Monday, April 5, 2021 5:47 AM To: PETSc <petsc-dev(a)mcs.anl.gov> Subject: [petsc-dev] DMNetwork static sizing Dowe really need a configure time constant for struct _p_DMNetworkComponentHeader { PetscInt index; /* index for user input global edge and vertex */ PetscInt subnetid; /* Id for subnetwork */ PetscInt ndata; /* number of components */ PetscInt size[PETSC_DMNETWORK_MAXIMUM_COMPONENTS_PER_POINT]; PetscInt key[PETSC_DMNETWORK_MAXIMUM_COMPONENTS_PER_POINT]; PetscInt offset[PETSC_DMNETWORK_MAXIMUM_COMPONENTS_PER_POINT]; PetscInt nvar[PETSC_DMNETWORK_MAXIMUM_COMPONENTS_PER_POINT]; /* Number of variables */ PetscInt offsetvarrel[PETSC_DMNETWORK_MAXIMUM_COMPONENTS_PER_POINT]; /* offset from the first variable of the network point */ } PETSC_ATTRIBUTEALIGNED(PetscMax(sizeof(double),sizeof(PetscScalar))); Can't we just allocate this struct when needed and carry the size along? This design seem to go against the rest of what we do in PETSc? Thanks, Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/<http://www.cse.buffalo.edu/~knepley/>

1 0

Re: [petsc-dev] -with-kokkos-cuda-arch=AMPERE80 nonsense
by Matthew Knepley 06 Apr '21

06 Apr '21

On Tue, Apr 6, 2021 at 2:08 PM Scott Kruger <kruger(a)txcorp.com> wrote: > > I wrote sent this yesterday but am having some strange mailing issues. > > On 2021-04-03 22:42, Barry Smith did write: > > > > It would be very nice to NOT require PETSc users to provide this flag, > how the heck will they know what it should be when we cannot automate it > ourselves? > > > > Any ideas of how this can be determined based on the current system? > NVIDIA does not help since these "advertising" names don't seem to > trivially map to information you can get from a particular GPU when you > logged into it. For example nvidia-smi doesn't use these names directly. Is > there some mapping from nvidia-smi to these names we could use? If we are > serious about having a non-trivial number of users utilizing GPUs, which we > need to be for future, we cannot have this absurd demands in our > installation process. > > The mapping of the Nvidia card to the gencodes and cuda arch is one of > those annoyances that is so ridiculous it is hard to believe. > The best reference I have found is this: > > https://arnon.dk/matching-sm-architectures-arch-and-gencode-for-various-nvi… > > To this end, the fact that Kokkos provides a mapping from colloquial > card name to gencode/arch is a real benefit and useful. The problem is > that this mapping is buried in their build system and lacks > introspection. > > > > > Barry > > > > Does spack have some magic for this we could use? > > > > spack developed the archspec repo to abstract all of these issues: > https://github.com/archspec/archspec I do not love it. Besides the actual code (you can always complain about code), they do not really do any tests. They go look in a few places that data should be. We can do the same thing in probably 10x less code. It would be great to actually test the hardware to verify. Thanks, Matt > This is a *great* idea and eventually BuildSystem should incorporate it as > the standard way of doing things; however, it is been focused mostly on > the CPU issues, and is still under active development (my understanding > is that the pulling it out of spack and getting those interop issues > sorted out is tangled up in how spack handles dependencies and > compilers). It'd be nice if someone would go in and port the Kokkos gpu > mappings to archspec as there is some great knowledge on these mapping > buried in the Kokkos build system (not volunteering); i.e., translating > that webpage to some real code (even if it is in make) is valuable. > > TL;DR: It's a known problem with currently no good solution AFAIK. > Waiting until archspec gets further along seems like the best solution. > > Scott > > P.S. ROCm has rocminfo which also doesn't solve the problem but is at > least sane. > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>

1 0