April 2012 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] Reminder: *every* symbol with extern linkage *must* be namespaced
by Jed Brown 15 Apr '12

15 Apr '12

On Sun, Apr 15, 2012 at 12:17, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > We use MPIU_XXX as a replacement for missing/broken/... MPI_XXX > > The reason they start with MPI is to indicate their very close > relationship with their "true" MPI cousins. > > I think the natural replacement is PETSC_MPI_XXX, if we make this > change then the "obvious" connection to MPI_XXX is weaker. But proper > naming spacing does dictate we make this change. > Note that there are also functions, e.g. MPIU_File_read_all(), which could become Petsc_MPI_File_read_all() or PetscMPI_File_read_all() as you prefer. I'm providing other implementations of MPIX_Iallreduce() which I'd like to namespace consistently. Is it PetscMPI_Iallreduce(), Petsc_MPI_Iallreduce(), or something else?

1 0

various dot products messed up for complex numbers?
by Barry Smith 15 Apr '12

15 Apr '12

/*@ VecDot - Computes the vector dot product. Collective on Vec Input Parameters: . x, y - the vectors Output Parameter: . val - the dot product Performance Issues: $ per-processor memory bandwidth $ interprocessor latency $ work load inbalance that causes certain processes to arrive much earlier than others Notes for Users of Complex Numbers: For complex vectors, VecDot() computes $ val = (x,y) = y^H x, where y^H denotes the conjugate transpose of y. Use VecTDot() for the indefinite form $ val = (x,y) = y^T x, where y^T denotes the transpose of y. Level: intermediate Concepts: inner product Concepts: vector^inner product .seealso: VecMDot(), VecTDot(), VecNorm(), VecDotBegin(), VecDotEnd() /*@ VecTDot - Computes an indefinite vector dot product. That is, this routine does NOT use the complex conjugate. Collective on Vec Input Parameters: . x, y - the vectors Output Parameter: . val - the dot product Notes for Users of Complex Numbers: For complex vectors, VecTDot() computes the indefinite form $ val = (x,y) = y^T x, where y^T denotes the transpose of y. Use VecDot() for the inner product $ val = (x,y) = y^H x, where y^H denotes the conjugate transpose of y. Level: intermediate Concepts: inner product^non-Hermitian Concepts: vector^inner product Concepts: non-Hermitian inner product .seealso: VecDot(), VecMTDot() @*/ But source code has PetscErrorCode VecDot_Seq(Vec xin,Vec yin,PetscScalar *z) { const PetscScalar *ya,*xa; PetscBLASInt one = 1,bn = PetscBLASIntCast(xin->map->n); PetscErrorCode ierr; PetscFunctionBegin; ierr = VecGetArrayRead(xin,&xa);CHKERRQ(ierr); ierr = VecGetArrayRead(yin,&ya);CHKERRQ(ierr); *z = BLASdot_(&bn,xa,&one,ya,&one); ierr = VecRestoreArrayRead(xin,&xa);CHKERRQ(ierr); ierr = VecRestoreArrayRead(yin,&ya);CHKERRQ(ierr); if (xin->map->n > 0) { ierr = PetscLogFlops(2.0*xin->map->n-1);CHKERRQ(ierr); } PetscFunctionReturn(0); } #undef __FUNCT__ #define __FUNCT__ "VecTDot_Seq" PetscErrorCode VecTDot_Seq(Vec xin,Vec yin,PetscScalar *z) { const PetscScalar *ya,*xa; PetscBLASInt one = 1, bn = PetscBLASIntCast(xin->map->n); PetscErrorCode ierr; PetscFunctionBegin; ierr = VecGetArrayRead(xin,&xa);CHKERRQ(ierr); ierr = VecGetArrayRead(yin,&ya);CHKERRQ(ierr); *z = BLASdot_(&bn,xa,&one,ya,&one); ierr = VecRestoreArrayRead(xin,&xa);CHKERRQ(ierr); ierr = VecRestoreArrayRead(yin,&ya);CHKERRQ(ierr); if (xin->map->n > 0) { ierr = PetscLogFlops(2.0*xin->map->n-1);CHKERRQ(ierr); } PetscFunctionReturn(0); } How can they both use BLASdot() when one is supposed to use the complex conjugate of y the second argument. Note also DOUBLE COMPLEX FUNCTION ZDOTC(N,ZX,INCX,ZY,INCY) * .. Scalar Arguments .. INTEGER INCX,INCY,N * .. * .. Array Arguments .. DOUBLE COMPLEX ZX(*),ZY(*) * .. * * Purpose * ======= * * ZDOTC forms the dot product of a vector. * * Further Details * =============== * * jack dongarra, 3/11/78. * modified 12/3/93, array(1) declarations changed to array(*) * IX = 1 IY = 1 IF (INCX.LT.0) IX = (-N+1)*INCX + 1 IF (INCY.LT.0) IY = (-N+1)*INCY + 1 DO I = 1,N ZTEMP = ZTEMP + DCONJG(ZX(IX))*ZY(IY) IX = IX + INCX IY = IY + INCY END DO END IF ZDOTC = ZTEMP uses the complex conjugate of the FIRST argument. So it looks like everything is major broken for the complex case. Barry

1 0

Re: [petsc-dev] Reminder: *every* symbol with extern linkage *must* be namespaced
by Barry Smith 15 Apr '12

15 Apr '12

On Apr 15, 2012, at 10:34 AM, Jed Brown wrote: > On Sat, Mar 17, 2012 at 13:16, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > I have fixed the logging variables and a few others. The following incantation will show you a list of the symbols that are not currently in a "claimed" namespace (either directly or through bundling). These are arguably already too promiscuous, but the symbols that this does not filter out should probably be fixed. > > $ nm --print-file-name mpich-static/lib/libpetsc.a |sort -k 3 | grep ' $B\|D\|C\|T$ ' | grep -i -v ' $XXT\|XYT\|Xi\|TetGen\|Vec\|Mat\|Petsc\|KSP\|PC\|SNES\|DM\|TS\|PF\|IS\|AO\|Characteristic\|ClassPerf\|EventPerfLog\|EventRegLog\|spbas_\|f90array\|mpifcmb\|SPARSEPACK\|MPIU_\|MINPACK\|admf_$' > > Why does PETSc use the MPIU_ prefix? MPICH2 uses it internally for "utility" routines. Should that stuff become PetscMPI_ or similar? > > http://wiki.mcs.anl.gov/mpich2/index.php/Function_Name_Prefix_Convention We use MPIU_XXX as a replacement for missing/broken/... MPI_XXX The reason they start with MPI is to indicate their very close relationship with their "true" MPI cousins. I think the natural replacement is PETSC_MPI_XXX, if we make this change then the "obvious" connection to MPI_XXX is weaker. But proper naming spacing does dictate we make this change. > > > Also, perhaps we should set up a board like this to highlight global naming violations. How about just catching the bad name in the repository after someone pushes an objectionable name and installing a virus onto their machine that taunts them for it? > > http://www.mcs.anl.gov/research/projects/mpich2/todo/globsyms.htm

1 0

Re: [petsc-dev] Reminder: *every* symbol with extern linkage *must* be namespaced
by Jed Brown 15 Apr '12

15 Apr '12

On Sat, Mar 17, 2012 at 13:16, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > I have fixed the logging variables and a few others. The following > incantation will show you a list of the symbols that are not currently in a > "claimed" namespace (either directly or through bundling). These are > arguably already too promiscuous, but the symbols that this does not filter > out should probably be fixed. > > $ nm --print-file-name mpich-static/lib/libpetsc.a |sort -k 3 | grep ' > $B\|D\|C\|T$ ' | grep -i -v ' > $XXT\|XYT\|Xi\|TetGen\|Vec\|Mat\|Petsc\|KSP\|PC\|SNES\|DM\|TS\|PF\|IS\|AO\|Characteristic\|ClassPerf\|EventPerfLog\|EventRegLog\|spbas_\|f90array\|mpifcmb\|SPARSEPACK\|MPIU_\|MINPACK\|admf_$' > Why does PETSc use the MPIU_ prefix? MPICH2 uses it internally for "utility" routines. Should that stuff become PetscMPI_ or similar? http://wiki.mcs.anl.gov/mpich2/index.php/Function_Name_Prefix_Convention Also, perhaps we should set up a board like this to highlight global naming violations. http://www.mcs.anl.gov/research/projects/mpich2/todo/globsyms.htm

1 0

Re: [petsc-dev] Configuring for atomics
by Barry Smith 14 Apr '12

14 Apr '12

On Apr 14, 2012, at 11:44 AM, Jed Brown wrote: > Lots of other compilers support GCC-style intrinsics like __sync_bool_compare_and_swap() and similar, so we really can't be doing this kind of logic in lockfree.c. > > #if __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ >= 1050 > #define PetscAtomicCompareandSwap(ptr, oldval, newval) (OSAtomicCompareAndSwapPtr(oldval,newval,ptr)) > #elif defined(_MSC_VER) > #define PetscAtomicCompareandSwap(ptr, oldval, newval) (InterlockedCompareExchange(ptr,newval,oldval)) > #elif (__GNUC__ * 10000 + __GNUC_MINOR__ * 100 + __GNUC_PATCHLEVEL__) > 40100 > #define PetscAtomicCompareandSwap(ptr, oldval, newval) (__sync_bool_compare_and_swap(ptr,oldval,newval)) > #else > # error No maping for PetscAtomicCompareandSwap > #endif > > > What does everyone else think about using Dave Goodell's Open Portable Atomics (https://trac.mcs.anl.gov/projects/openpa/, used by MPICH2) instead of maintaining our own suite of configuration tests? I guess it is worth trying. Barry

1 0

Configuring for atomics
by Jed Brown 14 Apr '12

14 Apr '12

Lots of other compilers support GCC-style intrinsics like __sync_bool_compare_and_swap() and similar, so we really can't be doing this kind of logic in lockfree.c. #if __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ >= 1050 #define PetscAtomicCompareandSwap(ptr, oldval, newval) (OSAtomicCompareAndSwapPtr(oldval,newval,ptr)) #elif defined(_MSC_VER) #define PetscAtomicCompareandSwap(ptr, oldval, newval) (InterlockedCompareExchange(ptr,newval,oldval)) #elif (__GNUC__ * 10000 + __GNUC_MINOR__ * 100 + __GNUC_PATCHLEVEL__) > 40100 #define PetscAtomicCompareandSwap(ptr, oldval, newval) (__sync_bool_compare_and_swap(ptr,oldval,newval)) #else # error No maping for PetscAtomicCompareandSwap #endif What does everyone else think about using Dave Goodell's Open Portable Atomics (https://trac.mcs.anl.gov/projects/openpa/, used by MPICH2) instead of maintaining our own suite of configuration tests?

1 0

Re: [petsc-dev] Need a reference for manual page for PCLSC - Preconditioning for Schur complements, based on Least Squares Commutators
by Jed Brown 12 Apr '12

12 Apr '12

http://petsc.cs.iit.edu/petsc/petsc-dev/rev/ff33760c1e46 On Thu, Apr 12, 2012 at 14:11, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > We need a reference for the manual page for PCLSC - Preconditioning for > Schur complements, based on Least Squares Commutators. What is the right > reference to put in? > > Barry > >

1 0

Need a reference for manual page for PCLSC - Preconditioning for Schur complements, based on Least Squares Commutators
by Barry Smith 12 Apr '12

12 Apr '12

We need a reference for the manual page for PCLSC - Preconditioning for Schur complements, based on Least Squares Commutators. What is the right reference to put in? Barry

1 0

Re: [petsc-dev] nvcc -g3 error
by Ben Jamroz 12 Apr '12

12 Apr '12

Yep, I hadn't hg updated. I have your fix now. Thanks Satish. -Ben On 4/12/12 11:55 AM, Satish Balay wrote: > I had changed the default flag for a cuda build from -g3 to > -g. Perhaps you don't have the latest petsc-dev/BuildSystem? > > http://petsc.cs.iit.edu/petsc/BuildSystem/rev/05b504243938 > > BTW: The defualt opt flags can be changed with --COPTFLAGS configure > option. > > Satish > > On Thu, 12 Apr 2012, Ben Jamroz wrote: > >> Hi, >> >> I've had trouble building a debug version of petsc-dev with nvcc. At compile >> time I was getting the following error. >> >> =============================================================== >> libfast in: /scr_0/jamroz/petsc-dev-jamroz/src/vec/vec/impls/seq/seqcusp >> nvcc -O3 -g -g -arch=sm_20 -c --compiler-options=-g -O3 -g3 -fno-inline -O0 >> -I/scr_0/jamroz/petsc-dev-jamroz/include >> -I/scr_0/jamroz/petsc-dev-jamroz/arch-linux2-c-debug/include >> -I/usr/local/cuda/4.1/cuda/include -I/scr_0/jamroz/gpuPETSc/txpetscgpu/include >> -I/scr_0/jamroz/ -I/usr/local/cuda/4.1/cuda/include/ >> -I/usr/mpi/gcc/openmpi-1.4.3/include >> -D__INSDIR__=src/vec/vec/impls/seq/seqcusp/ veccusp.cu >> <built-in>(1): error: "__STDC_HOSTED__" is predefined; attempted redefinition >> ignored >> <built-in>(1): error: "__WCHAR_TYPE__" is predefined; attempted redefinition >> ignored >> <built-in>(1): error: "__x86_64" is predefined; attempted redefinition ignored >> <built-in>(1): error: "__x86_64__" is predefined; attempted redefinition >> ignored >> <built-in>(1): error: "__linux__" is predefined; attempted redefinition >> ignored >> <built-in>(1): error: "__unix__" is predefined; attempted redefinition ignored >> <built-in>(1): error: "unix" is predefined; attempted redefinition ignored >> 7 errors detected in the compilation of >> "/tmp/tmpxft_00006e29_00000000-7_veccusp.cpp3.i". >> /usr/bin/ar: veccusp.o: No such file or directory >> =============================================================== >> >> It turns out that -g3 (debug level) in the --compiler-options was causing the >> trouble. Commenting out this line in BuildSystem/config/compilerOptions.py >> (line 24) fixed this error. According to >> http://gcc.gnu.org/onlinedocs/gcc/Debugging-Options.html the default debug is >> level 2. >> >> Just a heads up. >> >> -Ben >> >> ps. My config script was: >> >> ./configure --with-debugging=1 --with-mpi=1 >> --with-thrust-dir=/usr/local/cuda/4.1/cuda/include/ >> --with-cusp-dir=/scr_0/jamroz/ --CFLAGS="-g -O3" --with-cuda=1 >> --with-cuda-dir=/usr/local/cuda/4.1/cuda --CUDAFLAGS="-O3 -g" >> --with-cudac=nvcc --with-cuda-arch=sm_20 --with-precision=double >> --with-txpetscgpu=1 --download-txpetscgpu=no >> --with-txpetscgpu-dir=/scr_0/jamroz/gpuPETSc/txpetscgpu >> >> -- Ben Jamroz jamroz(a)txcorp.com Tech-X Corporation 5621 Arapahoe Ave, Suite A Boulder, CO 80303 Phone: (720) 974-1853, Fax: (303) 448-7756

1 0

Re: [petsc-dev] nvcc -g3 error
by Satish Balay 12 Apr '12

12 Apr '12

I had changed the default flag for a cuda build from -g3 to -g. Perhaps you don't have the latest petsc-dev/BuildSystem? http://petsc.cs.iit.edu/petsc/BuildSystem/rev/05b504243938 BTW: The defualt opt flags can be changed with --COPTFLAGS configure option. Satish On Thu, 12 Apr 2012, Ben Jamroz wrote: > Hi, > > I've had trouble building a debug version of petsc-dev with nvcc. At compile > time I was getting the following error. > > =============================================================== > libfast in: /scr_0/jamroz/petsc-dev-jamroz/src/vec/vec/impls/seq/seqcusp > nvcc -O3 -g -g -arch=sm_20 -c --compiler-options=-g -O3 -g3 -fno-inline -O0 > -I/scr_0/jamroz/petsc-dev-jamroz/include > -I/scr_0/jamroz/petsc-dev-jamroz/arch-linux2-c-debug/include > -I/usr/local/cuda/4.1/cuda/include -I/scr_0/jamroz/gpuPETSc/txpetscgpu/include > -I/scr_0/jamroz/ -I/usr/local/cuda/4.1/cuda/include/ > -I/usr/mpi/gcc/openmpi-1.4.3/include > -D__INSDIR__=src/vec/vec/impls/seq/seqcusp/ veccusp.cu > <built-in>(1): error: "__STDC_HOSTED__" is predefined; attempted redefinition > ignored > <built-in>(1): error: "__WCHAR_TYPE__" is predefined; attempted redefinition > ignored > <built-in>(1): error: "__x86_64" is predefined; attempted redefinition ignored > <built-in>(1): error: "__x86_64__" is predefined; attempted redefinition > ignored > <built-in>(1): error: "__linux__" is predefined; attempted redefinition > ignored > <built-in>(1): error: "__unix__" is predefined; attempted redefinition ignored > <built-in>(1): error: "unix" is predefined; attempted redefinition ignored > 7 errors detected in the compilation of > "/tmp/tmpxft_00006e29_00000000-7_veccusp.cpp3.i". > /usr/bin/ar: veccusp.o: No such file or directory > =============================================================== > > It turns out that -g3 (debug level) in the --compiler-options was causing the > trouble. Commenting out this line in BuildSystem/config/compilerOptions.py > (line 24) fixed this error. According to > http://gcc.gnu.org/onlinedocs/gcc/Debugging-Options.html the default debug is > level 2. > > Just a heads up. > > -Ben > > ps. My config script was: > > ./configure --with-debugging=1 --with-mpi=1 > --with-thrust-dir=/usr/local/cuda/4.1/cuda/include/ > --with-cusp-dir=/scr_0/jamroz/ --CFLAGS="-g -O3" --with-cuda=1 > --with-cuda-dir=/usr/local/cuda/4.1/cuda --CUDAFLAGS="-O3 -g" > --with-cudac=nvcc --with-cuda-arch=sm_20 --with-precision=double > --with-txpetscgpu=1 --download-txpetscgpu=no > --with-txpetscgpu-dir=/scr_0/jamroz/gpuPETSc/txpetscgpu > >

1 0