August 2012 - petsc-dev - mailman3.cels.anl.gov

Adjacency relations in Sieve and stencils
by Chris Eldred 23 Aug '12

23 Aug '12

I am working with a 2D unstructured mesh using sieve and I wanted to get some advice on how to determine adjacency relations using the topological operators provided by Sieve (cone, support, closure, star). I have figured out the following: Vertices for a given edge: cone(edge) Cells for a given edge: support(edge) Cells for a given vertex: star(vertex) \ support(vertex) Edges for a given vertex: support(vertex) Edges for a given cell: cone(cell) Vertices for a given cell: closure(cell) \ cone(cell) The set of edges associated with the cells for a given edge: U cone(support(edge)) - ie the union of the cones of the support of the edge Are there other topological operators that more naturally express these relations (especially the ones involving unions or complements)? -- Chris Eldred DOE Computational Science Graduate Fellow Graduate Student, Atmospheric Science, Colorado State University B.S. Applied Computational Physics, Carnegie Mellon University, 2009 chris.eldred(a)gmail.com

1 0

Re: [petsc-dev] Ghost values in sieve
by Jed Brown 23 Aug '12

23 Aug '12

On Thu, Aug 23, 2012 at 12:12 PM, Chris Eldred <chris.eldred(a)gmail.com>wrote: > Thanks- that helps a lot. If I need stencils that are larger than > closure(p) U star(p) (for a higher-order finite difference method, for > example), I assume that I need to create my own PetscSF's that > describe which points need to be ghosted? > Is this still a fully unstructured method? The Sieve formalism doesn't give you a very efficient way to do this for structured or semi-structured grids. Even so, if wider stencils are to be supported, I think it should be implemented within the library. Doing it outside with the current infrastructure is going to be quite a rabbit hole. > Is there some documentation or example code that explains the theory > behind star forests? > Docs for the basic operations: http://59A2.org/files/StarForest.pdf

1 0

Re: [petsc-dev] Ghost values in sieve
by Chris Eldred 23 Aug '12

23 Aug '12

Thanks- that helps a lot. If I need stencils that are larger than closure(p) U star(p) (for a higher-order finite difference method, for example), I assume that I need to create my own PetscSF's that describe which points need to be ghosted? Is there some documentation or example code that explains the theory behind star forests? On Sat, Aug 18, 2012 at 3:49 PM, Matthew Knepley <knepley(a)gmail.com> wrote: > On Sat, Aug 18, 2012 at 4:11 PM, Chris Eldred <chris.eldred(a)gmail.com> > wrote: >> >> Yes I think that makes sense. How do you access the associated PetscSF >> for the DMComplex? Is it possible to define multiple PetscSF's for a >> given DMComplex, each associated with different PetscSections? > > > 1) DMGetDefaultSF() for the one for the default Section > > 2) Yes, but you need to create them yourself using DMCreateDefaultSF() > >> >> My current understanding of using DMComplex is (for the serial case): >> >> 1) Create a DMComplex object, set its cones/supports, etc. > > > A good guide is to look at complexcreate.c, where I create a few default > meshes. > >> >> 2) Create PetscSections that describe the layout of data across the >> DMComplex >> 3) Set a PetscSection as the default section >> 4) Use DMCreateLocalVector to create the Vec objects needed for the >> computation >> >> Is this correct? > > > Yes > >> >> In the parallel case, what else needs to be done? > > > In parallel, > > a) I call DMComplexDistribute(). The alternative is to create your mesh > directly in parallel, > but this does not work with any mesh generator I know, and you would > have to create > the PetscSF for points that this function makes automatically. > > b) Now the section you create is local > > c) You can use the local section and pointSF to create a global section, > which knows about overlap > > d) You can use the local and global section to make the dofSF > > c) and d) are done automatically for the default section (data layout). > > Matt > >> >> >> -Chris >> >> On Fri, Aug 17, 2012 at 1:24 PM, Matthew Knepley <knepley(a)gmail.com> >> wrote: >> > On Fri, Aug 17, 2012 at 1:21 PM, Chris Eldred <chris.eldred(a)gmail.com> >> > wrote: >> >> >> >> I was wondering how ghost values were handled when creating a >> >> distributed (global) vector from a specific PetscSection over a >> >> DMComplex object- ie what defines the stencils used to figure out >> >> which values need to be ghosted? What about local vs. global indexes? >> > >> > >> > This answer will have to be long because the question is mixing levels >> > in >> > the interface. >> > >> > A PetscSection is only an offset structure, so it has no designated >> > fields >> > for overlap information. What I do for DM is to create 2 sections: one >> > describing the layout of the local vector, and one describing the global >> > vector and ghosts. The global section can be created automatically given >> > the local section and a PetscSF (successor to VecScatter). >> > >> > The PetscSF describes the parallel overlap, and also has scatters and >> > reductions. >> > This is created by the DMComplex instead of a scatter since it can >> > handle >> > arbitrary types, and has a better query interface. DMComplex creates a >> > (non-overlapping) partition of some stratum of points, usually cells for >> > FEM. >> > Then the closure(p) U star(p) for each point is added to that partition, >> > and >> > the dofs on any shared point are ghosted. >> > >> > This is all now done using the vanilla DM interface, so >> > DMGlobalToLocal() >> > and LocalToGlobal() and cousins work as normal. >> > >> > Does that make sense? >> > >> > Matt >> > >> >> >> >> >> >> -Chris >> >> >> >> -- >> >> Chris Eldred >> >> DOE Computational Science Graduate Fellow >> >> Graduate Student, Atmospheric Science, Colorado State University >> >> B.S. Applied Computational Physics, Carnegie Mellon University, 2009 >> >> chris.eldred(a)gmail.com >> > >> > >> > >> > >> > -- >> > What most experimenters take for granted before they begin their >> > experiments >> > is infinitely more interesting than any results to which their >> > experiments >> > lead. >> > -- Norbert Wiener >> >> >> >> -- >> Chris Eldred >> DOE Computational Science Graduate Fellow >> Graduate Student, Atmospheric Science, Colorado State University >> B.S. Applied Computational Physics, Carnegie Mellon University, 2009 >> chris.eldred(a)gmail.com > > > > > -- > What most experimenters take for granted before they begin their experiments > is infinitely more interesting than any results to which their experiments > lead. > -- Norbert Wiener -- Chris Eldred DOE Computational Science Graduate Fellow Graduate Student, Atmospheric Science, Colorado State University B.S. Applied Computational Physics, Carnegie Mellon University, 2009 chris.eldred(a)gmail.com

1 0

Re: [petsc-dev] [petsc-maint #129715] __float128 complex
by Jose E. Roman 23 Aug '12

23 Aug '12

El 23/08/2012, a las 18:27, Satish Balay escribió: > On Thu, 23 Aug 2012, Jose E. Roman wrote: > >> Anyway, it would be a good opportunity to upgrade to Lapack 3.4 (the >> toclapack.sh script should work for this version as well). > > Ok. Will do this. Hower I see a bunch of embeded code in toclapack.sh > - corresponding to different lapack versions. Would this be a problem? > > Satish > --------- > > asterix:/home/balay/spetsc/bin/maint>grep -i version toclapack.sh |grep LAPACK > /* -- LAPACK auxiliary routine (version 3.3.0) -- */ > /* -- LAPACK auxiliary routine (version 3.3.0) -- */ > /* -- LAPACK auxiliary routine (version 3.3.0) -- */ > /* -- LAPACK auxiliary routine (version 3.3.0) -- */ > /* -- LAPACK auxiliary routine (version 3.3.0) -- */ > /* -- LAPACK auxiliary routine (version 3.3.0) -- */ > /* -- LAPACK auxiliary routine (version 3.3.0) -- */ > /* -- LAPACK auxiliary routine (version 3.3.0) -- */ > /* -- LAPACK auxiliary routine (version 3.3.0) -- */ > /* -- LAPACK auxiliary routine (version 3.3.0) -- */ > /* -- LAPACK auxiliary routine (version 3.3.0) -- */ > /* -- LAPACK auxiliary routine (version 3.3.0) -- */ > /* -- LAPACK auxiliary routine (version 3.3.0) -- */ > /* -- LAPACK auxiliary routine (version 3.3.0) -- */ > /* -- LAPACK auxiliary routine (version 3.3.0) -- */ > /* -- LAPACK auxiliary routine (version 3.3.0) -- */ > /* -- LAPACK auxiliary routine (version 3.3.0) -- */ > /* -- LAPACK auxiliary routine (version 3.3.0) -- */ > /* -- LAPACK auxiliary routine (version 3.0) -- > /* -- LAPACK auxiliary routine (version 3.0) -- > /* -- LAPACK routine (version 3.2.2) -- */ > #/* -- LAPACK auxiliary routine (version 3.1) -- */ > asterix:/home/balay/spetsc/bin/maint> There should be no problem. Anyway, we will check this and modify toclapack.sh if necessary. We will also test the w* subrotines more. I will come back to you tomorrow. Jose

1 0

Re: [petsc-dev] [petsc-maint #129715] __float128 complex
by Satish Balay 23 Aug '12

23 Aug '12

On Thu, 23 Aug 2012, Jose E. Roman wrote: > Anyway, it would be a good opportunity to upgrade to Lapack 3.4 (the > toclapack.sh script should work for this version as well). Ok. Will do this. Hower I see a bunch of embeded code in toclapack.sh - corresponding to different lapack versions. Would this be a problem? Satish --------- asterix:/home/balay/spetsc/bin/maint>grep -i version toclapack.sh |grep LAPACK /* -- LAPACK auxiliary routine (version 3.3.0) -- */ /* -- LAPACK auxiliary routine (version 3.3.0) -- */ /* -- LAPACK auxiliary routine (version 3.3.0) -- */ /* -- LAPACK auxiliary routine (version 3.3.0) -- */ /* -- LAPACK auxiliary routine (version 3.3.0) -- */ /* -- LAPACK auxiliary routine (version 3.3.0) -- */ /* -- LAPACK auxiliary routine (version 3.3.0) -- */ /* -- LAPACK auxiliary routine (version 3.3.0) -- */ /* -- LAPACK auxiliary routine (version 3.3.0) -- */ /* -- LAPACK auxiliary routine (version 3.3.0) -- */ /* -- LAPACK auxiliary routine (version 3.3.0) -- */ /* -- LAPACK auxiliary routine (version 3.3.0) -- */ /* -- LAPACK auxiliary routine (version 3.3.0) -- */ /* -- LAPACK auxiliary routine (version 3.3.0) -- */ /* -- LAPACK auxiliary routine (version 3.3.0) -- */ /* -- LAPACK auxiliary routine (version 3.3.0) -- */ /* -- LAPACK auxiliary routine (version 3.3.0) -- */ /* -- LAPACK auxiliary routine (version 3.3.0) -- */ /* -- LAPACK auxiliary routine (version 3.0) -- /* -- LAPACK auxiliary routine (version 3.0) -- /* -- LAPACK routine (version 3.2.2) -- */ #/* -- LAPACK auxiliary routine (version 3.1) -- */ asterix:/home/balay/spetsc/bin/maint>

1 0

Re: [petsc-dev] [petsc-maint #129715] __float128 complex
by Barry Smith 23 Aug '12

23 Aug '12

Who cares, why not just live with the _ mangling? Barry On Aug 23, 2012, at 11:23 AM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > On Thu, Aug 23, 2012 at 11:21 AM, Jose E. Roman <jroman(a)dsic.upv.es> wrote: > Yes. But the "stdcall" case seems more tedious. Anyway, __float128 is only available in GCC which uses underscore mangling, right? > > True

1 0

Re: [petsc-dev] [petsc-maint #129715] __float128 complex
by Jed Brown 23 Aug '12

23 Aug '12

On Thu, Aug 23, 2012 at 11:21 AM, Jose E. Roman <jroman(a)dsic.upv.es> wrote: > Yes. But the "stdcall" case seems more tedious. Anyway, __float128 is only > available in GCC which uses underscore mangling, right? True

1 0

Re: [petsc-dev] [petsc-maint #129715] __float128 complex
by Jose E. Roman 23 Aug '12

23 Aug '12

El 23/08/2012, a las 18:13, Jed Brown escribió: > On Thu, Aug 23, 2012 at 11:02 AM, Jose E. Roman <jroman(a)dsic.upv.es> wrote: > > El 22/08/2012, a las 18:52, Jose E. Roman escribió: > > > f2cblaslapack is more or less prepared for this case already (it is not much tested though, we will make some tests for the w* subroutines tomorrow). > > > > What is not ready is PETSc itself. Some definitions are missing. For instance, PetscScalar is undefined, so one should add the following to petscmath.h:106: > > > > #elif defined(PETSC_USE_REAL___FLOAT128) > > typedef __complex128 PetscScalar; > > > > #define PetscRealPart(a) crealq(a) > > #define PetscImaginaryPart(a) cimagq(a) > > #define PetscAbsScalar(a) cabsq(a) > > #define PetscConj(a) conjq(a) > > #define PetscSqrtScalar(a) csqrtq(a) > > #define PetscPowScalar(a,b) cpowq(a,b) > > #define PetscExpScalar(a) cexpq(a) > > #define PetscLogScalar(a) clogq(a) > > #define PetscSinScalar(a) csinq(a) > > #define PetscCosScalar(a) ccosq(a) > > > > Similarly, MPIU_SCALAR should be defined. Maybe other as well. > > > > Jose > > I have pushed a change to petsc-dev that allows to configure --with-precision=__float128 --with-scalar-type=complex > > I have tested this with src/ksp/ksp/examples/tutorials/ex12.c with 1 and 2 processes. It seems to work. > > Please review the change, since I had to modify pinit.c > > It looks good to me. Thanks. > > > A couple of comments: > > 1) The f2cblaslapack interface to quad precision subroutines (q* and w*) is defined only in the case of "uscore" mangling. Maybe a test in configure should assure this. > > Don't we just need to copy and update the list of defines to the different mangling conventions? Seems easier to do than to test/document that it doesn't work. Yes. But the "stdcall" case seems more tedious. Anyway, __float128 is only available in GCC which uses underscore mangling, right? > > > 2) I could not make it work --with-clanguage=c++ - I get compilation errors like this. Any idea? > > C++, in all it's template metaprogramming glory, standardized that std::complex may *only* be used with float, double, and long double. Using std::complex<anything_else> is invalid. Just one of a very long list of reasons why std::complex should not be used.

1 0

Re: [petsc-dev] [petsc-maint #129715] __float128 complex
by Jose E. Roman 23 Aug '12

23 Aug '12

El 23/08/2012, a las 18:11, Barry Smith escribió: > > Jose, > > This is great, thanks so much! > > Does Satish need to roll a new http://ftp.mcs.anl.gov/pub/petsc/externalpackages/f2cblaslapack-3.1.1.q.tar… tar ball? I'm guessing yes? > No. I used the one that is currently brought by --download-f2cblaslapack Anyway, it would be a good opportunity to upgrade to Lapack 3.4 (the toclapack.sh script should work for this version as well). Jose > > > Barry > > > > > On Aug 23, 2012, at 11:02 AM, "Jose E. Roman" <jroman(a)dsic.upv.es> wrote: > >> >> El 22/08/2012, a las 18:52, Jose E. Roman escribió: >> >>> f2cblaslapack is more or less prepared for this case already (it is not much tested though, we will make some tests for the w* subroutines tomorrow). >>> >>> What is not ready is PETSc itself. Some definitions are missing. For instance, PetscScalar is undefined, so one should add the following to petscmath.h:106: >>> >>> #elif defined(PETSC_USE_REAL___FLOAT128) >>> typedef __complex128 PetscScalar; >>> >>> #define PetscRealPart(a) crealq(a) >>> #define PetscImaginaryPart(a) cimagq(a) >>> #define PetscAbsScalar(a) cabsq(a) >>> #define PetscConj(a) conjq(a) >>> #define PetscSqrtScalar(a) csqrtq(a) >>> #define PetscPowScalar(a,b) cpowq(a,b) >>> #define PetscExpScalar(a) cexpq(a) >>> #define PetscLogScalar(a) clogq(a) >>> #define PetscSinScalar(a) csinq(a) >>> #define PetscCosScalar(a) ccosq(a) >>> >>> Similarly, MPIU_SCALAR should be defined. Maybe other as well. >>> >>> Jose >> >> I have pushed a change to petsc-dev that allows to configure --with-precision=__float128 --with-scalar-type=complex >> >> I have tested this with src/ksp/ksp/examples/tutorials/ex12.c with 1 and 2 processes. It seems to work. >> >> Please review the change, since I had to modify pinit.c >> >> A couple of comments: >> >> 1) The f2cblaslapack interface to quad precision subroutines (q* and w*) is defined only in the case of "uscore" mangling. Maybe a test in configure should assure this. >> >> 2) I could not make it work --with-clanguage=c++ - I get compilation errors like this. Any idea? >> >> /usr/include/c++/4.6/complex: In function ‘_Tp std::__complex_abs(const std::complex<_Tp>&) [with _Tp = __float128]’: >> /usr/include/c++/4.6/complex:592:68: instantiated from ‘_Tp std::abs(const std::complex<_Tp>&) [with _Tp = __float128]’ >> /home/jroman/soft/petsc-dev/src/sys/utils/sortd.c:116:26: instantiated from here >> /usr/include/c++/4.6/complex:571:50: error: call of overloaded ‘abs(__float128&)’ is ambiguous >> /usr/include/c++/4.6/complex:571:50: note: candidates are: >> /usr/include/c++/4.6/cmath:82:3: note: double std::abs(double) >> /usr/include/c++/4.6/cmath:86:3: note: float std::abs(float) >> /usr/include/c++/4.6/cmath:90:3: note: long double std::abs(long double) >> /usr/include/c++/4.6/complex:571:50: error: call of overloaded ‘abs(__float128&)’ is ambiguous >> /usr/include/c++/4.6/complex:571:50: note: candidates are: >> /usr/include/c++/4.6/cmath:82:3: note: double std::abs(double) >> /usr/include/c++/4.6/cmath:86:3: note: float std::abs(float) >> /usr/include/c++/4.6/cmath:90:3: note: long double std::abs(long double) >> /usr/include/c++/4.6/complex:576:46: error: call of overloaded ‘sqrt(__float128)’ is ambiguous >> /usr/include/c++/4.6/complex:576:46: note: candidates are: >> /usr/include/x86_64-linux-gnu/bits/mathcalls.h:158:1: note: double sqrt(double) >> /usr/include/c++/4.6/cmath:416:3: note: float std::sqrt(float) >> /usr/include/c++/4.6/cmath:420:3: note: long double std::sqrt(long double) >> >> >> >

1 0

Re: [petsc-dev] [petsc-maint #129715] __float128 complex
by Jed Brown 23 Aug '12

23 Aug '12

On Thu, Aug 23, 2012 at 11:02 AM, Jose E. Roman <jroman(a)dsic.upv.es> wrote: > > El 22/08/2012, a las 18:52, Jose E. Roman escribió: > > > f2cblaslapack is more or less prepared for this case already (it is not > much tested though, we will make some tests for the w* subroutines > tomorrow). > > > > What is not ready is PETSc itself. Some definitions are missing. For > instance, PetscScalar is undefined, so one should add the following to > petscmath.h:106: > > > > #elif defined(PETSC_USE_REAL___FLOAT128) > > typedef __complex128 PetscScalar; > > > > #define PetscRealPart(a) crealq(a) > > #define PetscImaginaryPart(a) cimagq(a) > > #define PetscAbsScalar(a) cabsq(a) > > #define PetscConj(a) conjq(a) > > #define PetscSqrtScalar(a) csqrtq(a) > > #define PetscPowScalar(a,b) cpowq(a,b) > > #define PetscExpScalar(a) cexpq(a) > > #define PetscLogScalar(a) clogq(a) > > #define PetscSinScalar(a) csinq(a) > > #define PetscCosScalar(a) ccosq(a) > > > > Similarly, MPIU_SCALAR should be defined. Maybe other as well. > > > > Jose > > I have pushed a change to petsc-dev that allows to configure > --with-precision=__float128 --with-scalar-type=complex > > I have tested this with src/ksp/ksp/examples/tutorials/ex12.c with 1 and 2 > processes. It seems to work. > > Please review the change, since I had to modify pinit.c > It looks good to me. Thanks. > > A couple of comments: > > 1) The f2cblaslapack interface to quad precision subroutines (q* and w*) > is defined only in the case of "uscore" mangling. Maybe a test in configure > should assure this. > Don't we just need to copy and update the list of defines to the different mangling conventions? Seems easier to do than to test/document that it doesn't work. > > 2) I could not make it work --with-clanguage=c++ - I get compilation > errors like this. Any idea? > C++, in all it's template metaprogramming glory, standardized that std::complex may *only* be used with float, double, and long double. Using std::complex<anything_else> is invalid. Just one of a very long list of reasons why std::complex should not be used.

1 0