July 2015 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] --output-sync for nightly builds so that warnings can be parsed
by Satish Balay 30 Jul '15

30 Jul '15

Barry, This flag is a gnumake-4.0+ feature. We currently use gnumake-3.81 as the minimum version [OSX defaults to this version and will never upgrade due to GPL3 issue]. For now - I've: - updated --download-make to default to gnumake-4.1 - if configure detects gnumake =>4.0 - it sets "--output-sync=recurse" flag. - switched linux-cxx-cmplx-pkgs-64idx to use --download-make And we can switch selected builds [as needed] to --download-make - but not all builds. Satish On Sat, 18 Jul 2015, Barry Smith wrote: > > Satish, > > > Our make sometimes produces mangled test output such as: > > /sandbox/petsc/petsc.clone-2/include/petscviewer.h:348:22: warningIn file included from /sandbox/petsc/petsc.clone-2/src/ksp/ksp/interface/itfunc.c:6: > > This makes it difficult for the automatic tools to parse out and ignore irrelevant warning messages so we get sent email about code that is ok. > > gnumake has the option --output-sync https://www.gnu.org/software/make/manual/html_node/Parallel-Output.html that would prevent this problem but also means that no output is started until the job is complete. This is ok for the nightly builds but for normal builds we want the output to come out as soon as possible. > > Can you figure out a way so that the nightly builds (and Jenkins) use the --output-sync but other builds don't? Perhaps some extra argument to configure? > > Thanks > > Barry > > > > > >

1 0

Re: [petsc-dev] Julia Petsc Wrapper
by Jed Brown 28 Jul '15

28 Jul '15

Jared Crean <jcrean01(a)gmail.com> writes: > As for the MPI communicator, the julia MPI package uses a C int > to store it, so I will typealias to that to ensure consistency. If an > MPI implementation uses an 8 byte pointer, MPI.jl will have to change too. Open MPI and relatives (like the now-obsolete LAM-MPI) use a pointer for all handles. To my knowledge, there is no subset of the MPI interface that can reliably be called to determine the binary interface, so you'll need access to the headers there too.

1 0

Re: [petsc-dev] Julia Petsc Wrapper
by Barry Smith 24 Jul '15

24 Jul '15

> On Jul 19, 2015, at 2:25 PM, Jared Crean <jcrean01(a)gmail.com> wrote: > > Hello, > Yeah, Clang.jl was supposed to be the answer to that, but it doesn't handle preprocessor macros and a few other things quite right. > > Anyways, type detection is working, but when running the tests with PetscScalar as a double precision complex number, Petsc gives the 1 norm of the vector [1.0 + 0im; 2.0 + 1.0im; 3.0 + 2.0im] as 9.0, where as Julia calculates it as 6.841619252963779. It looks like Petsc isn't taking the modulus of the elements of the vector before summing them. Is this a bug or is there a particular reason for it? I have changed PETSc master branch to use the more traditional 1 norm for complex numbers. This is how it will work in future releases (but not in the patch releases). Thanks for reporting our nonstandard usage Barry > > Jared Crean > > On 07/14/2015 10:53 PM, Barry Smith wrote: >> The Julia folks should give up their delusion that a C package is characterized completely by its .so, it is not, it is defined by its .so and its .h; a .so without its .h it is fairly useless. >> >> Barry >> >> >> >>> On Jul 14, 2015, at 9:40 PM, Matthew Knepley <knepley(a)gmail.com> wrote: >>> >>> On Tue, Jul 14, 2015 at 9:35 PM, Jared Crean <jcrean01(a)gmail.com> wrote: >>> Hello, >>> PETSC_SCALAR is not a symbol either. I skimmed through the names in the shared library, and it doesn't look like any data type information is there. >>> >>> Damn, yes PETSC_SCALAR and PETSC_REAL are defines. I think we are careful about this so that you can use >>> a real and complex PETSc together by building multiple versions of the library (no symbol clash). However, I believe >>> you can use >>> >>> PetscDataTypeFromString("scalar", &stype, &found); >>> PetscDataTypeFromString("complex", &ctype, &found); >>> isComplex = stype == ctype; >>> >>> Thanks, >>> >>> Matt >>> >>> Jared Crean >>> >>> >>> On 7/14/2015 9:22 PM, Matthew Knepley wrote: >>>> On Tue, Jul 14, 2015 at 8:03 PM, Jared Crean <jcrean01(a)gmail.com> wrote: >>>> Hello, >>>> PETSC_USE_COMPLEX isn't a symbol in the shared library when Petsc is built with complex scalars, so I don't see a way to access it at runtime. I'll have to write a simple C program that uses sizeof() and write the value to a file. >>>> >>>> That is crazy. How about >>>> >>>> isComplex = PETSC_COMPLEX == PETSC_SCALAR >>>> >>>> Matt >>>> As for the MPI communicator, the julia MPI package uses a C int to store it, so I will typealias to that to ensure consistency. If an MPI implementation uses an 8 byte pointer, MPI.jl will have to change too. >>>> >>>> Jared Crean >>>> >>>> >>>> On 7/14/2015 1:04 PM, Matthew Knepley wrote: >>>>> On Tue, Jul 14, 2015 at 10:56 AM, Jared Crean <jcrean01(a)gmail.com> wrote: >>>>> Hello everyone, >>>>> I got the package in a reasonably working state and Travis testing setup, so I am putting the package up on Github. >>>>> >>>>> https://github.com/JaredCrean2/PETSc.jl >>>>> >>>>> There is still a lot more work to do, but its a start. >>>>> >>>>> A couple questions: >>>>> When looking though the code, I noticed the MPI communicator is being passed as a 64 bit integer. mpi.h typedefs it as an int, so shouldn't it be a 32 bit integer? >>>>> >>>>> Some MPI implementations store the communicator as a pointer, which may be 64 bits. I think the only thing the standard says is >>>>> that MPI_Comm should be defined. >>>>> Also, is there a way to find out at runtime what datatype a PetscScalar is? It appears PetscDataTypeGetSize does not accept PetscScalar as an argument. >>>>> >>>>> If PETSC_USE_COMPLEX is defined its PETSC_COMPLEX, otherwise its PETSC_REAL. You can also just use sizeof(PetscScalar). What do you >>>>> want to do? >>>>> >>>>> Thanks, >>>>> >>>>> Matt >>>>> >>>>> Jared Crean >>>>> >>>>> >>>>> >>>>> On 07/06/2015 09:02 AM, Matthew Knepley wrote: >>>>>> On Mon, Jul 6, 2015 at 4:59 AM, Patrick Sanan <patrick.sanan(a)gmail.com> wrote: >>>>>> I had a couple of brief discussions about this at Juliacon as well. I think it would be useful, but there are a couple of things to think about from the start of any new attempt to do this: >>>>>> 1. As Jack pointed out, one issue is that the PETSc library must be compiled for a particular precision. This raises some questions - should several versions of the library be built to allow for flexibility? >>>>>> 2. An issue with wrapping PETSc is always that the flexibility of using the PETSc options paradigm is reduced - how can this be addressed? Could/should an expert user be able to access the options database directly, or would this be too much violence to the wrapper abstraction? >>>>>> >>>>>> I have never understood why this is an issue. Can't you just wrap our interface level, and use the options just as we do? That >>>>>> is essentially what petsc4py does. What is limiting in this methodology? On the other hand, requiring specific types, ala FEniCS, >>>>>> is very limiting. >>>>>> >>>>>> Matt >>>>>> On Sat, Jul 4, 2015 at 11:00 PM, Jared Crean <jcrean01(a)gmail.com> wrote: >>>>>> Hello, >>>>>> I am a graduate student working on a CFD code written in Julia, and I am interested in using Petsc as a linear solver (and possibly for the non-linear solves as well) for the code. I discovered the Julia wrapper file Petsc.jl in Petsc and have updated it to work with the current version of Julia and the MPI.jl package, using only MPI for communication (I don't think Julia's internal parallelism will scale well enough, at least not in the near future). >>>>>> >>>>>> I read the discussion on Github [https://github.com/JuliaLang/julia/issues/2645] and it looks like >>>>>> there currently is not a complete package to access Petsc from Julia. With your permission, I would like to use the Petsc.jl file as the basis for developing a package. My plan is create a lower level interface that exactly wraps Petsc functions, and then construct a higher level interface, probably an object that is a subtype of Julia's AbstractArray, that allows users to store values into Petsc vectors and matrices. I am less interested in integrating tightly with Julia's existing linear algebra capabilities than ensuring good scalability. The purpose of the high level interface it simple to populate the vector or matrix. >>>>>> >>>>>> What do you think, both about using the Petsc.jl file and the overall approach? >>>>>> >>>>>> Jared Crean >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. >>>>>> -- Norbert Wiener >>>>> >>>>> >>>>> >>>>> -- >>>>> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. >>>>> -- Norbert Wiener >>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. >>>> -- Norbert Wiener >>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. >>> -- Norbert Wiener >

1 0

Re: [petsc-dev] [GPU] Cusp default configuration is broken
by Barry Smith 24 Jul '15

24 Jul '15

I have fixed the following: When configure options like --with-cusp-include=xxx are given and no corresponding --with-cusp-lib=yyy is needed configure will no longer print an inaccurate error message that you must also set a --with-cusp-lib=yyy but instead correctly use the provided value Reported-by: Massimiliano Leoni <Massimiliano.Leoni(a)Rolls-Royce.com> in the branch master cacadbe Thanks for reporting it, Barry > On Jul 24, 2015, at 9:28 AM, Karl Rupp <rupp(a)iue.tuwien.ac.at> wrote: > > Hi Massimiliano, > > > I came across this [alleged] bug when installing petsc with GPU support: >> if I run configure with option –with-cusp-include=… it complains I >> should also set –with-cusp-lib, but cusp is a pure template library, so >> there is no static or shared library to link against. > > Correct. The sample file in config/examples/arch-cuda-double.py uses --with-cusp-dir for exactly that reason. > > >> As an alternative, it offers to specify –with-cusp-dir=CUSP_DIR , and >> then it tries to guess the include directory as $CUSP_DIR/include, which >> is wrong as in current CUSP version the directory containing the headers >> is $CUSP_DIR/cusp. >> >> An easy workaround is to go to $CUSP_DIR and create a symbolic link >> $CUSP_DIR/include that points to $CUSP_DIR/cusp, but I think the >> configure script should be corrected [one might not always have >> permission to create this link. > > Thanks for the feedback, I will look into this. We have other libraries in use which use a similar folder organization as CUSP, so I suspect something system-specific. Could you send us configure.log for the case where you configure using –with-cusp-dir=CUSP_DIR? > > Thanks and best regards, > Karli >

1 0

Re: [petsc-dev] [GPU] Cusp default configuration is broken
by Barry Smith 24 Jul '15

24 Jul '15

> On Jul 24, 2015, at 9:28 AM, Karl Rupp <rupp(a)iue.tuwien.ac.at> wrote: > > Hi Massimiliano, > > > I came across this [alleged] bug when installing petsc with GPU support: >> if I run configure with option –with-cusp-include=… it complains I >> should also set –with-cusp-lib, but cusp is a pure template library, so >> there is no static or shared library to link against. Interestingly despite the lack of a library it is located at https://github.com/cusplibrary/cusplibrary.git > > > Correct. The sample file in config/examples/arch-cuda-double.py uses --with-cusp-dir for exactly that reason. > > >> As an alternative, it offers to specify –with-cusp-dir=CUSP_DIR , and >> then it tries to guess the include directory as $CUSP_DIR/include, which >> is wrong as in current CUSP version the directory containing the headers >> is $CUSP_DIR/cusp. >> >> An easy workaround is to go to $CUSP_DIR and create a symbolic link >> $CUSP_DIR/include that points to $CUSP_DIR/cusp, but I think the >> configure script should be corrected [one might not always have >> permission to create this link. > > Thanks for the feedback, I will look into this. We have other libraries in use which use a similar folder organization as CUSP, so I suspect something system-specific. Could you send us configure.log for the case where you configure using –with-cusp-dir=CUSP_DIR? > > Thanks and best regards, > Karli >

1 0

Re: [petsc-dev] [GPU] Cusp default configuration is broken
by Leoni, Massimiliano 24 Jul '15

24 Jul '15

Sure, here it is! > -----Original Message----- > From: Karl Rupp [mailto:[email protected]] > Sent: 24 July 2015 15:28 > To: Leoni, Massimiliano; petsc-dev(a)mcs.anl.gov > Subject: Re: [petsc-dev] [GPU] Cusp default configuration is broken > > Hi Massimiliano, > > > I came across this [alleged] bug when installing petsc with GPU support: > > if I run configure with option -with-cusp-include=... it complains I > > should also set -with-cusp-lib, but cusp is a pure template library, > > so there is no static or shared library to link against. > > Correct. The sample file in config/examples/arch-cuda-double.py uses -- > with-cusp-dir for exactly that reason. > > > > As an alternative, it offers to specify -with-cusp-dir=CUSP_DIR , and > > then it tries to guess the include directory as $CUSP_DIR/include, which > > is wrong as in current CUSP version the directory containing the headers > > is $CUSP_DIR/cusp. > > > > An easy workaround is to go to $CUSP_DIR and create a symbolic link > > $CUSP_DIR/include that points to $CUSP_DIR/cusp, but I think the > > configure script should be corrected [one might not always have > > permission to create this link. > > Thanks for the feedback, I will look into this. We have other libraries > in use which use a similar folder organization as CUSP, so I suspect > something system-specific. Could you send us configure.log for the case > where you configure using -with-cusp-dir=CUSP_DIR? > > Thanks and best regards, > Karli The data contained in, or attached to, this e-mail, may contain confidential information. If you have received it in error you should notify the sender immediately by reply e-mail, delete the message from your system and contact +44 (0) 1332 622800(Security Operations Centre) if you need assistance. Please do not copy it for any purpose, or disclose its contents to any other person. An e-mail response to this address may be subject to interception or monitoring for operational reasons or for lawful business practices. (c) 2015 Rolls-Royce plc Registered office: 62 Buckingham Gate, London SW1E 6AT Company number: 1003142. Registered in England.

1 0

Re: [petsc-dev] [GPU] Cusp default configuration is broken
by Karl Rupp 24 Jul '15

24 Jul '15

Hi Massimiliano, > I came across this [alleged] bug when installing petsc with GPU support: > if I run configure with option ï¿½with-cusp-include=ï¿½ it complains I > should also set ï¿½with-cusp-lib, but cusp is a pure template library, so > there is no static or shared library to link against. Correct. The sample file in config/examples/arch-cuda-double.py uses --with-cusp-dir for exactly that reason. > As an alternative, it offers to specify ï¿½with-cusp-dir=CUSP_DIR , and > then it tries to guess the include directory as $CUSP_DIR/include, which > is wrong as in current CUSP version the directory containing the headers > is $CUSP_DIR/cusp. > > An easy workaround is to go to $CUSP_DIR and create a symbolic link > $CUSP_DIR/include that points to $CUSP_DIR/cusp, but I think the > configure script should be corrected [one might not always have > permission to create this link. Thanks for the feedback, I will look into this. We have other libraries in use which use a similar folder organization as CUSP, so I suspect something system-specific. Could you send us configure.log for the case where you configure using ï¿½with-cusp-dir=CUSP_DIR? Thanks and best regards, Karli

1 0

Re: [petsc-dev] [SLEPc - GPU] Problems running SLEPc on GPUs
by Leoni, Massimiliano 24 Jul '15

24 Jul '15

> -----Original Message----- > From: Jose E. Roman [mailto:[email protected]] > Sent: 24 July 2015 09:20 > To: Leoni, Massimiliano > Cc: petsc-dev(a)mcs.anl.gov; Alejandro Lamas > Subject: Re: [petsc-dev] [SLEPc - GPU] Problems running SLEPc on GPUs > > As you can seen in section 8.3 of SLEPc’s users manual, the command-line > example adds -bv_type vecs, otherwise you get this error. In the short term > we plan to add GPU support for the default BV, but for the moment you > have to use this BV type. > > We would be interested in knowing how this works for you, send feedback > to slepc-maint. > > Thanks. > Jose Great, apologies for not going through the SLEPc manual first, I thought I was fine with PETSc's. Anyway, I just run the same test and I got a neat, out-of-the-box 3x speedup compared to the single processor version [NVIDIA Tesla C2070 vs Intel Xeon X5650]. I will go through more extensive testing if we decide to continue using PETSc-SLEPc. Best, Massimiliano The data contained in, or attached to, this e-mail, may contain confidential information. If you have received it in error you should notify the sender immediately by reply e-mail, delete the message from your system and contact +44 (0) 1332 622800(Security Operations Centre) if you need assistance. Please do not copy it for any purpose, or disclose its contents to any other person. An e-mail response to this address may be subject to interception or monitoring for operational reasons or for lawful business practices. (c) 2015 Rolls-Royce plc Registered office: 62 Buckingham Gate, London SW1E 6AT Company number: 1003142. Registered in England.

1 0

[GPU] Cusp default configuration is broken
by Leoni, Massimiliano 24 Jul '15

24 Jul '15

Hi all, I came across this [alleged] bug when installing petsc with GPU support: if I run configure with option -with-cusp-include=... it complains I should also set -with-cusp-lib, but cusp is a pure template library, so there is no static or shared library to link against. As an alternative, it offers to specify -with-cusp-dir=CUSP_DIR , and then it tries to guess the include directory as $CUSP_DIR/include, which is wrong as in current CUSP version the directory containing the headers is $CUSP_DIR/cusp. An easy workaround is to go to $CUSP_DIR and create a symbolic link $CUSP_DIR/include that points to $CUSP_DIR/cusp, but I think the configure script should be corrected [one might not always have permission to create this link. Best, Massimiliano The data contained in, or attached to, this e-mail, may contain confidential information. If you have received it in error you should notify the sender immediately by reply e-mail, delete the message from your system and contact +44 (0) 1332 622800(Security Operations Centre) if you need assistance. Please do not copy it for any purpose, or disclose its contents to any other person. An e-mail response to this address may be subject to interception or monitoring for operational reasons or for lawful business practices. (c) 2015 Rolls-Royce plc Registered office: 62 Buckingham Gate, London SW1E 6AT Company number: 1003142. Registered in England.

1 0

Re: [petsc-dev] [SLEPc - GPU] Problems running SLEPc on GPUs
by Jose E. Roman 24 Jul '15

24 Jul '15

> El 24/7/2015, a las 10:09, Leoni, Massimiliano <Massimiliano.Leoni(a)Rolls-Royce.com> escribió: > > Hi everybody, > > I have recently being trying to run SLEPc on GPUs, but I am experiencing some trouble. > I think I correctly installed PETSc to run on GPUs, as I can actually see an execution time difference in PETSc programs when run with or without –vec_type cusp –mat_type aijcusp. > When I try to do the same with SLEPc, I get the following error [running hands-on exercise 2] > [0]PETSC ERROR: No support for this operation for this object type > [0]PETSC ERROR: Cannot create a BVSVEC from a non-standard template vector > > From the documentation I understood this BVSVEC is a base SLEPc type, so I am not sure how to interpret this [wouldn’t know what to touch at high-level]. > Can anyone please point me to some examples/documentation on this? > > Thanks in advance > > Massimiliano Leoni > As you can seen in section 8.3 of SLEPc’s users manual, the command-line example adds -bv_type vecs, otherwise you get this error. In the short term we plan to add GPU support for the default BV, but for the moment you have to use this BV type. We would be interested in knowing how this works for you, send feedback to slepc-maint. Thanks. Jose

1 0