March 2013 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] Dealing with off-processor aij-matrices not having preallocation information
by Jed Brown 21 Mar '13

21 Mar '13

On Thu, Mar 21, 2013 at 7:50 AM, Karl Rupp <rupp(a)mcs.anl.gov> wrote: > Hi Jed, > > > > As the comment noted, this is quite a hack and actually breaks the > >> current implementation for GPU matrices. I think that the problem >> can be completely resolved by not checking for PETSC_DEFAULT and >> PETSC_DECIDE in the MPI preallocation above, but just forward d_nz >> and o_nz to the sequential preallocation routine. This way, the >> logic remains in MatSeqAIJSetPreallocation(), where >> MAT_NEW_NONZERO_ALLOCATION_ERR is only set if the user provided an >> explicit preallocation scheme. >> >> >> Hmm, the 'nonew' member is accessed in the MATMPIAIJ so we have to >> either change the code to avoid ever using it from the top level or we >> need to propagate the setting back to the parent. See also this >> discussion: >> >> http://lists.mcs.anl.gov/**pipermail/petsc-dev/2012-**January/007121.html<http://lists.mcs.anl.gov/pipermail/petsc-dev/2012-January/007121.html> >> >> https://bitbucket.org/petsc/**petsc/commits/7827cd58<https://bitbucket.org/petsc/petsc/commits/7827cd58> >> > > Yes, I want to preserve the behavior Barry described in 2), i.e. only > error if the user specified an allocation pattern explicitly. I'm only > questioning the way it is currently implemented: > > If the number of nonzeros is specified as PETSC_DECIDE to > MatSeqAIJSetPreallocation_**SeqAIJ(), > it does not set MAT_NEW_NONZERO_ALLOCATION_**ERR. If the number of > nonzeros is larger than zero, it does. So far, so good. > If nnz is nonnegative, yes. > MatMPIAIJSetPreallocation_**MPIAIJ(), however, first checks for the > number of nonzeros. If it is PETSC_DECIDE, it sets defaults and then calls > MatSeqAIJSetPreallocation_**SeqAIJ() with the defaults. Because > MatSeqAIJSetPreallocation_**SeqAIJ() no longer sees PETSC_DECIDE, it sets > MAT_NEW_NONZERO_ALLOCATION_**ERR, which is then reset right after in > MatMPIAIJSetPreallocation_**MPIAIJ(). The CUSP-implementation does not > reset the flag and thus leads to errors, even though the implementation of > the CUSP-Preallocation routine looks perfectly fine. I rather want to fix > the hack in MatMPIAIJSetPreallocation_**MPIAIJ() than to propagate it to > the GPU implementations as well. > I thought perhaps you were going to leave it to the _SeqAIJ parts to store the nonew state. If the _MPIAIJ part resolves PETSC_DECIDE independently from the _SeqAIJ part, then they have to *always* coincide exactly, otherwise we'll have very confusing bugs. You can likely forward PETSC_DECIDE down to the _SeqAIJ parts, then query a->nonew and b->nonew to set the mpiaij->nonew flag. (I'm not sure it's consistent now, but we want it to be consistent.)

1 0

Re: [petsc-dev] Dealing with off-processor aij-matrices not having preallocation information
by Karl Rupp 21 Mar '13

21 Mar '13

Hi Jed, > As the comment noted, this is quite a hack and actually breaks the > current implementation for GPU matrices. I think that the problem > can be completely resolved by not checking for PETSC_DEFAULT and > PETSC_DECIDE in the MPI preallocation above, but just forward d_nz > and o_nz to the sequential preallocation routine. This way, the > logic remains in MatSeqAIJSetPreallocation(), where > MAT_NEW_NONZERO_ALLOCATION_ERR is only set if the user provided an > explicit preallocation scheme. > > > Hmm, the 'nonew' member is accessed in the MATMPIAIJ so we have to > either change the code to avoid ever using it from the top level or we > need to propagate the setting back to the parent. See also this discussion: > > http://lists.mcs.anl.gov/pipermail/petsc-dev/2012-January/007121.html > > https://bitbucket.org/petsc/petsc/commits/7827cd58 Yes, I want to preserve the behavior Barry described in 2), i.e. only error if the user specified an allocation pattern explicitly. I'm only questioning the way it is currently implemented: If the number of nonzeros is specified as PETSC_DECIDE to MatSeqAIJSetPreallocation_SeqAIJ(), it does not set MAT_NEW_NONZERO_ALLOCATION_ERR. If the number of nonzeros is larger than zero, it does. So far, so good. MatMPIAIJSetPreallocation_MPIAIJ(), however, first checks for the number of nonzeros. If it is PETSC_DECIDE, it sets defaults and then calls MatSeqAIJSetPreallocation_SeqAIJ() with the defaults. Because MatSeqAIJSetPreallocation_SeqAIJ() no longer sees PETSC_DECIDE, it sets MAT_NEW_NONZERO_ALLOCATION_ERR, which is then reset right after in MatMPIAIJSetPreallocation_MPIAIJ(). The CUSP-implementation does not reset the flag and thus leads to errors, even though the implementation of the CUSP-Preallocation routine looks perfectly fine. I rather want to fix the hack in MatMPIAIJSetPreallocation_MPIAIJ() than to propagate it to the GPU implementations as well. I can create a pull request for the proposed changes if this is too complicated to be discussed via email. Best regards, Karli

1 0

PETSc/Cygwin and Intel Visual Studio
by Chen Shapira 21 Mar '13

21 Mar '13

Hi Everyone, While building PETSc in Cygwin, I've noticed the following warning : "You are using an Intel supplied intrinsic header file with a third-party compiler." It seems that the configure and 'make' were completed well, but when I'm testing the petsc tutorials/examples, I sometimes get strange errors/warnings such as : In ex2 and ex3 : "Error : Window Not Successfuly Created" In ex7 , ex10 and others : "Warning: win32fe: Library Path Not Found: /home/petsc-dev/PROGRA~2IntelCOMPOS~3b inia32" Any ideas on what to do ? Thanks Chen

1 0

Re: [petsc-dev] Dealing with off-processor aij-matrices not having preallocation information
by Jed Brown 21 Mar '13

21 Mar '13

On Wed, Mar 20, 2013 at 6:04 PM, Karl Rupp <rupp(a)mcs.anl.gov> wrote: > > As the comment noted, this is quite a hack and actually breaks the current > implementation for GPU matrices. I think that the problem can be completely > resolved by not checking for PETSC_DEFAULT and PETSC_DECIDE in the MPI > preallocation above, but just forward d_nz and o_nz to the sequential > preallocation routine. This way, the logic remains in > MatSeqAIJSetPreallocation(), where MAT_NEW_NONZERO_ALLOCATION_ERR is only > set if the user provided an explicit preallocation scheme. Hmm, the 'nonew' member is accessed in the MATMPIAIJ so we have to either change the code to avoid ever using it from the top level or we need to propagate the setting back to the parent. See also this discussion: http://lists.mcs.anl.gov/pipermail/petsc-dev/2012-January/007121.html https://bitbucket.org/petsc/petsc/commits/7827cd58

1 0

Re: [petsc-dev] Unable to find zlib library
by Chen Shapira 21 Mar '13

21 Mar '13

Thanks Satish, I've successfuly configured it now C. On Wed, Mar 20, 2013 at 5:51 PM, Satish Balay <balay(a)mcs.anl.gov> wrote: > Make sure you have gcc/g++ from cygwin installed for sowing. > > Satish > > On Wed, 20 Mar 2013, Chen Shapira wrote: > >> Thanks Satish, >> I've tried : >> ./config/configure.py --with-cc='win32fe cl' --with-fc='win32fe ifort' >> --with-cxx='win32fe cl' --with-mpi-dir="/cygdrive/C/Program Files >> (x86)/MPICH2" --download-f-blas-lapack=1 --with-clanguage=c >> --with-petsc-arch=arch-mswin-c-debug >> --CFLAGS=-D_CRT_SECURE_NO_DEPRECATE --with-debugging=0 >> --ignore-cygwin-link --with-valgrind=0 --with-c2html=0 >> --download-sowing=1 --with-hdf5=1 >> --with-hdf5-include=/cygdrive/c/libraries/hdf5/include >> --with-hdf5-lib=\[/cygdrive/c/libraries/hdf5/libs/hdf5dll.lib,/cygdrive/c/libraries/hdf5/libs/hdf5_fortrandll.lib,/cygdrive/c/libraries/hdf5/libs/szip.lib,/cygdrive/c/libraries/hdf5/libs/zlib.lib\] >> --LIBS=/cygdrive/c/libraries/hdf5/libs/zlib.lib >> >> and also with : --LIBS='/cygdrive/c/libraries/hdf5/libs/zlib.lib' >> >> and I got a different error : >> Error running make; make install on Sowing (install manually): Could >> not execute "cd /home/petsc-dev/externalpackages/sowing-1.1.16d && >> make && make install && make clean": >> (cd src/sys && make ) >> make[1]: Entering directory >> `/home/petsc-dev/externalpackages/sowing-1.1.16d/src/sys' >> cl.exe -I../../include -I../../include -c arch.c >> arch.c >> arch.c(22) : fatal error C1083: Cannot open include file: >> 'sys/utsname.h': No such file or directory >> ../../Makerules:39: recipe for target `arch.o' failed >> make[1]: Leaving directory >> `/home/petsc-dev/externalpackages/sowing-1.1.16d/src/sys' >> Makefile:18: recipe for target `libs' failed >> >> Thanks again >> Chen >> >> On Tue, Mar 19, 2013 at 4:16 PM, Satish Balay <balay(a)mcs.anl.gov> wrote: >> > zlib detection code is a bit wired. Try using the additional option: >> > >> > >> > LIBS=/cygdrive/c/libraries/hdf5/libs/zlib.lib >> > >> > Satish >> > >> > On Tue, 19 Mar 2013, Chen Shapira wrote: >> > >> >> Hi Again, >> >> >> >> While configuring PETSc in Cygwin, >> >> I recieve the following error : >> >> "Compression library [libz.a or equivalent] not found" >> >> >> >> The thing is, I already have the file : zlib.lib included in the >> >> configuration of petsc . >> >> >> >> My configure command is : >> >> ./config/configure.py --with-cc='win32fe cl' --with-fc='win32fe ifort' >> >> --with-cxx='win32fe cl' --with-mpi-dir="/cygdrive/C/Program Files >> >> (x86)/MPICH2" --download-f-blas-lapack=1 --with-clanguage=c >> >> --with-petsc-arch=arch-mswin-c-debug >> >> --CFLAGS=-D_CRT_SECURE_NO_DEPRECATE --with-debugging=0 >> >> --ignore-cygwin-link --with-valgrind=0 --with-c2html=0 >> >> --download-sowing=1 --with-hdf5=1 >> >> --with-hdf5-include=/cygdrive/c/libraries/hdf5/include >> >> --with-hdf5-lib=\[/cygdrive/c/libraries/hdf5/libs/hdf5dll.lib,/cygdrive/c/libraries/hdf5/libs/hdf5_fortrandll.lib,/cygdrive/c/libraries/hdf5/libs/szip.lib,/cygdrive/c/libraries/hdf5/libs/zlib.lib\] >> >> >> >> Thank you again.. >> >> Chen >> >> >> > >> >

1 0

Re: [petsc-dev] CMake-assisted CUDA builds ready
by Dave Nystrom 20 Mar '13

20 Mar '13

Hi Karli, Just now seeing this. I did my build from a clean slate. Thanks, Dave Karl Rupp writes: > Hi Dave, > > I also found that in some occasions one should issue > $> rm -rf PETSC_ARCH/CMake* > prior to running a reconfigure in order to circumvent a bug [1] in CMake. > > @Jed, Matt: The current cmakeboot.py removes > PETSC_ARCH/CMakeFiles/{version_string}, which does not seem to even > exist on my machine (CMake 2.8.7). Can we instead just delete the whole > CMakeFiles-folder and probably CMakeCache.txt? > > Best regards, > Karli > > [1] http://www.mail-archive.com/[email protected]/msg44765.html > > > > On 03/20/2013 10:17 AM, Nystrom, William D wrote: > > Hi Karli, > > > > I ran a test build last night and it failed. It was a relatively simple configuration where > > I was downloading mpich and cmake and building with threads, cuda and the txpetscgpu > > package. I was building on a Fedora 17 machine with Cuda 5. The failure seem to be > > associated with cmake. When I get home this evening, I could send you the configure > > and make logs. I did not have time this morning before leaving for work. I wanted to > > give you some early feedback even though the logs will have to wait till this evening. > > > > Thanks, > > > > Dave > > > > -- > > Dave Nystrom > > LANL HPC-5 > > Phone: 505-667-7913 > > Email: wdn(a)lanl.gov > > Smail: Mail Stop B272 > > Group HPC-5 > > Los Alamos National Laboratory > > Los Alamos, NM 87545 > > > > > > ________________________________________ > > From: petsc-dev-bounces(a)mcs.anl.gov [petsc-dev-bounces(a)mcs.anl.gov] on behalf of Karl Rupp [rupp(a)mcs.anl.gov] > > Sent: Tuesday, March 19, 2013 4:04 PM > > To: petsc-dev(a)mcs.anl.gov > > Subject: Re: [petsc-dev] CMake-assisted CUDA builds ready > > > > Hi Dave, > > > > perfect, thanks! > > I should add that a reconfigure is needed. It won't work with re-running > > make only. > > > > Best regards, > > Karli > > > > > > On 03/19/2013 04:59 PM, Nystrom, William D wrote: > >> Hi Karli, > >> > >> I will try this soon and report back when I have results. > >> > >> Thanks, > >> > >> Dave > >> > >> ________________________________________ > >> From: petsc-dev-bounces(a)mcs.anl.gov [petsc-dev-bounces(a)mcs.anl.gov] on behalf of Karl Rupp [rupp(a)mcs.anl.gov] > >> Sent: Tuesday, March 19, 2013 3:58 PM > >> To: For users of the development version of PETSc > >> Subject: [petsc-dev] CMake-assisted CUDA builds ready > >> > >> Dear all, > >> > >> I just merged the CMake-based builds for CUDA-packages into 'next', > >> reducing compilation times considerably compared to the old legacy > >> builds. While the legacy builds print something like > >> 'libfast in <DIR>' > >> the new CMake-based builds print the usual percentage counters in front. > >> > >> Any feedback on the proper functionality (both positive and negative) > >> are appreciated. > >> > >> Thanks and best regards, > >> Karli > >> > > >

1 0

Dealing with off-processor aij-matrices not having preallocation information
by Karl Rupp 20 Mar '13

20 Mar '13

Hi guys, if a distributed aij-matrix is created and no preallocation hints are given, the current behavior is to silently allow preallocation even if the user requested to error out if not preallocated. From MatMPIAIJSetPreallocation_MPIAIJ() in mpiaij.c:3285: (slightly modified using style violations for better visual appearance) if (d_nz == PETSC_DEFAULT || d_nz == PETSC_DECIDE) d_nz = 5; if (o_nz == PETSC_DEFAULT || o_nz == PETSC_DECIDE) o_nz = 2; /* Quite some more code here */ ierr = MatSeqAIJSetPreallocation(b->A,d_nz,d_nnz);CHKERRQ(ierr); ierr = MatSeqAIJSetPreallocation(b->B,o_nz,o_nnz);CHKERRQ(ierr); /* Do not error if the user did not give real preallocation information. Ugly because this would overwrite a previous user call to MatSetOption(). */ if (!d_realalloc) MatSetOption(b->A,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_FALSE); if (!o_realalloc) MatSetOption(b->B,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_FALSE); As the comment noted, this is quite a hack and actually breaks the current implementation for GPU matrices. I think that the problem can be completely resolved by not checking for PETSC_DEFAULT and PETSC_DECIDE in the MPI preallocation above, but just forward d_nz and o_nz to the sequential preallocation routine. This way, the logic remains in MatSeqAIJSetPreallocation(), where MAT_NEW_NONZERO_ALLOCATION_ERR is only set if the user provided an explicit preallocation scheme. Do I miss something? Best regards, Karli

1 0

Re: [petsc-dev] Reminder planning release soon
by Barry Smith 20 Mar '13

20 Mar '13

On Mar 20, 2013, at 3:59 PM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > > On Wed, Mar 20, 2013 at 3:57 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > I am not objecting to having an option to create a new branch. I am object to the use of "b" as the letter to indicate the option. > > You would prefer 'git checkout --branch new-branch-name [optional-commit-to-start-branch-from]'? Because that works. Or you want only 'git checkout --create-new-branch new-branch-name [optional-commit-to-start-branch-from]'? The problem is that "branch" is the git command to create a new branch, as well as the "root" name of a bunch of operations related to branches. You have git branch name (create new branch) git branch -a (list branches) git branch -d name (delete branch) other stuff you have internalized this strange user interface so you think "git checkout --branch name" makes perfect sense. A new user who has not internalized this strange user interface sees "git checkout --branch name" (or the short -b) and thinks "ok I'm checking out the branch named name", they sure would not conclude that a new branch is being made. Once you have this moronic starting point that "branch" is both the command to create a new branch and the "root" of operations related to branches any chance of a usable user interface is already gone. You need to revisit the higher level scheme and have for example git branch --help git branch --create git branch --list etc and have git branch behave just like git --help. Do this for all the "overloaded" git commands (note that, for example, git checkout is not an "overloaded" git command, the various options only change the specific behavior of checkout, they don't make it do completely different stuff like the options to branch). Once you have these consistently named "building blocks" you need to decide IF you want a way to compose them, and if you do, on a consistent way to do the composition. So, for example, the moronic git checkout --branch newname could become (under one possible composition model) git checkout --branch --create newname and of course git checkout --branch newname would be the same as git checkout --branch --help newname which result in the help message for branch being printed in the middle of the checkout operation, useful since you didn't know what you were doing. Now I don't like this composition syntax because "composed operations" are marked with the same "--" as options to operations so maybe a "++" could be used git checkout ++branch --create newname or the short form git c +b -c newname for people who do not like to type. Consistent and clean and it is immediately clear what is composition of base commands. Next you actually have to refine the model and try it out with all the functionality you want to provide in an iterative process (thinking not only about YOUR use of the tool but all kinds of peoples use of the tool), CHANGING things until you get something good. Now I am not claiming my design above is good (I suspect it has flaws that will come up in use), only time and use and evolution would only tell but I am submitting that I have already thought more about the user interface for git at the appropriate abstract level then the git people have in the last x years since it was started. Barry

1 0

Re: [petsc-dev] Reminder planning release soon
by Jed Brown 20 Mar '13

20 Mar '13

On Wed, Mar 20, 2013 at 3:57 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > I am not objecting to having an option to create a new branch. I am > object to the use of "b" as the letter to indicate the option. > You would prefer 'git checkout --branch new-branch-name [optional-commit-to-start-branch-from]'? Because that works. Or you want only 'git checkout --create-new-branch new-branch-name [optional-commit-to-start-branch-from]'?

1 0

Re: [petsc-dev] Reminder planning release soon
by Barry Smith 20 Mar '13

20 Mar '13

I am not objecting to having an option to create a new branch. I am object to the use of "b" as the letter to indicate the option. On Mar 20, 2013, at 3:53 PM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > > On Wed, Mar 20, 2013 at 3:48 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > Likely some allegedly smart person figured -b is short for branch so everyone will instinctively know that that -b means create a new branch. Of course to new users who don't know git they would never think -b meant create a new branch. The problem is that the developers of git have absolutely no taste in user interface design and likely consider any time spent thinking about user interface issues is wasted and just use the first stupid thing that pops into their head. > > It is very common to want to create a new branch starting from a particular commit. What syntax would you suggest for this combination? Or leave it to the user to write their own alias that does both?

1 0