June 2012 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] slow "ar Scq" on cygwin
by Chetan Jhurani 28 Jun '12

28 Jun '12

Matt, I was surprised too on both counts. Couldn't find any libpetsc.lib after the python build and I did not run "clean" in between that could have deleted the .o files. Here's one more try below to show it. $ ls test_8/lib/ libpetsc-obj $ ls test_8/lib/libpetsc-obj/ | wc -l 560 $ PETSC_DIR=/cygdrive/d/petsc-dev-2012-06-26 PETSC_ARCH=test_8 ./config/builder.py (The same happens (existing .o are deleted) if ./config/builder2.py build is used) Rebuilding Dependencies Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/bag Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/comm Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/dll I stopped the build using control-c. It would have deleted all files first. Traceback (most recent call last): File "./config/builder.py", line 1534, in <module> PETScMaker().run() File "./config/builder.py", line 1526, in run if self.buildLibraries('libpetsc', self.rootDir): File "./config/builder.py", line 1331, in buildLibraries objects += self.buildDir(root, files, objDir) File "./config/builder.py", line 1251, in buildDir objects.extend(getattr(self, 'compile'+language)(sourceMap[language], objDir)) File "./config/builder.py", line 1025, in compileC return self.compile(self.configInfo.languages.clanguage, source, objDir) File "./config/builder.py", line 1012, in compile (output, error, status) = self.executeShellCommand(cmd, checkCommand = noCheckCommand, log=self.log) File "/cygdrive/d/petsc-dev-2012-06-26/config/BuildSystem/script.py", line 254, in executeShellCommand (output, error, status) = runInShell(command, log, cwd) File "/cygdrive/d/petsc-dev-2012-06-26/config/BuildSystem/script.py", line 243, in runInShell thread.join(timeout) File "/usr/lib/python2.6/threading.py", line 655, in join self.__block.wait(delay) File "/usr/lib/python2.6/threading.py", line 258, in wait _sleep(delay) KeyboardInterrupt $ ls test_8/lib/libpetsc-obj/ bag.o All the previous .o were deleted. Perhaps I'm not doing something right? Chetan From: petsc-dev-bounces(a)mcs.anl.gov [mailto:[email protected]] On Behalf Of Matthew Knepley Sent: Thursday, June 28, 2012 2:46 PM To: For users of the development version of PETSc Subject: Re: [petsc-dev] slow "ar Scq" on cygwin On Thu, Jun 28, 2012 at 3:31 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> wrote: Thanks Matt, Satish. A few issues though. Like you said, the latest cygwin has python 2.6.8 which does not have argparse. When I place the following in path http://argparse.googlecode.com/svn/trunk/argparse.py I can run the python build with 2.6.8. However, the "ar" based build had worked fine and the python based build fails for these two files. src/snes/impls/MULTIB~1/multiblock.c src/snes/impls/vi/rsaug/virsaug.c Yep, it fails, but the build succeeds, so no problem. It looks like these two files were NOT compiled in the ar based build. The two make.log files are attached. One more question. I tried to run the python based build the second time (after it was unsuccessful the first time) and then it deleted all the previous .o files it had created in ARCH/lib/libpetsc-obj. How can that be avoided? It does not do that unless you run 'clean'. Matt Chetan From: petsc-dev-bounces(a)mcs.anl.gov [mailto:[email protected]] On Behalf Of Matthew Knepley Sent: Thursday, June 28, 2012 1:35 PM To: For users of the development version of PETSc Subject: Re: [petsc-dev] slow "ar Scq" on cygwin On Thu, Jun 28, 2012 at 2:23 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> wrote: Hi devs, Is there a way to avoid running the archiver (ar) every time a petsc directory is compiled? The reason I'm asking this is that when libpetsc.lib becomes large enough (say 100MB+, esp. for debug build), "ar Sqc" on cygwin takes very long in each directory. Depending on OS caching, very long can be 10 seconds per PETSc directory just to run ar. It creates a new temp file from scratch, puts stuff from libpetsc.lib and new stuff in it and then renames the temp file. As you can imagine, the compilation process is fast in the beginning but crawls by the end. I've seen the same IO sequence on Linux (with Sqc flags to ar), but since cygwin IO is slower the effect is more clearly visible. Building within a ramdisk leads to 25% faster compilation but it was more of a curiosity. Build with Python. It never uses antique things like ar :) python2.7 ./config/builder2.py build Matt Chetan -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] slow "ar Scq" on cygwin
by Matthew Knepley 28 Jun '12

28 Jun '12

On Thu, Jun 28, 2012 at 3:31 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com>wrote: > Thanks Matt, Satish. A few issues though.**** > > ** ** > > Like you said, the latest cygwin has python 2.6.8 which**** > > does not have argparse. When I place the following**** > > in path http://argparse.googlecode.com/svn/trunk/argparse.py**** > > I can run the python build with 2.6.8.**** > > ** ** > > However, the “ar” based build had worked fine and the python**** > > based build fails for these two files.**** > > ** ** > > src/snes/impls/MULTIB~1/multiblock.c**** > > src/snes/impls/vi/rsaug/virsaug.c > Yep, it fails, but the build succeeds, so no problem. > It looks like these two files were NOT compiled in the ar based**** > > build. The two make.log files are attached.**** > > ** ** > > One more question. I tried to run the python based build**** > > the second time (after it was unsuccessful the first time)**** > > and then it deleted all the previous .o files it had created in **** > > ARCH/lib/libpetsc-obj. How can that be avoided? > It does not do that unless you run 'clean'. Matt > Chetan**** > > ** ** > > ** ** > > *From:* petsc-dev-bounces(a)mcs.anl.gov [mailto: > petsc-dev-bounces(a)mcs.anl.gov] *On Behalf Of *Matthew Knepley > *Sent:* Thursday, June 28, 2012 1:35 PM > *To:* For users of the development version of PETSc > *Subject:* Re: [petsc-dev] slow "ar Scq" on cygwin**** > > ** ** > > On Thu, Jun 28, 2012 at 2:23 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> > wrote:**** > > Hi devs, > > Is there a way to avoid running the archiver (ar) > every time a petsc directory is compiled? > > The reason I'm asking this is that when libpetsc.lib > becomes large enough (say 100MB+, esp. for debug build), > "ar Sqc" on cygwin takes very long in each directory. > Depending on OS caching, very long can be 10 seconds per > PETSc directory just to run ar. > > It creates a new temp file from scratch, puts stuff from > libpetsc.lib and new stuff in it and then renames the > temp file. As you can imagine, the compilation process is > fast in the beginning but crawls by the end. I've seen > the same IO sequence on Linux (with Sqc flags to ar), but > since cygwin IO is slower the effect is more clearly visible. > Building within a ramdisk leads to 25% faster compilation > but it was more of a curiosity.**** > > ** ** > > Build with Python. It never uses antique things like ar :)**** > > ** ** > > python2.7 ./config/builder2.py build**** > > ** ** > > Matt**** > > **** > > > Chetan > > **** > > > > **** > > ** ** > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener**** > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Xcode
by Cai Durbin 28 Jun '12

28 Jun '12

Hey, I was wondering if anyone has had any success using PETSc with Xcode? Cheers, Cai

1 0

Re: [petsc-dev] slow "ar Scq" on cygwin
by Chetan Jhurani 28 Jun '12

28 Jun '12

Thanks Matt, Satish. A few issues though. Like you said, the latest cygwin has python 2.6.8 which does not have argparse. When I place the following in path http://argparse.googlecode.com/svn/trunk/argparse.py I can run the python build with 2.6.8. However, the "ar" based build had worked fine and the python based build fails for these two files. src/snes/impls/MULTIB~1/multiblock.c src/snes/impls/vi/rsaug/virsaug.c It looks like these two files were NOT compiled in the ar based build. The two make.log files are attached. One more question. I tried to run the python based build the second time (after it was unsuccessful the first time) and then it deleted all the previous .o files it had created in ARCH/lib/libpetsc-obj. How can that be avoided? Chetan From: petsc-dev-bounces(a)mcs.anl.gov [mailto:[email protected]] On Behalf Of Matthew Knepley Sent: Thursday, June 28, 2012 1:35 PM To: For users of the development version of PETSc Subject: Re: [petsc-dev] slow "ar Scq" on cygwin On Thu, Jun 28, 2012 at 2:23 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> wrote: Hi devs, Is there a way to avoid running the archiver (ar) every time a petsc directory is compiled? The reason I'm asking this is that when libpetsc.lib becomes large enough (say 100MB+, esp. for debug build), "ar Sqc" on cygwin takes very long in each directory. Depending on OS caching, very long can be 10 seconds per PETSc directory just to run ar. It creates a new temp file from scratch, puts stuff from libpetsc.lib and new stuff in it and then renames the temp file. As you can imagine, the compilation process is fast in the beginning but crawls by the end. I've seen the same IO sequence on Linux (with Sqc flags to ar), but since cygwin IO is slower the effect is more clearly visible. Building within a ramdisk leads to 25% faster compilation but it was more of a curiosity. Build with Python. It never uses antique things like ar :) python2.7 ./config/builder2.py build Matt Chetan -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] slow "ar Scq" on cygwin
by Satish Balay 28 Jun '12

28 Jun '12

On Thu, 28 Jun 2012, Matthew Knepley wrote: > > Build with Python. It never uses antique things like ar :) > > python2.7 ./config/builder2.py build builder2.py need python-2.7? For one its never tested on windows builds - esp with MS compilers. Now I see cygwin has sbalay@ps3 ~ $ python --version Python 2.6.7 Satish

1 0

Re: [petsc-dev] slow "ar Scq" on cygwin
by Satish Balay 28 Jun '12

28 Jun '12

On Thu, 28 Jun 2012, Chetan Jhurani wrote: > Hi devs, > > Is there a way to avoid running the archiver (ar) > every time a petsc directory is compiled? > > The reason I'm asking this is that when libpetsc.lib > becomes large enough (say 100MB+, esp. for debug build), > "ar Sqc" on cygwin takes very long in each directory. > Depending on OS caching, very long can be 10 seconds per > PETSc directory just to run ar. > > It creates a new temp file from scratch, puts stuff from > libpetsc.lib and new stuff in it and then renames the > temp file. As you can imagine, the compilation process is > fast in the beginning but crawls by the end. I've seen > the same IO sequence on Linux (with Sqc flags to ar), but > since cygwin IO is slower the effect is more clearly visible. > Building within a ramdisk leads to 25% faster compilation > but it was more of a curiosity. 'ar Scq' is a hack to speed up builds on Mac [as the previous default 'ar cr' was too slow on it] I don't know if 'ar cr' is also slow on windows. But perhaps it preferable to use '--with-ar='win32fe lib' - perhaps this is faster. ['ar' is unuseable with 64bit windows compilers anyway] Satish

1 0

Re: [petsc-dev] slow "ar Scq" on cygwin
by Matthew Knepley 28 Jun '12

28 Jun '12

On Thu, Jun 28, 2012 at 2:23 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com>wrote: > Hi devs, > > Is there a way to avoid running the archiver (ar) > every time a petsc directory is compiled? > > The reason I'm asking this is that when libpetsc.lib > becomes large enough (say 100MB+, esp. for debug build), > "ar Sqc" on cygwin takes very long in each directory. > Depending on OS caching, very long can be 10 seconds per > PETSc directory just to run ar. > > It creates a new temp file from scratch, puts stuff from > libpetsc.lib and new stuff in it and then renames the > temp file. As you can imagine, the compilation process is > fast in the beginning but crawls by the end. I've seen > the same IO sequence on Linux (with Sqc flags to ar), but > since cygwin IO is slower the effect is more clearly visible. > Building within a ramdisk leads to 25% faster compilation > but it was more of a curiosity. Build with Python. It never uses antique things like ar :) python2.7 ./config/builder2.py build Matt > > Chetan > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

slow "ar Scq" on cygwin
by Chetan Jhurani 28 Jun '12

28 Jun '12

Hi devs, Is there a way to avoid running the archiver (ar) every time a petsc directory is compiled? The reason I'm asking this is that when libpetsc.lib becomes large enough (say 100MB+, esp. for debug build), "ar Sqc" on cygwin takes very long in each directory. Depending on OS caching, very long can be 10 seconds per PETSc directory just to run ar. It creates a new temp file from scratch, puts stuff from libpetsc.lib and new stuff in it and then renames the temp file. As you can imagine, the compilation process is fast in the beginning but crawls by the end. I've seen the same IO sequence on Linux (with Sqc flags to ar), but since cygwin IO is slower the effect is more clearly visible. Building within a ramdisk leads to 25% faster compilation but it was more of a curiosity. Chetan

1 0

Re: [petsc-dev] [petsc-users] Loading Binary file problem
by Barry Smith 26 Jun '12

26 Jun '12

It would be nice if the error message could be related to the VecLoad() call so the error message could say that there is no vector next in the file. > > [d3m956@olympus tutorials]$ ./ex12 -f0 myA.dat > > This program expects a matrix followed by a vector in the file. You can use -rhs 0 to skip reading the right hand side vector from the file, it will use a vector of all ones. > > [0]PETSC ERROR: --------------------- Error Message ------------------------------------ > [0]PETSC ERROR: Read from file failed! > [0]PETSC ERROR: Read past end of file! > [0]PETSC ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 3.3.0, Patch 0, Tue Jun 5 14:20:42 CDT 2012 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: ./ex12 on a arch-linu named olympus.local by d3m956 Tue Jun 26 13:51:28 2012 > [0]PETSC ERROR: Libraries linked from /pic/projects/mca/ss/PETSC/petsc-3.3-p0/arch-linux2-c-debug/lib > [0]PETSC ERROR: Configure run at Thu Jun 14 17:00:19 2012 > [0]PETSC ERROR: Configure options --with-cc=gcc --with-fc=gfortran --download-f-blas-lapack --download-mpich > [0]PETSC ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: PetscBinaryRead() line 271 in src/sys/fileio/sysio.c > [0]PETSC ERROR: PetscBinarySynchronizedRead() line 558 in src/sys/fileio/sysio.c > [0]PETSC ERROR: PetscViewerBinaryRead() line 764 in src/sys/viewer/impls/binary/binv.c > [0]PETSC ERROR: PetscViewerBinaryReadVecHeader_Private() line 24 in src/vec/vec/utils/vecio.c > [0]PETSC ERROR: VecLoad_Binary() line 97 in src/vec/vec/utils/vecio.c > [0]PETSC ERROR: VecLoad_Default() line 348 in src/vec/vec/utils/vecio.c > [0]PETSC ERROR: VecLoad() line 1111 in src/vec/vec/interface/vector.c > [0]PETSC ERROR: main() line 92 in src/mat/examples/tutorials/ex12.c > > It said “Read from file failed! Read past end of file!” Anything wrong? I was using the example code without changing anything. Please help me to figure out the problem. > > Thanks, > Shuangshuang > > >

1 0

Problematic Merge of FieldSplit
by Matthew Knepley 26 Jun '12

26 Jun '12

It turns out that 'hg rollback' during an 'hg rebase' does not do what I thought it did. I think everything is cleaned up with this push, but if you made FS changes in the past month, please check that it is doing what you want with prefixes, etc. Now, nested fieldsplits from the command line work, ala -ksp_type fgmres -pc_type fieldsplit -pc_fieldsplit_type additive -pc_fieldsplit_0_fields 0,1 -fieldsplit_0_pc_type fieldsplit -fieldsplit_0_pc_fieldsplit_type schur -fieldsplit_0_pc_fieldsplitschur_factorization_type full -fieldsplit_0_fieldsplit_velocity_ksp_type preonly -fieldsplit_0_fieldsplit_velocity_pc_type lu -fieldsplit_0_fieldsplit_pressure_ksp_rtol 1e-10 -fieldsplit_0_fieldsplit_pressure_pc_type jacobi -pc_fieldsplit_1_fields 2 -fieldsplit_temperature_ksp_type preonly -fieldsplit_temperature_pc_type lu A split with only one field gets the field name, and otherwise a split number. Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0