June 2012 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] Xcode
by Barry Smith 30 Jun '12

30 Jun '12

On Jun 30, 2012, at 1:36 AM, Cai Durbin wrote: > Hey Barry, > > I just wanted to check that I've understood you correctly. Your message said that somebody has had some success in getting Xcode working with PETSc but with "limited experience". What did you mean by that last bit. > > Also, I checked the manual and it just directs me to the mailing list. Sorry, I had forgotten there is no information there. > > I've got PETSc working in the last build of Xcode, but if someone else has managed it then I'd be interested in comparing setups. I am actually the "someone with limited experience". I built the PETSc libraries and some examples for the iphone/ipad using xcode two years ago. The code is in the directory called xcode of PETSc. I don't know if it works now. Barry > > Cheers, > > Cai > > On 28 Jun 2012, at 18:29, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > >> >> Yes, but with limited experience. See the section in the users manual: http://www.mcs.anl.gov/petsc/petsc-current/docs/manual.pdf section 13.13 page 177 >> >> Feed back on improving the documentation on this etc would be appreciated. >> >> Barry >> >> >> On Jun 28, 2012, at 4:35 PM, Cai Durbin wrote: >> >>> Hey, >>> >>> I was wondering if anyone has had any success using PETSc with Xcode? >>> >>> Cheers, >>> >>> Cai >>

1 0

Re: [petsc-dev] slow "ar Scq" on cygwin
by Matthew Knepley 29 Jun '12

29 Jun '12

On Thu, Jun 28, 2012 at 8:26 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com>wrote: > > Look. It does not do that. Ever. In any universe. I don't know what you > think you are doing,**** > > > but it is definitely not what you wrote. Look at the code. It can't do > that.**** > > ** ** > > I don’t think you understand what your code is doing in a new**** > > environment of the same universe. > Yes, you are correct. I do not understand this code. I was wrong to tell you to use it. Don't use it. Matt > > > It works fine on Linux with the steps you had given.**** > > But on the cygwin side, no dependency files like**** > > ./lib/libpetsc-obj/adaptbasic.d are created. I found this after**** > > running the python build on Linux (I didn’t know before this**** > > where the dependencies were stored).**** > > ** ** > > Here is some debug info in the python code:**** > > ** ** > > In the buildLibraries function on the Linux box (when building in the > second step)**** > > ** ** > > self.sourceDatabase = DirectedGraph with 1172 vertices and 21894 edges* > *** > > ** ** > > However, on the cygwin side (again, when building the second time)**** > > ** ** > > self.sourceDatabase = <builder.NullSourceDatabase object at 0xffd833ac>* > *** > > ** ** > > So of course it is totalRebuild = true on cygwin.**** > > ** ** > > I don’t mind extra debugging on cygwin/Windows before reporting issues**** > > since I know the petsc project has many other priorities. But a few useful > **** > > debugging pointers will be serve both of us better.**** > > ** ** > > Chetan**** > > ** ** > > ** ** > > ** ** > > ** ** > > *From:* petsc-dev-bounces(a)mcs.anl.gov [mailto: > petsc-dev-bounces(a)mcs.anl.gov] *On Behalf Of *Matthew Knepley > *Sent:* Thursday, June 28, 2012 6:00 PM > > *To:* For users of the development version of PETSc > *Subject:* Re: [petsc-dev] slow "ar Scq" on cygwin**** > > ** ** > > On Thu, Jun 28, 2012 at 4:49 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> > wrote:**** > > *> > $ PETSC_DIR=/cygdrive/d/petsc-dev-2012-06-26 PETSC_ARCH=test_8 > ./config/builder.py > > > (The same happens (existing .o are deleted) if ./config/builder2.py > build is used)***** > > *> *This is not what I wrote. You used builder.py, which is python 2.4 > compatible, but has to do everything without options, > *> *so yes it deletes all the objects and rebuilds everything. That is > why I did not suggest you use it.**** > > *> *I said to use builder2.py, which does proper dependency tracking.**** > > **** > > That is what I used and I mentioned it above. I’ve tried both separately.* > *** > > **** > > Both give the same result that libpetsc.lib is not created. And**** > > whether I use builder.py or builder2.py, the existing objects**** > > files are deleted in the second build. I did not call for a cleanup.**** > > ** ** > > Look. It does not do that. Ever. In any universe. I don't know what you > think you are doing,**** > > but it is definitely not what you wrote. Look at the code. It can't do > that.**** > > ** ** > > matt**** > > **** > > To be sure, I tried the whole thing in a new ARCH with a fresh**** > > configure step, and this time builder.py was not used at all (by me)**** > > but it led to the same issues with builder2.py.**** > > **** > > I understand that this is not the most used platform, so let me**** > > know if I can be of help. And I forgot to mention that the**** > > python build was 4-5 times faster than “ar” based build and**** > > that’s great...**** > > **** > > Chetan**** > > **** > > **** > > **** > > **** > > **** > > *From:* petsc-dev-bounces(a)mcs.anl.gov [mailto: > petsc-dev-bounces(a)mcs.anl.gov] *On Behalf Of *Matthew Knepley > *Sent:* Thursday, June 28, 2012 3:21 PM > *To:* For users of the development version of PETSc > *Subject:* Re: [petsc-dev] slow "ar Scq" on cygwin**** > > **** > > On Thu, Jun 28, 2012 at 4:10 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> > wrote:**** > > Matt,**** > > **** > > I was surprised too on both counts. Couldn’t find any libpetsc.lib after > the python build**** > > and I did not run “clean” in between that could have deleted the .o files. > **** > > **** > > Here’s one more try below to show it.**** > > **** > > *$ ls test_8/lib/***** > > libpetsc-obj**** > > **** > > *$ ls test_8/lib/libpetsc-obj/ | wc -l***** > > *560***** > > **** > > *$ PETSC_DIR=/cygdrive/d/petsc-dev-2012-06-26 PETSC_ARCH=test_8 > ./config/builder.py***** > > *(The same happens (existing .o are deleted) if ./config/builder2.py > build is used)***** > > **** > > This is not what I wrote. You used builder.py, which is python 2.4 > compatible, but has to do everything without options,**** > > so yes it deletes all the objects and rebuilds everything. That is why I > did not suggest you use it.**** > > **** > > I said to use builder2.py, which does proper dependency tracking.**** > > **** > > Matt**** > > **** > > * ***** > > Rebuilding Dependencies**** > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys**** > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/bag**** > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/comm**** > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/dll**** > > **** > > *I stopped the build using control-c. It would have deleted all files > first.***** > > **** > > Traceback (most recent call last):**** > > File "./config/builder.py", line 1534, in <module>**** > > PETScMaker().run()**** > > File "./config/builder.py", line 1526, in run**** > > if self.buildLibraries('libpetsc', self.rootDir):**** > > File "./config/builder.py", line 1331, in buildLibraries**** > > objects += self.buildDir(root, files, objDir)**** > > File "./config/builder.py", line 1251, in buildDir**** > > objects.extend(getattr(self, 'compile'+language)(sourceMap[language], > objDir))**** > > File "./config/builder.py", line 1025, in compileC**** > > return self.compile(self.configInfo.languages.clanguage, source, > objDir)**** > > File "./config/builder.py", line 1012, in compile**** > > (output, error, status) = self.executeShellCommand(cmd, checkCommand = > noCheckCommand, log=self.log)**** > > File "/cygdrive/d/petsc-dev-2012-06-26/config/BuildSystem/script.py", > line 254, in executeShellCommand**** > > (output, error, status) = runInShell(command, log, cwd)**** > > File "/cygdrive/d/petsc-dev-2012-06-26/config/BuildSystem/script.py", > line 243, in runInShell**** > > thread.join(timeout)**** > > File "/usr/lib/python2.6/threading.py", line 655, in join**** > > self.__block.wait(delay)**** > > File "/usr/lib/python2.6/threading.py", line 258, in wait**** > > _sleep(delay)**** > > KeyboardInterrupt**** > > **** > > *$ ls test_8/lib/libpetsc-obj/***** > > *bag.o***** > > **** > > All the previous .o were deleted. Perhaps I’m not doing something right?* > *** > > **** > > Chetan**** > > **** > > **** > > *From:* petsc-dev-bounces(a)mcs.anl.gov [mailto: > petsc-dev-bounces(a)mcs.anl.gov] *On Behalf Of *Matthew Knepley > *Sent:* Thursday, June 28, 2012 2:46 PM > *To:* For users of the development version of PETSc > *Subject:* Re: [petsc-dev] slow "ar Scq" on cygwin**** > > **** > > On Thu, Jun 28, 2012 at 3:31 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> > wrote:**** > > Thanks Matt, Satish. A few issues though.**** > > **** > > Like you said, the latest cygwin has python 2.6.8 which**** > > does not have argparse. When I place the following**** > > in path http://argparse.googlecode.com/svn/trunk/argparse.py**** > > I can run the python build with 2.6.8.**** > > **** > > However, the “ar” based build had worked fine and the python**** > > based build fails for these two files.**** > > **** > > src/snes/impls/MULTIB~1/multiblock.c**** > > src/snes/impls/vi/rsaug/virsaug.c**** > > **** > > Yep, it fails, but the build succeeds, so no problem.**** > > **** > > It looks like these two files were NOT compiled in the ar based**** > > build. The two make.log files are attached.**** > > **** > > One more question. I tried to run the python based build**** > > the second time (after it was unsuccessful the first time)**** > > and then it deleted all the previous .o files it had created in **** > > ARCH/lib/libpetsc-obj. How can that be avoided?**** > > **** > > It does not do that unless you run 'clean'.**** > > **** > > Matt**** > > **** > > Chetan**** > > **** > > **** > > *From:* petsc-dev-bounces(a)mcs.anl.gov [mailto: > petsc-dev-bounces(a)mcs.anl.gov] *On Behalf Of *Matthew Knepley > *Sent:* Thursday, June 28, 2012 1:35 PM > *To:* For users of the development version of PETSc > *Subject:* Re: [petsc-dev] slow "ar Scq" on cygwin**** > > **** > > On Thu, Jun 28, 2012 at 2:23 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> > wrote:**** > > Hi devs, > > Is there a way to avoid running the archiver (ar) > every time a petsc directory is compiled? > > The reason I'm asking this is that when libpetsc.lib > becomes large enough (say 100MB+, esp. for debug build), > "ar Sqc" on cygwin takes very long in each directory. > Depending on OS caching, very long can be 10 seconds per > PETSc directory just to run ar. > > It creates a new temp file from scratch, puts stuff from > libpetsc.lib and new stuff in it and then renames the > temp file. As you can imagine, the compilation process is > fast in the beginning but crawls by the end. I've seen > the same IO sequence on Linux (with Sqc flags to ar), but > since cygwin IO is slower the effect is more clearly visible. > Building within a ramdisk leads to 25% faster compilation > but it was more of a curiosity.**** > > **** > > Build with Python. It never uses antique things like ar :)**** > > **** > > python2.7 ./config/builder2.py build**** > > **** > > Matt**** > > **** > > > Chetan**** > > > > **** > > **** > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener**** > > > > **** > > **** > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener**** > > > > **** > > **** > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener**** > > > > **** > > ** ** > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener**** > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] slow "ar Scq" on cygwin
by Chetan Jhurani 28 Jun '12

28 Jun '12

> -----Original Message----- > From: Satish Balay > Sent: Thursday, June 28, 2012 7:39 PM > To: For users of the development version of PETSc > Subject: Re: [petsc-dev] slow "ar Scq" on cygwin > > Hm - eventhough you added 'argparse.py' to python-2.6 on cygwin to get > builder.py working - perhaps Matt is using some other python-2.7 > feature - and thats causing this difference? Don't know. For what it's worth, my Linux box has python 2.6.6 and I had to put argparse.py there as well. Chetan > > Satish > > > > > > > > > I don't mind extra debugging on cygwin/Windows before reporting issues > > > > since I know the petsc project has many other priorities. But a few useful > > > > debugging pointers will be serve both of us better. > > > > > > > > Chetan > > > > > > > > > > > > > > > > > > > > From: petsc-dev-bounces(a)mcs.anl.gov [mailto:[email protected]] On Behalf Of Matthew > Knepley > > Sent: Thursday, June 28, 2012 6:00 PM > > To: For users of the development version of PETSc > > Subject: Re: [petsc-dev] slow "ar Scq" on cygwin > > > > > > > > On Thu, Jun 28, 2012 at 4:49 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> wrote: > > > > > > $ PETSC_DIR=/cygdrive/d/petsc-dev-2012-06-26 PETSC_ARCH=test_8 ./config/builder.py > > > > (The same happens (existing .o are deleted) if ./config/builder2.py build is used) > > > > > This is not what I wrote. You used builder.py, which is python 2.4 compatible, but has to do > everything without options, > > > so yes it deletes all the objects and rebuilds everything. That is why I did not suggest you use > it. > > > > > I said to use builder2.py, which does proper dependency tracking. > > > > > > > > That is what I used and I mentioned it above. I've tried both separately. > > > > > > > > Both give the same result that libpetsc.lib is not created. And > > > > whether I use builder.py or builder2.py, the existing objects > > > > files are deleted in the second build. I did not call for a cleanup. > > > > > > > > Look. It does not do that. Ever. In any universe. I don't know what you think you are doing, > > > > but it is definitely not what you wrote. Look at the code. It can't do that. > > > > > > > > matt > > > > > > > > To be sure, I tried the whole thing in a new ARCH with a fresh > > > > configure step, and this time builder.py was not used at all (by me) > > > > but it led to the same issues with builder2.py. > > > > > > > > I understand that this is not the most used platform, so let me > > > > know if I can be of help. And I forgot to mention that the > > > > python build was 4-5 times faster than "ar" based build and > > > > that's great... > > > > > > > > Chetan > > > > > > > > > > > > > > > > > > > > > > > > From: petsc-dev-bounces(a)mcs.anl.gov [mailto:[email protected]] On Behalf Of Matthew > Knepley > > Sent: Thursday, June 28, 2012 3:21 PM > > To: For users of the development version of PETSc > > Subject: Re: [petsc-dev] slow "ar Scq" on cygwin > > > > > > > > On Thu, Jun 28, 2012 at 4:10 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> wrote: > > > > Matt, > > > > > > > > I was surprised too on both counts. Couldn't find any libpetsc.lib after the python build > > > > and I did not run "clean" in between that could have deleted the .o files. > > > > > > > > Here's one more try below to show it. > > > > > > > > $ ls test_8/lib/ > > > > libpetsc-obj > > > > > > > > $ ls test_8/lib/libpetsc-obj/ | wc -l > > > > 560 > > > > > > > > $ PETSC_DIR=/cygdrive/d/petsc-dev-2012-06-26 PETSC_ARCH=test_8 ./config/builder.py > > > > (The same happens (existing .o are deleted) if ./config/builder2.py build is used) > > > > > > > > This is not what I wrote. You used builder.py, which is python 2.4 compatible, but has to do > everything without options, > > > > so yes it deletes all the objects and rebuilds everything. That is why I did not suggest you use it. > > > > > > > > I said to use builder2.py, which does proper dependency tracking. > > > > > > > > Matt > > > > > > > > > > > > Rebuilding Dependencies > > > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys > > > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/bag > > > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/comm > > > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/dll > > > > > > > > I stopped the build using control-c. It would have deleted all files first. > > > > > > > > Traceback (most recent call last): > > > > File "./config/builder.py", line 1534, in <module> > > > > PETScMaker().run() > > > > File "./config/builder.py", line 1526, in run > > > > if self.buildLibraries('libpetsc', self.rootDir): > > > > File "./config/builder.py", line 1331, in buildLibraries > > > > objects += self.buildDir(root, files, objDir) > > > > File "./config/builder.py", line 1251, in buildDir > > > > objects.extend(getattr(self, 'compile'+language)(sourceMap[language], objDir)) > > > > File "./config/builder.py", line 1025, in compileC > > > > return self.compile(self.configInfo.languages.clanguage, source, objDir) > > > > File "./config/builder.py", line 1012, in compile > > > > (output, error, status) = self.executeShellCommand(cmd, checkCommand = noCheckCommand, > log=self.log) > > > > File "/cygdrive/d/petsc-dev-2012-06-26/config/BuildSystem/script.py", line 254, in > executeShellCommand > > > > (output, error, status) = runInShell(command, log, cwd) > > > > File "/cygdrive/d/petsc-dev-2012-06-26/config/BuildSystem/script.py", line 243, in runInShell > > > > thread.join(timeout) > > > > File "/usr/lib/python2.6/threading.py", line 655, in join > > > > self.__block.wait(delay) > > > > File "/usr/lib/python2.6/threading.py", line 258, in wait > > > > _sleep(delay) > > > > KeyboardInterrupt > > > > > > > > $ ls test_8/lib/libpetsc-obj/ > > > > bag.o > > > > > > > > All the previous .o were deleted. Perhaps I'm not doing something right? > > > > > > > > Chetan > > > > > > > > > > > > From: petsc-dev-bounces(a)mcs.anl.gov [mailto:[email protected]] On Behalf Of Matthew > Knepley > > Sent: Thursday, June 28, 2012 2:46 PM > > To: For users of the development version of PETSc > > Subject: Re: [petsc-dev] slow "ar Scq" on cygwin > > > > > > > > On Thu, Jun 28, 2012 at 3:31 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> wrote: > > > > Thanks Matt, Satish. A few issues though. > > > > > > > > Like you said, the latest cygwin has python 2.6.8 which > > > > does not have argparse. When I place the following > > > > in path http://argparse.googlecode.com/svn/trunk/argparse.py > > > > I can run the python build with 2.6.8. > > > > > > > > However, the "ar" based build had worked fine and the python > > > > based build fails for these two files. > > > > > > > > src/snes/impls/MULTIB~1/multiblock.c > > > > src/snes/impls/vi/rsaug/virsaug.c > > > > > > > > Yep, it fails, but the build succeeds, so no problem. > > > > > > > > It looks like these two files were NOT compiled in the ar based > > > > build. The two make.log files are attached. > > > > > > > > One more question. I tried to run the python based build > > > > the second time (after it was unsuccessful the first time) > > > > and then it deleted all the previous .o files it had created in > > > > ARCH/lib/libpetsc-obj. How can that be avoided? > > > > > > > > It does not do that unless you run 'clean'. > > > > > > > > Matt > > > > > > > > Chetan > > > > > > > > > > > > From: petsc-dev-bounces(a)mcs.anl.gov [mailto:[email protected]] On Behalf Of Matthew > Knepley > > Sent: Thursday, June 28, 2012 1:35 PM > > To: For users of the development version of PETSc > > Subject: Re: [petsc-dev] slow "ar Scq" on cygwin > > > > > > > > On Thu, Jun 28, 2012 at 2:23 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> wrote: > > > > Hi devs, > > > > Is there a way to avoid running the archiver (ar) > > every time a petsc directory is compiled? > > > > The reason I'm asking this is that when libpetsc.lib > > becomes large enough (say 100MB+, esp. for debug build), > > "ar Sqc" on cygwin takes very long in each directory. > > Depending on OS caching, very long can be 10 seconds per > > PETSc directory just to run ar. > > > > It creates a new temp file from scratch, puts stuff from > > libpetsc.lib and new stuff in it and then renames the > > temp file. As you can imagine, the compilation process is > > fast in the beginning but crawls by the end. I've seen > > the same IO sequence on Linux (with Sqc flags to ar), but > > since cygwin IO is slower the effect is more clearly visible. > > Building within a ramdisk leads to 25% faster compilation > > but it was more of a curiosity. > > > > > > > > Build with Python. It never uses antique things like ar :) > > > > > > > > python2.7 ./config/builder2.py build > > > > > > > > Matt > > > > > > > > > > Chetan > > > > > > > > > > > > > > > >

1 0

Re: [petsc-dev] slow "ar Scq" on cygwin
by Satish Balay 28 Jun '12

28 Jun '12

On Thu, 28 Jun 2012, Chetan Jhurani wrote: > > Look. It does not do that. Ever. In any universe. I don't know what you think you are doing, > > > but it is definitely not what you wrote. Look at the code. It can't do that. > > > > I don't think you understand what your code is doing in a new > > environment of the same universe. > > > > It works fine on Linux with the steps you had given. > > But on the cygwin side, no dependency files like > > ./lib/libpetsc-obj/adaptbasic.d are created. I found this after > > running the python build on Linux (I didn't know before this > > where the dependencies were stored). > > > > Here is some debug info in the python code: > > > > In the buildLibraries function on the Linux box (when building in the second step) > > > > self.sourceDatabase = DirectedGraph with 1172 vertices and 21894 edges > > > > However, on the cygwin side (again, when building the second time) > > > > self.sourceDatabase = <builder.NullSourceDatabase object at 0xffd833ac> > > > > So of course it is totalRebuild = true on cygwin. Hm - eventhough you added 'argparse.py' to python-2.6 on cygwin to get builder.py working - perhaps Matt is using some other python-2.7 feature - and thats causing this difference? Satish > > > > I don't mind extra debugging on cygwin/Windows before reporting issues > > since I know the petsc project has many other priorities. But a few useful > > debugging pointers will be serve both of us better. > > > > Chetan > > > > > > > > > > From: petsc-dev-bounces(a)mcs.anl.gov [mailto:[email protected]] On Behalf Of Matthew Knepley > Sent: Thursday, June 28, 2012 6:00 PM > To: For users of the development version of PETSc > Subject: Re: [petsc-dev] slow "ar Scq" on cygwin > > > > On Thu, Jun 28, 2012 at 4:49 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> wrote: > > > > $ PETSC_DIR=/cygdrive/d/petsc-dev-2012-06-26 PETSC_ARCH=test_8 ./config/builder.py > > > (The same happens (existing .o are deleted) if ./config/builder2.py build is used) > > > This is not what I wrote. You used builder.py, which is python 2.4 compatible, but has to do everything without options, > > so yes it deletes all the objects and rebuilds everything. That is why I did not suggest you use it. > > > I said to use builder2.py, which does proper dependency tracking. > > > > That is what I used and I mentioned it above. I've tried both separately. > > > > Both give the same result that libpetsc.lib is not created. And > > whether I use builder.py or builder2.py, the existing objects > > files are deleted in the second build. I did not call for a cleanup. > > > > Look. It does not do that. Ever. In any universe. I don't know what you think you are doing, > > but it is definitely not what you wrote. Look at the code. It can't do that. > > > > matt > > > > To be sure, I tried the whole thing in a new ARCH with a fresh > > configure step, and this time builder.py was not used at all (by me) > > but it led to the same issues with builder2.py. > > > > I understand that this is not the most used platform, so let me > > know if I can be of help. And I forgot to mention that the > > python build was 4-5 times faster than "ar" based build and > > that's great... > > > > Chetan > > > > > > > > > > > > From: petsc-dev-bounces(a)mcs.anl.gov [mailto:[email protected]] On Behalf Of Matthew Knepley > Sent: Thursday, June 28, 2012 3:21 PM > To: For users of the development version of PETSc > Subject: Re: [petsc-dev] slow "ar Scq" on cygwin > > > > On Thu, Jun 28, 2012 at 4:10 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> wrote: > > Matt, > > > > I was surprised too on both counts. Couldn't find any libpetsc.lib after the python build > > and I did not run "clean" in between that could have deleted the .o files. > > > > Here's one more try below to show it. > > > > $ ls test_8/lib/ > > libpetsc-obj > > > > $ ls test_8/lib/libpetsc-obj/ | wc -l > > 560 > > > > $ PETSC_DIR=/cygdrive/d/petsc-dev-2012-06-26 PETSC_ARCH=test_8 ./config/builder.py > > (The same happens (existing .o are deleted) if ./config/builder2.py build is used) > > > > This is not what I wrote. You used builder.py, which is python 2.4 compatible, but has to do everything without options, > > so yes it deletes all the objects and rebuilds everything. That is why I did not suggest you use it. > > > > I said to use builder2.py, which does proper dependency tracking. > > > > Matt > > > > > > Rebuilding Dependencies > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/bag > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/comm > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/dll > > > > I stopped the build using control-c. It would have deleted all files first. > > > > Traceback (most recent call last): > > File "./config/builder.py", line 1534, in <module> > > PETScMaker().run() > > File "./config/builder.py", line 1526, in run > > if self.buildLibraries('libpetsc', self.rootDir): > > File "./config/builder.py", line 1331, in buildLibraries > > objects += self.buildDir(root, files, objDir) > > File "./config/builder.py", line 1251, in buildDir > > objects.extend(getattr(self, 'compile'+language)(sourceMap[language], objDir)) > > File "./config/builder.py", line 1025, in compileC > > return self.compile(self.configInfo.languages.clanguage, source, objDir) > > File "./config/builder.py", line 1012, in compile > > (output, error, status) = self.executeShellCommand(cmd, checkCommand = noCheckCommand, log=self.log) > > File "/cygdrive/d/petsc-dev-2012-06-26/config/BuildSystem/script.py", line 254, in executeShellCommand > > (output, error, status) = runInShell(command, log, cwd) > > File "/cygdrive/d/petsc-dev-2012-06-26/config/BuildSystem/script.py", line 243, in runInShell > > thread.join(timeout) > > File "/usr/lib/python2.6/threading.py", line 655, in join > > self.__block.wait(delay) > > File "/usr/lib/python2.6/threading.py", line 258, in wait > > _sleep(delay) > > KeyboardInterrupt > > > > $ ls test_8/lib/libpetsc-obj/ > > bag.o > > > > All the previous .o were deleted. Perhaps I'm not doing something right? > > > > Chetan > > > > > > From: petsc-dev-bounces(a)mcs.anl.gov [mailto:[email protected]] On Behalf Of Matthew Knepley > Sent: Thursday, June 28, 2012 2:46 PM > To: For users of the development version of PETSc > Subject: Re: [petsc-dev] slow "ar Scq" on cygwin > > > > On Thu, Jun 28, 2012 at 3:31 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> wrote: > > Thanks Matt, Satish. A few issues though. > > > > Like you said, the latest cygwin has python 2.6.8 which > > does not have argparse. When I place the following > > in path http://argparse.googlecode.com/svn/trunk/argparse.py > > I can run the python build with 2.6.8. > > > > However, the "ar" based build had worked fine and the python > > based build fails for these two files. > > > > src/snes/impls/MULTIB~1/multiblock.c > > src/snes/impls/vi/rsaug/virsaug.c > > > > Yep, it fails, but the build succeeds, so no problem. > > > > It looks like these two files were NOT compiled in the ar based > > build. The two make.log files are attached. > > > > One more question. I tried to run the python based build > > the second time (after it was unsuccessful the first time) > > and then it deleted all the previous .o files it had created in > > ARCH/lib/libpetsc-obj. How can that be avoided? > > > > It does not do that unless you run 'clean'. > > > > Matt > > > > Chetan > > > > > > From: petsc-dev-bounces(a)mcs.anl.gov [mailto:[email protected]] On Behalf Of Matthew Knepley > Sent: Thursday, June 28, 2012 1:35 PM > To: For users of the development version of PETSc > Subject: Re: [petsc-dev] slow "ar Scq" on cygwin > > > > On Thu, Jun 28, 2012 at 2:23 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> wrote: > > Hi devs, > > Is there a way to avoid running the archiver (ar) > every time a petsc directory is compiled? > > The reason I'm asking this is that when libpetsc.lib > becomes large enough (say 100MB+, esp. for debug build), > "ar Sqc" on cygwin takes very long in each directory. > Depending on OS caching, very long can be 10 seconds per > PETSc directory just to run ar. > > It creates a new temp file from scratch, puts stuff from > libpetsc.lib and new stuff in it and then renames the > temp file. As you can imagine, the compilation process is > fast in the beginning but crawls by the end. I've seen > the same IO sequence on Linux (with Sqc flags to ar), but > since cygwin IO is slower the effect is more clearly visible. > Building within a ramdisk leads to 25% faster compilation > but it was more of a curiosity. > > > > Build with Python. It never uses antique things like ar :) > > > > python2.7 ./config/builder2.py build > > > > Matt > > > > > Chetan > > > > > > > >

1 0

Re: [petsc-dev] slow "ar Scq" on cygwin
by Chetan Jhurani 28 Jun '12

28 Jun '12

> Look. It does not do that. Ever. In any universe. I don't know what you think you are doing, > but it is definitely not what you wrote. Look at the code. It can't do that. I don't think you understand what your code is doing in a new environment of the same universe. It works fine on Linux with the steps you had given. But on the cygwin side, no dependency files like ./lib/libpetsc-obj/adaptbasic.d are created. I found this after running the python build on Linux (I didn't know before this where the dependencies were stored). Here is some debug info in the python code: In the buildLibraries function on the Linux box (when building in the second step) self.sourceDatabase = DirectedGraph with 1172 vertices and 21894 edges However, on the cygwin side (again, when building the second time) self.sourceDatabase = <builder.NullSourceDatabase object at 0xffd833ac> So of course it is totalRebuild = true on cygwin. I don't mind extra debugging on cygwin/Windows before reporting issues since I know the petsc project has many other priorities. But a few useful debugging pointers will be serve both of us better. Chetan From: petsc-dev-bounces(a)mcs.anl.gov [mailto:[email protected]] On Behalf Of Matthew Knepley Sent: Thursday, June 28, 2012 6:00 PM To: For users of the development version of PETSc Subject: Re: [petsc-dev] slow "ar Scq" on cygwin On Thu, Jun 28, 2012 at 4:49 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> wrote: > > $ PETSC_DIR=/cygdrive/d/petsc-dev-2012-06-26 PETSC_ARCH=test_8 ./config/builder.py > > (The same happens (existing .o are deleted) if ./config/builder2.py build is used) > This is not what I wrote. You used builder.py, which is python 2.4 compatible, but has to do everything without options, > so yes it deletes all the objects and rebuilds everything. That is why I did not suggest you use it. > I said to use builder2.py, which does proper dependency tracking. That is what I used and I mentioned it above. I've tried both separately. Both give the same result that libpetsc.lib is not created. And whether I use builder.py or builder2.py, the existing objects files are deleted in the second build. I did not call for a cleanup. Look. It does not do that. Ever. In any universe. I don't know what you think you are doing, but it is definitely not what you wrote. Look at the code. It can't do that. matt To be sure, I tried the whole thing in a new ARCH with a fresh configure step, and this time builder.py was not used at all (by me) but it led to the same issues with builder2.py. I understand that this is not the most used platform, so let me know if I can be of help. And I forgot to mention that the python build was 4-5 times faster than "ar" based build and that's great... Chetan From: petsc-dev-bounces(a)mcs.anl.gov [mailto:[email protected]] On Behalf Of Matthew Knepley Sent: Thursday, June 28, 2012 3:21 PM To: For users of the development version of PETSc Subject: Re: [petsc-dev] slow "ar Scq" on cygwin On Thu, Jun 28, 2012 at 4:10 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> wrote: Matt, I was surprised too on both counts. Couldn't find any libpetsc.lib after the python build and I did not run "clean" in between that could have deleted the .o files. Here's one more try below to show it. $ ls test_8/lib/ libpetsc-obj $ ls test_8/lib/libpetsc-obj/ | wc -l 560 $ PETSC_DIR=/cygdrive/d/petsc-dev-2012-06-26 PETSC_ARCH=test_8 ./config/builder.py (The same happens (existing .o are deleted) if ./config/builder2.py build is used) This is not what I wrote. You used builder.py, which is python 2.4 compatible, but has to do everything without options, so yes it deletes all the objects and rebuilds everything. That is why I did not suggest you use it. I said to use builder2.py, which does proper dependency tracking. Matt Rebuilding Dependencies Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/bag Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/comm Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/dll I stopped the build using control-c. It would have deleted all files first. Traceback (most recent call last): File "./config/builder.py", line 1534, in <module> PETScMaker().run() File "./config/builder.py", line 1526, in run if self.buildLibraries('libpetsc', self.rootDir): File "./config/builder.py", line 1331, in buildLibraries objects += self.buildDir(root, files, objDir) File "./config/builder.py", line 1251, in buildDir objects.extend(getattr(self, 'compile'+language)(sourceMap[language], objDir)) File "./config/builder.py", line 1025, in compileC return self.compile(self.configInfo.languages.clanguage, source, objDir) File "./config/builder.py", line 1012, in compile (output, error, status) = self.executeShellCommand(cmd, checkCommand = noCheckCommand, log=self.log) File "/cygdrive/d/petsc-dev-2012-06-26/config/BuildSystem/script.py", line 254, in executeShellCommand (output, error, status) = runInShell(command, log, cwd) File "/cygdrive/d/petsc-dev-2012-06-26/config/BuildSystem/script.py", line 243, in runInShell thread.join(timeout) File "/usr/lib/python2.6/threading.py", line 655, in join self.__block.wait(delay) File "/usr/lib/python2.6/threading.py", line 258, in wait _sleep(delay) KeyboardInterrupt $ ls test_8/lib/libpetsc-obj/ bag.o All the previous .o were deleted. Perhaps I'm not doing something right? Chetan From: petsc-dev-bounces(a)mcs.anl.gov [mailto:[email protected]] On Behalf Of Matthew Knepley Sent: Thursday, June 28, 2012 2:46 PM To: For users of the development version of PETSc Subject: Re: [petsc-dev] slow "ar Scq" on cygwin On Thu, Jun 28, 2012 at 3:31 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> wrote: Thanks Matt, Satish. A few issues though. Like you said, the latest cygwin has python 2.6.8 which does not have argparse. When I place the following in path http://argparse.googlecode.com/svn/trunk/argparse.py I can run the python build with 2.6.8. However, the "ar" based build had worked fine and the python based build fails for these two files. src/snes/impls/MULTIB~1/multiblock.c src/snes/impls/vi/rsaug/virsaug.c Yep, it fails, but the build succeeds, so no problem. It looks like these two files were NOT compiled in the ar based build. The two make.log files are attached. One more question. I tried to run the python based build the second time (after it was unsuccessful the first time) and then it deleted all the previous .o files it had created in ARCH/lib/libpetsc-obj. How can that be avoided? It does not do that unless you run 'clean'. Matt Chetan From: petsc-dev-bounces(a)mcs.anl.gov [mailto:[email protected]] On Behalf Of Matthew Knepley Sent: Thursday, June 28, 2012 1:35 PM To: For users of the development version of PETSc Subject: Re: [petsc-dev] slow "ar Scq" on cygwin On Thu, Jun 28, 2012 at 2:23 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> wrote: Hi devs, Is there a way to avoid running the archiver (ar) every time a petsc directory is compiled? The reason I'm asking this is that when libpetsc.lib becomes large enough (say 100MB+, esp. for debug build), "ar Sqc" on cygwin takes very long in each directory. Depending on OS caching, very long can be 10 seconds per PETSc directory just to run ar. It creates a new temp file from scratch, puts stuff from libpetsc.lib and new stuff in it and then renames the temp file. As you can imagine, the compilation process is fast in the beginning but crawls by the end. I've seen the same IO sequence on Linux (with Sqc flags to ar), but since cygwin IO is slower the effect is more clearly visible. Building within a ramdisk leads to 25% faster compilation but it was more of a curiosity. Build with Python. It never uses antique things like ar :) python2.7 ./config/builder2.py build Matt Chetan -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] slow "ar Scq" on cygwin
by Satish Balay 28 Jun '12

28 Jun '12

On Thu, 28 Jun 2012, Barry Smith wrote: > > Satish, > > Is there a Windows machine we can ssh into running cygwin so we can actually try things on a reasonable schedule and debug issues that come up? Having you do everything windows-wise is an unneeded bottleneck. > ps3.mcs.anl.gov This is a VM where petsc nightly builds get run via ssh. I had created accounts for you an matt a while ago [and can create accounts for anyone who might need access]. Satish

1 0

Re: [petsc-dev] slow "ar Scq" on cygwin
by Barry Smith 28 Jun '12

28 Jun '12

Satish, Is there a Windows machine we can ssh into running cygwin so we can actually try things on a reasonable schedule and debug issues that come up? Having you do everything windows-wise is an unneeded bottleneck. Barry On Jun 28, 2012, at 7:59 PM, Matthew Knepley wrote: > On Thu, Jun 28, 2012 at 4:49 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> wrote: > > > $ PETSC_DIR=/cygdrive/d/petsc-dev-2012-06-26 PETSC_ARCH=test_8 ./config/builder.py > > > (The same happens (existing .o are deleted) if ./config/builder2.py build is used) > > > This is not what I wrote. You used builder.py, which is python 2.4 compatible, but has to do everything without options, > > so yes it deletes all the objects and rebuilds everything. That is why I did not suggest you use it. > > > I said to use builder2.py, which does proper dependency tracking. > > > > That is what I used and I mentioned it above. I’ve tried both separately. > > > > Both give the same result that libpetsc.lib is not created. And > > whether I use builder.py or builder2.py, the existing objects > > files are deleted in the second build. I did not call for a cleanup. > > > Look. It does not do that. Ever. In any universe. I don't know what you think you are doing, > but it is definitely not what you wrote. Look at the code. It can't do that. > > matt > > To be sure, I tried the whole thing in a new ARCH with a fresh > > configure step, and this time builder.py was not used at all (by me) > > but it led to the same issues with builder2.py. > > > > I understand that this is not the most used platform, so let me > > know if I can be of help. And I forgot to mention that the > > python build was 4-5 times faster than “ar” based build and > > that’s great... > > > > Chetan > > > > > > > > > > > > From: petsc-dev-bounces(a)mcs.anl.gov [mailto:[email protected]] On Behalf Of Matthew Knepley > Sent: Thursday, June 28, 2012 3:21 PM > To: For users of the development version of PETSc > Subject: Re: [petsc-dev] slow "ar Scq" on cygwin > > > > On Thu, Jun 28, 2012 at 4:10 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> wrote: > > Matt, > > > > I was surprised too on both counts. Couldn’t find any libpetsc.lib after the python build > > and I did not run “clean” in between that could have deleted the .o files. > > > > Here’s one more try below to show it. > > > > $ ls test_8/lib/ > > libpetsc-obj > > > > $ ls test_8/lib/libpetsc-obj/ | wc -l > > 560 > > > > $ PETSC_DIR=/cygdrive/d/petsc-dev-2012-06-26 PETSC_ARCH=test_8 ./config/builder.py > > (The same happens (existing .o are deleted) if ./config/builder2.py build is used) > > > > This is not what I wrote. You used builder.py, which is python 2.4 compatible, but has to do everything without options, > > so yes it deletes all the objects and rebuilds everything. That is why I did not suggest you use it. > > > > I said to use builder2.py, which does proper dependency tracking. > > > > Matt > > > > > > Rebuilding Dependencies > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/bag > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/comm > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/dll > > > > I stopped the build using control-c. It would have deleted all files first. > > > > Traceback (most recent call last): > > File "./config/builder.py", line 1534, in <module> > > PETScMaker().run() > > File "./config/builder.py", line 1526, in run > > if self.buildLibraries('libpetsc', self.rootDir): > > File "./config/builder.py", line 1331, in buildLibraries > > objects += self.buildDir(root, files, objDir) > > File "./config/builder.py", line 1251, in buildDir > > objects.extend(getattr(self, 'compile'+language)(sourceMap[language], objDir)) > > File "./config/builder.py", line 1025, in compileC > > return self.compile(self.configInfo.languages.clanguage, source, objDir) > > File "./config/builder.py", line 1012, in compile > > (output, error, status) = self.executeShellCommand(cmd, checkCommand = noCheckCommand, log=self.log) > > File "/cygdrive/d/petsc-dev-2012-06-26/config/BuildSystem/script.py", line 254, in executeShellCommand > > (output, error, status) = runInShell(command, log, cwd) > > File "/cygdrive/d/petsc-dev-2012-06-26/config/BuildSystem/script.py", line 243, in runInShell > > thread.join(timeout) > > File "/usr/lib/python2.6/threading.py", line 655, in join > > self.__block.wait(delay) > > File "/usr/lib/python2.6/threading.py", line 258, in wait > > _sleep(delay) > > KeyboardInterrupt > > > > $ ls test_8/lib/libpetsc-obj/ > > bag.o > > > > All the previous .o were deleted. Perhaps I’m not doing something right? > > > > Chetan > > > > > > From: petsc-dev-bounces(a)mcs.anl.gov [mailto:[email protected]] On Behalf Of Matthew Knepley > Sent: Thursday, June 28, 2012 2:46 PM > To: For users of the development version of PETSc > Subject: Re: [petsc-dev] slow "ar Scq" on cygwin > > > > On Thu, Jun 28, 2012 at 3:31 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> wrote: > > Thanks Matt, Satish. A few issues though. > > > > Like you said, the latest cygwin has python 2.6.8 which > > does not have argparse. When I place the following > > in path http://argparse.googlecode.com/svn/trunk/argparse.py > > I can run the python build with 2.6.8. > > > > However, the “ar” based build had worked fine and the python > > based build fails for these two files. > > > > src/snes/impls/MULTIB~1/multiblock.c > > src/snes/impls/vi/rsaug/virsaug.c > > > > Yep, it fails, but the build succeeds, so no problem. > > > > It looks like these two files were NOT compiled in the ar based > > build. The two make.log files are attached. > > > > One more question. I tried to run the python based build > > the second time (after it was unsuccessful the first time) > > and then it deleted all the previous .o files it had created in > > ARCH/lib/libpetsc-obj. How can that be avoided? > > > > It does not do that unless you run 'clean'. > > > > Matt > > > > Chetan > > > > > > From: petsc-dev-bounces(a)mcs.anl.gov [mailto:[email protected]] On Behalf Of Matthew Knepley > Sent: Thursday, June 28, 2012 1:35 PM > To: For users of the development version of PETSc > Subject: Re: [petsc-dev] slow "ar Scq" on cygwin > > > > On Thu, Jun 28, 2012 at 2:23 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> wrote: > > Hi devs, > > Is there a way to avoid running the archiver (ar) > every time a petsc directory is compiled? > > The reason I'm asking this is that when libpetsc.lib > becomes large enough (say 100MB+, esp. for debug build), > "ar Sqc" on cygwin takes very long in each directory. > Depending on OS caching, very long can be 10 seconds per > PETSc directory just to run ar. > > It creates a new temp file from scratch, puts stuff from > libpetsc.lib and new stuff in it and then renames the > temp file. As you can imagine, the compilation process is > fast in the beginning but crawls by the end. I've seen > the same IO sequence on Linux (with Sqc flags to ar), but > since cygwin IO is slower the effect is more clearly visible. > Building within a ramdisk leads to 25% faster compilation > but it was more of a curiosity. > > > > Build with Python. It never uses antique things like ar :) > > > > python2.7 ./config/builder2.py build > > > > Matt > > > > > Chetan > > > > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener > > > > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener > > > > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener > > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener

1 0

Re: [petsc-dev] slow "ar Scq" on cygwin
by Matthew Knepley 28 Jun '12

28 Jun '12

On Thu, Jun 28, 2012 at 4:49 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com>wrote: > *> > $ PETSC_DIR=/cygdrive/d/petsc-dev-2012-06-26 PETSC_ARCH=test_8 > ./config/builder.py > > > (The same happens (existing .o are deleted) if ./config/builder2.py > build is used)* > > *> *This is not what I wrote. You used builder.py, which is python 2.4 > compatible, but has to do everything without options, > *> *so yes it deletes all the objects and rebuilds everything. That is > why I did not suggest you use it.**** > > *> *I said to use builder2.py, which does proper dependency tracking.**** > > ** ** > > That is what I used and I mentioned it above. I’ve tried both separately.* > *** > > ** ** > > Both give the same result that libpetsc.lib is not created. And**** > > whether I use builder.py or builder2.py, the existing objects**** > > files are deleted in the second build. I did not call for a cleanup. > Look. It does not do that. Ever. In any universe. I don't know what you think you are doing, but it is definitely not what you wrote. Look at the code. It can't do that. matt > To be sure, I tried the whole thing in a new ARCH with a fresh**** > > configure step, and this time builder.py was not used at all (by me)**** > > but it led to the same issues with builder2.py.**** > > ** ** > > I understand that this is not the most used platform, so let me**** > > know if I can be of help. And I forgot to mention that the**** > > python build was 4-5 times faster than “ar” based build and**** > > that’s great...**** > > ** ** > > Chetan**** > > ** ** > > ** ** > > ** ** > > ** ** > > ** ** > > *From:* petsc-dev-bounces(a)mcs.anl.gov [mailto: > petsc-dev-bounces(a)mcs.anl.gov] *On Behalf Of *Matthew Knepley > *Sent:* Thursday, June 28, 2012 3:21 PM > *To:* For users of the development version of PETSc > *Subject:* Re: [petsc-dev] slow "ar Scq" on cygwin**** > > ** ** > > On Thu, Jun 28, 2012 at 4:10 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> > wrote:**** > > Matt,**** > > **** > > I was surprised too on both counts. Couldn’t find any libpetsc.lib after > the python build**** > > and I did not run “clean” in between that could have deleted the .o files. > **** > > **** > > Here’s one more try below to show it.**** > > **** > > *$ ls test_8/lib/***** > > libpetsc-obj**** > > **** > > *$ ls test_8/lib/libpetsc-obj/ | wc -l***** > > *560***** > > **** > > *$ PETSC_DIR=/cygdrive/d/petsc-dev-2012-06-26 PETSC_ARCH=test_8 > ./config/builder.py***** > > *(The same happens (existing .o are deleted) if ./config/builder2.py > build is used)***** > > ** ** > > This is not what I wrote. You used builder.py, which is python 2.4 > compatible, but has to do everything without options,**** > > so yes it deletes all the objects and rebuilds everything. That is why I > did not suggest you use it.**** > > ** ** > > I said to use builder2.py, which does proper dependency tracking.**** > > ** ** > > Matt**** > > **** > > * ***** > > Rebuilding Dependencies**** > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys**** > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/bag**** > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/comm**** > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/dll**** > > **** > > *I stopped the build using control-c. It would have deleted all files > first.***** > > **** > > Traceback (most recent call last):**** > > File "./config/builder.py", line 1534, in <module>**** > > PETScMaker().run()**** > > File "./config/builder.py", line 1526, in run**** > > if self.buildLibraries('libpetsc', self.rootDir):**** > > File "./config/builder.py", line 1331, in buildLibraries**** > > objects += self.buildDir(root, files, objDir)**** > > File "./config/builder.py", line 1251, in buildDir**** > > objects.extend(getattr(self, 'compile'+language)(sourceMap[language], > objDir))**** > > File "./config/builder.py", line 1025, in compileC**** > > return self.compile(self.configInfo.languages.clanguage, source, > objDir)**** > > File "./config/builder.py", line 1012, in compile**** > > (output, error, status) = self.executeShellCommand(cmd, checkCommand = > noCheckCommand, log=self.log)**** > > File "/cygdrive/d/petsc-dev-2012-06-26/config/BuildSystem/script.py", > line 254, in executeShellCommand**** > > (output, error, status) = runInShell(command, log, cwd)**** > > File "/cygdrive/d/petsc-dev-2012-06-26/config/BuildSystem/script.py", > line 243, in runInShell**** > > thread.join(timeout)**** > > File "/usr/lib/python2.6/threading.py", line 655, in join**** > > self.__block.wait(delay)**** > > File "/usr/lib/python2.6/threading.py", line 258, in wait**** > > _sleep(delay)**** > > KeyboardInterrupt**** > > **** > > *$ ls test_8/lib/libpetsc-obj/***** > > *bag.o***** > > **** > > All the previous .o were deleted. Perhaps I’m not doing something right?* > *** > > **** > > Chetan**** > > **** > > **** > > *From:* petsc-dev-bounces(a)mcs.anl.gov [mailto: > petsc-dev-bounces(a)mcs.anl.gov] *On Behalf Of *Matthew Knepley > *Sent:* Thursday, June 28, 2012 2:46 PM > *To:* For users of the development version of PETSc > *Subject:* Re: [petsc-dev] slow "ar Scq" on cygwin**** > > **** > > On Thu, Jun 28, 2012 at 3:31 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> > wrote:**** > > Thanks Matt, Satish. A few issues though.**** > > **** > > Like you said, the latest cygwin has python 2.6.8 which**** > > does not have argparse. When I place the following**** > > in path http://argparse.googlecode.com/svn/trunk/argparse.py**** > > I can run the python build with 2.6.8.**** > > **** > > However, the “ar” based build had worked fine and the python**** > > based build fails for these two files.**** > > **** > > src/snes/impls/MULTIB~1/multiblock.c**** > > src/snes/impls/vi/rsaug/virsaug.c**** > > **** > > Yep, it fails, but the build succeeds, so no problem.**** > > **** > > It looks like these two files were NOT compiled in the ar based**** > > build. The two make.log files are attached.**** > > **** > > One more question. I tried to run the python based build**** > > the second time (after it was unsuccessful the first time)**** > > and then it deleted all the previous .o files it had created in **** > > ARCH/lib/libpetsc-obj. How can that be avoided?**** > > **** > > It does not do that unless you run 'clean'.**** > > **** > > Matt**** > > **** > > Chetan**** > > **** > > **** > > *From:* petsc-dev-bounces(a)mcs.anl.gov [mailto: > petsc-dev-bounces(a)mcs.anl.gov] *On Behalf Of *Matthew Knepley > *Sent:* Thursday, June 28, 2012 1:35 PM > *To:* For users of the development version of PETSc > *Subject:* Re: [petsc-dev] slow "ar Scq" on cygwin**** > > **** > > On Thu, Jun 28, 2012 at 2:23 PM, Chetan Jhurani <chetan.jhurani(a)gmail.com> > wrote:**** > > Hi devs, > > Is there a way to avoid running the archiver (ar) > every time a petsc directory is compiled? > > The reason I'm asking this is that when libpetsc.lib > becomes large enough (say 100MB+, esp. for debug build), > "ar Sqc" on cygwin takes very long in each directory. > Depending on OS caching, very long can be 10 seconds per > PETSc directory just to run ar. > > It creates a new temp file from scratch, puts stuff from > libpetsc.lib and new stuff in it and then renames the > temp file. As you can imagine, the compilation process is > fast in the beginning but crawls by the end. I've seen > the same IO sequence on Linux (with Sqc flags to ar), but > since cygwin IO is slower the effect is more clearly visible. > Building within a ramdisk leads to 25% faster compilation > but it was more of a curiosity.**** > > **** > > Build with Python. It never uses antique things like ar :)**** > > **** > > python2.7 ./config/builder2.py build**** > > **** > > Matt**** > > **** > > > Chetan**** > > > > **** > > **** > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener**** > > > > **** > > **** > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener**** > > > > **** > > ** ** > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener**** > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] Patches like this make me want to cry
by Paul Mullowney 28 Jun '12

28 Jun '12

The TransplantPlaceArray, TransplantResetArray was very experimental and should have not have been committed. I removed it. > This is minutia now, but this statement is nonsense (it triggers > -Waddress and is meaningless). > > 38.66 + if (&y_array) y_array = PETSC_NULL; > > On Thu, Jun 28, 2012 at 2:53 PM, Jed Brown <jedbrown(a)mcs.anl.gov > <mailto:[email protected]>> wrote: > > On Thu, Jun 28, 2012 at 2:41 PM, Paul Mullowney <paulm(a)txcorp.com > <mailto:[email protected]>> wrote: > > I moved all CUSPARSE functions into seqcusparse/aijcusparse.cu > <http://aijcusparse.cu> (and a similar mpi version). I was > asked to do this some months ago. I did my best. I think it is > significantly improved from before. > > I was also asked by multiple people to attempt to get Complex > GPU capabilities working. I succeeded by making changes to > petscmath.h. As Jed pointed out, some of the changes may have > not been C99 compliant. > > > It needs to work with C89, not just C99. Global changes to handle > complex should also be done in separate patches. > > I hate the idea that CUSP leaks out into user code so I would > recommend casting internally rather than changing it globally. > That should also enable use of complex without requiring the user > to adopt a C++ compiler for their own code (C99 _Complex is > binary-compatible with std::complex and cusp::complex). > > > There is one commented out function in aijcusparse.cu > <http://aijcusparse.cu>. That can easily be fixed. > > Many of the other changes (especially to .cu) files were made > to protect against builds breaking because many of the CUSP > preconditioners do not support complex arithmetic. > > > Those should absolutely be in separate patches. Lumping it all > into one makes it impossible to review or test. > > > I have asked for input from multiple people on several > occasions and not gotten responses. > > What am I to do if I ask for input/feedback and don't get it????? > > > Next time, please provide the patch series. You could use "hg > email" or post your branch to bitbucket or elsewhere so that we > can comment. > > Also, please follow the coding guidelines. There is a developer's > manual and frequent discussion on petsc-dev about conventions. (If > you notice something that is adopted, but not in the developer's > manual, please point it out on the mailing list or make a patch.) > > > > -Paul > > > >> >> There are essentially no comments on this massive push. I was >> just reading it, getting mad. Jed mailed first. >> Pushes need enough comments that I can see exactly what >> functionality is being added and how it works. >> You will not be the maintainer of this functionality, we >> will. If we can't do it, it will be thrown out, and all your >> time will be wasted. >> >> Matt >> >> >> This is non-portable: >> >> http://petsc.cs.iit.edu/petsc/petsc-dev/rev/66ca8db0d5f8 >> >> >> Can we please institute some sort of policy on >> patch quality/reviewability? This one patch is >> going to take a significant amount of fix-up (not >> made easier by the several merges since) and/or >> generate several build failures and user >> inconvenience (petsc-maints). No doubt the >> functionality is important, but we just don't >> have time to fix these things line-by-line after >> they are pushed. >> >> >> >> >> >> >> -- >> What most experimenters take for granted before they begin >> their experiments is infinitely more interesting than any >> results to which their experiments lead. >> -- Norbert Wiener > > >

1 0

Re: [petsc-dev] Patches like this make me want to cry
by Jed Brown 28 Jun '12

28 Jun '12

This is minutia now, but this statement is nonsense (it triggers -Waddress and is meaningless). 38.66 + if (&y_array) y_array = PETSC_NULL; On Thu, Jun 28, 2012 at 2:53 PM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > On Thu, Jun 28, 2012 at 2:41 PM, Paul Mullowney <paulm(a)txcorp.com> wrote: > >> I moved all CUSPARSE functions into seqcusparse/aijcusparse.cu (and a >> similar mpi version). I was asked to do this some months ago. I did my >> best. I think it is significantly improved from before. >> >> I was also asked by multiple people to attempt to get Complex GPU >> capabilities working. I succeeded by making changes to petscmath.h. As Jed >> pointed out, some of the changes may have not been C99 compliant. >> > > It needs to work with C89, not just C99. Global changes to handle complex > should also be done in separate patches. > > I hate the idea that CUSP leaks out into user code so I would recommend > casting internally rather than changing it globally. That should also > enable use of complex without requiring the user to adopt a C++ compiler > for their own code (C99 _Complex is binary-compatible with std::complex and > cusp::complex). > > >> >> There is one commented out function in aijcusparse.cu. That can easily >> be fixed. >> >> Many of the other changes (especially to .cu) files were made to protect >> against builds breaking because many of the CUSP preconditioners do not >> support complex arithmetic. >> > > Those should absolutely be in separate patches. Lumping it all into one > makes it impossible to review or test. > > >> >> I have asked for input from multiple people on several occasions and not >> gotten responses. >> >> What am I to do if I ask for input/feedback and don't get it????? >> > > Next time, please provide the patch series. You could use "hg email" or > post your branch to bitbucket or elsewhere so that we can comment. > > Also, please follow the coding guidelines. There is a developer's manual > and frequent discussion on petsc-dev about conventions. (If you notice > something that is adopted, but not in the developer's manual, please point > it out on the mailing list or make a patch.) > > >> >> >> -Paul >> >> >> >> >> There are essentially no comments on this massive push. I was just >> reading it, getting mad. Jed mailed first. >> Pushes need enough comments that I can see exactly what functionality is >> being added and how it works. >> You will not be the maintainer of this functionality, we will. If we >> can't do it, it will be thrown out, and all your >> time will be wasted. >> >> Matt >> >> >>> >>>> This is non-portable: >>>>> >>>>> http://petsc.cs.iit.edu/petsc/petsc-dev/rev/66ca8db0d5f8 >>>>> >>>>> >>>>> Can we please institute some sort of policy on patch >>>>> quality/reviewability? This one patch is going to take a significant amount >>>>> of fix-up (not made easier by the several merges since) and/or generate >>>>> several build failures and user inconvenience (petsc-maints). No doubt the >>>>> functionality is important, but we just don't have time to fix these things >>>>> line-by-line after they are pushed. >>>>> >>>> >>>> >>> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> >> >

1 0